cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Support for GZIP Content-Encoding in HTTP responses

From: Dan Fandrich <dan_at_coneharvesters.com>
Date: Wed, 16 Apr 2003 14:51:17 -0700

On Fri, Apr 11, 2003 at 04:20:11PM +0100, niclists_at_nichines.com wrote:
> On Fri, 11 Apr 2003 Daniel Stenberg <daniel_at_haxx.se> wrote:
> >Content-Encoding method in HTTP responses.
> >
> >Whoa. This is next to unbelievable. Dan Fandrich posted a patch doing this,
> >and he beat you with ~11 hours! ;-)
>
> Damn! I searched around to see if it had been mentioned, I couldn't find
> anything.

I did the same thing, didn't see any mention of it and just quickly coded it
up. Great minds think alike!

[...]

> At first glance I think it is one point each *8-) Dan's handles extended
> headers but mine checks the CRC, which I don't think would be a big problem
> to add to Dan's code. I'll look closer at Dan's stuff and see how it
> compares. It's good to see that he had the same problem of having to cope
> with fragmentation when dealing with the GZIP header.

I was just tempted to assume the entire header would always be in the first
block and fail when it wasn't, but I decided to do it right. The only down
side is that it's possible to have a memory leak if the transfer ends for some
reason right after the block is malloced.

I discovered another problem with the decompressing code. It turns out that
the decompression buffer is too small in some cases. I found this after
downloading a 1 GB file with gzip -9 compression containing long runs of 0
bytes. Increasing the size of the buffer made the problem go away for me (at
the cost of a jump in the stack requirement).

I also found a problem with the buildconf script. It's nice enough to let the
user choose alternate autotools versions (which I needed to do on my Red Hat
7.2 box), but unfortunately doesn't actually use those version of the
tools. Also, the version checking script failed on my automake-1.5
which contains the version number twice in the --version output.

Attached is a patch against current CVS to fix all these problems.

>>> Dan

-- 
http://www.MoveAnnouncer.com              The web change of address service
          Let webmasters know that your web site has moved

-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf

Received on 2003-04-16