cURL / Mailing Lists / curl-library / Single Mail


Re: How does libCurl handle Content-Encoding GZIP + partial responses in respect to automatically decoding of compressed content

From: Dan Fandrich <>
Date: Mon, 30 Apr 2007 10:02:02 -0700

On Mon, Apr 30, 2007 at 04:53:31PM +0200, Stefan Krause wrote:
> I currently think about the use case where compressed content is
> requested by the client in ranges of a certain size, so that multiple
> requests for different ranges are needed to get the complete data. After
> all the (compressed) data was received, it has to be decompressed.
> 3) The clients saves the first 100 bytes to temporary storage and
> requests the next 100 bytes. The client knows about the compressed data,
> because the Content-Encoding header in the response is present and set
> to GZIP.
> 4) After the client has received the 1000 bytes of compressed data it
> uncompressed and data which results in the 10000 bytes of (uncompressed)
> data which is stored on the server.

That's almost right. curl (via zlib) will decompress the data as it
comes in--it's stored temporarily on the heap, not on disk. If the
first 100 bytes of compressed data can be decompressed on their own,
then the application will receive them immediately.

> And know the questions:
> 1) How does libCurl deal with compressed partial data (HTTP response
> code 206) ? According to the documentation there is some sort of
> "automatic" decompression. How does this work in detail when the
> CURLOPT_WRITEFUNCTION callback is used for data reception?

The app doesn't need to know if the data is compressed or not. libcurl
will handle the decompression transparently, so the app will only see
the uncompressed data.

> 2) I have the same use case but in the other direction. Compressed data
> should be uploaded in parts. Does libCurl provide here also some
> automation or does the libcurl client has to do the compression first
> and send send the data in appropriate data segments?

libcurl doesn't handle compressing data on uploads.

>>> Dan

--              The web change of address service
          Let webmasters know that your web site has moved
Received on 2007-04-30