cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Problem downloading files greater than 32Mb after upgrading from 7.14 to 7.20.1

From: David Tong <David.Tong_at_oracle.com>
Date: Tue, 22 Nov 2011 14:14:54 -0800

The first connection didn't exactly fail.
For some reason (I didn't write this) the code tries to download the
object in 32Mb chunks.
It sets the Content-Range field to 0-33554431/63353018
The download ends with the message
    transfer closed with 261952 bytes remaining to read
    Closing connection #0
It then tries to reconnect to download the next chunk.

I discovered the SSL_SESSIONID_CACHE option. Setting that to zero
disabled the cache,
forcing cURL to establish a new session on reconnecting - that resolved
the problem.
As far as any additional overhead goes none was noticeable on the client
side.

Since I have a solution I don't plan to investigate further. If there
are any other issues
I'll probably rewrite the code to invoke the curl executable directly
instead.

Cheers
Dave

On 11/22/11 13:41, Daniel Stenberg wrote:
> On Mon, 21 Nov 2011, David Tong wrote:
>
>> I have a legacy project that recently upgraded libcurl from 7.14 to
>> 7.20 (and yes I know that's not the latest)
>
> Still you switched to an almost 5 years newer version!
>
>> The project uses libcurl to download binary objects via SSL. I'm
>> finding that objects over 32Mb in size fail to download. Looking in
>> detail it seems that the first block (less than, but close to 32Mb)
>> downloads successfully. The code then attempts to connect again to
>> download more data, but the connection fails.
>
> So what's the exact error in the first fail? And when you say "The
> code" attempts to connect again, is this code your application you mean?
>
>> The issue appears to be related to re-using the connection
>
> If the connection previously ended with an error it should never be
> reused.
>
>> and may also be related to the fact that the SSL certificate is
>> self-signed.
>
> I don't think it matters.
>
>> And here's what I see with the same code using 7.20:
>>
>> < HTTP/1.0 200 Connection established^M
>> < ^M
>> * Proxy replied OK to CONNECT request
>> * Unknown SSL protocol error in connection to [SERVER]
>> * Closing connection #0
>>
>> Any suggestions would be appreciated
>
> Could I ask you to try the latest libcurl version first just an
> experiment to see if the problem remains? It seems pointless to chase
> for something that might already work...
>

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-11-22