cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: curl_easy_perform can't download huge file

From: Ray Satiro via curl-library <curl-library_at_cool.haxx.se>
Date: Thu, 3 Dec 2015 02:09:24 -0500

On 12/2/2015 9:27 PM, Sino wrote:
> I am using libcurl 7.35.0 on ubuntu 14.04 x86-64 platform.
>
> What I want to do is to download one huge file greater than 200M by
> using curl_easy_perform.
>
> It always downloads partial file, for example, one 433M file, it
> always ends at the size of 190M or so. curl_easy_perform returns
> without error.
>
> here are options I set.
>
> curl_easy_setopt(handle, CURLOPT_URL, pi->url);
> curl_easy_setopt(handle, CURLOPT_TCP_KEEPALIVE, 1L);
> curl_easy_setopt(handle, CURLOPT_TCP_KEEPIDLE, 15L);
> curl_easy_setopt(handle, CURLOPT_TCP_KEEPINTVL, 30L);
>
>
> But curl command tool can download the total file.

I tried Ubuntu 14.04 64-bit with system installed libcurl/7.35.0 and
built the url2file example [1] with the TCP keepalive options and
downloaded a 600MB file and it worked fine. Build url2file with the TCP
keepalive options and see if you can download this:

./url2file
https://mirror.umd.edu/ubuntu-iso/15.10/ubuntu-15.10-server-amd64.iso
md5sum page.out

Should be
fb4eef05edcabfc5cccd4cb44f3f9b48 *page.out

Then see if you can download your url, ./url2file <yoururl>

If all of that works then enable verbose mode in your program, that may
have some information to help:

curl_easy_setopt(handle, CURLOPT_VERBOSE, 1L);

Try to download the ubuntu ISO in your program and see if that works.
Also check that the content legnth is correct, and check your write
function. Are you checking for errors? Do you have enough space? If you
can't figure it out provide your write function or better a self
contained example that can be used to reproduce what you described.

[1]: http://curl.haxx.se/libcurl/c/url2file.html

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2015-12-03