cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: Partial Downloads

From: Dan Fandrich <dan_at_coneharvesters.com>
Date: Sat, 29 Jan 2011 11:01:37 -0800

On Sat, Jan 29, 2011 at 12:34:27PM -0500, Dejan Tolj wrote:
> Hi Dan, to answer your question i had selinux disabled, and the source
> was taken from the original downloads
> http://curl.haxx.se/download/curl-7.21.3.tar.gz
> not SRPM.
>
> I followed Kamils link and was able to install the latest curl on RHEL5.
> However the range param still dosn't work fully, except "-XXXXX"
>
> $curl -v ftp://eeeeeee/aaaa.wmv -r 0-100000 > asdf
>
> * About to connect() to proxy 10.224.135.86 port 3128 (#0)

A proxy? You never said anything about using a proxy. That changes the
problem completely. Rather than a limitation of libcurl or the remote
ftp server, it's most likely that this limitation is the fault of the
proxy.

> * Trying 10.224.135.86... % Total % Received % Xferd Average
> Speed Time Time Time Current
> Dload Upload Total Spent Left Speed
> 0 0 0 0 0 0 0 0 --:--:-- --:--:--
> --:--:-- 0connected
> * Connected to 10.224.135.86 (10.224.135.86) port 3128 (#0)
> * Proxy auth using Basic with user 'user1'
> > GET ftp://eeeeeee/aaaa.wmv HTTP/1.1
> > Proxy-Authorization: Basic cmVkaGF0LXVwZGF0ZTp0YWhkZXI=
> > Range: bytes=0-100000
> > User-Agent: curl/7.21.3 (i686-redhat-linux-gnu) libcurl/7.21.3 OpenSSL/0.9.8b zlib/1.2.3 c-ares/1.7.4 libidn/1.19 libssh2/1.2.7
> > Host: eeeeeee:21
> > Accept: */*
> > Proxy-Connection: Keep-Alive
> >
> * HTTP 1.0, assume close after body
> < HTTP/1.0 200 Gatewaying
> < Server: squid/3.0.STABLE19
> < Mime-Version: 1.0
> < Date: Sat, 29 Jan 2011 17:27:43 GMT
> < Content-Type: text/plain
> < Content-Length: 254056909
> < Last-Modified: Mon, 04 Oct 2010 20:40:46 GMT
> < X-Cache: MISS from 10.224.135.86
> < Via: 1.0 10.224.135.86 (squid/3.0.STABLE19)
> * HTTP/1.0 proxy connection set to keep alive!
> < Proxy-Connection: keep-alive

As you can see from the logs, curl is requesting the correct Range:, but the
proxy ignores it and serves the whole file. You could make an argument
that libcurl should kill the connection after the first 100001 bytes, but
for HTTP requests (including HTTP proxy requests), it currently passes the
entire payload to the application.

>>> Dan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-01-29