cURL
Haxx ad
libcurl

curl's project page on SourceForge.net

Sponsors:
Haxx

cURL > Mailing List > Monthly Index > Single Mail

curl-tracker mailing list Archives

[ curl-Bugs-2913886 ] proxy and ftp - never issues request

From: SourceForge.net <noreply_at_sourceforge.net>
Date: Sun, 13 Dec 2009 23:14:32 +0000

Bugs item #2913886, was opened at 2009-12-13 16:07
Message generated for change (Comment added) made by jnelson
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=2913886&group_id=976

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: libcurl
Group: wrong behaviour
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Jon Nelson (jnelson)
Assigned to: Daniel Stenberg (bagder)
Summary: proxy and ftp - never issues request

Initial Comment:
curl 7.19.6 on openSUSE 11.2, connecting via ftp through a squid proxy.

I noticed a problem attempting to fetch from an ftp resources that curl appeared to connect and then do nothing until it eventually timed out.
Suspecting some other problem, I tried aria2, wget, and firefox -- all of which worked just fine.

So I strace'd curl and this is what I get (the proxy is 192.168.1.1 port 3128):
...

10137 16:03:52.937144 socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3 <0.000020>
10137 16:03:52.937229 setsockopt(3, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0 <0.000015>
10137 16:03:52.937290 fcntl(3, F_GETFL) = 0x2 (flags O_RDWR) <0.000012>
10137 16:03:52.937340 fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0 <0.000011>
10137 16:03:52.937394 connect(3, {sa_family=AF_INET, sin_port=htons(3128), sin_addr=inet_addr("192.168.1.1")}, 16) = -1 EINPROGRESS (Operation now in progress) <0.000057>
10137 16:03:52.937516 poll([{fd=3, events=POLLOUT}], 1, 299999) = 1 ([{fd=3, revents=POLLOUT}]) <0.000879>
10137 16:03:52.938452 getsockopt(3, SOL_SOCKET, SO_ERROR, [0], [4]) = 0 <0.000012>
10137 16:03:52.938516 poll([{fd=3, events=POLLIN|POLLPRI}], 1, 3600000) = 1 ([{fd=3, revents=POLLIN}]) <29.348688>
10137 16:04:22.287387 recvfrom(3, "", 16384, 0, NULL, NULL) = 0 <0.000028>
10137 16:04:22.287525 close(3) = 0 <0.000079>

followed by curl grumping (error 56).

What I don't understand is why, after the connect is successful, curl doesn't send the proxy anything.

----------------------------------------------------------------------

>Comment By: Jon Nelson (jnelson)
Date: 2009-12-13 17:14

Message:
recv frequently gets turned into recvfrom by the C library.

Look at the timestamps. The recv[from] returning 0 is perfectly
reasonable: the proxy accepted a request, waited 30 seconds, never got a
request, and closed the connection.

The core of the problem is still this: curl isn't ever asking the proxy
for anything. it connects, and then waits for a response. it won't get one.
ever. it's not asked for anything.

I'm invoking curl like this:

curl --trace-ascii - --trace-time
ftp://ftp.suse.com/pub/projects/kernel/kotd/openSUSE-11.2/x86_64/kernel-desktop-base-2.6.31.7-0.0.0.8.a22d080.x86_64.rpm

the environment looks like this:

frank:~ # env | grep -i prox
http_proxy=http://192.168.1.1:3128/
ftp_proxy=http://192.168.1.1:3128/
https_proxy=http://192.168.1.1:3128/
frank:~ #

remember: wget works perfectly in the same environment. so does aria2.

wget is doing what one would expect:
connect to the proxy
write a request.
read response

Here is the wget strace:

socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(3128),
sin_addr=inet_addr("192.168.1.1")}, 16) = 0
write(2, "connected.\n", 11) = 11
select(4, NULL, [3], NULL, {900, 0}) = 1 (out [3], left {899, 999996})
write(3, "GET ftp://ftp.suse.com/pub/proje"..., 195) = 195
write(2, "Proxy request sent, awaiting res"..., 41) = 41
....

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2009-12-13 16:47

Message:
I disagree with your conclusion.

First, I assume that libcurl's use of recv() somehow gets translated into
a recvfrom() (which in my book is strange in itself but lets ignore that
for now). But if that recvfrom() returns 0 after having poll() told the
socket is readable, that's an end of the connection. Let me quote the man
page: "The return value will be 0 when the peer has performed an orderly
shutdown.". So the question is then why the other end end closes down the
connection that fast.

Second, what makes me very curious is the comment "FTP response reading
failed" (a "normal" ftp over HTTP proxy does no such thing as it no longer
does FTP then but is doing HTTP and then it never waits for FTP responses).
Are you by any chance doing "tunneling" through the proxy? Can you show us
a full command line that you use when you experience these problems.

----------------------------------------------------------------------

Comment By: Jon Nelson (jnelson)
Date: 2009-12-13 16:40

Message:
The recvfrom:
10137 16:04:22.287387 recvfrom(3, "", 16384, 0, NULL, NULL) = 0 <0.000028>

is receiving from file descriptor no 3. fd 3 was created here:

10137 16:03:52.937144 socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
<0.000020>

(clearly a TCP socket) and the subsequent statements clearly show it to be
the socket which curl is using to communicate with the proxy:

10137 16:03:52.937394 connect(3, {sa_family=AF_INET,
sin_port=htons(3128),
sin_addr=inet_addr("192.168.1.1")}, 16) = -1 EINPROGRESS (Operation now
in
progress) <0.000057>

--trace-ascii and --trace-time added:

18:21:57.436169 == Info: About to connect() to proxy 192.168.1.1 port 3128
(#0)
18:21:57.437620 == Info: Trying 192.168.1.1... 18:21:57.439207 == Info:
connected
18:21:57.439770 == Info: Connected to 192.168.1.1 (192.168.1.1) port 3128
(#0)

18:22:26.845603 == Info: FTP response reading failed
18:22:26.845939 == Info: Closing connection #0
curl: (56) FTP response reading failed

The strace is pretty clear to me that curl (or libcurl) is simply waiting
for a response that will never come. The response will never come because
it never asks for anything!

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2009-12-13 16:31

Message:
recvfrom() ?? Is that from the DNS resolve? It doesnät make sense to me.

Can you run curl with --trace-ascii and get a dump of that to show us?

We run ftp over http proxy in our test suite (in several test cases even),
which runs fine on every release before we ship it and it still runs fine
in our current dev version.

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=2913886&group_id=976
Received on 2009-12-14

These mail archives are generated by hypermail.

donate! Page updated November 12, 2010.
web site info

File upload with ASP.NET