Bugs item #2878004, was opened at 2009-10-13 11:51
Message generated for change (Settings changed) made by jnelson
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=2878004&group_id=976

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: http
>Group: new feature request
>Status: Open
>Resolution: None
>Priority: 3
Private: No
Submitted By: Jon Nelson (jnelson)
Assigned to: Daniel Stenberg (bagder)
Summary: 5 range support waits on keepalive timeout [2]

Initial Comment:
Sorry for creating a duplicate bug, but I've tried and I can't figure out how to re-open an issue (or even comment on an existing one!)

Specifically, I'm talking about: range support waits on keepalive timeout - ID: 2853575

I'll reproduce the relevant bits here:

me:

If one issues a range request, the Connection: close header *should* be
sent but it is not. Since the default protocol is http/1.1, what happens is
that curl makes a (perfectly valid!) request without any "Connection"
header, which in http/1.1 normally indicates a passive request for
keep-alive. When the server is done completing the response, it will hold
the socket open and wait for more requests on the same socket, typically
timing out eventually and closing the socket.

Since curl does not send a "Connection: close" header with the http/1.1
request, curl will also dutifully wait until the server times out and
closes the connection. This can be quite some time, although usually it's
between 15 and 30 seconds.

Fix: send a "Connection: close" header when making http/1.1 requests.

Side note: when curl is used to make the same exact request *without*
--range, it (curl) *closes* the connection after doing so. As an indication
that it intends to do so, it really should (but this is not a bug) send a
connection: close header.

Sample Request:

curl -D - --range 0-0,10-10,-30 http://.....

badger:

Why would a range: request need a connection: close header any more than
any other request?

The fix is not that simple as you make it sound like. curl and libcurl are
designed to do requests using persistent connections so that subsequent
requests to the same host will re-use the connection. The curl tool would
need to detect that no further requests is desired to the given host and
tell that to libcurl which then would use Connection: close.

But "connection: close" when libcurl is closing the socket anyway is
rather pointless.

curl should not wait for the server to close the socket for any requests.
That's either a bug in libcurl or a bug in the server. Can you show us the
exact sent/received headers with curl 7.19.6?

And now some new data:

1. regarding a connection: close header when curl is going to close the socket anyway being pointless:

rfc2616 clearly state: In case the client does not want to maintain a connection for more than that request, it SHOULD send a Connection header including the connection-token close.

It's a "be polite to the server" issue. Server implementations can optimize for the situation if they see a connection: close header. It's not much of an optimization, but it does help.

2. regarding libcurl "waiting" for the server to close the connection: I'll have to get back to you on that one.

3. regarding curl (the binary) and sending a connection: close header. curl (the binary) should send a connection: close header on the *last* (or only) request made.

It's true that libcurl should generally be configured to pipeline requests, however, in the case of curl (the binary) it should detect the *last* request being made (in most cases, probably the only request) and send the connection: close header.

----------------------------------------------------------------------

>Comment By: Jon Nelson (jnelson)
Date: 2009-10-13 23:05

Message:
Yes, it turns out to have been a server bug, and has nothing to do with
range requests.

However, your followup doesn't address points 1 or 3 above.

As I see it, curl (the binary) is a convenient user interface to libcurl.
*curl* knows which request is the last, and it can tell libcurl to issue
the connection: close header. The rfc is pretty clear that this is a
"should" and not a "must".

You are also right in that this issue should be changed to be a feature
request and not a bug.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2009-10-13 21:45

Message:
Then let me again say that this works for me.

HTTP 1.1 does persistant connections by default. The fact that a Range:
header is included makes no difference. A HTTP request using Range: can be
done using a persistant connection just fine without requiring a close or
disconnect. So your entire prelude for this bug report seems to be based on
a wrong assumption.

libcurl cannot "detect" the last request as it doesn't know the last one
by itself, only the app using it can know it.

Now, if you get curl to "hang" waiting for the server to close the
connection that is a sign of a bug. The bug is either in the server end or
in libcurl, but it is NOT a sign that we should make libcurl close the
connection immediately after having got the response from a request that
uses Range:

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=2878004&group_id=976
Received on 2009-10-14