Bugs item #3214223, was opened at 2011-03-15 21:06
Message generated for change (Settings changed) made by
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=3214223&group_id=976

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: libcurl
>Group: hang
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Chris Smowton ()
Assigned to: Nobody/Anonymous (nobody)
Summary: Pipelined HTTP requests with a zero-length body broken

Initial Comment:
The scenario:

cURL is running pipelined. It's trying to get a series of three documents from the server, call them http://server/one, two and three. 'one' is zero-length.

What happens:

1. cURL sends all three requests
2. Server sends two replies in quick succession.
3. cURL enters Curl_readwrite --> readwrite_data --> Curl_read
4. We're at the HTTP-read-headers stage, so we don't have an expected length. We read too much, saving the buffer in conn->master_buffer.
5. It's still header-time, so we enter Curl_http_readwrite_headers.
6. The first response's headers indicate Content-Length: 0, so we set "stop_reading"
7. ...and consequently bail out of the do-while loop.

The problem: that buffer saved in conn->master_buffer contains not only the zero-length request's headers, but also the headers and body of its successor requests. However it will never yield them unless we set conn->read_pos using read_rewind, but we only do that if we read a short body, *not* if we set stop_reading, which bails out immediately.

Consequently the responses which were in that buffer are silently dropped, and subsequent requests get paired with replies which are not their own! Example: Requests 1, 2 and 3 meet reply-packets [1, 2] and [3]. Request 1 is zero-length, triggering the bug and silently discarding the rest of its packet. Request 2 is then answered using reply [3]... curse HTTP's lack of a transaction-identifier field!

Solution: read_rewind whenever we've read too much data, not just in the special cases dealt with further down readwrite.

I attach as always a pair of test cases, somewhat tricky this time: a server and a client. They'll prompt you to press enter a few times: the drill should be:

1. Start server ("dumb_server 9000")
2. Start client, no params. We immediately spin up a pipelining, maxconnects=1 multi-context and submit 4 requests, "init", "zero_length", "nonzero_length_one", "nonzero_length_two".
3. We'll immediately submit the first request and the server will answer. So,e purpose is to convince cURL it's dealing with HTTP/1.1 and can pipeline the rest of them.
4. cURL will automatically send the other three as a pipeline and quiesce.
5. Now press return at the server: it'll send a single-packet (hopefully!) reply to the first two requests
6. Press return at cURL's console: it'll consume the first reply and silently eat the second one.
7. Press return at the server: it'll reply to the third request.
8. Press return at the client: it'll consume the third reply (but match it to the second request!) and end.

To check, see /tmp/curl_*_out.

If it works right, you'll get:

curl_zero_out: "init"
curl_first_out: ""
curl_second_out: "nz1"
curl_third_out: "nz2"

If the bug manifests, instead you'll get:

curl_zero_out: "init"
curl_first_out: ""
curl_second_out: "nz2"
curl_third_out: ""

Note that the third request has not returned, whilst the second has been given the value that rightfully belongs to the third.

I'd really appreciate a fix to this one, as I can't find a workaround that isn't disabling pipelining.

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=3214223&group_id=976
Received on 2011-03-15