Bugs item #3208760, was opened at 2011-03-13 11:05
Message generated for change (Comment added) made by
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=3208760&group_id=976

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: libcurl
Group: wrong behaviour
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Chris Smowton ()
Assigned to: Daniel Stenberg (bagder)
Summary: Bad HTTP POST retry with READFUNCTION or READDATA set

Initial Comment:
cURL version: curl 7.21.0 (x86_64-pc-linux-gnu) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18

The scenario:

1. cURL sends an HTTP POST to a server
2. Server realised on reading that (e.g.) the backend has hung up, and sends a FIN
3. cURL notices the socket is dead after sending, but before receiving anything
4. cURL attempts a retransmit from Curl_retry_request
4a. If the socket we were using was fresh (i.e., not re-used), at this point we die with error 52 / server sent empty reply
5. cURL neglects to try to rewind the input file
6. On the retry, cURL calls fread or a custom READFUNCTION and immediately gets EOF
7. cURL sends an empty request body which doesn't match the Content-Length header
8. Server waits a while for the promised POST body
9. Server gives up; silently hangs up the connection
10. cURL decides that since it transmitted a POST but received nothing... it had better retry!
11. Go to step 4

Note that this doesn't happen with the cURL command-line program, as it seems to use POSTFIELDS instead of READDATA or READFUNCTION.

I attach an example libcurl client program which exhibits the problem as follows:
1. It makes a small POST request against the server localhost:9000. This is to leave a re-usable socket around.
2. It makes another small POST against the same server. The socket gets re-used.

If invoked without arguments it does this using a small file on disk and READDATA; if invoked with argument 'func' it will use READFUNCTION and a fake fread function that prints what it's doing.

I also attach a dummy server that exhibits the problem. It behaves as follows:
1. Read the first request
2. Send a dummy response
3. Read a second request
4. Hang up the socket
5. Accept a second socket
6. Read the request

If client and server are used together, we see the following:
1. cURL connects
2. cURL sends POST
3. Server responds
4. cURL reuses connection for second POST
5. Server hangs up
6. cURL notes that the connection died and goes for a retry
7. cURL forms a new connection
8. Server accepts
9. Server reads
10. We note that the request body does not appear in the retried request.
11. Server hangs up
12. cURL fails with server-sent-empty-document.

Finally I attach

I'd suggest that the answer here is that we retry an HTTP POST for which we have called fread and don't try to rewind. If it's a FILE* or we have a SEEKFUNCTION this could work; otherwise we should probably fail with cant-rewind. The function http_perhaps_rewind looks like the right thing, but isn't used here.

----------------------------------------------------------------------

>Comment By: Chris Smowton ()
Date: 2011-03-13 11:10

Message:
BTW, my workaround at the moment: assume that a further fread() after EOF
means that libcurl is retrying the connection and rewind myself.

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=3208760&group_id=976
Received on 2011-03-13