Bugs item #1945240, was opened at 2008-04-17 23:07
Message generated for change (Comment added) made by bagder
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=1945240&group_id=976

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: libcurl
Group: wrong behaviour
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Ben Van Hof (bvanhof)
Assigned to: Daniel Stenberg (bagder)
Summary: libcurl sometimes sends body twice when using CURL_AUTH_ANY

Initial Comment:
Hi,

I've encountered an intermittent problem when using libcurl to send requests with a body to a server which requires authentication, using the CURL_AUTH_ANY option. Normally, it works as expected, but occasionally the server returns a 400 Bad Request instead of succeeding. The (failing) transaction looks like this:

client says: PUT blah
             [headers]
             Expect: 100-continue

server says: HTTP/1.1 100 Continue

client says: [request body]

server says: HTTP/1.1 401 Authorization Required

client says: [request body] *** very bad
             PUT blah
             [headers]

server says: HTTP/1.1 400 Bad Request WTF!!

Near as I can tell, here's what's happening: when sending the request body the first time, Curl_readwrite gets called repeatedly until Curl_fillreadbuffer reports that it sent 0 bytes (see lines ~1445 ff of transfer.c), at which point it clears the KEEP_WRITE flag and calls it a day. The implication is that one "extra" call to Curl_readwrite is required AFTER the entire body has been sent off to the server (in order to get the 'zero bytes written' condition). If the server happens to respond before that final call to Curl_readwrite, though, it winds up getting into the read code first, where it will call Curl_http_auth_act(), and, through it, perhapsrewind(), which decides to rewind the input file. Oops. Now when it gets back into the write code, it winds up re-sending the body, which of course confuses the heck out of the poor server.

The couple of quick fixes I've tried have failed something in the test cases, so rather than submit a half-assed patch I'll leave this to someone who knows this code well enough not to break it. You can reproduce the problem by using the curl client to send a request with a body to any server requiring authentication, using the '--anyauth' flag (the problem doesn't appear if you request a specific auth method, since that bypasses the extra negotiation step.) E.g.:

curl --anyauth -u user:passwd -X PUT -T body.txt http://localhost/some/url

It does happen much more frequently if the server is local (since it's dependent on the server responding quickly), but you can reproduce the bug reliably in any case by running it under the debugger and setting a breakpoint on Curl_readwrite (the third call is the critical one), which creates a delay sufficient to trigger it.

curl -V
curl 7.18.1 (i686-pc-linux-gnu) libcurl/7.18.1 OpenSSL/0.9.8c zlib/1.2.3 libidn/0.6.5 libssh2/0.17
Protocols: tftp ftp telnet dict ldap http file https ftps scp sftp
Features: Debug IDN IPv6 Largefile NTLM SSL libz

Let me know if you need any other info.

Ben Van Hof

----------------------------------------------------------------------

>Comment By: Daniel Stenberg (bagder)
Date: 2008-04-25 00:44

Message:
Logged In: YES
user_id=1110
Originator: NO

Any chance you could try creating a test case in the curl test suite that
repeats the problem? Test 154, 155 and 156 all seem to be pretty similar to
your case.

----------------------------------------------------------------------

Comment By: Ben Van Hof (bvanhof)
Date: 2008-04-18 04:19

Message:
Logged In: YES
user_id=2065988
Originator: YES

Basic auth, though from what I've seen it would probably happen with other
auth methods, too.

Ben

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2008-04-17 23:18

Message:
Logged In: YES
user_id=1110
Originator: NO

One important piece of missing info: what authentication(s) does your test
server require?

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=1945240&group_id=976
Received on 2008-04-25