Bugs item #2210686, was opened at 2008-10-30 23:10
Message generated for change (Comment added) made by boytheouk
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=2210686&group_id=976

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: libcurl
Group: bad behaviour
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Theodore H. Smith (boytheouk)
Assigned to: Daniel Stenberg (bagder)
Summary: Using NTLM proxy will lose form-data. Makes NTLM unusable.

Initial Comment:
I've proved that libcurl IS NOT sending parameters! In certain conditions, when using NTLM proxy.

It basically makes libcurl's NTLM proxy unusable for us, because while it doesn't happen every connection, it happens more like 1/10 connections, and we fire off hundreds of connections. And sometimes it will reliably happen on a certain URL, meaning certain URLs are unaccessable.

This bug appears more often using the multi interface, especially when firing off one connection quickly after the last has been completed.

The timing seems to be the most important thing. A delay before one connection and the next, seems to make the bug less likely to occur. However, sometimes the bug always appears even if we are talking about the first connection. So this behaviour is quite random, but it appears to have something to do with timing, and reusing of connections.

This bug also appears in the curl command line tool. But it's much rarer probably because it doesn't reuse connections quite as often as an application will, due to the fact that the tool will quit and be reopened once per connection.

Here is my debug trace. I did this using libcurl, as a C API, so I tried to make my debug function output text like libcurl's although it's not exactly the same. What's the problem? No form data! I've removed sensitive information from this example by replacing with ****

== Info: Expire cleared
== Info: Connection #0 to host **.**.**.** left intact
== Info: About to connect() to proxy **.**.**.** port 808 (#0)
== Info: Trying **.**.**.**... == Info: connected
== Info: Connected to **.**.**.** (**.**.**.**) port 808 (#0)
== Info: Proxy auth using NTLM with user ********
=> Send header: POST http://*******.com HTTP/1.1

Proxy-Authorization: NTLM *************************

Host: *******.com

Pragma: no-cache

Accept: */*

Proxy-Connection: Keep-Alive

Content-Length: 0

<= Recv header: HTTP/1.1 200 OK

<= Recv header: Date: Thu, 30 Oct 2008 13:45:44 GMT

<= Recv header: Server: Apache/2.2.8 (Ubuntu) mod_jk/1.2.25 mod_ssl/2.2.8 OpenSSL/0.9.8g

<= Recv header: Keep-Alive: timeout=15, max=100

<= Recv header: Transfer-Encoding: chunked

<= Recv header: Content-Type: text/plain

<= Recv header: Proxy-connection: Keep-Alive

<= Recv header:

=> Send data
10

Sometimes, with the exact same connection code... I see this instead amoungst the (long) debug output.

"

------------------------------9ea4b9344a58

Content-Disposition: form-data; name="lea"

Content-Type: application/binary

le2

"

----------------------------------------------------------------------

>Comment By: Theodore H. Smith (boytheouk)
Date: 2008-10-31 00:50

Message:
File Added: 1~555_AlfieConnection.pl

----------------------------------------------------------------------

Comment By: Theodore H. Smith (boytheouk)
Date: 2008-10-31 00:07

Message:
OK... there was a mistake in my explanation below.

I have curl in two places on the PC. I have curl.exe the command line
tool. And libcurl.dll.

my curl.exe says: curl 7.19.0 (i586-pc-mingw32msvc) libcurl/7.19.0
zlib/1.2.3

my libcurl says: libcurl/7.19.0 OpenSSL/0.9.8h zlib/1.2.3 libssh2/0.18

Hopefully it doesn't make a difference. I am experiencing this
"disappearing form-data" problem with libcurl.dll.

I am unable to find out if the problem appears with curl.exe, right this
second, but I am working on it. The main problem is our servers have
passwords that are hashed with the current unix time, so I can't just type
up a testing string in curl.exe because the password will be wrong. I am
working on a system that will let me expose development areas on the server
so timehashing isn't needed for passwords. I'll get that done by tomorrow.
then I'll be able to test for this bug under curl.exe, tomorrow.

I'm not sure this will help, because I am seeing the problem in
libcurl.dll, (and I don't yet know if it appears in curl.exe), but here is
what curl.exe gives me.

C:\Documents and Settings\Administrator\Desktop>curl.exe -V
curl 7.19.0 (i586-pc-mingw32msvc) libcurl/7.19.0 zlib/1.2.3
Protocols: tftp ftp telnet dict ldap http file
Features: Largefile NTLM SSPI libz

The trace I gave you was a complete trace of a failiure. I didn't give
any complete trace from a successful run.

I'll have to work on getting you the code to replicate this. My current
code isn't actually C, but I can write C easily enough. I'll update this
bug report when I have it.

Thanks a lot.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2008-10-30 23:41

Message:
That Windows version doesn't have OpenSSL but I guess it says "NTLM" as a
feature in the curl -V output? If so it uses the windows internal SSPI
interface and that then of course rules out that this is a problem in our
NTLM code.

I would like a more complete trace dump from the POST. Does it really only
issue a single POST?

I'd also like to see the code for a full app (as small as possible) that
you can use to reproduce this problem.

Unfortunately, I don't personally have a NTLM proxy to try against.

----------------------------------------------------------------------

Comment By: Theodore H. Smith (boytheouk)
Date: 2008-10-30 23:35

Message:
Here's the version and platform:

curl 7.19.0 (i386-apple-darwin9.5.0) libcurl/7.19.0 OpenSSL/0.9.7l
zlib/1.2.3

I also tried this on the PC. Same result.

curl 7.19.0 (i586-pc-mingw32msvc) libcurl/7.19.0 zlib/1.2.3

Please let me know what other information is needed.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2008-10-30 23:17

Message:
You didn't mention what libcurl version on what platform this is.

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=2210686&group_id=976
Received on 2008-10-31