cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: question on https through proxy

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Fri, 4 Oct 2002 17:14:15 +0200 (MET DST)

On Fri, 4 Oct 2002 RBramante_at_on.com wrote:

> Here is what I got. Included the http and https for comparison. They are
> more different then I thought they would be.

They do differ on how the request is sent, yes. For HTTPS, curl makes CONNECT
request to the server, through the proxy, waits for a 200 OK reply and then
makes a GET to the remote host. For normal HTTP, curl just sends a GET to the
proxy.

> == Info: About to connect() to 127.0.0.1:8000
> == Info: Connected to 127.0.0.1 (127.0.0.1) port 8000
> == Info: Establish HTTP proxy tunnel to 207.18.219.214:8443
> => Send data , 103 bytes (0x67)
> 0000: 43 4f 4e 4e 45 43 54 20 32 30 37 2e 31 38 2e 32 CONNECT 207.18.2
> 0010: 31 39 2e 32 31 34 3a 38 34 34 33 20 48 54 54 50 19.214:8443 HTTP
> 0020: 2f 31 2e 30 0d 0a 55 73 65 72 2d 41 67 65 6e 74 /1.0..User-Agent
> 0030: 3a 20 63 75 72 6c 2f 37 2e 39 2e 38 20 28 77 69 : curl/7.9.8 (wi
> 0040: 6e 33 32 29 20 6c 69 62 63 75 72 6c 20 37 2e 39 n32) libcurl 7.9
> 0050: 2e 38 20 28 4f 70 65 6e 53 53 4c 20 30 2e 39 2e .8 (OpenSSL 0.9.
> 0060: 36 64 29 0d 0a 0d 0a 6d)....
> <= Recv data, 36 bytes (0x24)
> 0000: 48 54 54 50 2f 31 2e 30 20 32 30 30 20 43 6f 6e HTTP/1.0 200 Con
> 0010: 6e 65 63 74 69 6f 6e 20 65 73 74 61 62 6c 69 73 nection establis
> 0020: 68 65 64 0a hed.
> <= Recv data, 22 bytes (0x16)
> 0000: 50 72 6f 78 79 2d 41 67 65 6e 74 3a 20 49 4a 2f Proxy-Agent: IJ/
> 0010: 32 2e 30 2e 32 0a 2.0.2.
> <= Recv data, 1 bytes (0x1)
> 0000: 0a .
> == Info: Closing connection #0

The response from the proxy doesn't seem to return correct CRLF newlines, but
only LF (0a only, not 0d 0a).

This is a violation of the HTTP spec, and curl doesn't understand that the
end of the proxy reply has been received, but instead it hangs waiting for
the reply "terminator sequence" to arrive. The proxy is supposed to send back
a 200 OK response for curl to know that the connection is done fine to the
remote host.

Of course, we should probably make an effort to detect this anyway and make
curl be happy with "bad" newlines too.

I would be happy if you could verify this, but setting a break-point in
lib/http.c:Curl_ConnectHTTPProxyTunnel() and just single-step down a bit
until it reads the response and checks for the terminating stuff to "arrive".

On what platform runs this "Proxy-Agent: IJ" ? Is it possible to download for
free/trial? Is it possible for me to try some curl commands on it, or are you
capable of fixing this yourself?

-- 
 Daniel Stenberg -- curl related mails on curl related mailing lists please
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
Received on 2002-10-04