Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

User-Agent not set when proxying SSL request #2381

Closed
coffeegist opened this issue Mar 14, 2018 · 4 comments
Closed

User-Agent not set when proxying SSL request #2381

coffeegist opened this issue Mar 14, 2018 · 4 comments

Comments

@coffeegist
Copy link

coffeegist commented Mar 14, 2018

The issue

The man page specifies that the user-agent set from using -A can also be set by using -H. This leads to misconceptions, because -A will set proxy-header user-agents, and -H will not. I'm proposing a documentation change, based on the following outcomes.

The Test Setup

Simulate a proxy with socat so we can see the requests:

kali# socat - TCP-LISTEN:80,crlf,reuseaddr,fork 

Make an https request using curl with a special user-agent:

user@kali:~# curl -v --proxy http://127.0.0.1:80 -H "User-Agent: Mozilla/5.0 (Curl Proxy Test)" https://www.google.com/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.google.com:443
> CONNECT www.google.com:443 HTTP/1.1
> Host: www.google.com:443
> User-Agent: curl/7.58.0
> Proxy-Connection: Keep-Alive
>
^C

user@kali:~# curl -v --proxy http://127.0.0.1:80 -A "Mozilla/5.0 (Curl Proxy Test)" https://www.google.com/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.google.com:443
> CONNECT www.google.com:443 HTTP/1.1
> Host: www.google.com:443
> User-Agent: Mozilla/5.0 (Curl Proxy Test)
> Proxy-Connection: Keep-Alive
>
^C

I expected the following

I expected the user-agent's to be the same, since the man page specifies that the UA can be set with either the -A or -H flags.

Expected output below:

user@kali:~# curl -v --proxy http://127.0.0.1:80 -H "User-Agent: Mozilla/5.0 (Curl Proxy Test)" https://www.google.com/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.google.com:443
> CONNECT www.google.com:443 HTTP/1.1
> Host: www.google.com:443
> User-Agent: Mozilla/5.0 (Curl Proxy Test)
> Proxy-Connection: Keep-Alive
>
^C

user@kali:~# curl -v --proxy http://127.0.0.1:80 -A "Mozilla/5.0 (Curl Proxy Test)" https://www.google.com/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.google.com:443
> CONNECT www.google.com:443 HTTP/1.1
> Host: www.google.com:443
> User-Agent: Mozilla/5.0 (Curl Proxy Test)
> Proxy-Connection: Keep-Alive
>
^C

curl/libcurl version

curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.0.2n zlib/1.2.8 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) libssh2/1.8.0 nghttp2/1.29.0 librtmp/2.3
Release-Date: 2018-01-24
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL

operating system

Linux Kali 4.14.0-kali3-amd64 #1 SMP Debian 4.14.12-2kali2 (2018-01-11) x86_64 GNU/Linux

@bagder
Copy link
Member

bagder commented Mar 14, 2018

The man page section for -H says:

Starting in 7.37.0, you need --proxy-header to send custom headers intended for a proxy.

@coffeegist
Copy link
Author

coffeegist commented Mar 15, 2018

@bagder thanks for the response. My confusion came from this line:

-A, --user-agent
(HTTP) Specify the User-Agent string to send to the HTTP server. To encode blanks
in the string, surround the string with single quote marks. This can also be set
with the -H, --header option of course.

That made it sound like -A and -H do the same thing, which I realize now is obviously not the case.

EDIT: If this isn't really seen as an issue by the community, feel free to close it. Personally, it just didn't make sense to a few of us that -A and -H had different effects.

@bagder
Copy link
Member

bagder commented Mar 15, 2018

They once had the exact same behavior, until we realized that sending -H headers unconditionally to both the server and the proxy has security implications and you may very well decide to send headers to just one of the parties...

bagder added a commit that referenced this issue Mar 15, 2018
@bagder
Copy link
Member

bagder commented Mar 15, 2018

I added a mention of --proxy-header to the man page section for -A.

@bagder bagder closed this as completed Mar 15, 2018
@lock lock bot locked as resolved and limited conversation to collaborators Jun 13, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Development

No branches or pull requests

3 participants