Out of memory is returned from command line for gssapi auth with missing krb5 ticket #3726

lslebodn · 2019-04-03T21:32:16Z

I did this

Ensure there is not any valid krb5 ticket

sh-5.0# klist -l
Principal name                 Cache name
--------------                 ----------

Try to use gssapi authentication for some site.
It will obviously fail but it should continue with basic auth

sh-5.0# curl -v --negotiate -u : -k https://kvm-guest.example.com/application/login
*   Trying 10.37.153.81...
* TCP_NODELAY set
* Connected to kvm-guest.example.com (10.37.153.81) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; O=Unspecified; CN=kvm-guest.example.com; emailAddress=root@kvm-guest.example.com
*  start date: Apr  3 14:59:21 2019 GMT
*  expire date: Apr  7 16:39:21 2020 GMT
*  issuer: C=US; O=Unspecified; OU=ca-64193668093928912; CN=kvm-guest.example.com; emailAddress=root@kvm-guest.example.com
*  SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
* gss_init_sec_context() failed: SPNEGO cannot find mechanisms to negotiate. 
* Connection #0 to host kvm-guest.example.com left intact
curl: (27) Out of memory
* Closing connection 0

I expected the following

sh-5.0# curl -v --negotiate -u : -k https://kvm-02-guest07.rhts.eng.brq.redhat.com/application/login
*   Trying 10.37.153.81...
* TCP_NODELAY set
* Connected to kvm-02-guest07.rhts.eng.brq.redhat.com (10.37.153.81) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; O=Unspecified; CN=kvm-02-guest07.rhts.eng.brq.redhat.com; emailAddress=root@kvm-02-guest07.rhts.eng.brq.redhat.com
*  start date: Apr  3 14:59:21 2019 GMT
*  expire date: Apr  7 16:39:21 2020 GMT
*  issuer: C=US; O=Unspecified; OU=ca-64193668093928912; CN=kvm-02-guest07.rhts.eng.brq.redhat.com; emailAddress=root@kvm-02-guest07.rhts.eng.brq.redhat.com
*  SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
> GET /application/login HTTP/1.1
> Host: kvm-02-guest07.rhts.eng.brq.redhat.com
> User-Agent: curl/7.64.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/1.1 401 Unauthorized
< Date: Wed, 03 Apr 2019 21:28:08 GMT
< Server: Apache/2.4.38 (Fedora) OpenSSL/1.1.1b mod_auth_gssapi/1.6.1
* gss_init_sec_context() failed: SPNEGO cannot find mechanisms to negotiate.
< WWW-Authenticate: Negotiate
< Content-Length: 127
< Content-Type: text/html; charset=iso-8859-1
<
* Connection #0 to host kvm-02-guest07.rhts.eng.brq.redhat.com left intact
<html><meta http-equiv="refresh" content="0; URL=/application/login2"><body>Kerberos authentication did not pass.</body></html>sh-5.0#

curl/libcurl version

sh-5.0# curl -V
curl 7.64.1 (x86_64-redhat-linux-gnu) libcurl/7.64.1 OpenSSL/1.1.1b zlib/1.2.11 brotli/1.0.7 libidn2/2.1.1 libpsl/0.20.2 (+libidn2/2.0.5) libssh/0.8.7/openssl/zlib nghttp2/1.37.0
Release-Date: 2019-03-27
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS brotli GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz Metalink NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets

operating system

sh-5.0# rpm -q curl
curl-7.64.1-1.fc31.x86_64
sh-5.0# cat /etc/os-release 
NAME=Fedora
VERSION="31 (Rawhide)"
ID=fedora
VERSION_ID=31
VERSION_CODENAME=""
PLATFORM_ID="platform:f31"
PRETTY_NAME="Fedora 31 (Rawhide)"
ANSI_COLOR="0;34"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:31"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/rawhide/system-administrators-guide/"
SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=rawhide
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=rawhide
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"

The text was updated successfully, but these errors were encountered:

lslebodn · 2019-04-03T21:47:55Z

It was changed in #1975

Following diff fixed that but I am not sure whether it is right solution

diff --git a/lib/http_negotiate.c b/lib/http_negotiate.c
index 9415236fb..08d85e253 100644
--- a/lib/http_negotiate.c
+++ b/lib/http_negotiate.c
@@ -143,6 +143,12 @@ CURLcode Curl_output_negotiate(struct connectdata *conn, bool proxy)
     }
     if(!neg_ctx->context) {
       result = Curl_input_negotiate(conn, proxy, "Negotiate");
+      if(result == CURLE_OUT_OF_MEMORY) {
+        authp->done = FALSE;
+        /*neg_ctx->havenegdata = FALSE;*/
+
+        return CURLE_OK;
+      }
       if(result)
         return result;
     }

Just test323 failed for me but it seems to be unrelated to spnego or gssapi

jay · 2019-04-03T22:24:48Z

Ref: https://github.com/curl/curl/blob/curl-7_64_1/lib/vauth/spnego_gssapi.c#L150-L174

It's returned OOM there for a long time, not sure why it's different now.

/cc @dhoelzl @iboukris @MarcelRaad @kdudka

dhoelzl · 2019-04-04T07:45:56Z

The call to Curl_input_negotiate at this point (https://github.com/curl/curl/blob/curl-7_64_1/lib/http_negotiate.c#L144-L148) is new. The reason for this is to initiate negotiating already on the first request, if authentication is already known to be negotiate (as curl was set to do so). Prior to my changes negotiating was not started before the first request without negotating failed (and the server said it needs negotiate), even if it was already known that negotiate authentication needs to be used in advance.
I changed this because all browsers behave like this and otherwise there would be one extra unnecessary round trip (especially sending huge amounts of POST/PUT data this causes needless delay).

That is why prior to my changes it switched to basic as negotiate was never getting initiated in your case.

~~Probably there is missing some error handling logic at that point. The fix above seems valid to me (but I have not checked the details).~~

Or this can be seen as intended behaviour, as curl is set to use negotiate (only) in this case?
There are some options to curl to automatically switch over to another authentication (--anyauth), maybe this needs to be considered as well? Does the version without your fix work with --anyauth in your case?

Besides:
CURLE_OUT_OF_MEMORY is "misused" here (probably due to no appropriate error code):

curl/lib/vauth/spnego_gssapi.c

Line 124 in 6c60355

return CURLE_OUT_OF_MEMORY;

There is no real "out of memory", just an error from the negotiate API.

kdudka · 2019-04-04T11:23:29Z

This bug is triggered by 6c60355. I agree that CURLE_OUT_OF_MEMORY is inappropriate in this case. It looks like a copy/paste error. It should probably return CURLE_RECV_ERROR as done in krb5_gssapi.c. However, just changing the error code does not fix the bug. The transfer is still terminated prematurely.

MarcelRaad · 2019-04-04T11:53:59Z

Maybe I don't fully understand the problem, but is this really a bug? --negotiate is not documented to fall back to Basic in https://curl.haxx.se/docs/manpage.html#--negotiate.

dhoelzl · 2019-04-04T12:55:08Z

I also cannot understand why curl should fall back to basic if it is directed to use negotiate authentication.
Maybe it was a bug that it did this before my fixes for #1975.

kdudka · 2019-04-04T13:32:52Z

I am not sure what you are talking about. In which version does curl --negotiate fallback to basic?

dhoelzl · 2019-04-04T13:36:07Z

I assume before #1975: 7.64.0

How does this behave with e.g. NTLM? Is there also a fallback to basic? I have never tried this.

MarcelRaad · 2019-04-04T13:38:05Z

@kdudka The issue description says:

It will obviously fail but it should continue with basic auth

So probably before #1975 . Isn't that what this issue is about? 😕

kdudka · 2019-04-04T14:02:31Z

Sorry, I somehow missed this expectation from the original report. It sounds incorrect to me and I do not think that curl --negotiate has ever worked like that. I tried curl --negotiate with 7.64.0 and it just sent an HTTP request without any Authorization header when gss_init_sec_context() failed.

You can see that the expected verbose output from the original report, which I believe was captured by an unaffected version of curl, does not contain any Authorization header either.

@lslebodn Could you please clarify your expectations?

bagder · 2019-04-04T14:42:38Z

curl never "falls back" to another authentication method. It picks one and goes with that. If that fails to work/authenticate, it returns error. (I would suggest anything else is a bug.)

lslebodn · 2019-04-04T15:07:03Z

curl never "falls back" to another authentication method. It picks one and goes with that. If that fails to work/authenticate, it returns error. (I would suggest anything else is a bug.)

It worked in curl <= 7.64.0 :-) if --negotaite was used but there was not any krb5 ticket for GSS-API

curl 7.64.0 (x86_64-redhat-linux-gnu) libcurl/7.64.0 OpenSSL/1.1.1b zlib/1.2.11 brotli/1.0.7 libidn2/2.1.1 libpsl/0.20.2 (+libidn2/2.0.5) libssh/0.8.7/openssl/zlib nghttp2/1.37.0
Release-Date: 2019-02-06
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz brotli TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL Metalink 
sh-5.0# 
sh-5.0# curl -v --negotiate -u : -k https://kvm-guest.example.com/application/login
*   Trying 10.37.153.128...
* TCP_NODELAY set
* Connected to kvm-guest.example.com (10.37.153.128) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; O=Unspecified; CN=kvm-guest.example.com; emailAddress=root@kvm-guest.example.com
*  start date: Apr  4 12:27:02 2019 GMT
*  expire date: Apr  8 14:07:02 2020 GMT
*  issuer: C=US; O=Unspecified; OU=ca-6605801528768118960; CN=kvm-guest.example.com; emailAddress=root@kvm-guest.example.com
*  SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
> GET /application/login HTTP/1.1
> Host: kvm-guest.example.com
> User-Agent: curl/7.64.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/1.1 401 Unauthorized
< Date: Thu, 04 Apr 2019 14:58:04 GMT
< Server: Apache/2.4.39 (Fedora) OpenSSL/1.1.1b mod_auth_gssapi/1.6.1
* gss_init_sec_context() failed: SPNEGO cannot find mechanisms to negotiate. 
< WWW-Authenticate: Negotiate
< Content-Length: 127
< Content-Type: text/html; charset=iso-8859-1
< 
* Connection #0 to host kvm-guest.example.com left intact
<html><meta http-equiv="refresh" content="0; URL=/application/login2"><body>Kerberos authentication did not pass.</body></html>

Which is the same as with --basic

sh-5.0# curl -v --basic -u : -k https://kvm-guest.example.com/application/login
*   Trying 10.37.153.128...
* TCP_NODELAY set
* Connected to kvm-guest.example.com (10.37.153.128) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; O=Unspecified; CN=kvm-guest.example.com; emailAddress=root@kvm-guest.example.com
*  start date: Apr  4 12:27:02 2019 GMT
*  expire date: Apr  8 14:07:02 2020 GMT
*  issuer: C=US; O=Unspecified; OU=ca-6605801528768118960; CN=kvm-guest.example.com; emailAddress=root@kvm-guest.example.com
*  SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
* Server auth using Basic with user ''
> GET /application/login HTTP/1.1
> Host: kvm-guest.example.com
> Authorization: Basic Og==
> User-Agent: curl/7.64.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/1.1 401 Unauthorized
< Date: Thu, 04 Apr 2019 14:59:43 GMT
< Server: Apache/2.4.39 (Fedora) OpenSSL/1.1.1b mod_auth_gssapi/1.6.1
< WWW-Authenticate: Negotiate
< Content-Length: 127
< Content-Type: text/html; charset=iso-8859-1
< 
* Connection #0 to host kvm-guest.example.com left intact
<html><meta http-equiv="refresh" content="0; URL=/application/login2"><body>Kerberos authentication did not pass.</body></html>

I tried to use --negotiate --anyauth or --negotiate --basic with 7.64.1 but it does not work as in previous version

sh-5.0# curl -V
curl 7.64.1 (x86_64-redhat-linux-gnu) libcurl/7.64.1 OpenSSL/1.1.1b zlib/1.2.11 brotli/1.0.7 libidn2/2.1.1 libpsl/0.20.2 (+libidn2/2.0.5) libssh/0.8.7/openssl/zlib nghttp2/1.37.0
Release-Date: 2019-03-27
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS brotli GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz Metalink NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets
sh-5.0# 
sh-5.0# curl -v --negotiate --anyauth -u : -k https://kvm-guest.example.com/application/login
*   Trying 10.37.153.128...
* TCP_NODELAY set
* Connected to kvm-guest.example.com (10.37.153.128) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; O=Unspecified; CN=kvm-guest.example.com; emailAddress=root@kvm-guest.example.com
*  start date: Apr  4 12:27:02 2019 GMT
*  expire date: Apr  8 14:07:02 2020 GMT
*  issuer: C=US; O=Unspecified; OU=ca-6605801528768118960; CN=kvm-guest.example.com; emailAddress=root@kvm-guest.example.com
*  SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
> GET /application/login HTTP/1.1
> Host: kvm-guest.example.com
> User-Agent: curl/7.64.1
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/1.1 401 Unauthorized
< Date: Thu, 04 Apr 2019 15:03:32 GMT
< Server: Apache/2.4.39 (Fedora) OpenSSL/1.1.1b mod_auth_gssapi/1.6.1
< WWW-Authenticate: Negotiate
< Content-Length: 127
< Content-Type: text/html; charset=iso-8859-1
<
* Ignoring the response-body
* Connection #0 to host kvm-guest.example.com left intact
* Issue another request to this URL: 'https://kvm-guest.example.com/application/login'
* Found bundle for host kvm-guest.example.com: 0x55ea157c8a20 [can pipeline]
* Could pipeline, but not asked to!
* Re-using existing connection! (#0) with host kvm-guest.example.com
* Connected to kvm-guest.example.com (10.37.153.128) port 443 (#0)
* gss_init_sec_context() failed: SPNEGO cannot find mechanisms to negotiate.
* Connection #0 to host kvm-guest.example.com left intact
curl: (27) Out of memory
* Closing connection 0

sh-5.0# curl -v --negotiate --basic -u : -k https://kvm-guest.example.com/application/login
*   Trying 10.37.153.128...
* TCP_NODELAY set
* Connected to kvm-guest.example.com (10.37.153.128) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; O=Unspecified; CN=kvm-guest.example.com; emailAddress=root@kvm-guest.example.com
*  start date: Apr  4 12:27:02 2019 GMT
*  expire date: Apr  8 14:07:02 2020 GMT
*  issuer: C=US; O=Unspecified; OU=ca-6605801528768118960; CN=kvm-guest.example.com; emailAddress=root@kvm-guest.example.com
*  SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
> GET /application/login HTTP/1.1
> Host: kvm-guest.example.com
> User-Agent: curl/7.64.1
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/1.1 401 Unauthorized
< Date: Thu, 04 Apr 2019 15:03:48 GMT
< Server: Apache/2.4.39 (Fedora) OpenSSL/1.1.1b mod_auth_gssapi/1.6.1
< WWW-Authenticate: Negotiate
< Content-Length: 127
< Content-Type: text/html; charset=iso-8859-1
<
* Ignoring the response-body
* Connection #0 to host kvm-guest.example.com left intact
* Issue another request to this URL: 'https://kvm-guest.example.com/application/login'
* Found bundle for host kvm-guest.example.com: 0x56144b571a20 [can pipeline]
* Could pipeline, but not asked to!
* Re-using existing connection! (#0) with host kvm-guest.example.com
* Connected to kvm-guest.example.com (10.37.153.128) port 443 (#0)
* gss_init_sec_context() failed: SPNEGO cannot find mechanisms to negotiate.
* Connection #0 to host kvm-guest.example.com left intact
curl: (27) Out of memory
* Closing connection 0

lslebodn · 2019-04-04T15:08:46Z

Anyway, returning curl: (27) Out of memory
Is far from ideal return code if --negotiate fails.

curl never "falls back" to another authentication method

I am fine if you do not want to support that and it "worked" just by a change in older version.
but I would appreciate a different and stable return code if --negotiate fails.
So I can rewrite test from and implement fall back from curl --negotiate to curl --basic

dhoelzl · 2019-04-04T15:13:17Z

What about curl --anyauth alone (without --negotiate)?
Can these parameters be combined?

lslebodn · 2019-04-04T15:29:18Z

What about curl --anyauth alone (without --negotiate)?
Can these parameters be combined?

I cannot see any difference between

curl -v --anyauth -u : -k https://kvm-guest.example.com/application/login

and

curl -v --anyauth --basic -u : -k https://kvm-guest.example.com/application/login

They all behave the same as with --negitiate.
It is described in man page and there is not any fall-back

       --anyauth
              (HTTP) Tells curl to figure out authentication method by itself, and use the most  secure
              one the remote site claims to support. This is done by first doing a request and checking
              the response-headers, thus possibly inducing an extra network round-trip.  This  is  used
              instead  of  setting  a  specific  authentication  method, which you can do with --basic,
              --digest, --ntlm, and --negotiate.

              Using --anyauth is not recommended if you do uploads from stdin,  since  it  may  require
              data  to  be  sent  twice  and then the client must be able to rewind. If the need should
              arise when uploading from stdin, the upload operation will fail.

              Used together with -u, --user.

              See also --proxy-anyauth and --basic and --digest.

As I already mention; I am fine with implement fall-back myself in the test.
I agree that running --negotiate without any valid krb5 ticket is not a typical use-case.
I just do not want to rely on return code 27 in the test, which is "Out of memory"

dhoelzl · 2019-04-04T15:39:20Z

I think the only bug here is the CURLE_OUT_OF_MEMORY instead of CURLE_RECV_ERROR (or whatever appropriate error code) which would be easy to fix as just the wrong error code is returned at some places. This behavior was already before #1975.

As stated in the docs, "...and use the most secure one the remote site claims to support" --anyauth will use negotiate in your case as this is more secure than basic, and no "fall back" to basic will be applied.

kdudka · 2019-04-04T15:42:18Z

@lslebodn

Which is the same as with --basic

I am trying to explain you that it is not the same. There is no Authorization: Basic ... header in the HTTP request with --negotiate, only with --basic.

@bagder

curl never "falls back" to another authentication method. It picks one and goes with that.

True, except with --anyauth, where this behavior is documented as a known bug:
10c91b66#diff-b97570cc6a6d2d180356338a9ccffee0R451

If that fails to work/authenticate, it returns error. (I would suggest anything else is a bug.)

The question is whether curl should send the request without authentication or not. This behavior has been consistent at least since curl-7.19.7. There might be some scripts out there relying on the fact that --negotiate without having a Kerberos ticket is a no-op.

dhoelzl · 2019-04-04T16:00:08Z

So if --anyauth or --proxy-anyauth (or even --negotiate with a "backward compatible" fallback to basic) should try alternatives/fallbacks, this is a curl command line issue but not libcurl itself?
#1975 was a fix primary for libcurl, and curl_easy_setopt(curl, CURLOPT_HTTPAUTH, CURLAUTH_NEGOTIATE); should just immediately start with a negotiate request and do nothing else but negotiate.
Maybe the code around that (curl command line, ...) may handle things different.

kdudka · 2019-04-04T16:14:57Z

@dhoelzl Nope. curl --anyauth just maps to curl_easy_setopt(..., CURLOPT_HTTPAUTH, CURLAUTH_ANY) in libcurl API, as you can easily check by curl --libcurl ....

bagder · 2019-04-04T21:07:12Z

True, except with --anyauth, where this behavior is documented as a known bug

Thanks, that was the little tidbit I had totally missed/forgot...

thrix · 2019-05-06T09:07:22Z

This change that curl fails on GSSAPI requires to start "if-ing" our test code that worked before for all Fedora and RHEL releases we test, with or without Kerberos authentication. Please consider at least providing a backward compatible solution, so we do not need to start workarounding the introduced behavior change.

bagder · 2019-05-06T09:26:27Z

Someone will have to do something...

…mory; return CURLE_RECV_ERROR in case of other errors

Fixes curl#3726

kdudka · 2019-05-07T22:20:29Z

Someone will have to do something...

There is pull request #3849 to change curl's behavior to be compatible with previous releases up to 7.64.0: If --negotiate is specified and no Kerberos ticket is available, curl sends an HTTP request without any authentication.

jay · 2019-05-08T04:44:52Z

There is pull request #3849 to change curl's behavior to be compatible with previous releases up to 7.64.0

Great so let's do that. Is there any reason why we shouldn't?

kdudka · 2019-05-09T07:52:40Z

Pushing now. Thanks for review!

Fixes #3726 Closes #3849

nicowilliams · 2019-06-03T19:35:52Z

I left a comment on the change itself. The fix is incomplete:

Setting conn->data->state.authproblem = TRUE still causes failure. This fix is incomplete. This is easy to test because you don't need any Kerberos setup or credentials to test it -- just attempt to use --negotiate to GET a resource from a simple HTTP service on localhost.

kdudka · 2019-06-04T07:32:09Z

@nicowilliams What exactly do you mean by causes failure? curl will not authenticate you with --negotiate when you have no credentials. Are you using --fail or CURLOPT_FAILONERROR?

nicowilliams · 2019-06-04T18:27:11Z

@kdudka I mean that if gss_init_sec_context() fails (in my case due to the service not existing in the Kerberos database), then the request fails as far as curl goes even though the server returned a 200 response. Yes, I was --fail (actually, libcurl, with CURLOPT_FAILONERROR enabled, but I can reproduce with the CLI).

$ curl-7.60.0 -f -u: --negotiate http://localhost:37701/x
...
$ echo $?
0
$ curl-7.65.0 -vvv -f -u --negotiate http://localhost:37701/x
...
* STATE: SENDPROTOCONNECT => DO handle 0x972df8; line 1526 (connection #0)
* gss_init_sec_context() failed: Server not found in Kerberos database
* Server auth using Negotiate with user ''
> GET /x HTTP/1.1
> Host: localhost:37701
> User-Agent: curl/7.65.0
> Accept: */*
> 
* STATE: DO => DO_DONE handle 0x972df8; line 1597 (connection #0)
* STATE: DO_DONE => PERFORM handle 0x972df8; line 1719 (connection #0)
* Mark bundle as not supporting multiuse
* HTTP 1.1 or later with persistent connection
< HTTP/1.1 200 OK
< Date: ...
< Server: ...
< ETag: ...
< Content-Type: ...
* Marked for [closure]: Connection: close used
< Connection: close
< Last-Modified: ...
< Transfer-Encoding: chunked
* multi_done
* Closing connection 0
* The cache now contains 0 members
* Expire cleared (transfer 0x972df8)
curl: (22) HTTP response code said error
$ echo $?
22
$

At the very least the error and verbose messages are not very useful. The HTTP response code was 200, so not an error, yet curl says curl: (22) HTTP response code said error. The reason for the uninformative and confusing messages is that setting conn->data->state.authproblem = TRUE causes curl to give up later where contextual information is missing.

In our case we're using TLS (via HTTPS) to authenticate the server using PKIX certificates, and we want HTTP/Negotiate for authenticating the user to the service when feasible -- if a service has no Kerberos credentials, then we want curl to continue. Perhaps we ought to be using --anyauth, as that doesn't fail in this case even with --fail, but if so we have hundreds of uses of --negotiate to remediate.

nicowilliams · 2019-06-04T18:53:11Z

@kdudka also, perhaps you should only treat gss_init_sec_context() failure as fatal when consuming a GSS token from the server. In any case, at the very least the curl error code and messages need to be more useful.

kdudka · 2019-06-05T10:26:16Z

@nicowilliams I agree that the behavior of curl --fail --negotiate ... has changed. However, it is yet another side effect of 6c60355. I do not think that f460370 makes it any worse. curl --fail had always treated the failure of gss_init_sec_context() as error. The only difference is that, before 6c60355, gss_init_sec_context() was not called unless the server requested the authentication, whereas now curl calls gss_init_sec_context() even before it sends the initial HTTP request. I would suggest to open a separate issue for this, as it is unrelated to the original CURLE_OUT_OF_MEMORY issue.

nicowilliams · 2019-06-05T17:59:36Z

@kdudka I've opened #3992. Thanks!

nicowilliams · 2019-06-05T21:18:49Z

f460370 does not make it worse, indeed, but f460370 was supposed to fix it given its subject, http_negotiate: do not treat failure of gss_init_sec_context() as fatal (it does still treat initial gss_init_sec_context() failures as fatal.

lslebodn changed the title ~~Out of memory is returned deom command line for gssapi auth with missing krb5 ticket~~ Out of memory is returned from command line for gssapi auth with missing krb5 ticket Apr 3, 2019

jay added the authentication label Apr 3, 2019

bagder added the help wanted label May 6, 2019

dhoelzl pushed a commit to dhoelzl/curl that referenced this issue May 6, 2019

curl#3726 vauth: return CURLE_OUT_OF_MEMORY only in case of out of me…

4a58c63

…mory; return CURLE_RECV_ERROR in case of other errors

dhoelzl mentioned this issue May 6, 2019

#3726 vauth: return CURLE_OUT_OF_MEMORY only in case of out of memory #3848

Closed

dhoelzl pushed a commit to dhoelzl/curl that referenced this issue May 6, 2019

curl#3726 vauth: return CURLE_OUT_OF_MEMORY only in case of out of me…

700e052

…mory; return CURLE_RECV_ERROR in case of other errors

kdudka added a commit to kdudka/curl that referenced this issue May 6, 2019

spnego_gssapi: fix return code on gss_init_sec_context() failure

609f8b1

Fixes curl#3726

kdudka added a commit to kdudka/curl that referenced this issue May 6, 2019

http_negotiate: do not treat failure of gss_init_sec_context() as fatal

c9ab6fa

Fixes curl#3726

kdudka closed this as completed in f65845c May 9, 2019

kdudka added a commit that referenced this issue May 9, 2019

http_negotiate: do not treat failure of gss_init_sec_context() as fatal

f460370

Fixes #3726 Closes #3849

nicowilliams mentioned this issue Jun 5, 2019

curl --fail --negotiate behavior changed between 7.64.0 and 7.65.0 #3992

Closed

lock bot locked as resolved and limited conversation to collaborators Sep 3, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Out of memory is returned from command line for gssapi auth with missing krb5 ticket #3726

Out of memory is returned from command line for gssapi auth with missing krb5 ticket #3726

lslebodn commented Apr 3, 2019

lslebodn commented Apr 3, 2019

jay commented Apr 3, 2019

dhoelzl commented Apr 4, 2019 •

edited

kdudka commented Apr 4, 2019

MarcelRaad commented Apr 4, 2019

dhoelzl commented Apr 4, 2019

kdudka commented Apr 4, 2019

dhoelzl commented Apr 4, 2019

MarcelRaad commented Apr 4, 2019

kdudka commented Apr 4, 2019

bagder commented Apr 4, 2019

lslebodn commented Apr 4, 2019

lslebodn commented Apr 4, 2019 •

edited

dhoelzl commented Apr 4, 2019

lslebodn commented Apr 4, 2019

dhoelzl commented Apr 4, 2019

kdudka commented Apr 4, 2019

dhoelzl commented Apr 4, 2019 •

edited

kdudka commented Apr 4, 2019

bagder commented Apr 4, 2019

thrix commented May 6, 2019

bagder commented May 6, 2019

kdudka commented May 7, 2019

jay commented May 8, 2019

kdudka commented May 9, 2019

nicowilliams commented Jun 3, 2019

kdudka commented Jun 4, 2019

nicowilliams commented Jun 4, 2019

nicowilliams commented Jun 4, 2019

kdudka commented Jun 5, 2019

nicowilliams commented Jun 5, 2019

nicowilliams commented Jun 5, 2019

Out of memory is returned from command line for gssapi auth with missing krb5 ticket #3726

Out of memory is returned from command line for gssapi auth with missing krb5 ticket #3726

Comments

lslebodn commented Apr 3, 2019

I did this

I expected the following

curl/libcurl version

operating system

lslebodn commented Apr 3, 2019

jay commented Apr 3, 2019

dhoelzl commented Apr 4, 2019 • edited

kdudka commented Apr 4, 2019

MarcelRaad commented Apr 4, 2019

dhoelzl commented Apr 4, 2019

kdudka commented Apr 4, 2019

dhoelzl commented Apr 4, 2019

MarcelRaad commented Apr 4, 2019

kdudka commented Apr 4, 2019

bagder commented Apr 4, 2019

lslebodn commented Apr 4, 2019

lslebodn commented Apr 4, 2019 • edited

dhoelzl commented Apr 4, 2019

lslebodn commented Apr 4, 2019

dhoelzl commented Apr 4, 2019

kdudka commented Apr 4, 2019

dhoelzl commented Apr 4, 2019 • edited

kdudka commented Apr 4, 2019

bagder commented Apr 4, 2019

thrix commented May 6, 2019

bagder commented May 6, 2019

kdudka commented May 7, 2019

jay commented May 8, 2019

kdudka commented May 9, 2019

nicowilliams commented Jun 3, 2019

kdudka commented Jun 4, 2019

nicowilliams commented Jun 4, 2019

nicowilliams commented Jun 4, 2019

kdudka commented Jun 5, 2019

nicowilliams commented Jun 5, 2019

nicowilliams commented Jun 5, 2019

dhoelzl commented Apr 4, 2019 •

edited

lslebodn commented Apr 4, 2019 •

edited

dhoelzl commented Apr 4, 2019 •

edited