Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

macOS: FTP + DoH fails with '(15) Could not DOH-resolve: {IPv4-address}' #5454

Closed
vszakats opened this issue May 25, 2020 · 2 comments
Closed
Labels
name lookup DNS and related tech

Comments

@vszakats
Copy link
Member

vszakats commented May 25, 2020

I understand it's an odd combination, but bumped into it anyhow. Initially I was looking into why the macOS-bundled curl 7.64.1 is doing a Segmentation fault: 11 with these options. I could not repeat it with a custom build of the latest source tree, but found this issue instead. Looks like curl wants to resolve the server IP via DoH.

I did this

./curl -q -vv \
  --doh-url https://dns.cloudflare.com/dns-query \
  ftp://ftp.thp.uni-duisburg.de/pub/source/pstree-2.39.tar.gz
stderr
* Found bundle for host dns.cloudflare.com: 0x7fe83240bfa0 [serially]
* Server doesn't support multiplex (yet)
*   Trying 2606:4700::6810:85e5:443...
* Hostname 'dns.cloudflare.com' was found in DNS cache
*   Trying 2606:4700::6810:85e5:443...
* Connected to dns.cloudflare.com (2606:4700::6810:85e5) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Connected to dns.cloudflare.com (2606:4700::6810:85e5) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* TLS 1.2 connection using TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
* Server certificate: cloudflare.com
* Server certificate: CloudFlare Inc ECC CA-2
* Server certificate: Baltimore CyberTrust Root
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fe83281d600)
> POST /dns-query HTTP/2
Host: dns.cloudflare.com
accept: */*
content-type: application/dns-message
content-length: 41

* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
* We are completely uploaded and fine
* TLS 1.2 connection using TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
* Server certificate: cloudflare.com
* Server certificate: CloudFlare Inc ECC CA-2
* Server certificate: Baltimore CyberTrust Root
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fe832823200)
> POST /dns-query HTTP/2
Host: dns.cloudflare.com
accept: */*
content-type: application/dns-message
content-length: 41

* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
* We are completely uploaded and fine
< HTTP/2 200 
< date: Mon, 25 May 2020 20:51:07 GMT
< content-type: application/dns-message
< content-length: 98
< access-control-allow-origin: *
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< server: cloudflare
< cf-ray: 599221f3bbf8ceab-GVA
< cf-request-id: 02ef338c560000ceabe3906200000001
< 
* Connection #0 to host dns.cloudflare.com left intact
* a DOH request is completed, 1 to go
< HTTP/2 200 
< date: Mon, 25 May 2020 20:51:07 GMT
< content-type: application/dns-message
< content-length: 123
< access-control-allow-origin: *
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< server: cloudflare
< cf-ray: 599221f3cc06ceab-GVA
< cf-request-id: 02ef338c5c0000ceabe3907200000001
< 
* Connection #1 to host dns.cloudflare.com left intact
* a DOH request is completed, 0 to go
* DOH Host name: ftp.thp.uni-duisburg.de
* TTL: 169695 seconds
* DOH A: 134.91.141.1
* CNAME: op2.thp.uni-duisburg.de
* CNAME: op2.thp.uni-duisburg.de
*   Trying 134.91.141.1:21...
* Connected to ftp.thp.uni-duisburg.de (134.91.141.1) port 21 (#0)
< 220 (vsFTPd 2.0.3)
> USER anonymous
< 331 Please specify the password.
> PASS ftp@example.com
< 230 Login successful.
> PWD
< 257 "/"
* Entry path is '/'
> CWD pub
* ftp_perform ends with SECONDARY: 0
< 250 Directory successfully changed.
> CWD source
< 250 Directory successfully changed.
> EPSV
* Connect data stream passively
< 229 Entering Extended Passive Mode (|||53657|)
* Could not DOH-resolve: 134.91.141.1
* Can't resolve new host 134.91.141.1:53657
* Closing connection 0
curl: (15) Could not DOH-resolve: 134.91.141.1

I expected the following

Start downloading the file at the URL while using the specified DoH resolver.

Omitting --doh-url fixes it. Adding --ftp-pasv doesn't.

curl/libcurl version: commit

curl 7.71.0-DEV (x86_64-apple-darwin19.4.0) libcurl/7.71.0-DEV SecureTransport zlib/1.2.11 brotli/1.0.7 libidn2/2.3.0 libpsl/0.21.0 (+libidn2/2.3.0) nghttp2/1.40.0 librtmp/2.3
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS brotli HTTP2 IDN IPv6 Largefile libz NTLM NTLM_WB PSL SSL UnixSockets

operating system

macOS 10.15.4

@vszakats vszakats added FTP name lookup DNS and related tech labels May 25, 2020
@bagder
Copy link
Member

bagder commented May 26, 2020

I believe this is a macOS-specific issue because of this logic:

curl/lib/hostip.c

Lines 530 to 544 in 96a822f

#ifndef USE_RESOLVE_ON_IPS
/* First check if this is an IPv4 address string */
if(Curl_inet_pton(AF_INET, hostname, &in) > 0)
/* This is a dotted IP address 123.123.123.123-style */
addr = Curl_ip2addr(AF_INET, &in, hostname, port);
#ifdef ENABLE_IPV6
if(!addr) {
struct in6_addr in6;
/* check if this is an IPv6 address string */
if(Curl_inet_pton(AF_INET6, hostname, &in6) > 0)
/* This is an IPv6 address literal */
addr = Curl_ip2addr(AF_INET6, &in6, hostname, port);
}
#endif /* ENABLE_IPV6 */
#endif /* !USE_RESOLVE_ON_IPS */

USE_RESOLVE_ON_IPS is defined in curl_setup.h if __APPLE is defined because we want to call getaddrinfo() even on plain IP addresses there. The logic just isn't adjusted to avoid DoH in this case!

@vszakats vszakats changed the title FTP + DoH fails with '(15) Could not DOH-resolve: {IPv4-address}' macOS: FTP + DoH fails with '(15) Could not DOH-resolve: {IPv4-address}' May 26, 2020
bagder added a commit that referenced this issue May 26, 2020
USE_RESOLVE_ON_IPS (defined on macOS) means that addresses still need to
get "resolved" but not with DoH.

Reported-by: Viktor Szakats
Fixes #5454
@bagder
Copy link
Member

bagder commented May 26, 2020

Let me know how that PR works for you!

@vszakats vszakats removed the FTP label May 26, 2020
@bagder bagder closed this as completed in 67d2802 May 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
name lookup DNS and related tech
Development

Successfully merging a pull request may close this issue.

2 participants