Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
"Could not resolve host" error only on the multi interface
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Luis Carlos Chalaca Figueira via curl-library <curl-library_at_lists.haxx.se>
Date: Tue, 31 Oct 2023 17:07:52 +0000
Hello everybody,
I'm building a web crawler.
Started with python but the need for speed led me to libcurl.
The obstacle I'm having is: when using the multi interface a big part of
the the connections cloe because of the error "Could not resolve host:"
What I've tried:
1 - Confirmed that all URLs are correct.
2 - Confirmed that using the easy interface URLs are resolved.
3 - Confirmed that other programs like dig, resolvectl, nslookup can
resolve URLs that multi interface couldn't.
4 - Set a "reputable" DNS server doing: curl_easy_setopt(handles[i],
CURLOPT_DNS_SERVERS, "8.8.8.8");.
5 - In the file /etc/nsswitch.conf, line "hosts: files
nDNS[NOTFOUNDED=return] dns" I deleted [NOTFOUNDED=return].
6 - I've turned verbose on - Doesn't help me much.
I don't know what to do more as it seems a problem inside the library,
so i posted here before going digging in the library.
System info:
O.S. - Ubuntu 22.04.3 LTS
Harware - HP HP Pavilion Laptop 14-ce3xxx
Processor - Intel® Core™ i7-1065G7 CPU _at_ 1.30GHz × 8
libcurl version - 7.81.0
Attached I send the source code file I'm using do debug this problem.
Best Regards,
Luis Figueira
Received on 2023-10-31
Date: Tue, 31 Oct 2023 17:07:52 +0000
Hello everybody,
I'm building a web crawler.
Started with python but the need for speed led me to libcurl.
The obstacle I'm having is: when using the multi interface a big part of
the the connections cloe because of the error "Could not resolve host:"
What I've tried:
1 - Confirmed that all URLs are correct.
2 - Confirmed that using the easy interface URLs are resolved.
3 - Confirmed that other programs like dig, resolvectl, nslookup can
resolve URLs that multi interface couldn't.
4 - Set a "reputable" DNS server doing: curl_easy_setopt(handles[i],
CURLOPT_DNS_SERVERS, "8.8.8.8");.
5 - In the file /etc/nsswitch.conf, line "hosts: files
nDNS[NOTFOUNDED=return] dns" I deleted [NOTFOUNDED=return].
6 - I've turned verbose on - Doesn't help me much.
I don't know what to do more as it seems a problem inside the library,
so i posted here before going digging in the library.
System info:
O.S. - Ubuntu 22.04.3 LTS
Harware - HP HP Pavilion Laptop 14-ce3xxx
Processor - Intel® Core™ i7-1065G7 CPU _at_ 1.30GHz × 8
libcurl version - 7.81.0
Attached I send the source code file I'm using do debug this problem.
Best Regards,
Luis Figueira
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
- text/x-csrc attachment: crawler.c