cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: blocking on dns resolving

From: Pedro Larroy <pedro.larroy.lists_at_gmail.com>
Date: Sun, 18 Dec 2011 20:04:38 +0100

This is what i mean:

(gdb) bt#0  0x00007ffff5aad6d0 in poll () from
/lib/x86_64-linux-gnu/libc.so.6#1  0x0000000000497505 in Curl_poll
(ufds=0x7fffffffd1c0, nfds=1, timeout_ms=0) at select.c:371#2
0x000000000049a57f in waitperform (conn=0xa05df8, timeout_ms=0) at
asyn-ares.c:287#3  0x000000000049a67f in Curl_resolver_is_resolved
(conn=0xa05df8, dns=0x7fffffffd328) at asyn-ares.c:324#4
0x000000000049b80f in Curl_resolv (conn=0xa05df8, hostname=0xa06418
"romavictrix.blogspot.com", port=80, entry=0x7fffffffd3e0) at
hostip.c:463#5  0x000000000049b915 in Curl_resolv_timeout
(conn=0xa05df8, hostname=0xa06418 "romavictrix.blogspot.com", port=80,
entry=0x7fffffffd3e0, timeoutms=60000) at hostip.c:628#6
0x00000000004b1bed in resolve_server (data=0x97b8e8, conn=0xa05df8,
async=0x7fffffffd726) at url.c:4583#7  0x00000000004b2d37 in
create_conn (data=0x97b8e8, in_connect=0xa05510, async=0x7fffffffd726)
at url.c:5066#8  0x00000000004b2feb in Curl_connect (data=0x97b8e8,
in_connect=0xa05510, asyncp=0x7fffffffd726,
protocol_done=0x7fffffffd725) at url.c:5197#9  0x0000000000492db1 in
multi_runsingle (multi=0x74def8, now=..., easy=0xa054f8) at
multi.c:1045#10 0x0000000000495023 in multi_socket (multi=0x74def8,
checkall=false, s=-1, ev_bitmask=0, running_handles=0x7fffffffdc24) at
multi.c:2194#11 0x0000000000495480 in curl_multi_socket_action
(multi_handle=0x74def8, s=-1, ev_bitmask=0,
running_handles=0x7fffffffdc24) at multi.c:2274#12 0x0000000000470cfb
in timer_cb (fd=-1, kind=1, userp=0x7fffffffd9d0) at
src/crawler.cc:767#13 0x00007ffff704f97c in event_base_loop () from
/usr/lib/libevent-2.0.so.5#14 0x00000000004737da in main (argc=1,
argv=0x7fffffffdec8) at src/crawler.cc:1133

Curl was built with:

        '--disable-ldap',
        '--without-libssh2',
        '--without-librtmp',
        '--enable-debug',
        '--enable-ares={0}'.format(os.path.join(old_dir,
'3rd_party','c-ares_install')),
        '--prefix={0}'.format(os.path.join(old_dir, '3rd_party',
'curl_install')),

On Sun, Dec 18, 2011 at 7:34 PM, Daniel Stenberg <daniel_at_haxx.se> wrote:
> On Sun, 18 Dec 2011, Pedro Larroy wrote:
>
>> After some flustration with blocking on dns requests while using the multi
>> interface with libevent (https://github.com/larroy/mycelium) I investigated
>> how curl uses ares and I came to the conclusion that the design of
>> asyn-ares.c is very limiting.
>
>
> Can you elaborate on when exactly it is "very limiting" ?
>
>
>> What's the point of having all the goodies of the multi interface to use
>> epoll etc, when waitperform function in asyn-ares.c just polls waiting for
>> DNS queries to finish?
>
>
> It doesn't. The multi interface tells your app about the socket c-ares uses
> and your app waits for actions on that socket before it tells libcurl to
> work on it.
>
>
>> I think the proper way would be to expose the resolving sockets up to the
>> multi interface so the whole system is really asynchronous. Is there any
>> reason why it wasn't done this way?
>
>
> It _is_ done like that. There are just a few exceptions to that rule. They
> are documented and will hopefully all be fixed one by one until gone one day
> in the future.
>
> --
>
>  / daniel.haxx.se
>
> -------------------------------------------------------------------
> List admin: http://cool.haxx.se/list/listinfo/curl-library
> Etiquette:  http://curl.haxx.se/mail/etiquette.html

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-12-18