cURL / Mailing Lists / curl-library / Single Mail


Re: Reducing the number of DNS queries in the multi handle.

From: Tsachi <>
Date: Wed, 26 Nov 2008 14:16:50 +0200

On Wed, Nov 26, 2008 at 1:18 PM, Daniel Stenberg <> wrote:
> On Wed, 26 Nov 2008, Tsachi wrote:
>> Now, many of the urls I am fetching belongs to the same domain.
>> Example: , ,
>> I can see that all DNS requests are sent at the beginning (probably right
>> after calling curl_multi_perform()) that's b/c I am using asynchronous name
>> resolves.
> And you're fetching these pages in parallell, right?


>> I have noticed that the libcurl issues a different dns request for every
>> URL in the multi handle although many of them share the same domain. This
>> means that for the example above I get to see 3 DNS requests for
> libcurl has an internal DNS cache that by default is used for all easy
> handles within a multi handle, but that of course requires that the host
> name has already been resolved fine at the time a new connection is about to
> be setup.
>> I am wondering if there is an option that I can use to tell the libcurl to
>> issue only one DNS request for and only when response arrives
>> perform the HTTP requests for all the URL's sharing the same domain ?
> There's no such option. But you could delay the following requests to not
> get started until the first one delivers data as then you know it has been
> resolved and will be dealt with by the cache...
> --
> /

Thanks, that is what I will probably do.

Is there a way to know that I got a DNS response before the actual
http connection starts (callback / info) ?
This way I could add the others easy handles (with the same domain
URLs) at this point.

Received on 2008-11-26