cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: Slow DNS lookups compared to wget

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Mon, 18 Feb 2002 09:54:24 +0100 (MET)

On Fri, 15 Feb 2002, Rick Richardson wrote:

> DNS lookups of some (but not all) URLs are extremely slow using cURL 7.8,
> but are not slow when using wget:

Judging from other comments in your mail, I'm guessing that you're using a
default Redhat 7.2 curl installation, right? That would mean that curl says
it is "ipv6 enabled", right (in the curl -V output)?

> # wget is nice and fast....
>
> $ time wget -q -O- "http://quote.bloomberg.com/markets/earnings/ecal.cgi" > xxx
> real 0m2.511s
> user 0m0.000s
> sys 0m0.010s
>
>
> # cURL 7.8 is picking lint from its bellybutton...
>
> $ time curl -s "http://quote.bloomberg.com/markets/earnings/ecal.cgi" > xxx
> real 0m31.857s
> user 0m0.010s
> sys 0m0.000s

This certainly indicates a problem somewhere, and I'll be much suprised if
this is anything that the curl source code does wrong! :-/

The main difference between wget and curl in the DNS resolving parts, is that
wget uses the good old traditional gethostbyname() for name resolves, while
curl (if compiled "IPv6 enabled") uses getaddrinfo() and if not IPv6 enabled,
it uses gethostbyname_r() (on systems that offer it).

Is your kernel IPv6 enabled at all?

Does anyone else have more clues to fill in on this subject here? We could
check for glibc-related notes, or Redhat erratas or whatever...

> Are there any workarounds besides a painful domino-like upgrade of the RPM
> and the several co-dependant RPMs (libssl, libcrypto, php)?

You either do as Troy suggests, or you grab a source archive and build
yourself. It really isn't hard. Even for someone who never did it before
(which doesn't necessarily mean you, I just mean that it is easy).

-- 
    Daniel Stenberg -- curl groks URLs -- http://curl.haxx.se/
Received on 2002-02-18