cURL / Mailing Lists / curl-library / Single Mail

curl-library

Performance fix for dns lookup timeouts

From: Daniel Speyer <dspeyer_at_gmail.com>
Date: Sun, 26 Apr 2015 00:11:47 -0400

Hello curl folks,

I was using curl in the test cases of the new profiling tool I was
developing and I stumbled on a small bug. When curl spins off a
separate thread for dns lookups, it waits using poll. However, the
child thread doesn't write to any file descriptor, so the parent waits
for a timeout. This costs several milliseconds (either when nscd has
the ip in cache, but also if the lookup time modulo the timeout is
small). If the actual remote host is over a local network, this
timeout could be a large fraction of the total run time, and if the
curl command was inside a loop in a shell script, it could add up.

I've written a fix (attached) that has the child thread write to a
pipe that gets included in the poll command. I realize pipes aren't
the most popular form of interthread communication, but they're simple
and they play nicely with poll.

I tested it on some simple downloads and it gets the expected
performance improvement. I haven't tested it on anything really
complicated (I'm not really sure what a complicated usage of curl
would be).

Hope you like it,
--Daniel Speyer

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html

Received on 2015-04-26