cURL / Mailing Lists / curl-library / Single Mail


Re: libcurl 7.9.8: Bottleneck on multhithread HTTP fetches?

From: Daniel Stenberg <>
Date: Wed, 15 Oct 2003 08:52:34 +0200 (CEST)

On Tue, 14 Oct 2003, Andrew Fuller wrote:

> By the way, if it's any use, this time the longest path (timewise) according
> to Rational Quantify was :
> curl_easy_perform->Curl_perform->Curl_connect->SetupConnection->ConnectPlease->Curl_connecthost->socketerror->getsockopt->EnterCriticalSection
> Next longest was the same to above except ended :
> ...->getsockopt->NtRemoveIoCompletion

Thanks for the provided info. Here's my take on analyzing it:

The VERBOSE output log (from your previous mail) is a mess to read and really
doesn't help much since all the different threads' output mix in a seemingly
random way. You would need to prefix all output lines with a thread id/number
or something for it to start becoming useful. (Thus you'd need to use

How many threads do you need to get this problem? You used 80 here, but what's
the minimum amount? What's your network connection to the test server?

We could try to learn from history. See what Chicco wrote back in 2002:

Bjorn Reese filled in some more info on why libcurl does this:

The relevant question here and now is of course: can we modify this behavior
on Windows to make it run better? Chicco said that he didn't need to invoke
'socketerror' if connect() returns 0. Perhaps this is always true on windows?
This page on seems to claim this:

I don't use windows enough myself to try this out.

 Daniel Stenberg -- curl: been grokking URLs since 1998
This email is sponsored by: Giveback Program. hosts over 70,000 Open Source Projects.
See the people who have HELPED US provide better services:
Click here:
Received on 2003-10-15