cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Libcurl returns timeout error when used from multiple threads, but not from a single thread

From: Dan Fandrich <dan_at_coneharvesters.com>
Date: Tue, 13 Dec 2011 23:19:05 -0800

On Wed, Dec 14, 2011 at 10:27:06AM +0400, Alex wrote:
> I'm using libcurl with C++. I have made a thread-safe class for downloading webpages. Each call to static download method creates "easy" handle, performs the job and frees the handle. When I use it from the single thread - everything's fine. But when I spawn several threads to download several pages in parallel I sometimes (not fro every download, but quite often) get error saying "timeout". I have a reasonably high timeout configured (5 sec connection timeout and 25 sec global timeout).
>
> Any ideas as to what might be the cause and how to fix it?

There's a reasonable chance that it's a server limitation, either on the
maximum number of concurrent connections, or deliberate throttling.
You'd have to contact the server administrator to verify that, though.
You could also look at a network trace when that happens to verify that
your app is sending the correct data and that the response (either DNS or
TCP/HTTP) isn't received.

>>> Dan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-12-14