cURL / Mailing Lists / curl-and-php / Single Mail

curl-and-php

Re: multi curl working very slow when threads > 10

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Mon, 24 Jan 2011 17:21:57 +0100 (CET)

On Mon, 24 Jan 2011, far Fraser wrote:

> Curl: 7.21.3

Using which resolver backend in libcurl?

> I'm trying simple multi curl script (GET list of URLs in 10-100 threads).

The libcurl multi interface doesn't use threads at all. It does multiple
transfers in a single thread.

> It does work on any windows, but when I try the same code on CentOS: if
> threads = 10, everything works almost fine, but when I set threads = 50 or
> 100 the process is almost stuck, it takes minutes for "multi_curl_exec"

That sounds like you're using the stock resolver, as that is synchronous all
the other transfers stall during each name resolve. 100 name resolves in
serial can take a long time. libcurl for windows defaults to build with a
threader resolver.

-- 
  / daniel.haxx.se
_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
Received on 2011-01-24