Hi Nick,

On Fri, Jun 20, 2008 at 2:34 PM, Daniel Stenberg <daniel_at_haxx.se> wrote:

> On Fri, 20 Jun 2008, Nick George wrote:
>
> I ended up giving up on the multithreaded approach. I was having other
>> problems with my i386 box not being able to create more than 300 threads
>> anyway. So, instead I tried extending the multi-app.c example to perform the
>> same tests. It works fine up to about 150 threads, where performance starts
>> to degrade massively.
>>
>
>
I would suggest you to look at the code paths and what is leading it
to the locking contentions.

From an architectural point of view, since your HW has 4-8 cpu cores,
more threads than about 20-30 can scarcely deliver any performance gains.
Each extra thread is only consuming memory and scheduling resources
(although very effective in kernels-2.6).

You may wish to consider the great curl_multi_socket() API as proposed by
Daniel,
whereas several threads can be run each with it's own independent load to
utilize
multi-core hardware.
This is what is doing curl-loader for example.

Yet another option to consider is Leader-Followers design pattern of Dr.
Douglas Schmidt
and his great ACE-library team. However, you will need to work hard to
interface libcurl
in a proper way.

This is jm2c. Take care.

Truly,
Robert Iakobashvili
......................................................................
www.ghotit.com
Assistive technology that understands you
......................................................................
Received on 2008-06-20