On Tue, 8 Nov 2005, Cory Nelson wrote:

> The current multi interface of having the user wait for events is not an
> efficient design when it comes to IOCP.
>
> IOCP uses threading to get full scalabilty on SMP systems.

But does it really scale that well on the ordinary plain non-SMP system? Is
your normal Windows box up to serving 10,000 threads? (if you have the RAM for
it). And I would expect that using a thread for each connection will be more
memory consuming that using a single thread for all transfers (due to stack
and thread context overhead).

> It could certainly be done from one thread, but then you would be losing
> many of the benefits. This is aimed at being high performance so libcurl
> (imho) should definately add the minimal threading awareness for IOCP, if
> only for this hiper API.
>
> The API I gave was complete. You open a hiper handle and push your easy
> handles into it. The hiper handle would open one or more threads in the
> background and callback when something significant happens to an easy
> handle. Simple as that. If you wanted to wait until all the easy handles
> are finished, you can call the wait function.

(I see a problem to merge a Windows-optimzed concept with the currently
planned event-optimized concept...)

If that is what you want (using a thread for each transfer), won't it suffice
to simply start a new thread and fire off a separate curl_easy_perform() in
there? In what way would this suggested interface be an improvement to that?

HTTP pipelining might be hard to add nicely for such a use case.

> If hiper is meant to abstract away all the stuff needed for high performance
> http, it should also be in charge of threading efficiently as needed.

I want libcurl to abstract away all protocol and transfer related matters. I
want it to know as little as possible about event systems and threading
models.

> That's where the confusion is :) My proposal moves the blocking into
> libcurl where it could be efficiently managed and out of the users reach.

Now I believe I understand your proposal a lot better.

I'll have to give this some more thoughts...

-- 
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html