cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Project hiper - High Performance libcurl

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Mon, 7 Nov 2005 20:34:22 +0100 (CET)

On Mon, 7 Nov 2005, Cory Nelson wrote:

> Sorry for the late reply, been busy...

No worries at all. This project won't be "done" in several months anyway, and
I won't get up to full speed on it until December.

> Here is a small sample that should give you the idea. Unfortunately I don't
> know *nix APIs very well, so I'm unable to guess the best API that would
> work for the both of them.

You don't have to have that knowledge. But I would rather like us to discuss
around and based on the current API proposal and the psuedo code app as I've
published on the site, as it has been developed over time through discussions
to become a fairly generic approach that would fit "random" event systems.

> I'd make something like:
>
> // called on completion, error, etc..
> typedef void (*hiper_callback)(int status, CURL *easy);
>
> // creates the handler.
> hiper_handle hiper_create(hiper_callback);
> void hiper_destroy(hiper_handle);
>
> // pushes an easy handle onto the hyper handle.
> void hiper_push(hiper_handle, CURL *easy);
>
> // blocks until all easy handles are done processing.
> void hiper_wait(hiper_handle);
>
> The big change is making the wait function optional- blocking doesn't belong
> in a high perf app. I'm assuming that function currently processes all the
> sockets- I believe that should all be done in the background for the user so
> CURL can make proper use of SMP if it's available. And maybe add another
> function to pull the current status of an easy handle.
>
> What are your thoughts?

First I sould say that I don't know much about how to design this to make it
work as good as possible in an SMP environment, and I haven't given that so
much consideration so far.

A few things about your proposal:

I don't see how that is a fully working API. My idea is to extend/modify the
current multi interface, which thus allows simultaneous transfers in the same
thread. I have no plans to add any kind of multi-thread awareness in libcurl,
but of course i should still function correctly when used multi-threaded.

That said, your "should all be done in the background" comment sounds weird in
my ears. Sure you can start a new thread and have libcurl do all the work
there and thus it works "in the background", but I don't see how libcurl must
be modified to deal with that.

The current multi interface is not blocking. The only blocking in a typical
app should be done by the event system call that waits for action on one or
more sockets. And that is made by the app, outside of libcurl's control.

So I guess in the end I don't fully understand your proposal. Can you expand
it to a fully "working" pseudo code example?

-- 
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html
Received on 2005-11-07