cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: a curl_multi_fdset() replacement? (TODO-RELEASE #55)

From: Daniel Stenberg <daniel-curl_at_haxx.se>
Date: Mon, 31 Jan 2005 09:08:51 +0100 (CET)

On Mon, 31 Jan 2005, Jamie Lokier wrote:

> On many platforms, it's possible to overcome the FD_SETSIZE limitation while
> still using select(). You have to break the rules a bit, but on nearly
> every platform fd_set is a bit vector using some word size and the FD_SET et
> al. macros work with larger fd numbers, provided enough memory is allocated
> and zeroed. You can't guarantee this is portable, though, just that it
> works in practice on many platforms.

I prefer not to walk that path. libcurl compiles on runs on a large amount of
systems. Doing like this would bite us, one way or another.

Besides, when using the multi interface, FD_SET() would also most likely be
used by the app using libcurl so the work-around would be to be done by all
libcurl-using apps that care about this. Not so nice.

> You're limiting the API to something that's slow when there are lots of
> descriptors and only a few are active at each poll.

This function would only return descriptors that are "active" - that is any
one of the returned descriptors may turn out readable/writeable.

In a typical app, you'd call this function immediately before you poll(),
select() or whatever.

> This means that each call to select() or poll() or equivalent is going to
> take O(n) in the number of descriptors being polled.

This interface doesn't differ from the current function in this aspect.
AFAICT.

> For applications with thousands of descriptors, the time spent in select()
> or poll() can be considerable.
>
> This becomes more and more of a problem when handling large numbers of
> descriptors.
>
> Ideally, you want an API which takes O(n) in the number of descriptors which
> are _ready_ at the poll, so that you can take advantage of faster
> readiness-monitoring system calls such as epoll (Linux), kqueue (FreeBSD),
> or /dev/poll (Solaris/IRIX/HPUX).

I'm not sure what "_ready_ at the poll" means. Nor am I familiar with these
faster system calls. Can you elaborate?

> That means an API where the caller can register an fd to be monitored, is
> notified when the fd is ready, and a "poll" operation takes an opaque data
> structure which remembers which fds are being monitored.

You mean something like curl_multi_poll() that would wait for all libcurl's
descriptors plus the applications' own ones? I fail to see how this makes
anything faster/better.

-- 
      Daniel Stenberg -- http://curl.haxx.se -- http://daniel.haxx.se
       Dedicated custom curl help for hire: http://haxx.se/curl.html
Received on 2005-01-31