Daniel Stenberg wrote:
> >For applications with thousands of descriptors, the time spent in select()
> >or poll() can be considerable.
> >
> >This becomes more and more of a problem when handling large numbers of
> >descriptors.
> >
> >Ideally, you want an API which takes O(n) in the number of descriptors
> >which are _ready_ at the poll, so that you can take advantage of faster
> >readiness-monitoring system calls such as epoll (Linux), kqueue (FreeBSD),
> >or /dev/poll (Solaris/IRIX/HPUX).
>
> I'm not sure what "_ready_ at the poll" means. Nor am I familiar with these
> faster system calls. Can you elaborate?

Imagine what happens when you have 10000 open file descriptors
(extreme to illustrate) and after each call to select() or poll(),
only 20 descriptors are ready for reading.

select() or poll() will scan 10000 descriptors, find the 20 ready ones,
and return the 20.

The slow part is scanning the 10000 descriptors just to find they are
not ready. (It's especially slow on Linux unfortunately). It is
surprisingly slow: several milliseconds.

If at each select() or poll() call, the number of ready descriptors is
very small compared with the number being polled, then a lot of time
is spent scanning not-ready descriptors, and relatively little time is
spent doing actual I/O.

In other words, the time spend in select() or poll() comes to
dominate. That typically happens when you have lots of connections
over lots of slow links - for example in big web servers - but also in
a big http spider (to pick a relevant example for curl...).

All of the replacement APIs fix this problem.
All of them follow this basic principle:

    - a call to add or remove an fd from an "interest set".
    - a call to poll everything in an interest set.

The speed difference comes from the fact that an fd is added to an
interest set once, and then you leave it there across many poll calls
until it's ready for I/O.

So in the example of 10000 connections, that's 10000 fds added
initially, but then every call to e.g. epoll_wait() will return the 20
ready descriptors _without_ scanning all the not-ready ones.

It's an algorithmic difference, that becomes significant with large
numbers of connections where relatively few are ready for I/O at each
polling time - that's the sort of pattern I'd expect if curl is used
for a big web spider.

> >That means an API where the caller can register an fd to be monitored, is
> >notified when the fd is ready, and a "poll" operation takes an opaque data
> >structure which remembers which fds are being monitored.
>
> You mean something like curl_multi_poll() that would wait for all libcurl's
> descriptors plus the applications' own ones? I fail to see how this makes
> anything faster/better.

No. (You could do that, but I warn you that writing code to use the
various other APIs at their best isn't easy, and as someone else
mentioned, it wouldn't fit comfortably with programs that need to call
select() or whatever themselves, such as Gnome programs).

No: the important thing is that your API treats the application's
polling method as having two function calls:

    1. Add/change/remove a file descriptor.
    2. Poll to get list of ready descriptors.

So curl should either (a) call an application callback function to do
the add/change/remove operation, or (b) rather than returning arrays
of descriptors that curl needs to poll, it should return an array of
the _changes_ in the set of descriptors being polled.

A good all-round summary is found at "The C10K problem" at:

    http://www.kegel.com/c10k.html

-- Jamie
Received on 2005-01-31