cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: libcurl multithreading

From: Jamie Lokier <jamie_at_shareable.org>
Date: Fri, 22 Sep 2006 19:00:03 +0100

Török Edvin wrote:
> >Beware that pthread_getspecific() is not async-signal-safe on some
> >platforms, and in the POSIX standards.
> >
> >One platform where it is definitely not safe is LinuxThreads when the
> >program uses sigaltstack().
> >
> >Async-signal-safety applies to functions called from the alarm signal.
>
> You are right, combining pthred_getspecific with signals doesn't work.
> Libcurl should then use some sort of per-thread alarms as described in:
> http://www.cs.umu.se/kurser/TDBC64/VT03/pthreads/pthread-primer.pdf#search=%22per-thread%20alarm%20pthread%22
>
> Since you know more about signals&pthread than I do, could you suggest
> such an implementation?

Thanks for the Pthreads primer link; it's quite a good read.

Unfortunately, per-thread alarms as described in that document are
quite fiddly to implement, and not well suited to a library such as
libcurl, which avoid using global process resources as much as
possible.

You would need to create one (and only one) "timer handling" thread
for the whole process, separate from the ordinary processing threads;
you would need to ensure that SIGALRM is blocked in all other threads
(not just those using libcurl), and you would need to reserve a signal
(SIGUSR1 is used in that document) to deliver the timers to each
thread, as well as maintaining an ordered queue of per-thread timers.

It is possible to do it without a "timer handling" thread, and without
needing a second signal - just allowing SIGALRM to be handled by any
thread, which then relays it to the thread that is at the head of the
(global) ordered queue. However I don't think it can be done portably,
as neither pthread_self() nor pthread_kill() are async-signal-safe.

And after all that, I'm not convinced it's safe anyway to interrupt
DNS resolution at an arbitrary time. How does curl (when
single-threaded) ensure that it's safe to longjmp from the SIGALRM
handler interrupting DNS resolution? What happens to file descriptors
that may have been opened by the resolver code, and resolver data
structures that might be in a temporary state?

I favour, instead of all that, just fork/exec process(es) to do DNS
resolving, communicating with it over a pipe. Kill processes that
don't reply in time. That can be easily thread-safe and portable.
It's not perfect, because fork/exec races with file descriptor
creation in other threads (an unfortunate unix API fault), so those
child processes would sometimes inherit file descriptors that you'd
rather were closed.

-- Jamie
Received on 2006-09-22