cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: libcurl multithreading

From: Török Edvin <edwintorok_at_gmail.com>
Date: Fri, 22 Sep 2006 21:21:07 +0300

On 9/22/06, Jamie Lokier <jamie_at_shareable.org> wrote:
> Unfortunately, per-thread alarms as described in that document are
> quite fiddly to implement, and not well suited to a library such as
> libcurl, which avoid using global process resources as much as
> possible.

You don't have to implement exactly as described inthere,
particularly, you are not required to use signals at all.
Just have one global thread, which does the following:
- you can queue timeouts to it, lets say
queue_timeout_for_current_thread(timeout)
- when one of the timeouts expire, it either pthread_cancel, or
pthread_kill that thread
- waiting for a timeout doesn't need to use SIGUSR1, you can use
pthread_cond_timedwait, sleep

So you can:
pthread_cond_timedwait for a condition (call it queue_cond), with the
timeout being the timeout of the thread that should timeout the
soonest.
Whenever somebody queue a new timeout, it also
pthread_cond_signal(queue_cond,...).

>
> You would need to create one (and only one) "timer handling" thread
> for the whole process, separate from the ordinary processing threads;

Yes

> you would need to ensure that SIGALRM is blocked in all other threads
> (not just those using libcurl), and you would need to reserve a signal
> (SIGUSR1 is used in that document) to deliver the timers to each
> thread, as well as maintaining an ordered queue of per-thread timers.

See my suggestion above, you don't need to use the 2 signals approach.

>
> It is possible to do it without a "timer handling" thread, and without
> needing a second signal - just allowing SIGALRM to be handled by any
> thread, which then relays it to the thread that is at the head of the
> (global) ordered queue. However I don't think it can be done portably,
> as neither pthread_self() nor pthread_kill() are async-signal-safe.

use pthread_cond_timedwait(), that is use wait instead of signals.

>
> And after all that, I'm not convinced it's safe anyway to interrupt
> DNS resolution at an arbitrary time. How does curl (when
> single-threaded) ensure that it's safe to longjmp from the SIGALRM
> handler interrupting DNS resolution? What happens to file descriptors
> that may have been opened by the resolver code, and resolver data
> structures that might be in a temporary state?

Good question. Would it be safe to kill a resolver thread?

>
> I favour, instead of all that, just fork/exec process(es) to do DNS
> resolving, communicating with it over a pipe. Kill processes that
> don't reply in time. That can be easily thread-safe and portable.

If pthread is not available, yes a better solution than longjmp anyway.

> It's not perfect, because fork/exec races with file descriptor
> creation in other threads (an unfortunate unix API fault), so those
> child processes would sometimes inherit file descriptors that you'd
> rather were closed.

After forking, just close all file descriptors, other than the file
descriptor used for communcation with the main process. Or use
clone/unshare if you are under Linux.

Edwin
Received on 2006-09-22