curl / Mailing Lists / curl-library / Single Mail


Re: curl handles and multi thread

From: surya chandrika via curl-library <>
Date: Tue, 21 Nov 2017 13:39:48 +0530

Hi All,

Actually i have multiple message queues say for eg 50 queues(shared bw
threads) with messages that needs to be delivered without affecting the
sequence of messages in a particular queue.

Multiple threads are supposed to serve this purpose.
Each thread keeps polling this message queue ,which ever is open,thread
takes a message and deliver them one by one.

in current implementation its a sync request so if the destination host is
down, this thread has a waiting period, so in order to make them faster,
we planning to switch to async where thread_a (from a pool of threads)
perform multi_perform when a message queue is available and puts them to a
pending messages queue. which is then processed by other thread_b(from a
pool of threads) once ready.
This thread_a will now be open to poll the other message queues. so handles
prcoessed by multiple threads would be different , so i assume threads
should not be blocking except for the select()/timeout in thread_b.

Please share your thoughts for the same.


On Sat, Nov 11, 2017 at 3:52 AM, Daniel Stenberg <> wrote:

> On Wed, 8 Nov 2017, surya chandrika via curl-library wrote:
> You must never share the same handle in multiple threads. You can pass the
>> handles around among threads, but you must never use a single handle from
>> more than one thread at any given time.
>> Usecase: (cpp)
> 1. Get a multi handle and post an async req
>> 2. If still_running = 1 push them into a global map
> Them?
> 3. another thread which actively parse this map to select() the handles.
>> (one thread dedicated for selection for pending request)
> Why another thread? The thread in (2) can't do anything anyway as long as
> (3) uses the same handle.
> 4. If an activity is detected another thread from threadpool perform multi
>> perform.
> Yet another? It's still the same multi handle,so it can't do anything with
> the handle unless the other threads using the handle all hold off.
> So using a proper locking mechanism i understand that i should be able to
>> perform the above without any exception.
> Sure, but the question is then left why you feel you need to use so many
> threads...
> --
> /

Received on 2017-11-21