cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Multithreaded use of libcurl

From: Ray Satiro via curl-library <curl-library_at_cool.haxx.se>
Date: Sun, 20 Sep 2015 03:20:29 -0400

On 9/18/2015 6:47 AM, doa379 wrote:
> On 18/09/15 01:22, Aaron Meriwether wrote:
>>
>>> On Sep 17, 2015, at 1:06 PM, doa379 <doa379_at_gmail.com> wrote:
>>>
>>> I think my issue has resolved itself by putting in a delay to some
>>> of the operations in the threads.
>>
>> If you use delays to "fix" a bug in a threaded program, you are doing
>> it wrong, and the bug will almost certainly resurface later to bite you.
>>
>> -Aaron
>> -------------------------------------------------------------------
>> List admin: http://cool.haxx.se/list/listinfo/curl-library
>> Etiquette: http://curl.haxx.se/mail/etiquette.html
>>
>
>
> On 18/09/15 05:53, Ray Satiro via curl-library wrote:
> >
> > That doesn't sound right. I think your solution is possibly covering up
> > a problem with the design of your program.
> > -------------------------------------------------------------------
> > List admin: http://cool.haxx.se/list/listinfo/curl-library
> > Etiquette: http://curl.haxx.se/mail/etiquette.html
>
>
> The program follows this example:
>
> http://softpixel.com/~cwright/programming/threads/threads.c.php

The example has two threads and is using usleep like a yield to allow
for a task switch but I don't think he adequately explains that. The
order in which the output appears in your console is unpredictable,
however it will usually alternate.

"The output will be (mostly) alternating lines as the main() and
threadFunc() threads execute and pause. Without the usleep()'s they'll
not switch because we aren't doing anything that takes long enough to
consume our whole time slice."

He is trying to show how a scheduler will run threads by multiplexing
them in time slices on a core. Scheduler says do some of this thread
then time's up and scheduler says do some of that thread and so on. But
the whole time slice isn't necessarily consumed and also both threads
may be running at the same time in parallel truly concurrent if they're
running on different cores. They may alternate because they're both
calling printf and there is lock contention to obtain ownership of
stdout. In other words one thread has to wait to write if the other is
writing to the same. Also, usleep guarantees at a minimum suspension of
n microseconds if no signal, not maximum so it could be any amount of
microseconds >= n before a thread resumes.

I think we are treading outside the scope of this list now but how have
you followed the example in your program? My take so far is you have one
thread that is performing the transfer using libcurl and receiving a
continuous stream of records, and those records are then processed by
other threads concurrently. I don't understand why you would fork the
libcurl thread; also there seem to be some problems with forking and
multi-threading. I am mostly in Windows so someone experienced in
forking multi-threaded apps would be better to explain why you should or
shouldn't do that.

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2015-09-20