> On Sep 20, 2015, at 1:20 AM, Ray Satiro via curl-library <curl-library_at_cool.haxx.se> wrote:
>
> On 9/18/2015 6:47 AM, doa379 wrote:
>> On 18/09/15 01:22, Aaron Meriwether wrote:
>>>
>>>> On Sep 17, 2015, at 1:06 PM, doa379 <doa379_at_gmail.com> wrote:
>>>>
>>>> I think my issue has resolved itself by putting in a delay to some of the operations in the threads.
>>>
>>> If you use delays to "fix" a bug in a threaded program, you are doing it wrong, and the bug will almost certainly resurface later to bite you.
>>>
>>> -Aaron
>>
>>
>> On 18/09/15 05:53, Ray Satiro via curl-library wrote:
>> >
>> > That doesn't sound right. I think your solution is possibly covering up
>> > a problem with the design of your program.
>>
>>
>> The program follows this example:
>>
>> http://softpixel.com/~cwright/programming/threads/threads.c.php
>
> The example has two threads and is using usleep like a yield to allow for a task switch but I don't think he adequately explains that. The order in which the output appears in your console is unpredictable, however it will usually alternate.
>
> "The output will be (mostly) alternating lines as the main() and threadFunc() threads execute and pause. Without the usleep()'s they'll not switch because we aren't doing anything that takes long enough to consume our whole time slice."
>
> He is trying to show how a scheduler will run threads by multiplexing them in time slices on a core. Scheduler says do some of this thread then time's up and scheduler says do some of that thread and so on. But the whole time slice isn't necessarily consumed and also both threads may be running at the same time in parallel truly concurrent if they're running on different cores. They may alternate because they're both calling printf and there is lock contention to obtain ownership of stdout. In other words one thread has to wait to write if the other is writing to the same. Also, usleep guarantees at a minimum suspension of n microseconds if no signal, not maximum so it could be any amount of microseconds >= n before a thread resumes.
>
> I think we are treading outside the scope of this list now but how have you followed the example in your program? My take so far is you have one thread that is performing the transfer using libcurl and receiving a continuous stream of records, and those records are then processed by other threads concurrently. I don't understand why you would fork the libcurl thread; also there seem to be some problems with forking and multi-threading. I am mostly in Windows so someone experienced in forking multi-threaded apps would be better to explain why you should or shouldn't do that.
>

Right, the softpixel page mentions several different thread-related concepts. The "usleep" in the examples is not being used to reliably resolve a race condition, but as a hack to give a higher probability that the example will exhibit interesting behavior. That page also mentions the use of mutexes, which is the proper path if you need reliable inter-thread resource-sharing. However, neither of the multithreading examples given on that page actually make use of a mutex.

If you want to use curl in the background, writing to an in-memory buffer, and to access that buffer from another thread, you will need to use a mutex to ensure that the accesses from your two different threads do not conflict. I've written a quick example to show how you can use a background worker thread and a mutex in order to safely accomplish this:

https://gist.github.com/p120ph37/906fa4af65da7948d69b

-Aaron

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2015-09-22