cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Recommended architecture for streaming over multiple sockets

From: Boutin Maël <mael.boutin_at_gmail.com>
Date: Thu, 10 Mar 2016 15:43:56 +0100

Hi,

On Wed, 9 Mar 2016, Boutin Maël wrote:

Ok for curl_multi_info_read. I changed my code as follow :
>
>
> curlret = curl_multi_perform(m_pstCurlMultiHandle, &iRunningFDs);
>> if (!iRunningFDs)
>> {
>> usleep(10000);
>> }
>
>
Why do you sleep when there's no more transfers running?

Because there is another thread that may add new connection. Then in the
> sending thread i add a new CURL handle in the multihandle to manage this
> connection. But right maybe i should not call multi_perform if there are no
> connection to be managed

       /* wait for activity, timeout or "nothing" */
>
> curlret = curl_multi_wait(m_pstCurlMultiHandle, NULL, 0, 10000,
>> &iWaitFDs);
>> if (iWaitFDs)
>> {
>> struct CURLMsg *m;
>> do {
>> int msgq = 0;
>> m = curl_multi_info_read(m_pstCurlMultiHandle, &msgq);
>
>
Why do you check the queue only when curl_multi_wait() told you there's
> socket acivity? Transfers can end without that (due to timeouts etc)!

 Hmm then maybe i don't get what is a "socket activity". For me if the
> transfer ended there is an activity ?

       However this does not work, as multi_wait will here "wait" for the
> duration.
>

If there's no socket activity, sure.

Same as previously. What is considered a socket activity, this generic term
> is way too broad for me :)

> I dont want this behaviour because the data to be sent are coming
> from another thread and i want to send them as soon as they are available.
>

Make it wait for another socket/pipe that you signal on from that other
> thread when data arrives? Or perhaps if you want it simpler: make the max
> waiting time fairly short and then check for data and then loop.

Ok this is what i had in mind.
> Again thank you for the time you spend on this. This really helps me a lot.
> Regards,

Me again...

Based on your advices i changed my code. It seems to work but do you see
any flaws there ?

1. Call multi_perform
2. If still running FDs is >= previous running FDs: call multi_wait if
there are no more data to be sent
3. Check the message queue to get ended transfers and reinitiate connection
if it ended due to an error (by removing the curl handle from multi_handle
then add a new one).

Regards

2016-03-10 10:00 GMT+01:00 Boutin Maël <mael.boutin_at_gmail.com>:

> Hi,
>
> On Wed, 9 Mar 2016, Boutin Maël wrote:
>
> Ok for curl_multi_info_read. I changed my code as follow :
>>
>>
>> curlret = curl_multi_perform(m_pstCurlMultiHandle, &iRunningFDs);
>>> if (!iRunningFDs)
>>> {
>>> usleep(10000);
>>> }
>>
>>
> Why do you sleep when there's no more transfers running?
>
>
> Because there is another thread that may add new connection. Then in the
> sending thread i add a new CURL handle in the multihandle to manage this
> connection. But right maybe i should not call multi_perform if there are no
> connection to be managed
>
> /* wait for activity, timeout or "nothing" */
>>
>> curlret = curl_multi_wait(m_pstCurlMultiHandle, NULL, 0, 10000,
>>> &iWaitFDs);
>>> if (iWaitFDs)
>>> {
>>> struct CURLMsg *m;
>>> do {
>>> int msgq = 0;
>>> m = curl_multi_info_read(m_pstCurlMultiHandle, &msgq);
>>
>>
> Why do you check the queue only when curl_multi_wait() told you there's
>> socket acivity? Transfers can end without that (due to timeouts etc)!
>
>
> Hmm then maybe i don't get what is a "socket activity". For me if the
> transfer ended there is an activity ?
>
>
>
> However this does not work, as multi_wait will here "wait" for the
>> duration.
>>
>
> If there's no socket activity, sure.
>
>
> Same as previously. What is considered a socket activity, this generic
> term is way too broad for me :)
>
>
>
>> I dont want this behaviour because the data to be sent are coming
>> from another thread and i want to send them as soon as they are available.
>>
>
> Make it wait for another socket/pipe that you signal on from that other
>> thread when data arrives? Or perhaps if you want it simpler: make the max
>> waiting time fairly short and then check for data and then loop.
>
>
> Ok this is what i had in mind.
>
> Again thank you for the time you spend on this. This really helps me a lot.
>
> Regards,
>
> 2016-03-09 23:31 GMT+01:00 Daniel Stenberg <daniel_at_haxx.se>:
>
>> On Wed, 9 Mar 2016, Boutin Maël wrote:
>>
>> Ok for curl_multi_info_read. I changed my code as follow :
>>>
>>> curlret = curl_multi_perform(m_pstCurlMultiHandle, &iRunningFDs);
>>> if (!iRunningFDs)
>>> {
>>> usleep(10000);
>>> }
>>>
>>
>> Why do you sleep when there's no more transfers running?
>>
>> /* wait for activity, timeout or "nothing" */
>>> curlret = curl_multi_wait(m_pstCurlMultiHandle, NULL, 0, 10000,
>>> &iWaitFDs);
>>> if (iWaitFDs)
>>> {
>>> struct CURLMsg *m;
>>> do {
>>> int msgq = 0;
>>> m = curl_multi_info_read(m_pstCurlMultiHandle, &msgq);
>>>
>>
>> Why do you check the queue only when curl_multi_wait() told you there's
>> socket acivity? Transfers can end without that (due to timeouts etc)!
>>
>> However this does not work, as multi_wait will here "wait" for the
>>> duration.
>>>
>>
>> If there's no socket activity, sure.
>>
>> I dont want this behaviour because the data to be sent are coming from
>>> another thread and i want to send them as soon as they are available.
>>>
>>
>> Make it wait for another socket/pipe that you signal on from that other
>> thread when data arrives? Or perhaps if you want it simpler: make the max
>> waiting time fairly short and then check for data and then loop.
>>
>> --
>>
>> / daniel.haxx.se
>> -------------------------------------------------------------------
>> List admin: https://cool.haxx.se/list/listinfo/curl-library
>> Etiquette: https://curl.haxx.se/mail/etiquette.html
>>
>
>
>
> --
> Maël BOUTIN
>
>

-- 
Maël BOUTIN

-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2016-03-10