cURL / Mailing Lists / curl-library / Single Mail

curl-library

RE: curl_easy vs curl_multi - performance

From: Paolo Piacentini <paolopiace_at_hotmail.com>
Date: Mon, 13 Jun 2011 10:51:02 -0700

I thought with that subject the posting was going in the thread I initiated on June 10th with the same subject, which it refers to.
Sorry about that.

-----------------------------------------------------------------------------------------------------------------
From: paolopiace_at_hotmail.com
To: curl-library_at_cool.haxx.se
Subject: curl_easy vs curl_multi - performance
Date: Mon, 13 Jun 2011 10:41:49 -0700

I dug more into the issue and here is what I found. Here below is the relevant code.

It seems that most of the excess time taken by this implementation with curl_multi is due to the wait(seconds=0.1) call in the 'do' loop that runs until curl_multi_perform does not return something different than CURLM_CALL_MULTI_PERFORM.

With wait(seconds=0.1) everything works very fine but, as explained, the download process takes a very long time.

So, why not setting seconds< 0.1 in that wait(.)?

Because any seconds< 0.1 takes *exactly* curl_multi_fdset in segmentation-fault ALTHOUGH curl_multi_perform has returned zero.

Any hint/feedback/comment would be extremely appreciated.

P.S. Sorry about this: due to non-disclosure reasons it's nearly impossible to set up significant test cases to submit.

==============================================================

void wait(float seconds)
{
if (seconds > 1) seconds = 1.0;

struct timeval tv = { 0, (int)(seconds*1000000) }; // microsec

select( 0, 0, 0, 0, &tv ); // delay
}

==============================================================

                fd_set fdread;

                struct timeval timeout;

                long int curlTimeout;

                float seconds;

                int count, maxFD, selectRC = 1, stillRunning = 1;

                while (stillRunning) { // Adapted from multi-app.c

                        if (selectRC >=0) {

                                count = 0; // 'count' is for debugging only.

                                do {
                                        wait(seconds=0.1);
                                        multiResult = curl_multi_perform(multiSession, &stillRunning);
                                        // cout << ++count << " =count (debug)\n";
                                }
                                while (multiResult == CURLM_CALL_MULTI_PERFORM);
                        }

                        multiResult = curl_multi_timeout(multiSession, &curlTimeout);

                        timeout.tv_sec = 1;
                        timeout.tv_usec = 0;
                        //
                        if (curlTimeout >= 0) {
                                timeout.tv_sec = curlTimeout / 1000;
                                if (timeout.tv_sec > 10) // setting max timeout.
                                        timeout.tv_sec = 10;
                                else
                                if (timeout.tv_sec == 0)
                                        timeout.tv_usec = curlTimeout * 1000;
                        }

                        FD_ZERO(&fdread);

                        multiResult = curl_multi_fdset(multiSession, &fdread, NULL, NULL, &maxFD);

                        selectRC = select(maxFD+1, &fdread, NULL, NULL, &timeout);
                }

==============================================================

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-06-13