Hello all,

My problem is the following:
I have a multithread application. Each threads are independant and calls libcurl functionalities to fetch resources on the net.
As i am using the curl_easy_perform, I noticed that the connexion time increase considerably with the number of thread.
Up to 5, it is ok (a few seconds).
But with 30 threads, it can take more than a minute.
Furthermore, the cpu is always at 100%.
Here is the code that will execute each thread:

before executing the threads:
        curl_global_init(CURL_GLOBAL_DEFAULT);

each thread will execute the following code:
        CURL_STATUS status;
        
        CURL *curl;
          char buf[] = "Expect:";
          struct curl_slist *headerlist=NULL;

          
          CURL_Write_UserData *my_userdata = new CURL_Write_UserData;
          my_userdata->object = this;
          my_userdata->req = request;
            CURLcode res;
                struct HttpPost *formPost=NULL;

        //Get a handle for the request
        curl = curl_easy_init();

        //Set all options

        //***************************************************
        //Common options
        //***************************************************

        //To follow the location in case of redirection
        curl_easy_setopt(curl,CURLOPT_FOLLOWLOCATION,1);

        //Setup cookies
        CURLcode code2 = curl_easy_setopt(curl,CURLOPT_COOKIEFILE,"D:\\DocumentAccessAccelerator\\cookies.txt");
        curl_easy_setopt(curl,CURLOPT_COOKIEJAR,"D:\\DocumentAccessAccelerator\\cookies.txt");
        

        //How to write the received datas
        curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, CSD_CURL_Write);
         curl_easy_setopt(curl, CURLOPT_WRITEDATA,my_userdata);
   
        //How to retrieved the headers sent by the web server
        curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION,CSD_CURL_WriteHeader);
        curl_easy_setopt(curl, CURLOPT_WRITEHEADER,request);

          /* initalize custom header list (stating that Expect: 100-continue is not
             wanted */
        headerlist = curl_slist_append(headerlist, buf);
            curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerlist);

        //***********************************
        //End of Common Options
          
           
           string l_newURL = "";
           string FormatedParam;

        if(request->get_MimeType()=="multipart/form-data")
        {
                formPost = FormatPostMultipart(request);
                    curl_easy_setopt(curl, CURLOPT_HTTPPOST, formPost);
                curl_easy_setopt(curl, CURLOPT_URL, request->get_m_URL().c_str());

        }
        else if(request->isGet()==false)
        {
                curl_easy_setopt(curl,CURLOPT_POST,1);
                FormatedParam = FormatParam(request);
                curl_easy_setopt(curl,CURLOPT_POSTFIELDS,FormatedParam.c_str());
                curl_easy_setopt(curl, CURLOPT_URL, request->get_m_URL().c_str());
        }
        else
        {
                FormatedParam = FormatParam(request);
                l_newURL = request->get_m_URL();
                if(!FormatedParam.empty())
                {
                        if(l_newURL.find("?")==-1)
                        {
                                l_newURL.append("?");
                        }
                        l_newURL.append(FormatedParam);
                }
                curl_easy_setopt(curl, CURLOPT_URL, l_newURL.c_str());

                
        }

        long l_timeout = request->get_m_timeout();
        l_timeout = l_timeout/1000;
        l_timeout = (long) (l_timeout*9/10);
        curl_easy_setopt(curl,CURLOPT_TIMEOUT,l_timeout);

        (*p_currentLog)<<start_msg(LEV_2)<<"Starting Download"<<end_msg;

        //Perform the request

        res = curl_easy_perform(curl);
        (*p_currentLog)<<start_msg(LEV_2)<<"res perform = "<<res<<end_msg;

        if(res==CURLE_OK)
        {
                status = CURL_NO_ERROR;
        }
        else
        {
                status = CURL_ERROR;
        }
        
        double returnCode;
        CURLcode code;

        
        long httpcode;

        code = curl_easy_getinfo(curl,CURLINFO_HTTP_CODE,&httpcode);

        if(code==CURLE_OK)
        {
                (*p_currentLog)<<start_msg(LEV_2)<<"HTTP return code = "<<httpcode<<end_msg;
                p_returnedHTTPCode = httpcode;
        }

        code = curl_easy_getinfo(curl,CURLINFO_TOTAL_TIME,&returnCode);
        (*p_currentLog)<<start_msg(LEV_2)<<"Total Downloading Time: "<<returnCode<<end_msg;

        (*p_currentLog)<<start_msg(LEV_2)<<"Cleaning CURL Session"<<end_msg;
    

        curl_easy_cleanup(curl);

        /* then cleanup the formpost chain */
        if(formPost!=NULL)
        {
                curl_formfree(formPost);
        }

    /* free slist */
    curl_slist_free_all (headerlist);
        delete my_userdata;

Here is my configuration:
- os win2000.

I do not use any debug! (i noticed it could be longer with debug information as it is mentioned in the doc)

The object request contains all the parameter for the request (url, parameters...)
- Am I doing something wrong?
- Is there a limitation that one of you notice when making more than x request without using the multi handle? Or is there something to pay attention to?
- my write functionsimply copy the received content in a file. Do not think my problem comes from here since it seems like it it is a connexion issue.
- does anybody know if there could be some limitations with the number of simultaneous connexion on windows?

thanks your help.

Regards,

Arnaud.

--------------------------------------------------------------------------------------------
 <<Arnaud VALLEE.vcf>>

-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Stuff, things, and much much more.
http://thinkgeek.com/sf