It looks like there is some confusion regarding the thread pool. 1. There is a queue of jobs (structure having a pointer to a function and values for calling the function) 2. A few worker threads are running, each of which remove one job from the queue and call the job's function. 3. One thread which periodically calls epoll_wait and processes the epoll_event by just pushing the job to the job queue. There is no call to curl_multi_socket_action here. Call to curl_multi_socket_action is made when the job is picked up by one of the worker threads. > > 4. I add this job to the IO queue. > > > 5. When the job is executed again, call curl_multi_socket_action with the > > proper socket descriptor and ev_bitmask. > > I don't understand this statement. You already added the handle, what you mean > with "when it is executed again"? By "executing a job", I mean when the job is removed from the job queue and its function is called. > > curl_multi_socket_action() is of course called by you whenever there is > action to do on a specific socket that you monitor or when you deem that the > timeout has triggered. Right? Yes, but the call does not take place until the worker thread executes the job. > > > 6. If the HTTP transfer is not completed, add the job back to the IO queue. > > I don't get this either. You already added the handle, surely you'll just keep > transferring until curl_multi_info_read() tells you that the particular > transfer is complete? Yes, I will. But the entire process (of calling curl_multi_socket_action and listening for required events) is delayed because of jobs being queued up. > > > 1. I am using HTTPS to connect to the server. So every time I add the easy > > handle, steps 5 and 6 are executed multiple times. This is fine if the > > number of "jobs" is less, but as they increase, I get a > > CURLE_OPERATION_TIMEDOUT. > > I don't understand this description either. I think you should consider > providing us with a full example source code showing what you do and ideally > one that repeats the problem you have. I think the description I have provided should make it clearer. I will work on writing a sample code and will mail it by tomorrow. > > > I am guessing that 30s is not enough for the SSL certificate verification. > > If that is really the case, it sounds like exceptionally slow machines > involved. Modern machines do SSL handshakes counted in milliseconds. Again, there is a confusion here. The 30s is inclusive of the delay in queuing the job, waiting for an IO event (which IS a few ms), pushing to job queue and waiting for a free worker thread to execute the job (which then calls curl_multi_socket_action) > > > Is there anyway for me to force libcurl to finish the send operation? > > What exactly would that mean? libcurl will always do as much as it can without > blocking, then wait until it "gets permission" to try again and then continue > from there. Would would a "force" operation even be able to do? The operation I am interested in requires to wait for the response, which could take a couple of minutes before it arrives. Hence, I am looking for an interface where I can tell curl to do send() in blocking manner, and recv() in a non-blocking manner. I am not sure if this is even technically feasible. > > > 2. In step 6, if I use curl_easy_perform to do some HTTP transfers on the > > same server, before going to step 1 again. A new connection is established > > to the server. Is that expected? > > Not if it can re-use an idle existing connection, no. Please find attached sample to reproduce this.

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html