On Wed, 16 Jul 2014, David Meyer wrote:

> Profiling shows that curl_multi_perform() is spending all its time in
> Curl_multi_process_pending_handles(). This function is called every time a
> request completes, and it iterates over every running request.

Ugh, yes. That's not very nice. A linear scan over all attached easy
handles...

> I am able to completely eliminate the performance problem by commenting out
> the body of Curl_multi_process_pending_handles(). It appears this code is
> only needed when CURLMOPT_MAX_TOTAL_CONNECTIONS is set.

The functionality is for pipelining so it should only be necessary if
pipelining is enabled on the multi handle.

Are you using pipelning?

> Would it make sense to store the list of pending handles as a separate
> linked list, to avoid iterating through every easy_handle?

Yes!

It should actually do both I think: only scan for such a handle if pipelining
is enabled, and use a separate list of pending handles so that it never has to
scan through all handles for this.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html