cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: CURL handle re-use with different query parameters

From: Ray Satiro via curl-library <curl-library_at_cool.haxx.se>
Date: Mon, 13 Jul 2015 23:53:56 -0400

On 7/13/2015 5:56 AM, Rajalakshmi Iyer wrote:
> In my application, there are several HTTP GET requests to the same
> external server but with different query parameters in each request.
>
> As a result, for every request, I need to do a curl_easy_setopt for
> the CURLOPT_URL.
>
> Would having a single curl_easy_setopt for CURLOPT_URL and passing the
> query params in the POST body (assuming the external server allows for
> this) for every request provide any performance improvement?

Whether GET is faster than POST to send the queries depends on your
server. I doubt there's any significant difference in libcurl performance.

Significant improvements in performance can be seen though by utilizing
connection caching, aka persistent connections, which you can do via
easy handle reuse or a multi handle.

curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_URL, "http://www.example.com/test?1");
res = curl_easy_perform(curl);
curl_easy_setopt(curl, CURLOPT_URL, "http://www.example.com/test?2");
res = curl_easy_perform(curl);
curl_easy_setopt(curl, CURLOPT_URL, "http://www.example.com/test?3");
res = curl_easy_perform(curl);

This accesses each of the urls sequentially over the same connection
(assuming server supports it). POSTFIELDS would be similar, just set
that each time and the URL once before the initial perform.

multi is a bit different. it has a connection cache shared among the
owned easy handles but is concurrent. let's say you had 20 easy handles
so you're adding http://www.example.com/?test{1..20} to multi then
perform, what will happen is since none are available it's going to open
all 20 connections concurrently by default unless you have set a limit [1].

Having several persistent connections to your server will likely be
beneficial given your number of requests (I'm assuming this is the same
project described in your prior e-mail threads). If you're going the
multi route I'd set CURLMOPT_MAX_HOST_CONNECTIONS to a considerate
number like 6 unless the administrator is ok with more.

[1]: http://curl.haxx.se/libcurl/c/CURLMOPT_MAX_HOST_CONNECTIONS.html

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2015-07-14