libcurl http2 connection reuse
Date: Wed, 10 Jan 2018 13:51:40 +0530
I would like to apologise first that due to my oversight I posted the same
message on curl-users mailing list whereas I should have posted it here.
Here is the message :-
I am using libcurl's multi interface(library version *7.57*) along with
epoll for getting data in parallel from n http2 servers(assume that n = 2).
I want to give a total of say 200 ms to receive all the responses(from n
http2 servers), after which I process the responses. Find attached a file
that contains the code that uses multi_socket_action API with epoll.
Ouput of curl_version()* libcurl/7.57.0 OpenSSL/1.0.2g zlib/1.2.8
Here is how I use it :-
1) Create a multi handle and set options on it.
2) Create n easy handles
3) Set options on the easy handles
4) Add the n easy handles in the multi handle
5) Call multi_socket_action with CURL_SOCKET_TIMEOUT to start everything. I
wait a maximum of 200ms for the entire process using epoll.
6) Call curl_multi_remove_handle on each of the n easy handles.
7) *Goto Step 3*.
If all responses received within 200ms, I process them, else(atleast one of
them could not complete) we process the ones we received, i.e 200 ms is the
maximum time I want my application to wait to get response from a
designated server, after which I continue to work with responses I received.
Now the problem is that whenever a server is unable to send a response in
200ms(lets call this event a timeout), libcurl tearsdown the underlying TCP
connection established with this server, which adds the overhead of
connection establishment(which increases a lot considering that I am using
https) for a subsequent transaction with the same server.
Technically, when the *curl_multi_remove_handle* function is called for the
easy_handle the connection is in not in the state *CURLM_STATE_COMPLETED* and
is marked "*premature*"(premature = (data->mstate < CURLM_STATE_COMPLETED)
? TRUE : FALSE;) and this premature connection is then closed in the
Here is the comment before calling *Curl_disconnect* on the connection in
the function *multi_done*
if premature is TRUE, it means this connection was said to be DONE
the entire request operation is complete and thus we can't know in what
state it is for re-using, so we're forced to close it. In a perfect
we can add code that keep track of if we really must close it here or
but currently we have no such detail knowledge.
I can understand why connection teardown is needed in HTTP/1.1, but why
can't this behaviour be done away in HTTP/2 considering that HTTP/2
supports multiple streams(multiplexed) over the same connection; so
*theoretically* I can just "*discard*" the old stream(which had timed out
from my application's perspective and could contain old/invalid data) and
start a new stream for subsequent transactions over the same ESTABLISHED
connection instead of tearing down the TCP connection and bearing the
overhead of connection establishment again(which is certainly prohibitive
in my case) for a subsequent transaction with the same server. Is this
change technically difficult or not supported by the protocol? Request you
to elaborate the reasoning as I am not a networking expert. Please feel
free to correct me.
- text/x-csrc attachment: curl_http2_epoll.c