cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: HTTP Pipelining Contributions

From: George Rizkalla <grizkalla_at_rim.com>
Date: Wed, 25 Jul 2012 16:45:18 +0000

On 7/24/12 6:13 PM, Daniel Stenberg wrote:
>
>Awesome. You are most welcome here and I'm sure I'm not the only one who
>will
>appreciate this!

Thanks! We're certainly looking forward to the opportunity to work on
this :).

>
>> 1. MAX SOCKETS CHANGES
>
>I know this has been requested by others in the past - exactly to prevent
>the
>application from having to do the queing. I suspect you want to do this
>by not
>altering the API and simply just not start new handles if there aren't
>any
>available connections ?

Exactly.

>
>> 2. HTTP PIPELINING CHANGES
>>
>> It is possible to determine whether a socket should be penalized by
>>either:
>>
>> a) A Content-Length header specifying a content-length greater than the
>> proposed new curl option CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE
>> b) A chunk size larger than the proposed new curl option
>> CURLMOPT_CHUNK_PENALTY_SIZE where a chunked transfer-encoding is used
>>
>This sounds fine too. I'm a bit doubtful about the ncesssity for the
>CURLMOPT_*_PENALTY_SIZE options though. They seem like options that
>simply
>nobody would touch if we would claim the defaults are somewhat sensible.
>Or do
>you see any use-case in your world where you will actually change these
>according to some scheme or acquired knowledge in the application?

The issue I can foresee in our domain is that depending on the underlying
radio access technology, "sensible" might vary. For example, a 512 KB
limit on an archaic GPRS network might make less sense on LTE. And any
value considered legitimate over slower wireless technologies might make
little sense on a high-speed wired network.

>
>> -Pipeline-able Sockets
>
>> In addition to the obvious checks for the use of an HTTP/1.1 server,
>>before
>> assuming that pipelining is possible on a particular socket, libcurl
>>could
>> check for older/incompatible IIS servers before deciding to pipeline.
>
>Ouch. How would that be done? Scanning for certain magic keywords in
>response
>Server: headers?

Right. I agree with the "Ouch" sentiment-- I wish it didn't seem like
there was a need to do this :). That said, with the need for backwards
compatibility with the large volume of buggy/old servers on the internet,
I'm not aware of a better way to reduce the probability of failure when
dealing with unfamiliar hosts... On the bright side, this would only need
to be done on the first HTTP response on a socket.

>
>> -Basic Socket Selection Algorithm
>[snip]
>
>Wow. Sounds like you've been giving this some thoughts and I am
>impressed. It
>certainly sounds like a working way that will *greatly* enhance the way
>libcurl does pipelining.

Thanks for all of the feedback-- it's greatly appreciated!

George

---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2012-07-25