curl / Mailing Lists / curl-library / Single Mail

curl-library

Pipelining is a pain, can we ditch it now?

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Sat, 30 Jun 2018 12:16:36 +0200 (CEST)

Hi all,

We got a fresh new HTTP pipelining bug submitted today [1]. We already have
other pipelining bugs listed in the KNOWN_BUGS document [2].

I'm pretty sure there are more pipelining quirks that just aren't
documented/stored anywhere.

HTTP Pipelining is badly supported by curl in the sense that we have bugs and
it is a fragile feature without enough tests. Also, when something turns out
to have problems it is really tricky to debug due to the timing sensitivy so
very often enabling debug outputs or similar completely changes the nature of
the behavior and things are not reproducing anymore!

HTTP Pipelining was never enabled by default by the large desktop browsers due
to all the issues with it. Both Firefox and Chrome have also dropped
Pipelining support entirely since a long time back now. We are in fact over
time becoming more and more lonely in supporting pipelining.

The bad state of HTTP pipelining was a primary driving factor behind HTTP/2
and its multiplexing feature. HTTP/2 multiplexing is truly and really
"pipelining done right". It is way more solid, practical and solves the use
case in a better way with better performance and fewer downsides and problems.

In 2018, pipelining *should* be abandoned and HTTP/2 should be used instead.

How many users do we have that use Pipelining?

In the annual curl survey we ask users and a shockingly large amount of users
claim they use pipelining. Over 20% in the latest poll. I've never trusted
this number though, since we would have way more bug reports and work done on
the code if it truly was used that much.

What would a convincing argument for keeping it?

I don't know. I'm sure there will be some amount of users who say they use it,
it works for them and they can't switch to HTTP/2 for various reasons (often
because they don't control the other end).

How important pipelining is for these users? If we are going to keep
pipelining because we deem these users' use cases are important enough, how
are we going to proceed to shape up the support/quality? I'm afraid we don't
see a lot of willing developers joining the ranks to fix the problems.

I certainly don't have all the answers. Thoughts anyone?

[1] = https://github.com/curl/curl/issues/2701
[2] = https://curl.haxx.se/docs/knownbugs.html#Disabling_HTTP_Pipelining

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html
Received on 2018-06-30