RE: Roadmap 2023 ?

From: Dmitry Karpov via curl-library <curl-library_at_lists.haxx.se>
Date: Mon, 13 Feb 2023 23:10:15 +0000

Thanks for the reply! I think that we are more or less aligned on most of the items. 😊

> I think this is a premature design idea, but I would fold this into a larger "make sure HTTP/3 transfers are as fast as possible" item once they actually are stable and made non-experimental.

Yes, I agree. This is a performance optimization, which definitely should be considered after H3 support becomes a quite stable feature.
I mentioned it in my list, because the H3 performance is also a very important aspect of adopting it in production environment, which we shouldn't forget about.
If H3 performs worse than H1/H2 then it can limit the stimuli of using it in a wide scale.

> We've touched on this before, yes. I think it would be helpful to come up with more details and write down how this would work, perhaps from a user's perspective.
> I think that could help potentially getting more people interested.

Yes, it is a good idea to outline what the "system" layer functionality will be about, and how it can help to some libcurl-based applications.
So, here is the brief description of the major points from our discussions:

Context and necessity:

There can be cases when one large "host" application using libcurl may include numerous "plugin" modules also using libcurl, which cannot be easily modified or completely closed for modifications.
In such scenarios, the "plugin" modules can use either some default transfer options or set their own, which are either not optimal in the "host" application context,
or are prohibited by some internal polices which the "host" application may impose.

For example, the "plugin" modules can use IPv4/IPv6 dual-stack where IPv6 doesn't work very well or use redirections or weak ciphers where the "host" application wants to prohibit such behavior or has some strict security rules.

To perform such control, the "host" application should have some global "system" level capabilities, which should allow it to define behavior for all libcurl transfers that occur in the "host" app.
Such capabilities are currently missing, which make it very difficult to support such scenarios.


Potential Solution (based on our thoughts in discussions):
The libcurl may provide an opt-in mechanism which will allow the "host" application to create and set up a "system" handle with libcurl.

The "system" handle (i.e. named CURLS or CURLSYS) will allow:
- Register a "pre-transfer" callback, which will allow the "host" application to check the transfer options it cares about and modify the ones which are not optimal or against "host" application polices.
- Register other callbacks like "is IPv6 enabled in the system", which will provide ways for additional "system" tuning.
   
Note: Even though the "pre-transfer" callback may handle special cases when IPv6 is not enabled in the "system", it will be called for each transfer,
whereas the "is IPv6 enabled in the system" callback mentioned above is called just once per "multi" handle.

So, for application-specific "IPv6 enabled" checks it still will be helpful if the "system" handle provides a way to setup a custom mechanism to check just that.

All multi-threading issues, which may potentially occur when multiple transfers run in multiple threads, should be handled by the "host" application which should provide a proper thread
protection in all callbacks it will register in the "system" handle.

Thanks,
Dmitry Karpov



-----Original Message-----
From: Daniel Stenberg <daniel_at_haxx.se>
Sent: Saturday, February 11, 2023 12:59 AM
To: Dmitry Karpov via curl-library <curl-library_at_lists.haxx.se>
Cc: Dmitry Karpov <dkarpov_at_roku.com>
Subject: [EXTERNAL] RE: Roadmap 2023 ?

On Fri, 10 Feb 2023, Dmitry Karpov via curl-library wrote:

> - Provide the "system" layer functionality.

We've touched on this before, yes. I think it would be helpful to come up with more details and write down how this would work, perhaps from a user's perspective. I think that could help potentially getting more people interested.

> - Provide TTL honoring for DNS cache entries for those DNS backends
> that provide the TTL information (like c-ares).

I consider that more or less just a bugfix, but yeah we should do that.
Perhaps with a lower end cap, as sometimes the set TTL is just a couple of seconds.

Firefox has (had?) an interesting implementation solution where they use the system resolver to get the name but they also fire off a separate DNS lookup (like we could do with c-ares) to get only the TTL for improved cache time information from that separate API.

> - Provide background buffering for H3/QUIC to match TCP socket background buffering.

I think this is a premature design idea, but I would fold this into a larger "make sure HTTP/3 transfers are as fast as possible" item once they actually are stable and made non-experimental.

> - Eventually provide "message" layer support for WebSockets in libcurl.

I hope to get to this within 2023.

--

  / daniel.haxx.se
  | Commercial curl support up to 24x7 is available!
  | Private help, bug fixes, support, ports, new features
  | https://curl.se/support.html

-- 
Unsubscribe: https://lists.haxx.se/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Received on 2023-02-14