Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: Fwd: Adding IPFS Trustless Gateway Protocol Questions
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Daniel Stenberg via curl-library <curl-library_at_lists.haxx.se>
Date: Thu, 26 Oct 2023 16:56:42 +0200 (CEST)
On Thu, 26 Oct 2023, Hugo Valtier via curl-library wrote:
> This would make curl or libcurl capable of downloading ipfs:// content
> from any reachable IPFS node, not just a localhost trusted one.
> If anything, do we agree that doing this is a desirable improvement ?
> assuming implemented in curl next to the current ipfs:// URL rewriting
I know very little about IPFS but it certainly sounds like the right way.
Would this not rather *replace* the URL rewrite approach? I mean, isn't
verifying the content always better than not verifying? And if not, how would
a user know when to use which?
> For me the main draw of having it in libcurl is that we can provide the same
> GET semantics to consumers while having self validating incrementally
> verified requests.
Right, for application authors who want to add IPFS support next to some other
protocols that libcurl already supports that seems like a decent benefit. I
just don't see the user demand for that while at the same time it would add a
significant cost and complexity to the library - plus the fact that
"hierachically" it is not a clear-cut fit for libcurl.
> If I make my own C library which consumes libcurl either I have a different
> API (creates work for consumers) or I make my own fork but I believe me
> tracking you is work overall than if libcurl would know how to reach out to
> the IPFS validation code, it's also annoying for distributions.
To provide IPFS-over-HTTP as a library, I would probably argue that doing this
as a separate library that itself uses libcurl is the way forward. The API
could even be designed to work as a libcurl companion perhaps rather than a
straight layer on top.
> I don't understand why precisely drawing the line at over-HTTP, Trustless
> Gateway is used in production to transfer data between IPFS implementations.
One of my most important tasks in this project is to resist creeping featurism
- and everyone who has been around here a while know that we are still adding
features and changes at a rather high pace. To make sure that the line in the
sand that is drawn between what is suitable for libcurl and what is NOT
suitable for libcurl remains intact and untouched as much as possible. I think
sticking "core" transport protocols could be considered one of those things
that so far has defined if a protocol is meant for libcurl or not.
Lines in sand are by definition not distinct or always very clear, but I think
we do everyone a service when we resist changing how that line is drawn.
Of course, sometimes we end up clarifying or changing how we view the world
after discussions and deliberating and then we adapt and move on with an
updated view. But until that moment, we resist it.
> Would the new Trustless-Gateway-over-HTTP-over-Libp2p protocol be a suitable
> candidate for libcurl ?
I'm sorry but I don't understand the question. If IPFS-over-HTTP is not for
libcurl, then how is Trustless-Gateway-over-HTTP-over-Libp2p any better?
And why do HTTP-over-libp2p at all?
> AFAIT it can't just be done on top since it requires running HTTP over yamux
> + tls + tcp (or other libp2p transports).
Are you proposing that libcurl would use another library for IPFS that itself
would have a *separate* HTTP implementation and this, because you're doing
HTTP in a non-standard way? That sounds so crazy I must have misunderstood!
Date: Thu, 26 Oct 2023 16:56:42 +0200 (CEST)
On Thu, 26 Oct 2023, Hugo Valtier via curl-library wrote:
> This would make curl or libcurl capable of downloading ipfs:// content
> from any reachable IPFS node, not just a localhost trusted one.
> If anything, do we agree that doing this is a desirable improvement ?
> assuming implemented in curl next to the current ipfs:// URL rewriting
I know very little about IPFS but it certainly sounds like the right way.
Would this not rather *replace* the URL rewrite approach? I mean, isn't
verifying the content always better than not verifying? And if not, how would
a user know when to use which?
> For me the main draw of having it in libcurl is that we can provide the same
> GET semantics to consumers while having self validating incrementally
> verified requests.
Right, for application authors who want to add IPFS support next to some other
protocols that libcurl already supports that seems like a decent benefit. I
just don't see the user demand for that while at the same time it would add a
significant cost and complexity to the library - plus the fact that
"hierachically" it is not a clear-cut fit for libcurl.
> If I make my own C library which consumes libcurl either I have a different
> API (creates work for consumers) or I make my own fork but I believe me
> tracking you is work overall than if libcurl would know how to reach out to
> the IPFS validation code, it's also annoying for distributions.
To provide IPFS-over-HTTP as a library, I would probably argue that doing this
as a separate library that itself uses libcurl is the way forward. The API
could even be designed to work as a libcurl companion perhaps rather than a
straight layer on top.
> I don't understand why precisely drawing the line at over-HTTP, Trustless
> Gateway is used in production to transfer data between IPFS implementations.
One of my most important tasks in this project is to resist creeping featurism
- and everyone who has been around here a while know that we are still adding
features and changes at a rather high pace. To make sure that the line in the
sand that is drawn between what is suitable for libcurl and what is NOT
suitable for libcurl remains intact and untouched as much as possible. I think
sticking "core" transport protocols could be considered one of those things
that so far has defined if a protocol is meant for libcurl or not.
Lines in sand are by definition not distinct or always very clear, but I think
we do everyone a service when we resist changing how that line is drawn.
Of course, sometimes we end up clarifying or changing how we view the world
after discussions and deliberating and then we adapt and move on with an
updated view. But until that moment, we resist it.
> Would the new Trustless-Gateway-over-HTTP-over-Libp2p protocol be a suitable
> candidate for libcurl ?
I'm sorry but I don't understand the question. If IPFS-over-HTTP is not for
libcurl, then how is Trustless-Gateway-over-HTTP-over-Libp2p any better?
And why do HTTP-over-libp2p at all?
> AFAIT it can't just be done on top since it requires running HTTP over yamux
> + tls + tcp (or other libp2p transports).
Are you proposing that libcurl would use another library for IPFS that itself
would have a *separate* HTTP implementation and this, because you're doing
HTTP in a non-standard way? That sounds so crazy I must have misunderstood!
-- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://curl.se/support.html -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2023-10-26