cURL / Mailing Lists / curl-library / Single Mail


Re: questions about multi API

From: Daniel Stenberg <>
Date: Wed, 8 Jul 2009 10:51:35 +0200 (CEST)

On Tue, 7 Jul 2009, Mohun Biswas wrote:

> I'd say the meta-problem is precisely due to the deliberately vague
> phrasing. I understand the value of leaving certain things unspecified, but
> there's such a thing as being too imprecise.

Oh indeed. The problem is to find the right ballance. To give enough info to
let the user understand exactly what to expect and how to use it, but still
not give info than isn't completely backed up or is subject to change.

> Unfortunately, the vague phrasing means we have to know what sockets are
> open along with their current state and semantics.

I disagree with that, but I realize I'm so biased here so it's not really
relevant how I interpret the user docs.

> For instance, even a non-expert like me knows that in order to transfer a
> file from here to there we need two file descriptors: one from which to read
> the file and one to write to. The libcurl-multi document makes no
> distinction between the two, which I guess is because you assume we
> understand that one is a socket which can block and the other is likely to
> be a file which will not.

The libcurl docs makes no distinction because it has no knowledge about one of
them - the file reading one - and you never tell libcurl anything about it.
Thus libcurl doesn't care about it and the docs doesn't mention it.

How do you suggest we would?

> The libcurl-multi doc says things like "libcurl will then transfer data if
> there is anything available to transfer". Since I'm doing uploads I read
> this, as I think any native speaker of English would, to mean "if there's
> anything ready *on the input side*". I'm guessing you really mean "libcurl
> will then transfer data if any socket is ready". Or maybe even "... if doing
> so would not cause the program to sleep". Or something - since I don't
> actually _know_ what it does, it's hard to suggest language.

Upload is callback based (too). libcurl calls your callback function to get
data when it already knows that the server socket is ready to receive. It
doesn't know any file descriptor or similar for reading that data when
uploading. Your app provides the data when libcurl asks for it.

> Next, wrt CURLM_CALL_MULTI_PERFORM you might want to be more clear in saying
> "you probably don't want to do anything which would cause your program to
> block while curl_multi_perform still has data ready to transfer". You state
> it in terms of select() but there are more ways to block and that - you
> could wait for a child, for instance. Again, this assumes I now understand
> it.

Perhaps, but we can't cover it all in the docs. And other blocking ways will
just delay the progress, while using libcurl's own API to wait for further
actions before the already reported actions haven't yet been dealt with feels
much more like an actual error to me.

> Here's another thing I don't know about sockets: is there any correlation
> between a socket being "ready" and the speed of the physical connection?

Yes, but hard to figure out. In general it'll become ready more often if the
network speed is high so that it empties the internal send buffers faster.

> What I'm afraid of is that a socket on a slow connection will continually
> signal ready.

It won't. libcurl will send() data when the socket is ready to receive data
and then the TCP stack will send that buffer to the peer and the socket will
not be "ready" again until the stack is again ready to get a send().

> If I need to send a large file to it, it seems to me I could still be
> tied up for a very long time sending data there because the socket might
> remain ready even though the connection is slow.

No. There's also a limited buffer in the TCP stack that will prevent libcurl
from passing on very large amounts and it'll instead get EWOUDBLOCK (or
similar) returned and then go back to waiting for the socket to be "ready" to
receive more data.

> But maybe I misunderstand and the socket will become unready until its
> buffers are drained? This is the sort of low-level detail that I think
> should either be handled internally by an API or spelled out in its
> documentation.

To me that's the sort of lowlevel socket behaviors that are just universally
known! ;-)

But more seriously, it is hard to document this as then we're stepping into
documenting how sockets work and it is hard to know where to draw the line
then. I'm not saying I know where it would be drawn best.

Received on 2009-07-08