curl / Mailing Lists / curl-library / Single Mail

curl-library

Re: Multiple READFUNCTION callbacks not happening

From: Dan Fandrich <dan_at_coneharvesters.com>
Date: Wed, 11 Jan 2017 22:18:42 +0100

On Thu, Jan 12, 2017 at 01:34:04AM +0530, Pushkar Kulkarni wrote:
> I am working with the URLSession implementation in open source Swift,which is
> based on libcurl. I have a question to ask. But a code snippet is not really
> feasible, so kindly bear with me. 

I'm not familiar with URLSession so my answers will be from the perspective of
the native libcurl C binding. URLSession could do something completely
different.

> An HTTP POST request is made using the CURLOPT_READFUNCTION callback, through
> which the HTTP request body is passed. In my tests, I have seen that the
> READFUNCTION callback allows writing a maximum of 16K (CURL_WRITE_MAX_SIZE)
> bytes of data to the buffer. So, for request body sizes of more than 16K, I’d
> expect this callback to happen multiple times until the READFUNCTION  returned
> a 0. Is that right? 

Correct. However, a full POST consists of sending data and receiving the
response so both CURLOPT_READFUNCTION and CURLOPT_WRITEFUNCTION would be needed
to handle both directions. But libcurl will keep calling the read callback
function until it returns end-of-file or an error occurs.

> However, for body sizes more than 16K bytes in size I see the callback
> happening only once.  Since the subsequent callbacks don’t happen, the server
> eventually times out. Hence, large requests or file uploads with sizes > 16K
> simply fail.

Is the read callback function returning the correct number of bytes it's
supplying? Is it returning too many or too few in total compared to what the
program sets with CURLOPT_INFILESIZE? It sounds like you've checked that
already, though.

> I can confirm that the first (and only) time the callback happens, the
> READFUNCTION does NOT return 0. I can also confirm that we do set the
> CURLOPT_INFILESIZE value to the expected size of the body. Though I am not sure
> if the CURLOPT_POSTFIELDSIZE option would make a difference here, I did try
> setting it to the size of the request body but that didn’t help. I have also
> fiddled with a few HTTP headers as well, but I can’t get the callback to happen
> for more than once. 
>
>
> Am I missing some config option here? Can someone help please?  

Without seeing code it's hard to say. You can look at the various example
programs to see how they do it, but sounds like you have the basics already.
There may be something specific to URLSession involved here. You could try
installing a debug callback function and seeing if there are any clues in
there.

>>> Dan
-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2017-01-11