Re: Streaming data using Libcurl
Date: Fri, 28 Feb 2014 20:32:24 +0100
On Fri, Feb 28, 2014 at 09:02:35AM -0800, Sunny Khatri wrote:
> Thanks for the response.
> Let me clarify the situation a bit more,
> What we have is a libcurl client talking and sending data to our server (Jetty/
> SOLR). Now as soon as a chunk is posted via curl_easy_perform, SOLR tries to
> parse it and errors out in case the data was not a valid csv (not containing
> all the required expected csv fields). What we want is Server to wait for the
> next chunk which would give the remaining expected fields and then process
> that csv.
So, the server is processing data as though it's all been received before it's
actually all available? That's definitely a server bug. Fix the server instead
of trying to hack around the problem client-side!
> For example, we wish to send: 1,value1,value2,value3
> chunk1: 1,value1
> chunk2: value2,value3
> I hope this makes things more clear.
> On Thu, Feb 27, 2014 at 9:19 PM, Fabian Frank <fabian.frank.de_at_gmail.com>
> On Thu, Feb 27, 2014 at 3:43 PM, Sunny Khatri <sunny.kh03_at_gmail.com> wrote:
> I'm trying to use libcurl to stream data to the server. However, the
> problem is that as soon as server receives a chunk (cur_easy_perform),
> it tries to process that data, even though it's partial, which is
> expected given chunked encoding is used which sets the the
> content-length for that particular chunk.
> What do you mean by "tries to process the data"? Do you control the server?
> What kind of server/API are you talking to?
> Is there anything that needs to be set in the header to allow streaming
> of data. Ideally we want data to be processed at server only when we
> have send all the required data. Setting content-length is not an
> option as we don't know in advance what the total size of the data
> would be.
> Is this a binary upload? If so, you might use a PUT request.
List admin: http://cool.haxx.se/list/listinfo/curl-library
Received on 2014-02-28