curl / Mailing Lists / curl-users / Single Mail

curl-users

Re: bug/problem: curl sending extra data with `--upload-file - --header "Content-Lenght: XXX"` ?

From: Tomas Pospisek <tpo2_at_sourcepole.ch>
Date: Tue, 3 Jan 2017 08:41:30 +0100

Am 02.01.2017 um 17:33 schrieb Dan Fandrich:
> On Mon, Jan 02, 2017 at 12:45:54PM +0100, Tomas Pospisek wrote:
>> TLDR; curl seem to be adding extra data to the payload when using
>> `--upload-file -` together with `--header "Content-Length: XXX"`.
>>
>> The below is a reduced example. In reality we need to upload a file,
>> which is very large, whose size we know, but that is not available
>> locally on disk. Therefore we pass the contents to upload to curl via
>> STDIN and we set the (known) content length header.
> [...]
>> You'll note in the trace above there's "31 61 0d 0a" preceeding the
>> "Hello, my name is...".
>
> The trace you supplied shows that the transfer is being performed with chunked
> encoding, which adds the "31 61 0d 0a" header. To quote the HTTP RFC:
>
> The Content-Length header field MUST NOT be sent
> if these two lengths are different (i.e., if a Transfer-Encoding
> header field is present).
>
> so the condition set up by the flags is invalid. Why are you setting the
> Content-Length? Just let curl handle that and the chunked transfer should work
> fine.

The problem is, that the receiving side *requires* (as specified by the
API) the content length to be given in the HTTP headers (supposedly to
allocate space for the image that it will receive in advance).

So I googled on how to disable "chunked transfer" and found this post
[1] by Daniel Stenberg.

So my command line now looks like this:

    # Do send file via stdin via curl
        $ cat /tmp/hello_world.txt | \
          curl --upload-file - \
               --request POST \
               --user secret:secret \
               --header 'Content-Type: application/octet-stream' \
               --header 'Transfer-Encoding:' \
               --header 'Content-Length: 26' \
               https://example.org/secret

which according to my preliminary tests seems to work. The "Expect:
100-continue" mechanism that Daniel is mentioning on how to switch of
doesn't pose no problem to neither the client nor the server so I left
that in.

So *thanks a lot* Dan for your good advice, and a good new year to you
and everybody else here,
*t

[1] https://curl.haxx.se/mail/lib-2007-06/0187.html
-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-users
FAQ: https://curl.haxx.se/docs/faq.html
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2017-01-03