Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CURLE_WRITE_ERROR shadows CURLE_BAD_CONTENT_ENCODING #4310

Closed
PersDep opened this issue Sep 9, 2019 · 4 comments
Closed

CURLE_WRITE_ERROR shadows CURLE_BAD_CONTENT_ENCODING #4310

PersDep opened this issue Sep 9, 2019 · 4 comments
Assignees

Comments

@PersDep
Copy link

PersDep commented Sep 9, 2019

What is going on

curl_multi_info_read returns msg->data.result == CURLE_WRITE_ERROR if received Content-Encoding is not supported.

I expected the following

I expect curl_multi_info_read to return CURLE_BAD_CONTENT_ENCODING.

Code

if(!conn->data->set.http_te_skip && !k->ignorebody) {
    if(!conn->data->set.http_ce_skip && k->writer_stack)
        result = Curl_unencode_write(conn, k->writer_stack, datap, piece);
    else
        result = Curl_client_write(conn, CLIENTWRITE_BODY, datap, piece);

    if(result)
        return CHUNKE_WRITE_ERROR;
}

http_chunks.c 194-202

Curl_unencode_write returns CURLE_BAD_CONTENT_ENCODING and then it is being shadowed by CHUNKE_WRITE_ERROR which later is turning into CURLE_WRITE_ERROR.

curl/libcurl version

curl 7.65.3-DEV (x86_64-pc-linux-gnu) libcurl/7.65.3-DEV OpenSSL/1.1.1 zlib/1.2.11

operating system

Ubuntu 18.04.3 LTS

@bagder
Copy link
Member

bagder commented Sep 10, 2019

Do you have an example of how to reproduce this?

@PersDep
Copy link
Author

PersDep commented Sep 10, 2019

Do you have an example of how to reproduce this?

Here is a test based on tests/libtest/lib597.c

#include "test.h"

#include <limits.h>

#include "testutil.h"
#include "warnless.h"
#include "memdebug.h"

#define TEST_HANG_TIMEOUT 5 * 1000

int test(char *URL)
{
  CURL *easy = NULL;
  CURLM *multi = NULL;
  int res = 0;
  int running;
  int msgs_left;
  CURLMsg *msg;

  start_test_timing();

  global_init(CURL_GLOBAL_ALL);

  easy_init(easy);

  multi_init(multi);

  /* go verbose */
  easy_setopt(easy, CURLOPT_VERBOSE, 1L);

  /* specify target */
  easy_setopt(easy, CURLOPT_URL, URL);

  /* ask for unsupported encoding */
  easy_setopt(easy, CURLOPT_ACCEPT_ENCODING, "br");

  multi_add_handle(multi, easy);

  for(;;) {
    struct timeval interval;
    fd_set fdread;
    fd_set fdwrite;
    fd_set fdexcep;
    long timeout = -99;
    int maxfd = -99;

    multi_perform(multi, &running);

    abort_on_test_timeout();

    if(!running)
      break; /* done */

    FD_ZERO(&fdread);
    FD_ZERO(&fdwrite);
    FD_ZERO(&fdexcep);

    multi_fdset(multi, &fdread, &fdwrite, &fdexcep, &maxfd);

    /* At this point, maxfd is guaranteed to be greater or equal than -1. */

    multi_timeout(multi, &timeout);

    /* At this point, timeout is guaranteed to be greater or equal than
       -1. */

    if(timeout != -1L) {
      int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeout;
      interval.tv_sec = itimeout/1000;
      interval.tv_usec = (itimeout%1000)*1000;
    }
    else {
      interval.tv_sec = TEST_HANG_TIMEOUT/1000 + 1;
      interval.tv_usec = 0;
    }

    select_test(maxfd + 1, &fdread, &fdwrite, &fdexcep, &interval);

    abort_on_test_timeout();
  }

  msg = curl_multi_info_read(multi, &msgs_left);
  if(msg)
    res = msg->data.result;
  printf("RES: %d CURLE_WRITE_ERROR: %d\n", res, CURLE_WRITE_ERROR);

  multi_remove_handle(multi, easy);

test_cleanup:

  /* undocumented cleanup sequence - type UA */

  curl_multi_cleanup(multi);
  curl_easy_cleanup(easy);
  curl_global_cleanup();

  return res;
}

You can run it with URL == https://www.programcreek.com/python/example/103281/brotli.compress to see the described behavior. Also curl has to be configured with --without-brotli flag

@bagder
Copy link
Member

bagder commented Sep 10, 2019

Thank you, reproduced perfectly. Now I just need to figure out how to deal with this the best way...

@bagder bagder self-assigned this Sep 13, 2019
@bagder
Copy link
Member

bagder commented Oct 1, 2019

Here's a stand alone version to repro: debug-4310.c

bagder added a commit that referenced this issue Oct 1, 2019
Unknown content-encoding would get returned as CURLE_WRITE_ERROR if the
response is chunked-encoded.

Reported-by: Ilya Kosarev
Fixes #4310
@bagder bagder closed this as completed in f0f053f Oct 2, 2019
PersDep added a commit to tarantool/tarantool that referenced this issue Nov 6, 2019
libcurl-7.66.0 and older returns CURLE_WRITE_ERROR when a server
responds with unknown or unsupported Content-Encoding (see [1] and
[2]). This was fixed in future libcurl-7.67.0 and proper
CURLE_BAD_CONTENT_ENCODING code will be returned in this case.

We should process the code in the same way as we do for
CURLE_WRITE_ERROR.

[1]: curl/curl#4310 
[2]: curl/curl#4449

Closes #4579
Totktonada pushed a commit to tarantool/tarantool that referenced this issue Dec 23, 2019
libcurl-7.66.0 and older returns CURLE_WRITE_ERROR when a server
responds with unknown or unsupported Content-Encoding (see [1] and [2]).
This was fixed in future libcurl-7.67.0 and proper
CURLE_BAD_CONTENT_ENCODING code will be returned in this case.

We should process the code in the same way as we do for
CURLE_WRITE_ERROR.

[1]: curl/curl#4310
[2]: curl/curl#4449

Closes #4579

Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>
Totktonada pushed a commit to tarantool/tarantool that referenced this issue Dec 23, 2019
libcurl-7.66.0 and older returns CURLE_WRITE_ERROR when a server
responds with unknown or unsupported Content-Encoding (see [1] and [2]).
This was fixed in future libcurl-7.67.0 and proper
CURLE_BAD_CONTENT_ENCODING code will be returned in this case.

We should process the code in the same way as we do for
CURLE_WRITE_ERROR.

[1]: curl/curl#4310
[2]: curl/curl#4449

Closes #4579

Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>
(cherry picked from commit e343263)
Totktonada pushed a commit to tarantool/tarantool that referenced this issue Dec 23, 2019
libcurl-7.66.0 and older returns CURLE_WRITE_ERROR when a server
responds with unknown or unsupported Content-Encoding (see [1] and [2]).
This was fixed in future libcurl-7.67.0 and proper
CURLE_BAD_CONTENT_ENCODING code will be returned in this case.

We should process the code in the same way as we do for
CURLE_WRITE_ERROR.

[1]: curl/curl#4310
[2]: curl/curl#4449

Closes #4579

Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>
(cherry picked from commit e343263)
@lock lock bot locked as resolved and limited conversation to collaborators Jan 1, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Development

Successfully merging a pull request may close this issue.

2 participants