Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re: working around status code 429
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Ray Satiro via curl-users <curl-users_at_lists.haxx.se>
Date: Thu, 1 Feb 2024 03:58:34 -0500
On 1/31/2024 7:08 AM, Dirk Hünniger via curl-users wrote:
> sorry for the delay. I updated the output file in the file curlrun1 to
> point to tmp, which is a path that exists. I attached a 7zip file
> "files.7z" of the input file "curlrun1" and the output file "out". I
> used the following curl command:
>
> curl --retry 30 -v -w "%{http_code} %{url} %{filename_effective}\n"
> --compressed -K ./curlrun1 --parallel -L > out 2>&1
>
> When I do
>
> cat out | grep "429 h"
>
> I get the output:
>
> 429 https://de.wikibooks.org/File:Parathion-skeletal.png /tmp/626.2.html
> 429 https://de.wikibooks.org/wiki/Wikibooks:Sammlungen/File:Betain.svg
> /tmp/637.3.html
> and more.
>
> So to my understanding the are 429 errors, which are not retried,
> although I used --retry in the call to curl.
Try:
grep "Warning: Problem : HTTP error. Will retry in" out
This most recent log shows transfers are retried (you can see the
decreasing numbers in the barrage of output). Why do you think the
transfers are not retried? This may be an unusual case because the
server sets the retry delay time to 1 second using the header
`retry-after: 1` and curl complies with that directive [1], but it is
probably an impractical time.
To address this I am proposing a new command line option that will allow
ignoring the server's retry delay time. [2]
[1]: https://curl.se/docs/manpage.html#--retry
[2]: https://github.com/curl/curl/pull/12837
Date: Thu, 1 Feb 2024 03:58:34 -0500
On 1/31/2024 7:08 AM, Dirk Hünniger via curl-users wrote:
> sorry for the delay. I updated the output file in the file curlrun1 to
> point to tmp, which is a path that exists. I attached a 7zip file
> "files.7z" of the input file "curlrun1" and the output file "out". I
> used the following curl command:
>
> curl --retry 30 -v -w "%{http_code} %{url} %{filename_effective}\n"
> --compressed -K ./curlrun1 --parallel -L > out 2>&1
>
> When I do
>
> cat out | grep "429 h"
>
> I get the output:
>
> 429 https://de.wikibooks.org/File:Parathion-skeletal.png /tmp/626.2.html
> 429 https://de.wikibooks.org/wiki/Wikibooks:Sammlungen/File:Betain.svg
> /tmp/637.3.html
> and more.
>
> So to my understanding the are 429 errors, which are not retried,
> although I used --retry in the call to curl.
Try:
grep "Warning: Problem : HTTP error. Will retry in" out
This most recent log shows transfers are retried (you can see the
decreasing numbers in the barrage of output). Why do you think the
transfers are not retried? This may be an unusual case because the
server sets the retry delay time to 1 second using the header
`retry-after: 1` and curl complies with that directive [1], but it is
probably an impractical time.
To address this I am proposing a new command line option that will allow
ignoring the server's retry delay time. [2]
[1]: https://curl.se/docs/manpage.html#--retry
[2]: https://github.com/curl/curl/pull/12837
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2024-02-01