Buy commercial curl support from WolfSSL. We help you work
out your issues, debug your libcurl applications, use the API, port to new
platforms, add new features and more. With a team lead by the curl founder
himself.
Re-running curl tests
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: Jacob Hoffman-Andrews via curl-library <curl-library_at_lists.haxx.se>
Date: Tue, 24 Oct 2023 10:56:06 -0700
Hi all,
I often have trouble efficiently debugging curl tests, and I wanted to ask
if I'm doing it wrong, and/or if there are improvements possible.
My workflow is like this: If I notice a set of tests are failing, I will
usually run, e.g.:
make; cd tests; ./runtests.pl 719
One issue I run into is that `make` must be run in the curl repository
root, while `runtests.pl` must be run in the `tests` directory. Is it
possible to make `runtests.pl` agnostic as to the current directory from
which it is run? This would save some overhead changing directory back and
forth.
Another issue is that if tests are running long (for instance, if I ran
`make test` and then changed my mind), hitting Ctrl-C takes a very long
time to shut down the test runner. At least 10 seconds, and often much
more. Sometimes the Ctrl-C doesn't register with the test runner at all.
Lastly, when tests fail in CI, it can be hard to find the cause. For
instance, in this test run (
https://github.com/curl/curl/actions/runs/6628686909/job/18006346659), I
saw that test 719 failed, so I did Ctrl-F 719 to find it, with no results.
Later I realized that GitHub cuts off super long log output, so my browser
did not have a copy of the lines related to test case 719. Using the
"Search logs" text box in GitHub's log viewer did turn up the relevant log
lines. A couple possible fixes here: When tests fail we could emit a final
log line that says to use the search box rather than the browser's built-in
search. Also, one of the major culprits was the (failing) test1474, which
emits ~6000 lines of mostly "xxxxxx". Perhaps we could suppress or
summarize output for certain tests that have super long output?
Thanks,
Jacob
Date: Tue, 24 Oct 2023 10:56:06 -0700
Hi all,
I often have trouble efficiently debugging curl tests, and I wanted to ask
if I'm doing it wrong, and/or if there are improvements possible.
My workflow is like this: If I notice a set of tests are failing, I will
usually run, e.g.:
make; cd tests; ./runtests.pl 719
One issue I run into is that `make` must be run in the curl repository
root, while `runtests.pl` must be run in the `tests` directory. Is it
possible to make `runtests.pl` agnostic as to the current directory from
which it is run? This would save some overhead changing directory back and
forth.
Another issue is that if tests are running long (for instance, if I ran
`make test` and then changed my mind), hitting Ctrl-C takes a very long
time to shut down the test runner. At least 10 seconds, and often much
more. Sometimes the Ctrl-C doesn't register with the test runner at all.
Lastly, when tests fail in CI, it can be hard to find the cause. For
instance, in this test run (
https://github.com/curl/curl/actions/runs/6628686909/job/18006346659), I
saw that test 719 failed, so I did Ctrl-F 719 to find it, with no results.
Later I realized that GitHub cuts off super long log output, so my browser
did not have a copy of the lines related to test case 719. Using the
"Search logs" text box in GitHub's log viewer did turn up the relevant log
lines. A couple possible fixes here: When tests fail we could emit a final
log line that says to use the search box rather than the browser's built-in
search. Also, one of the major culprits was the (failing) test1474, which
emits ~6000 lines of mostly "xxxxxx". Perhaps we could suppress or
summarize output for certain tests that have super long output?
Thanks,
Jacob
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2023-10-24