curl-library
Re: curl test coverage report
Date: Sat, 3 Mar 2007 22:33:49 +0100 (CET)
On Fri, 2 Mar 2007, Dan Fandrich wrote:
> I just tried running curl's regression test suite with gcc's test coverage
> feature to see how thorough a job the test suite is doing. The results
> below came from ggcov. Here are the enabled curl features in the version I
> tested:
Very *very* cool report Dan. Thanks for this!
> curl 7.16.2-CVS (i686-pc-linux-gnu) libcurl/7.16.2-CVS OpenSSL/0.9.7a zlib/1.2.1.2 libidn/0.5.6
> Protocols: tftp ftp telnet dict ldap http file https ftps
> Features: IDN IPv6 Largefile NTLM SSL libz
>
> SSH and C-Ares are the notable omissions. This combination of features
> allowed 320 of 328 tests to run.
--enable-debug would run a few more and use a few more lines I guess.
> What this means is that 84% of the C functions within libcurl and curl got
> exercised by the test suite.
Given that the test cases have just more or less randomly been added over
time, I'd say that is a very good number. With just some effort and attention
on your report, we should be able to up those quite a bit.
> A good chunk of the uncovered lines in curl are error handling code that's
> tricky to exercise, especially out of memory error handling code.
Running the "torture" tests with "runtests.pl -t" and adding up those lines
ought to add quite a few executed branches to the summary. But I haven't run
-t for a while so I guess we need to fix the code on a few places first to
make that actually proceed fine over all test cases.
> Also, three of the protocols in the curl version under test (telnet,
> dict and ldap) have no tests.
...and ftps.
Anyway, can you reproduce this report with a script or similar on demand? I
hope to be able to use that list as input and write up a few new test cases to
test more code, and it then it would be nice to see that the additions
actually have an effect on the coverage.
-- Commercial curl and libcurl Technical Support: http://haxx.se/curl.htmlReceived on 2007-03-03