cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: curl test coverage report

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Sat, 3 Mar 2007 22:33:49 +0100 (CET)

On Fri, 2 Mar 2007, Dan Fandrich wrote:

> I just tried running curl's regression test suite with gcc's test coverage
> feature to see how thorough a job the test suite is doing. The results
> below came from ggcov. Here are the enabled curl features in the version I
> tested:

Very *very* cool report Dan. Thanks for this!

> curl 7.16.2-CVS (i686-pc-linux-gnu) libcurl/7.16.2-CVS OpenSSL/0.9.7a zlib/1.2.1.2 libidn/0.5.6
> Protocols: tftp ftp telnet dict ldap http file https ftps
> Features: IDN IPv6 Largefile NTLM SSL libz
>
> SSH and C-Ares are the notable omissions. This combination of features
> allowed 320 of 328 tests to run.

--enable-debug would run a few more and use a few more lines I guess.

> What this means is that 84% of the C functions within libcurl and curl got
> exercised by the test suite.

Given that the test cases have just more or less randomly been added over
time, I'd say that is a very good number. With just some effort and attention
on your report, we should be able to up those quite a bit.

> A good chunk of the uncovered lines in curl are error handling code that's
> tricky to exercise, especially out of memory error handling code.

Running the "torture" tests with "runtests.pl -t" and adding up those lines
ought to add quite a few executed branches to the summary. But I haven't run
-t for a while so I guess we need to fix the code on a few places first to
make that actually proceed fine over all test cases.

> Also, three of the protocols in the curl version under test (telnet,
> dict and ldap) have no tests.

...and ftps.

Anyway, can you reproduce this report with a script or similar on demand? I
hope to be able to use that list as input and write up a few new test cases to
test more code, and it then it would be nice to see that the additions
actually have an effect on the coverage.

-- 
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html
Received on 2007-03-03