curl / Mailing Lists / curl-library / Single Mail

curl-library

code coverage and its usefulness

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Fri, 2 Jun 2017 10:15:35 +0200 (CEST)

Hi,

Our test suite is lovely and fantastic, but how much code does it test?

We've done occasional code coverage runs in the past and those have showned
some numeric percentages that weren't too shabby.

I think the time has come to up our game and get this done on a regular basis
so that we can get a proper sense for what code we never test (and hence is a
great idea to write tests for).

My first idea was to use the coveralls.io service for this, since I've seen
other projects use it successfully. Unfortunately I'm too stupid to graps how
it is supposed to work so after me not finding enough documentation anywhere
on how it should be done, I scrapped that idea.

I instead decided I'd fix our build to offer everyone an easy way to run
coverage tests locally. So yesterday I submitted my first take on this as PR
#1528: https://github.com/curl/curl/pull/1528

This adds a configure option called --enable-code-coverage and an example
script that makes a whole coverage configure, build, test run and code
coverage output as html. It seems to work.

As a taste of what this looks like, here's the output I got from my run last
night: https://curl.haxx.se/coverage/ (I'll probably update that with updated
data as my experiments develop.)

Using this coverage data I've already submitted two PRs to add tests and to
remove unused code, and I think there will be more coming. This is useful!

Additionally, this morning I'm adjusting the coverage.sh script to also run
our "torture tests" as well as the "event tests" that should increase code
coverage considerably.

Ideally we'll get this integrated into travis and our CI somehow going
forward, so if there's someone who has experience of that I can use a little
help.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:   https://curl.haxx.se/mail/etiquette.html
Received on 2017-06-02