cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: nightly skips a bucket of tests .. how to reduce that a bit

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Sun, 22 May 2016 15:40:54 +0200 (CEST)

On Sat, 21 May 2016, dev_user wrote:

> So it looks like I have sorted out the issue with nightly builds and now I'd
> like to reduce the number of tests being skipped.

Okej, let's do them one by one:

> TESTINFO: "curl lacks debug support" 78 times (67, 68, 69, 81, 89, 90, 91,
> 150, 155 and 69 more)

Add --enable-debug to configure. It will build with more checks, more verbose
output, some "hooks" to changing internals and it allows more access to
internals for unit tests etc.

> TESTINFO: "curl lacks Metalink support" 16 times (2005, 2008, 2009, 2010,
> 2011, 2012, 2013, 2014, 2015 and 7 more)

Add --with-libmetalink to configure, also requires that you actually have
libmetalink installed.

> TESTINFO: "curl lacks http2 support" 1 times (1800)

Requires nghttp2 to be installed at build time. configure --with-nghttp2

> TESTINFO: "failed starting HTTP-pipe server" 4 times (1900, 1901, 1902, 1903)

To test HTTP pipelining we have a dedicated HTTP server that is written in
python and it failed to start. Possibly due to not having python installed?

> TESTINFO: "no stunnel" 34 times (300, 301, 302, 303, 304, 305, 306, 307, 308
> and 25 more)

Lots of TLS based testing requires stunnel to be installed for the test suite
to use. Install stunnel.

> TESTINFO: "failed starting SSH server" 56 times (564, 582, 583, 600, 601,
> 602, 603, 604, 605 and 47 more)

We have a bunch of SSH and SOCKS related tests that actually use the standard
(open)ssh. Presumably you don't have it installed or at least it failed to
work properly.

> TESTINFO: "rlimit problem: fds needed 65562 > system limit 65536" 1 times
> (518)

Test 518 has its own "pre check" that checks that you can create a certain
number of file descriptors since the test checks what happens when we go over
the limit, and I believe that limit is hardcoded so it has to work or that
check fails like that and prevents the test from being run.

> TESTINFO: "curl lacks PSL support" 1 times (1136)

You need libpsl installed so configure can find it and enable its use.

> TESTINFO: "configured as DISABLED" 10 times (594, 836, 882, 938, 1209, 1211,
> 1316, 1510, 1512 and 1 more)

Those are explictly disabled in tests/data/DISABLED. We do that when we have a
test that is added but it turns out the test is unreliable or otherwise not
good enough so we disable it in the hope that someone will fix the test at
some future point in time...

> TESTINFO: "Resolving IPv6 'ip6-localhost' didn't work" 2 times (241, 1083)

I guess it says it all. We use 'ip6-localhost' as a way to resolve a host name
to a local ipv6 address, and then we check that first outside of curl that it
actually works without curl and in this case it didn't!

> TESTINFO: "curl lacks WinSSL support" 1 times (2043)

No surprise there! =)

> TESTINFO: "curl lacks unittest support" 21 times (1300, 1301, 1302, 1303,
> 1304, 1305, 1306, 1307, 1308 and 12 more)

A bit weird, but if you build debug-enabled that means it can build and run
unittests too.

> TESTINFO: "curl lacks TrackMemory support" 2 times (96, 558)

You either enable the full debug build with --enable-debug or you enable only
the memory debugging things with configure's --enable-curldebug option.

> TESTINFO: "precheck command error" 17 times (831, 834, 877, 880, 933, 936,
> 2023, 2024, 2025 and 8 more)

Various kinds of pre checks that check if the conditiona is right before the
test is run, and apparently in these cases they weren't. Some of these I
believe try to do LD_PRELOAD tricks.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:  https://curl.haxx.se/mail/etiquette.html
Received on 2016-05-22