We also use Cirrus-CI, right now only for two FreeBSD builds which means it is underutilized.

Overview of our (con)current CI capacity before this PR:

• travis-ci: 31/20 (big problem, even though the recent limit increase)
• gh-actions: 2/20 (CIFuzz and Ubuntu default build which only ran on push, not for PRs)
• azure-pipelines: 20/20
• appveyor: 16/10 (not that big of a problem, because 8 jobs take less than 3 minutes)
• cirrus-ci:
  • 0/8 Linux Containers
  • 0/2 Windows Containers
  • 2/2 FreeBSD
  • 0/1 macOS

@bagder maybe it is a good idea to include the limits in your stats script and have a check for them?

include the limits in your stats script and have a check for them?

I don't think they belong in there. The limits also vary over time and the script can't figure out what limits that are set for what service when.

I also don't think the limits themselves is the problem as much as the total time the jobs take to complete.

Note to myself: turn build combinations into matrix strategy elements to reduce YAML duplication.

@bagder can we drop cpp.yml from GH Actions? I think that is already covered by other CIs.

can we drop cpp.yml from GH Actions

Sure, if the same build runs elsewhere it isn't necessary to have there!

@bagder do you think the number of macOS jobs is crazy or would it be okay to have those?

do you think the number of macOS jobs is crazy

How many are they and why do you suggest we need that amount?

do you think the number of macOS jobs is crazy

How many are they and why do you suggest we need that amount?

Previously it came to 29 (9 autotools builds with clang, gcc-8 and gcc-9, plus 2 cmake builds).
Now I switched the various compiler testing to the cmake builds and have 15 builds (9 autotools and 6 cmake builds).

I basically transferred and merged, but also logically combined/expanded upon the build variants we had on travis-ci and azure-pipelines to form this list.

@bagder please give a Go for merge. I will do the other CI changes in separate PRs.

BTW-- the reason I am moving macOS builds away from travis-ci is the backlog on there:

During peak hours the amount of macOS jobs on their platform results in delayed job execution.

so the same build combos as before still run, just spread out differently over the services? And then some new combos are added? This is 14 commits and the yaml is impossible to understand. It's hard to review....

so the same build combos as before still run, just spread out differently over the services?

Yes, but instead of spreading them over travis-ci and azure-pipelines, macOS is for now only used with gh-actions.

And then some new combos are added?

Yes, to compensate for some gcc to clang transitions. See below for details.

This is 14 commits and the yaml is impossible to understand. It's hard to review....

I am sorry for that, I guess you can see the summarized changes here, but in order to make a review and overview of the changes easier, I created the following illustration (click for original/full size):

• on the left = before this PR: top = travis-ci & bottom = azure-pipelines
• on the right = after this PR: gh-actions only
• blue lines = moved and merged to other CI without build configuration change
• orange lines = moved to other CI while switching from gcc to the macOS default clang
• green dots = compensate for switch to clang by adding similar builds using gcc with cmake

I don't think they belong in there. The limits also vary over time and the script can't figure out what limits that are set for what service when.

@bagder Okay, but what about splitting the dashboard graph per CI platform? and make the graph lines stacked so that it is possible to tell the amount of jobs per CI platform?

Once this is merged, I need to double-check that the stats script actually counts the jobs correctly still...

Thanks, I just merged this PR and also did the other tidy-ups we talked about.

Regarding 7e8a1a0 I forgot to mention in the commit description that a similar job is also running on Azure Pipelines besides Travis CI.

Once this is merged, I need to double-check that the stats script actually counts the jobs correctly still...

I guess it won't since I am now making use of the matrix feature and also plan to do the same for Azure Pipelines to reduce duplication clutter.

I guess you could adapt it to count the combinations of - install and - CC lines, though the later is only relevant for the 2nd job related to cmake. So the script will need to be aware of jobs.

cc @bagder

Overview of our (con)current CI capacity after this PR was merged:

• travis-ci: 24/20 (Ubuntu variants only, less of a problem now due to job limit increase)
• gh-actions: 16/20 (15 macOS + 1 CIFuzz)
• azure-pipelines: 16/20 (6 Ubuntu + 10 Windows msys)
• appveyor: 16/10 (Windows variants only, not a problem since 8 jobs take less than 3 minutes)
• cirrus-ci:
  • 0/8 Linux Containers
  • 0/2 Windows Containers
  • 2/2 FreeBSD
  • 0/1 macOS

My next plan is to fix the randomly breaking AppVeyor jobs with #5034, move some Linux jobs to CirrusCI and also add some more Windows jobs (most likely WSL-based ones which are currently only possible on AppVeyor, because PRs for actions/runner-images#50 are still pending). So I may also try to do some rebalancing of Windows jobs if AppVeyor becomes too slow or utilized.

cc @bagder @MarcelRaad

Improving Azure Pipelines is in the works with #5468. Next up: move more builds to CirrusCI.

most likely WSL-based ones which are currently only possible on AppVeyor, because PRs for actions/runner-images#50 are still pending

That issue got fixed.
To use WSL you first have to install a distribution.
You can do so using my shiny new setup-wsl action: https://github.com/marketplace/actions/setup-wsl if you like :-)

Definitely on my todo list, thanks.