Skip to content

Add coverage reporting support for the testsuite - #315

Draft
jajik wants to merge 31 commits into
modcluster:mainfrom
jajik:coverage
Draft

jajik wants to merge 31 commits into
modcluster:mainfrom
jajik:coverage

Conversation

@jajik

@jajik jajik commented Jan 14, 2025

Copy link
Copy Markdown
Member

Opening as a draft because I think the generated report is incomplete (well, I guess the build is not executed properly and because of that not everything is marked as executed). I just want to have it out there.

Both lcov & gcov reports are generated, but the source data are the same.

@jajik
jajik requested a review from jfclere January 14, 2025 15:06
@rhusar

rhusar commented Jan 17, 2025

Copy link
Copy Markdown
Member

I think this is starting to look good! I will be curious to review the reports.

@jajik

jajik commented Jan 17, 2025

Copy link
Copy Markdown
Member Author

Except it's not reported properly. I'll talk to the compiler a little bit more, maybe he'll change his mind... 🙂

@jajik jajik linked an issue Jan 17, 2025 that may be closed by this pull request
@jajik jajik self-assigned this Mar 24, 2025
@jajik
jajik removed the request for review from jfclere March 24, 2025 07:48
@jajik
jajik force-pushed the coverage branch 2 times, most recently from ebd46fb to 9eaeaee Compare May 15, 2025 08:22
@jajik
jajik force-pushed the coverage branch 4 times, most recently from bf96da7 to 7b23f7a Compare May 20, 2026 12:02
@jajik
jajik force-pushed the coverage branch 5 times, most recently from 15f4498 to c317291 Compare August 31, 2026 16:57
@jajik

jajik commented Sep 1, 2026 •

Copy link
Copy Markdown
Member Author

It's my pleasure to make this ready for review. The coverage is working (at least it seems so) now.

I used multiple AIs for debugging my original solution, because the coverage reported was low and clearly incorrect (functions that must have been executed in order for tests to pass were marked as not executed). The best results I got were from Claude, which overengineered a "fix" that worked. It was needlessly complicated, so I didn't use it; however, it pointed out the issue I had in my solution – missing permissions for the workers, which are fixed using chmod with umask. It was an interesting exercise though. 🙂

Main changes are:

  • code coverage reporting for tests
  • by default ON in CI (-> coverage artifacts with html reports)
  • tests compile modules using CMake now

@jajik
jajik marked this pull request as ready for review September 1, 2026 07:59
@jajik
jajik requested review from jfclere and rhusar September 1, 2026 07:59
@jajik jajik removed their assignment Sep 1, 2026
jajik added 26 commits October 6, 2026 19:04
Also have one job to gather and evaluate tests as needed
`expr` returns 1 (a failure) when the result is 0 which may lead to
some failures even though nothing bad happenned (see testsuite.sh)
We will require only the gather-results job succeeding, so we can make
things a little bit simpler.
@jajik

jajik commented Oct 6, 2026 •

Copy link
Copy Markdown
Member Author

MOVING THIS BACK TO A DRAFT

I went back to this and rebased it to #403. It would be needed anyway and having it ready and serialized seems like a better approach from reviewer perspective. And it also allowed me to gather coverage from all testsuites.

It is ready for a review, but we should merge (and also review) it in the right order.

@jajik
jajik marked this pull request as draft October 6, 2026 19:36
@jajik

jajik commented Oct 6, 2026

Copy link
Copy Markdown
Member Author

Just to get the idea, this is what we get here:

coverage-report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Explore how to add code coverage tests/reporting to the repository

2 participants