Brimyr¶
Brimyr is quality assurance for a pull request: it gates on the coverage of the
lines that PR changed, and on the net-new quality findings that PR introduced. Point
it at any repo and it figures out the rest — it detects the ecosystem, runs the test
suite with coverage instrumentation on, and gates on the coverage of the changed lines
(diff-cover semantics), blocking below a threshold (default 80%). Pre-existing
uncovered code never blocks. Turn quality: 'true' on and the same run also classifies
the linters' findings against that same diff. One job, one summary, one PR comment.
- uses: magmamoose/brimyr@v1 # that is the whole configuration
That is the part nothing else does. Patch-coverage maths is well-trodden; what every other tool has in common is that you bring your own coverage report: you work out the right test command for each language and wire it per repo. Across an estate in several languages, that wiring is the project.
Brimyr is quality assurance; Chargate is security assurance, and Diatreme builds and releases. Brimyr and Chargate are twins, not competitors — Chargate gates net-new security findings, Brimyr gates coverage and net-new quality findings. The line between them is the subject, not the tool: Brimyr's quality half runs Chargate's net-new engine rather than growing its own (see below).
What it detects¶
| Ecosystem | Markers | Test command | Coverage format |
|---|---|---|---|
| Python | pyproject.toml, setup.py, tox.ini |
pytest --cov |
Cobertura |
| JavaScript / TypeScript | package.json |
jest, or vitest run when the repo uses vitest |
lcov |
| .NET | *.sln, *.csproj, *.fsproj, *.vbproj |
dotnet test --collect |
Cobertura |
| Java / JVM | pom.xml |
mvn ... jacoco:prepare-agent test jacoco:report |
JaCoCo |
A polyglot repo matches more than one, produces one report per language, and Brimyr
merges them into a single number. Override any of it with ecosystem,
test_command, or coverage_file. See Action reference.
The same run can also ship coverage to SonarQube for the long-run trend, non-blocking. It does
that only when sonar_url and sonar_token are both set and a sonar-scanner is on PATH;
otherwise the step is skipped with a reason and nothing fails. See SonarQube.
Three faces, kept separate¶
- Blocking: the patch-coverage gate. The percentage of changed executable
lines the tests covered, diffed against
merge-base(base, head). Blocks below the threshold. Computed locally; no SonarQube involvement. - Report-only by default: the net-new quality gate. The quality findings this
PR introduced, classified against that same diff. Off unless
quality: 'true', and even thenquality_fail_ondefaults tonone— it counts and reports, and blocks only at a SARIF level you choose. - Non-blocking, and opt-in: one
sonar-scannerrun. Off unlesssonar_urlandsonar_tokenare set. Sonar's native quality analysis plus ingesting the coverage file → SonarQube, for history and the coverage/quality trend. Sonar derives new-vs-old code itself (its New Code Period); you never feed it "net-new".
The two gates share one job, one summary and one PR comment, and the job exits on the worse of the two.
Broken test run ≠ 0% coverage
If the test run failed, timed out, or produced no coverage, that is an error (build red): never reported as "0% patch coverage" that hard-fails the gate.
Coverage is a byproduct of the test run¶
You run the tests with instrumentation on (pytest --cov, jest --coverage,
dotnet test --collect) and that single run emits the coverage file. There is no
separate "measure coverage" pass. Brimyr detects the ecosystem and runs the right
command; polyglot repos produce one coverage file per language, merged.
Coverage is half of quality assurance¶
The other half is what the linters say. Set quality: 'true' and Brimyr also gates the
net-new quality findings this pull request introduced, classified against the same
diff the coverage gate uses, and folds that verdict into the same job summary and the
same PR comment as coverage. The job exits on the worse of the two.
Brimyr implements none of that classification. Chargate already owns a finished net-new
engine, so Brimyr does not import it, vendor it or re-implement it — it calls it as a
nested step: a shared package would buy version skew, lockfile drift and a diamond
dependency inside one job, and a subprocess in its own environment has none of those
properties. That is
ADR 0002.
Chargate's own fail_on is pinned to none there, so it can never set the job's exit
code — it reports, Brimyr decides.
quality_fail_on defaults to none, which makes it report-only until you say
otherwise. MegaLinter's quality half over a mature repo is far denser than its security
half, and a first PR that goes red with hundreds of findings is how a gate becomes
decoration nobody reads. Ship it reporting, measure a release cycle, then pick a SARIF
level. See Quality findings.
A scan that never ran is not a clean one. The nested step is continue-on-error, so it
cannot take the coverage gate down with it, but Brimyr goes by that step's outcome
rather than by the counts file it may have left behind — a failed scan can leave a
well-formed row of zeros on disk, which is what a clean PR looks like — and reports a
tool error instead: exit 2, quality_gate_result error. Nor is a scan that
completed necessarily a full one: when Chargate could not start a linter it says which,
and Brimyr states that shortfall beside the count rather than passing a smaller scan off
as the whole answer.
Two surfaces, one CLI¶
| Surface | What it is | When to use |
|---|---|---|
| Composite action | action.yml |
When you compose your own steps. |
| pre-push hook | .pre-commit-hooks.yaml (brimyr hook) |
Catch a shortfall locally before pushing. |
See Setup & usage to wire one up, Action reference for every input and output, Architecture for how it fits together, Patch coverage for the precise classification rules, and Quality findings for the other half. When something goes wrong, Troubleshooting.
The PR comment¶
Opt in with pr_comment and the verdict lands on the pull request as one
comment, updated in place on every push rather than stacked, and carrying both
halves: the coverage number, the threshold and the changed lines the tests never
executed, then the net-new findings count, its per-level breakdown and what it blocks
on. Set token_broker_url as well and it is authored by Brimyr[bot] rather than
the shared github-actions[bot]. Neither can fail the gate: a comment is a
convenience, and a convenience must never turn a green PR red. See
PR comment.
Modes¶
- PR events → run tests → patch-coverage gate, plus the quality gate when
quality: 'true'→ ship to SonarQube. - Push to default branch / scheduled → run tests → ship to SonarQube as the trend baseline → no gate. Quality findings are still counted and shown; a baseline run has no diff to gate against, and says so.
mode: auto (default) picks this from the event; force it with mode: pr|baseline.
Why patch coverage?¶
Requiring 80% coverage on a whole legacy codebase is a non-starter; ignoring coverage on new code lets it rot. Patch coverage splits the difference: hold new and changed lines to a bar, leave the back-catalogue alone.
- Gate on what this PR changed → actionable, no legacy-debt noise.
- Ship the full coverage to SonarQube → the long-run trend, and Sonar's own quality gate (not to be confused with Brimyr's net-new quality gate above).
Rollout cost is the point¶
The comparison that matters is not "which tool computes coverage" — it is what it takes to switch this on across an estate. For eight repos in four languages:
| SonarQube | Brimyr | |
|---|---|---|
| Server to host / licence | a server, or a per-committer plan | none |
| Per repo | a project, a token, a quality gate, workflow YAML | one uses: line |
| Per ecosystem | a different scanner — dotnet sonarscanner for .NET, mvn sonar:sonar for Java, the CLI for the rest |
detected |
| Producing the coverage report | yours to work out, per language | it runs your tests |
| Per-PR diff coverage | Developer Edition and above | included |
License¶
MIT.