ferramenta
Skip to content

Test Coverage

Coverage goals, local commands, CI thresholds, and artifact locations for Ferrocat.

Ferrocat uses coverage reporting to keep regression work visible, especially in the parser, serializer, catalog API, and ICU parser crates. Coverage is not treated as the only quality signal, but it gives maintainers a concrete guardrail for day-to-day changes.

The repository exposes workspace-local Cargo aliases for coverage reporting through cargo-llvm-cov.

The long-term goal for the shipped library surface is:

  • 85%+ line coverage for ferrocat-cli, with room to ratchet toward the library threshold as more CLI workflows ship
  • 95%+ line coverage for ferrocat-po
  • 95%+ line coverage for ferrocat-icu
  • 95%+ line coverage for ferrocat once the umbrella crate contains measurable executable logic beyond re-exports

The coverage commands intentionally exclude:

  • ferrocat-bench, which is a benchmark harness rather than shipped library code
  • ferrocat-conformance, which is fixture data and expectations for the upstream snapshot

That keeps the report focused on the public and internal library crates that matter most in day-to-day development:

  • ferrocat
  • ferrocat-cli
  • ferrocat-po
  • ferrocat-icu

Current Gate

The CI coverage job measures the current percentages on every run, exports the machine-readable summary, and enforces these crate-level thresholds through scripts/coverage.sh, the single source for the llvm-cov filters and the thresholds:

  • ferrocat-cli >= 85%
  • ferrocat-po >= 95%
  • ferrocat-icu >= 95%
  • ferrocat is still reported, but not percentage-gated until llvm-cov has measurable lines for the crate

Exact percentages are intentionally read from the current CI artifact instead of being copied into this page. They move whenever covered lines or tests change, while the thresholds above are the stable policy this documentation tracks.

The next high-yield test targets are:

  • crates/ferrocat-cli/src/main.rs
  • ferrocat-po/src/api/catalog.rs
  • ferrocat-po/src/api/compile.rs
  • ferrocat-po/src/api/audit.rs
  • ferrocat-po/src/merge.rs
  • ferrocat-po/src/borrowed.rs
  • ferrocat-icu/src/parser.rs

Local Setup

Install the required tooling once:

rustup component add llvm-tools-preview
cargo install cargo-llvm-cov

Local Commands

Print a terminal summary:

cargo coverage-summary

Build an HTML report:

cargo coverage

The HTML output is written to:

target/coverage/html/index.html

Generate an LCOV file for external tooling:

cargo coverage-lcov

The LCOV output is written to:

target/lcov.info

Run the gate exactly as CI runs it:

scripts/coverage.sh

That script writes target/lcov.info and target/coverage-summary.json, then prints the measured percentage per crate against its threshold and exits non-zero when a gated crate is below it.

CI

The GitHub Actions CI workflow runs a dedicated coverage job on Ubuntu, installs the required LLVM tooling, and runs scripts/coverage.sh — the same command contributors run locally. The thresholds are enforced per crate rather than through one global blended percentage, so a well-covered crate cannot hide a regression in another one.

The job publishes its evidence in three places, on a failing run as much as on a passing one:

  • the run's job summary, which lists the measured percentage per crate next to the threshold it was checked against
  • lcov.info as a workflow artifact
  • coverage-summary.json, the machine-readable llvm-cov summary, as a second workflow artifact

Status note (2026-09-07): the gate lives entirely in this repository's CI. The previous Codecov upload and its CODECOV_TOKEN secret were removed; no external coverage service is involved, and no coverage token is required to run or fork the build.