Mutation testing — gremlins and go-mutesting — Library

Overview

Mutation testing judges the test suite itself: seed small bugs (mutants) into the code, run the tests, and count how many mutants the suite kills. Line coverage says the code ran; mutation score says its behaviour was asserted. In Go the practice exists but is niche — the tools (gremlins, go-mutesting) work at the AST level like their JVM cousins, yet none has PIT's decade of maturity, ecosystem integration or incremental-analysis features, so the pragmatic play is targeted runs on critical packages rather than whole-repo gating.

Key points

gremlins (go-gremlins/gremlins) is the current front-runner: mutates the source AST, uses coverage profiles to skip mutants no test even reaches, reports killed / lived / not-covered per mutant with a mutation-score summary.
go-mutesting (zimmski/go-mutesting, with maintained forks e.g. avito-tech's) is the older tool: applies AST mutators, runs an exec script per mutant — more manual, more scriptable.
Mutant catalogue: the usual suspects — conditional-boundary swaps (< ↔ <=), negations, arithmetic operator swaps, removed statements, inverted branch conditions.
Score vs coverage: 90 % line coverage with a 50 % mutation score means the suite executes plenty and checks little — assertion-free tests, over-broad table cases, tests asserting only "no error".
Lived mutants are actionable: each one names a file, line and mutation your suite cannot see; fix by adding the missing assertion, not by chasing the number.
Cost is the constraint: naively, every mutant is a full go test run of the package — minutes become hours. Control it or abandon it.
Why rarer than on the JVM: younger tools, no bytecode-level trick (PIT mutates bytecode; Go tools recompile per mutant batch), no incremental history, no IDE/build plugins — the friction is real, the idea is identical.

Details

Cost controls

Scope to what matters: run on the money packages (parsing, pricing, authz) — not on main, generated code or thin glue. gremlins takes package patterns like go test.
Use the coverage pre-pass: gremlins skips mutants on uncovered lines by default — keep it; killing an uncovered mutant is impossible and running it is waste.
Limit mutant classes: start with conditionals-boundary and negation mutators; the exotic operators mostly add runtime, not insight.
Time-box in CI: nightly or weekly job, not per-PR; set per-mutant test timeouts so an infinite-loop mutant cannot hang the run (a mutant that causes a timeout counts as killed in gremlins' model).
Fast tests are a prerequisite: mutation testing multiplies your unit-suite runtime by the mutant count — a suite that leans on containers is disqualified before it starts; keep integration suites out of scope with -short/build tags.

Reading the report

Verdict	Meaning	Action
Killed	A test failed under the mutant	Good — the suite sees this behaviour
Lived	Tests passed with the bug in place	Missing/weak assertion — the real signal
Not covered	No test executes the line	Coverage gap, cheaper to see with `go test -cover`
Timed out	Mutant caused non-termination	Counted killed; verify the timeout is sane

Examples

$ go install github.com/go-gremlins/gremlins/cmd/gremlins@latest
$ gremlins unleash ./internal/pricing/... --tags=unit
...
Killed: 42, Lived: 5, Not covered: 3
Test efficacy: 89.36%
Mutator coverage: 94.00%

Testing in Go — strategies & tooling map — parent map of the strategy portfolio.
Unit testing — the fast suite mutation testing evaluates (and depends on).
go tool cover — the coverage floor that mutation score refines.
Mutation testing in Java — PIT, the maturity benchmark Go's tools chase.