Performance & load testing — testing.B, benchstat, vegeta

Overview

Go bakes microbenchmarking into the same stdlib as testing: func BenchmarkXxx(b *testing.B) runs under go test -bench, reports ns/op and (with ReportAllocs) allocations, and feeds profiles straight into pprof. The discipline is knowing the traps — dead-code elimination, run-to-run noise — and answering them with sink patterns and benchstat. System-level load testing is a separate exercise with separate tools (vegeta, k6, hey) and a shared methodology problem: percentiles and coordinated omission.

Key points

• testing.B mechanics: the framework calibrates iteration count; classic idiom for i := 0; i < b.N; i++ { ... }, and since Go 1.24 the preferred for b.Loop() { ... }, which also keeps the compiler from optimising the loop body away and excludes setup/teardown from timing automatically.
• b.ResetTimer / b.StopTimer: exclude expensive setup from the measurement in the b.N style (largely unnecessary with b.Loop).
• b.ReportAllocs() (or -benchmem): allocs/op and B/op — for hot-path Go code the allocation count is often the headline number, since it prices GC pressure.
• The dead-code trap: an unused result lets the compiler delete your workload and report a fantasy number. Sink the result — assign to a package-level variable or use runtime.KeepAlive — or use b.Loop, which prevents it by design.
• benchstat is non-negotiable: run benchmarks ≥10 times (-count=10), compare old vs new with benchstat (golang.org/x/perf); it reports deltas with p-values and flags noise as "~". A single before/after pair is an anecdote.
• Stabilise the machine: quiesce background load, pin CPU frequency where possible; -benchtime to lengthen runs when variance stays high.
• Profile under benchmark: go test -bench=. -cpuprofile=cpu.out -memprofile=mem.out then go tool pprof — the shortest path from "it's slow" to "this line is slow"; see pprof.
• Load tools: vegeta (Go-native, constant-rate attacks, library + CLI, latency histograms), k6 (JS-scripted scenarios, thresholds for CI gating), hey (quick ad-hoc). Constant-rate open-model tools like vegeta avoid coordinated omission by construction.

Details

Microbenchmark traps

Latency methodology (system level)

• Percentiles, not means: report p50/p95/p99/p99.9; the mean hides the tail your users live in. Never average percentiles across runs or instances.
• Coordinated omission: closed-loop tools that wait for each response before sending the next silently pause the clock during stalls, wildly understating tail latency. Open-model, constant-arrival-rate load (vegeta's -rate, k6's constant-arrival-rate executor) measures what an independent client population would see.
• Warm up, then measure: discard the first seconds (JIT-free Go still has cold caches, connection pools, page cache).
• Gate in CI: k6 thresholds (e.g. http_req_duration p(99)<250ms) fail the job — load testing as regression testing, same portfolio as the rest of the suite; the deeper methodology lives in Performance engineering.

Examples

var sink int

func BenchmarkDecode(b *testing.B) {
    payload := loadFixture(b, "payload.json")
    b.ReportAllocs()
    for b.Loop() { // Go 1.24+; use for i := 0; i < b.N; i++ before
        v, err := Decode(payload)
        if err != nil {
            b.Fatal(err)
        }
        sink = v.N // sink defeats dead-code elimination in the b.N style
    }
}

$ go test -bench=Decode -count=10 -benchmem ./codec/ > new.txt
$ benchstat old.txt new.txt
              │   old.txt    │               new.txt               │
Decode-8        1.842µ ± 2%    1.291µ ± 1%   -29.91% (p=0.000 n=10)

$ echo "GET http://localhost:8080/orders" | vegeta attack -rate=500 -duration=60s | vegeta report
Latencies  [mean, 50, 95, 99, max]  2.1ms, 1.8ms, 4.2ms, 9.7ms, 41ms

Related

• Testing in Go — strategies & tooling map — parent map of the strategy portfolio.
• pprof — profiling the hotspots the benchmark exposes.
• Performance engineering — the system-level methodology (USE/RED, tail latency) behind the tools.
• Performance & load testing in Java — JMH/Gatling as the JVM mirror, with JIT warm-up worries Go skips.