Performance & load testing — JMH and Gatling

Overview

Performance testing splits into two families that answer different questions: microbenchmarks (JMH) measure the cost of a code path in isolation, while load tests (Gatling, k6, JMeter) measure a running system under concurrent traffic. Both are easy to get numerically wrong — the JVM's JIT invalidates naive timing loops, and naive load-test maths invalidates latency percentiles — so methodology matters more than tooling. Profiling (JFR, async-profiler) runs alongside both to explain why the numbers are what they are.

Key points

• Never hand-roll System.nanoTime() loops: dead-code elimination, constant folding, on-stack replacement and warmup effects make naive benchmarks measure the JIT, not the code. JMH exists because of this.
• JMH (OpenJDK's harness): annotation-driven benchmarks, forked JVMs, controlled warmup/measurement iterations, Blackhole to defeat dead-code elimination, @State for fixtures, modes for throughput/average time/sample percentiles.
• Gatling: code-as-scenario load tool with a Java DSL (Scala originally); simulations express injection profiles (ramps, constant rates), assertions gate CI, HTML reports include response-time percentile distributions.
• k6 (JS scenarios, Go engine) and JMeter (GUI/XML, huge plugin base) are the common alternatives; pick per team — the methodology transfers.
• Coordinated omission: closed-model tools that wait for each response before sending the next silently drop the samples that would have occurred during stalls, flattening tail latency. Use open-model injection (arrival rate, as in Gatling's constantUsersPerSec / k6 arrival-rate executors) and percentile-safe recording (HdrHistogram).
• Report percentiles, never averages: p50/p95/p99/p99.9 — the tail is the user experience; averages hide it entirely.
• Profile alongside: JFR is always-on-capable in production (-XX:StartFlightRecording), analysed in JMC; async-profiler gives low-overhead CPU/alloc/lock flame graphs and plugs into JMH via -prof async.
• Framework note: measure Spring/Quarkus/Micronaut services at the system level (load test the HTTP surface); use JMH only for hot library code — benchmarking through a DI container mostly measures the container.

Details

JMH pitfalls checklist

Load-test methodology

1. Define the question: capacity (max sustainable rate), latency under expected load, or soak (degradation over hours).
2. Model traffic as an open system: specify arrival rate, not virtual-user count; ramp to target and hold.
3. Assert on percentiles and error rate (Gatling assertions), fail CI on regression.
4. Watch the system under test (GC logs, JFR recording, saturation metrics) during the run — a load test without server-side observation only tells you that it is slow.
5. Load generators saturate too: monitor the injector, scale it out before believing bad numbers.

Examples

@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@Warmup(iterations = 5, time = 1)
@Measurement(iterations = 5, time = 1)
@Fork(2)
@State(Scope.Benchmark)
public class CsvParseBench {

    private String line;

    @Setup
    public void setup() { line = "2026-08-09;42.50;EUR;order-123"; }

    @Benchmark
    public Order parse() {          // returned => not dead code
        return Order.parseCsv(line);
    }
}

// Gatling Java DSL — open-model injection
setUp(
  scenario("checkout")
      .exec(http("create").post("/orders").body(StringBody(payload)))
      .injectOpen(rampUsersPerSec(10).to(200).during(120), constantUsersPerSec(200).during(600))
).assertions(
  global().responseTime().percentile(99.0).lt(500),
  global().failedRequests().percent().lt(1.0)
);

Related

• Testing in Java — strategies & tooling map — parent map of the strategy portfolio.
• JFR & JMC — the profiling pair to run alongside every load test.
• Performance engineering — the system-level methodology these tools implement.
• One Billion Row Challenge — measurement discipline applied to a concrete optimisation exercise.