rgoussu.dev
rgoussu@goussu: ~/library/applicative-architecture
❯ ~/library/applicative-architecture cat testing-strategies.md

Testing strategies

# Testing as a strategy, not a ritual — the pyramid and its critics, contract and property-based testing, load and chaos, and testing in production.

Conceptsaved 2026-08-08updated 2026-08-20 #testing#strategy#contract-testing#quality#craft

Overview

TDD tells you how to write a test; a testing strategy decides which kinds of tests, in what proportion, catching which risks, at what cost. The classic answer is the test pyramid — many fast unit tests, fewer integration tests, a thin crust of end-to-end — but the interesting engineering is in its exceptions and its blind spots: contracts between services, properties instead of examples, load, failure injection, and the tests that can only run in production.

Key points

  • The pyramid, and why it's shaped that way: cost and feedback speed rise as you climb; flakiness rises too (e2e tests fail for reasons unrelated to the change). The critiques matter: the "testing trophy" argues integration tests catch the most real bugs per line for API/UI code; testcontainers-style real-dependency tests have made "integration" much cheaper than when the pyramid was drawn. The honest rule: test each behavior at the lowest level that can actually falsify it.
  • Unit vs. integration is a seam question: sociable units (real collaborators, fake I/O) vs. solitary (mock everything) — over-mocking couples tests to internals and rots them (TDD's London/Detroit trade). Ports from hexagonal architecture are the natural mocking line.
  • Contract testing: consumer-driven contracts (Pact-style) verify that provider and consumer agree on the API without spinning both up together — each side tests against the recorded contract in its own pipeline. This is what makes independently deployable microservices honest, and it replaces the integration-environment bottleneck that otherwise reappears.
  • Property-based testing: state invariants ("encode∘decode = identity", "sorted output is a permutation of input") and let the framework (QuickCheck lineage: jqwik, Hypothesis, proptest) generate and shrink counterexamples. Finds the edge cases nobody writes examples for; ideal for parsers, serializers, and anything with a round-trip.
  • Mutation testing (PIT, Stryker): seed artificial bugs and check the suite catches them — the only honest measure of suite strength; coverage only proves code ran.
  • Load & resilience: performance tests belong in the strategy, not in launch week — k6/Gatling with realistic scenarios, percentile assertions, and coordinated-omission awareness (performance engineering); chaos engineering (failure injection, starting in staging with a hypothesis) tests the system's error handling the way unit tests test a function's.
  • Testing in production is a tier, not a sin: smoke tests post-deploy, canary analysis, synthetic monitoring — some risks (config, data shape, scale) only exist there; progressive delivery is the safety harness that makes it responsible.
  • Suite hygiene is strategy too: flaky tests quarantined and fixed (a suite people rerun is a suite people ignore), test data builders over shared fixtures, deterministic time/randomness injection.
  • To explore: approval/snapshot testing (and its rubber-stamp failure mode), fuzzing as security-grade property testing, test impact analysis at scale.

Practice

  • Diamond kata, property-based (source) — solve it with properties instead of examples ("symmetric", "widest at the middle") and let the framework shrink the counterexamples.
  • Mutation-test a real suite (source) — run PIT (or Stryker) on a project whose suite you trust; every surviving mutant is a test you thought you had.
  • Pact contract-testing workshop (source) — consumer-driven contracts between two small services; retire the shared integration environment they replace.
  • Build Your Own JSON parser (source) — an ideal property-based target: parse∘serialize round-trips plus a torture suite of malformed inputs.
  • Load-test with k6 (source) — script a realistic scenario against a side project with percentile assertions, not averages.

Related