Overview
Consensus is the problem of getting a set of unreliable machines to agree on a value — the primitive beneath leader election, distributed locks, configuration stores, and every strongly consistent replicated database. Paxos proved it possible; Raft made it teachable, and Raft's design (replicated log + leader + terms) is now the shape of the thing in practice (etcd, Consul, CockroachDB, Kafka's KRaft). Understanding it explains both what coordination costs and why systems avoid it where they can.
Key points
- The core guarantee: safety always (no two nodes decide differently), liveness usually (FLP impossibility: with a fully asynchronous network you cannot guarantee progress — real systems use timeouts and randomization to make stalls improbable).
- Raft in one paragraph: nodes are follower/candidate/leader; time divides into numbered terms; a candidate wins election by majority vote (log-completeness check ensures the winner has all committed entries); the leader appends client commands to its log, replicates them, and an entry is committed once a majority holds it — followers apply committed entries in order. Everything is majorities: any two quorums intersect, so a new leader always knows the committed prefix.
- Quorums generalize: majority reads/writes (R + W > N) give the same intersection property without a leader — the Dynamo-style alternative with weaker guarantees (no ordering, sloppy quorums trade safety for availability).
- What consensus costs: every decision is a round-trip to a majority — latency floors and throughput ceilings; hence the design idiom of using consensus sparingly: metadata, leases, leader election, membership — with bulk data flowing outside it (leader/follower replication, per-partition leaders).
- Leases & fencing: a leader/lock granted by consensus expires (lease) and every grant carries a monotonic fencing token the downstream must check — the piece everyone forgets, and the reason "distributed lock" alone doesn't prevent split-brain writes from a paused process.
- In the wild: etcd under Kubernetes (all cluster state), ZooKeeper's ZAB, KRaft replacing ZooKeeper in Kafka, per-range Raft groups in CockroachDB/TiKV, S3-strong consistency — the pattern is always "consensus for coordination, replication for data".
- To explore: Multi-Paxos vs. Raft differences (mostly pedagogy), Byzantine fault tolerance (when nodes lie — blockchain territory), flexible quorums, the Raft paper itself (genuinely readable).
Raft's node roles, as the one-paragraph summary above gives them:
stateDiagram-v2
state "Follower" as Follower
state "Candidate" as Candidate
state "Leader" as Leader
[*] --> Follower
Follower --> Candidate: stands for election in a new numbered term
Candidate --> Leader: wins a majority vote (log-completeness check passed)
Leader --> Leader: appends and replicates client commands, an entry commits once a majority holds it
Practice
- Raft, animated (source) — step through elections, terms, and log replication visually before writing a line; the raft.github.io guide and paper sit one click away.
- Implement Raft with Gossip Glomers (source) — climb the Fly.io challenges to a Raft-backed linearizable KV store, with Maelstrom injecting the partitions the safety argument is about.
- MIT 6.5840 Raft labs (source) — the full treatment: leader election, log replication, persistence, and snapshots against a deliberately brutal test harness.
Related
- System design fundamentals — the consistency spectrum consensus anchors.
- Asynchronous and distributed system patterns — no-global-clock reasoning; consensus is the escape hatch when order must be total.
- Message brokers & event streaming — KRaft and ISR are applied consensus.
- Databases and other storage systems — consensus-backed vs. eventually consistent stores.