rgoussu.dev
rgoussu@goussu: ~/library/system-design
❯ ~/library/system-design cat websockets-and-bidirectional-protocols.md

WebSockets & bidirectional protocols

# Pushing data to clients — WebSockets, SSE, long polling, WebTransport, gRPC streaming, webhooks — and how to choose and scale them.

Conceptsaved 2026-08-08updated 2026-08-09 #websockets#sse#realtime#http#push#system-design

Overview

Plain HTTP is request/response: the client asks, the server answers, the connection's job is done. The moment the server has something to say first — chat, live dashboards, notifications, collaborative editing — you need a push mechanism, and there is a whole ladder of them: polling, long polling, Server-Sent Events, WebSockets, WebTransport, gRPC streams, and webhooks for the server-to-server case. Choosing is a trade between directionality, infrastructure friendliness, and operational cost — and the real engineering usually starts at scaling the connections, not opening them.

Key points

  • The ladder, simplest first: short polling (repeat requests; cheap to build, wasteful, latency = interval) → long polling (server holds the request until there's data; near-realtime over plain HTTP, one message per round trip) → SSE (one-way stream) → WebSocket (full duplex) → WebTransport (the HTTP/3 successor).
  • WebSocket: starts as an HTTP/1.1 GET with Upgrade: websocket (101 Switching Protocols), then becomes a framed, message-oriented, full-duplex TCP channel — no HTTP semantics afterward (no status codes, no caching, no content negotiation). Subprotocols (e.g. STOMP, GraphQL-WS) put structure back on top. Heartbeats (ping/pong) are on you: intermediaries silently kill idle connections.
  • Server-Sent Events: a long-lived text/event-stream response over ordinary HTTP — server→client only. Auto-reconnect with Last-Event-ID resume is built into the browser's EventSource; works through proxies, CDNs, and HTTP/2 multiplexing with no special handling. The right default when clients only receive (feeds, notifications, LLM token streaming) — many teams reach for WebSockets when SSE would do.
  • Directionality is the first question: server→client only → SSE; genuinely bidirectional and chatty (chat, games, collaborative cursors) → WebSocket; client→server only → plain HTTP was fine.
  • The wider family: HTTP/2 server push (deprecated and removed — not a messaging mechanism); gRPC server/client/bidi streaming (the service-to-service equivalent); WebTransport (QUIC-based: multiple streams without head-of-line blocking, plus unreliable datagrams for games/media); WebRTC data channels (peer-to-peer, UDP-flavored); webhooks (push between servers: HTTP callbacks + retries + signatures + consumer idempotency).
  • Scaling is the actual hard part: long-lived connections are per-node state — you need connection-count-aware load balancing (and sticky routing or a connection-id registry), a pub/sub backplane (Redis, Kafka, NATS) so any node can reach any client, graceful drain on deploys (mass reconnect = thundering herd — jitter the backoff), and explicit reconnect-with-resume semantics so missed messages replay. Managed layers (Pusher, Ably, Socket.IO with its fallback dance) exist because of exactly this.
  • Ops details that bite: L7 proxies/LBs need explicit Upgrade support and generous idle timeouts; auth happens at the handshake (cookies or a ticket — headers aren't settable from browser WebSocket APIs) and needs a story for expiry mid-connection; buffering middleboxes break SSE unless flushing/X-Accel-Buffering is handled.
  • To explore: GraphQL subscriptions, MQTT (the IoT sibling), CRDTs/OT for the collaborative-editing layer above the transport, backpressure on slow consumers.

The upgrade dance, and where auth has to happen:

sequenceDiagram
    participant B as Browser client
    participant S as Server
    Note over B,S: Auth happens at the handshake - cookies or a ticket (headers not settable from browser WebSocket APIs)
    B->>S: HTTP/1.1 GET with Upgrade: websocket
    S-->>B: 101 Switching Protocols
    Note over B,S: Framed, message-oriented, full-duplex TCP channel - no HTTP semantics afterward
    B->>S: Message frames
    S->>B: Message frames
    B->>S: Ping
    S-->>B: Pong

Scaled out, the fan-out topology looks like this:

flowchart TD
    C1[Client] --> LB["Load balancer - connection-count-aware, sticky routing"]
    C2[Client] --> LB
    C3[Client] --> LB
    LB --> N1["Connection-holding node A"]
    LB --> N2["Connection-holding node B"]
    N1 <-->|publish and subscribe| BP["Pub/sub backplane - Redis, Kafka, NATS"]
    N2 <-->|publish and subscribe| BP

Practice

  • Protohackers: Budget Chat (source) — a TCP chat room from spec: framing, joins and leaves, concurrent connections — the raw material under every realtime protocol.
  • SSE with resume (source) — a live feed over text/event-stream with Last-Event-ID replay; prove a dropped client misses nothing, and feel why SSE is the right default for one-way push.
  • WebSocket server from RFC 6455 (source) — implement the handshake, frame parsing (masking, fragmentation), and ping/pong straight from the RFC, then run the Autobahn testsuite against it.
  • Load balancer, upgraded (source) — the stretch goals add Upgrade passthrough, idle timeouts, and sticky routing: long-lived connections meet the LB, exactly where production pain lives.

Related