rgoussu.dev
rgoussu@goussu: ~/library/platform
❯ ~/library/platform cat kubernetes-and-containers.md

Kubernetes & containers deep dive

# From namespaces and images to the reconciliation loop — what containers actually are and how Kubernetes turns them into a platform.

Conceptsaved 2026-08-08updated 2026-08-20 #kubernetes#containers#docker#platform#orchestration

Overview

A container is not a lightweight VM — it's an ordinary Linux process wearing namespaces (isolation of view) and cgroups (limits on resources), started from a layered filesystem image. Kubernetes is a control plane over many such processes, built on one idea applied everywhere: you declare desired state, controllers reconcile actual state toward it, forever. Grasp those two foundations and the whole surface — pods, deployments, services, operators — becomes derivable rather than memorized.

Key points

  • Containers = kernel features: PID/net/mount/user namespaces + cgroup v2 limits + a union filesystem (image layers, copy-on-write) — see Linux deep dive. One process per container is convention because the container is the process; OCI images/runtimes (runc, containerd) are the standardized pieces — Docker is a UX over them.
  • Image discipline: layers cache by instruction order (dependencies before code), multi-stage builds keep toolchains out of runtime images, distroless/slim bases cut attack surface, pin digests not latest.
  • The reconciliation loop is Kubernetes: every object (Deployment, Service…) is a record in etcd (consensus); controllers watch, diff desired vs. actual, and act; the scheduler is just the controller that assigns pods to nodes; kubelet is the node-side reconciler. Self-healing is nothing more than this loop never stopping.
  • The object model in layers: Pod (co-scheduled containers sharing network/volumes) ← ReplicaSet (N copies) ← Deployment (rolling updates) — plus StatefulSet (stable identity/storage), DaemonSet (one per node), Job/CronJob. Services give pods a stable virtual IP + DNS name with load balancing; Ingress/Gateway handles L7 routing in.
  • Config & operations: ConfigMaps/Secrets injected as env or files; requests/limits drive scheduling and eviction (set requests honestly or the bin-packing lies); liveness vs. readiness probes (confusing them causes restart storms); HPA for autoscaling; RBAC + namespaces for multi-tenancy.
  • The extension pattern: CRDs + custom controllers ("operators") let you declare your domain objects (a PostgresCluster) and reconcile them — the same loop, user-defined; this is why the ecosystem (cert-manager, ArgoCD, Prometheus operator) feels native.
  • To explore: networking model (CNI, kube-proxy/eBPF, why every pod gets an IP), service mesh, GitOps deployment flows, when Kubernetes is overkill (it often is — managed PaaS/serverless first for small estates).

The reconciliation loop as a cycle — self-healing is nothing more than this loop never stopping:

flowchart TD
    E["Desired state declared - every object a record in etcd"] --> W["Controllers watch"]
    W --> D["Diff desired vs actual state"]
    D --> A["Act to converge"]
    A -->|the loop never stops| W
    Sch["Scheduler"] -. "the controller that assigns pods to nodes" .-> W
    K["kubelet"] -. "the node-side reconciler" .-> W

And the object model's containment in one line:

flowchart LR
    Dep["Deployment - rolling updates"] --> RS["ReplicaSet - N copies"] --> Pod["Pod - co-scheduled containers sharing network and volumes"]

Practice

  • Killercoda scenarios (source) — browser-based Kubernetes playgrounds: kubectl drills, broken-cluster debugging, CKA/CKAD-style tasks; teaches the object model with zero setup cost.
  • Image-slimming kata (source) — take a fat single-stage Dockerfile, reorder layers for caching, convert to multi-stage on a distroless base, pin digests, and compare sizes; teaches image discipline measurably.
  • Probe & limits lab (source) — deploy a service with wrong requests/limits and a liveness probe on a slow-start path, watch the eviction and restart storm, then fix both; teaches the config that causes most production k8s incidents.
  • Build your own container runtime (exercise) — implement run from namespaces, pivot_root, and cgroups up; teaches that a container is just a dressed-up Linux process.
  • Kubernetes the Hard Way (exercise) — bootstrap a cluster with no installers: PKI, etcd, control plane, kubelets, pod network, all by hand; teaches what every arrow in the architecture diagram actually is.
  • Write an operator (source) — define a CRD for a domain object of yours and write the controller that reconciles it; teaches the reconciliation loop by making you implement one.

Related