rgoussu.dev
rgoussu@goussu: ~/library/security
❯ ~/library/security cat secrets-management.md

Secrets management

# Keeping credentials out of code and alive in production — vaults, rotation, short-lived credentials, and the leak-response playbook.

Conceptsaved 2026-08-08 #secrets#security#credentials#devsecops

Overview

Every system holds secrets — database passwords, API keys, signing keys, TLS private keys — and the default failure mode is depressingly consistent: committed to a repo, baked into an image, or living unrotated for years in an env file. Secrets management is the discipline of giving workloads the credentials they need without humans ever seeing them, and making every credential short-lived enough that leaking one is an incident, not a catastrophe.

Key points

  • Never in the repo, never in the image: git history is forever (a removed secret is still a leaked secret — rotate, don't just delete); secret scanning (gitleaks, GitHub push protection) as a pre-commit and CI gate.
  • The hierarchy of maturity: env vars from a deploy system → encrypted-at-rest secret stores (Kubernetes Secrets + KMS, SOPS) → a real secrets manager (Vault, cloud secret managers) with access control, audit logs, and dynamic issuance.
  • Short-lived beats encrypted: dynamic secrets (Vault issuing per-workload database credentials with a TTL) and workload identity federation (pods/VMs exchanging their platform identity for cloud tokens — no stored keys at all) beat any static secret, however well vaulted.
  • Rotation must be designed in: dual-secret acceptance windows so rotation is a routine no-downtime operation; if rotating a credential is scary, that's the finding.
  • Scope and audit: one credential per consumer (blast radius + attribution), least privilege per credential, audit trail on access; shared "the API key" in a team vault is an anti-pattern.
  • Leak response: assume exploited — rotate immediately, then hunt usage in audit logs, then fix the path it leaked through. Speed of rotation is the metric that matters.
  • To explore: envelope encryption & KMS key hierarchies, sealed secrets for GitOps, SPIFFE/SVID for service identity, HSMs for root keys.

Practice

  • gitleaks (source) — scan a repo's full history for leaked secrets and wire it as a pre-commit hook; feel why "removed" ≠ "not leaked".
  • GitHub secret scanning & push protection (source) — enable the CI gate on a repo and watch it block a planted credential before it lands.
  • sops + age (source) — encrypt secrets at rest inside a repo with age keys and decrypt only in the deploy step — the GitOps-safe pattern.
  • Vault dev-server lab (source) — run a dev server, then issue dynamic, TTL'd database credentials — short-lived beats vaulted, made concrete.

Related