Goal
Every path in this curriculum assumes AI-assisted work; this one makes the assistance itself an engineering subject. The goal is a sound mental model of LLMs (capabilities and failure modes), the systems layer over them (retrieval, tools, the agent loop) built once with no framework, and the eval discipline that separates engineering from demo-ware. Short path, deliberately: the field churns, but these fundamentals are the stable core.
Prerequisites
- Foundations; software craft module 1 — evals are TDD wearing a lab coat.
Modules
1. The working model
- Read: LLM fundamentals for engineers.
- Do: the note's drills — tokenizer archaeology, sampling-parameter sweeps, a failure-mode gallery collected from your own usage.
- Proof: you can predict which kinds of task a model will flub before running it, and explain temperature to a colleague without hand-waving.
2. The agent, from first principles
- Read: Agents, RAG & tool use.
- Do: Build an agent from scratch.
- Proof: the subject's gates — raw HTTP to the API, schema-validated tools, a bounded loop, full tracing, the scored eval set green.
3. Retrieval, measured
- Read: the same concept note's RAG half, now with the agent build behind you.
- Do: RAG over the knowledge base — over this very repository.
- Proof: the retrieval eval set's hit-rate target met, and the bottleneck (chunking? retrieval? generation?) identified from evidence, per the subject.
Related
- The path to excellence — the master path; Stage 5, as the force multiplier on everything before it.
- Software craft — the eval discipline is its testing culture transplanted.
- Distributed systems & data — retrieval pipelines are data systems and inherit their failure modes.
- Theme drawn on: AI Engineering.