Nick Gustafson
I'm a software engineer and data scientist in DC. This blog is where I write about stuff and share things I find interesting. More about me →
The Series
Holding the LLM Stack in Your Head
A dependency-ordered walk through the full LLM stack. Twelve arcs, about a hundred posts. The goal isn't rigor, it's intuition that survives contact with real systems.
Start at 1.1 and let every post earn the next one. Twelve arcs, about a hundred posts, dependency-ordered.
The whole series as a dependency graph. Find the concept you need and walk its chain backwards.
Forty interactive pieces pulled out of their posts. No reading required — drag things, press play.
A few good first posts
Vectors as lists of activations, matrix multiplication as a linear map, and why every neural network operation bottoms out in matmuls.
Queries, keys, and values as learned linear projections, how scaled dot-product computes soft token-to-token lookups, and exactly why we divide by sqrt(d_k).
Walking through the exact matrix operations at each decode step to see why generation is memory-bound, not compute-bound.
Deriving the key-value cache by noticing which attention computations are redundant, then watching memory grow linearly with sequence length.
Tracing a query from ingestion through retrieval, reranking, prompt assembly, and generation — covering naive RAG, multi-hop, and agentic RAG patterns.
Dissecting the core execute-observe-decide cycle that every agent framework implements, including error handling and retry logic.