Compute-bound?
- GPU FLOPs saturated?
- Long prompts in prefill?
- Big batches?
“TTFT is high even when the queue is empty.”
AI infrastructure · interactive field guide
Watch an LLM
system think.
Type a prompt and watch it become tokens, fill a KV cache, and stream back one token at a time. Then follow that same request through GPUs, routers, vLLM and Kubernetes until you can explain every hop in an interview.
Try it: watch your own prompt
Try it · your prompt, step by step
Type anything. Watch prefill fill the KV cache, then decode read it.
idle
Time to first token—
Inter-token latency—
KV cache size0 B
Token passes (work)0
Tokensprompt → answer
Why#29829
·is#18783
·my#67377
·Kub#97823
ernetes#43929
·pod#21266
·stuck#97129
·in#82688
·Pending#63298
?#47258
The#73736
·pod#21266
·is#18783
·Pending#63298
·because#89769
·no#82664
·node#80211
·has#7953
·a#88398
·free#45657
·GPU#73211
⟨eos⟩#2
Model4 layers
KV cacheK | V per layer
Attentionwhat the query reads
GPU compute (math)
Memory bandwidth (reading weights + KV)
KeyValueQuery / attention readOutput token
Arrive
A request arrives: “Why is my Kubernetes pod stuck in Pending?”. A GPU can’t read text, so the words must become numbers first.
Tokenizer, timings and answers are a teaching simulation. The KV size uses Llama-3.1-8B in BF16 (128 KiB per token).
Your route through the atlas.
Go top to bottom, or jump in. Each page mixes narrated animations, simulators you can break, and drag-and-drop checks. Mark lessons as understood and your progress is saved in this browser.
Beginner
No background needed. What actually happens when you press Enter.
Intermediate
Scaling, optimizing and running it on a cluster.
02 / SCALE OUT30 min
Scale out
Split models across GPUs with TP, PP, DP, EP and CP, route requests intelligently, and see NCCL move data.
Play with: will-it-fit · router simulator · collectives
03 / OPTIMIZE35 min
Optimize
PagedAttention, prefix caching, chunked prefill, P/D disaggregation, speculative decoding, quantization and KV tiers.
Play with: memory pager · hash chain · token budget
04 / PLATFORM35 min
Kubernetes platform
vLLM and llm-d on Kubernetes: gateway, endpoint picker, pods, GPU scheduling, autoscaling and observability.
Play with: request walk · YAML explorer · incident game
Advanced
Frontier workloads, the vLLM engine itself, and keeping GPU clusters healthy.
05 / FRONTIER25 min
Frontier serving
Mixture-of-experts, multimodal pipelines, RL rollout loops and agentic sessions with 96% cache hits.
Play with: expert router · agent session sim
06 / VLLM DEEP DIVE45 min
Inside vLLM
Engine internals from the vLLM Office Hours: scheduler, CUDA graphs, KV connectors, wide EP, capacity maths.
Play with: scheduler stepper · GPU calculator · serve builder
07 / K8S FOR GPUS40 min
Kubernetes for GPU workloads
Keep GPUs scheduled, patched and serving: the four states of GPU capacity, why pods stay Pending, quota gates, and node rotation without an outage.
Play with: Pending debugger · quota gates · node-rotation simulator
Prove it
Test yourself and get interview-ready.
08 / INTERVIEW KITongoing
Interview kit
Staff-level answers for 20 core topics, 63 vLLM questions with self-rating, and numbers to memorize.
Play with: practice mode · flashcards
09 / PLAYGROUND20 min
Playground
Drag-and-drop checks: build the request path, sort concepts, diagnose bottlenecks, match fixes, explore the map.
Play with: 5 drag-and-drop games
10 / GLOSSARYreference
Glossary
Every term in plain English, with a link to the lab where you can see it working.
Play with: search
One request, top to bottom.
Every layer of the stack does one job for your request. Press play to trace it down the stack, or click any layer.
Trace · the full serving stack
Where does my request go?
Layer 1/11
Workload — Everything starts with the traffic shape: how long are prompts and answers, how much is shared, and how many turns does a session take?
Something is slow? Ask five questions.
An interview framework that shows you reason from workload → bottleneck → architecture → trade-off instead of listing buzzwords. Practise it in the diagnosis game.
Memory-bound?
- KV cache full?
- Preemptions?
- HBM bandwidth in decode?
“Stream slows as more users join; preemptions climb.”
Communication-bound?
- NCCL all-reduce time?
- All-to-all for experts?
- Cross-node links?
“Throughput dropped when the model moved from 1 node to 2.”
Scheduling-bound?
- Queue depth?
- Bad placement?
- Lost KV locality?
“GPUs look fine but P99 TTFT spikes and hit rate fell.”
Workload-bound?
- Context length?
- Output length?
- Agent loops, multimodal, RL?
“The traffic changed shape, not the system.”