Front Matter

Preface

This book is a field-first mastery text on modern ML systems — a holy-grail reference for interview preparation and deep production work. It covers ML foundations, training and fine-tuning, inference internals, production serving, retrieval and RAG, agents, distributed systems, data plane, observability, build/deploy/operate, and a complete ML system design interview playbook.

The ML and inference track (Parts I–III) ramps Beginner → Expert → PhD → Production. Each layer is built on the one beneath it, so the production choices read as the natural answer to the research-level questions. The systems and MLOps tracks (Parts IV–IX) assume a strong generalist baseline and go deep on tradeoffs, design rationale, and "why this and not that." Part X is the capstone interview playbook.

Every chapter ends with Read it yourself (curated references) and Practice (exercises including a stretch problem). Use them.

Reading paths

The mastery path (slowest, deepest)

Front to back, every chapter. The "I want to be the best" path.

The interview-prep path (focused)

56 chapters that cover what interviewers actually test. Start here if you have 2–4 weeks.

Part III — Inference
21 Prefill vs decode: the two-phase nature of LLM inference 22 The KV cache: the single most important optimization in LLM inference 23 Batching: static, dynamic, continuous (Orca) 24 PagedAttention and vLLM as a virtual-memory system for KV cache 25 FlashAttention and the GPU memory hierarchy 26 Quantization: INT8, INT4, FP8, AWQ, GPTQ, SmoothQuant 30 Cost modeling for inference: tokens, GPUs, dollars 31 Latency budgets, tail latency, and the p99 problem 33 The attention compression family: MHA, MQA, GQA, MLA 34 Mixture of Experts: routing, balancing, and the inference cost story 36 Disaggregated prefill/decode: production reality with workload-dependent payoff 44 The serving framework landscape: vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, MLC, Triton 45 Inference servers and orchestration: KServe, BentoML, Seldon, Ray Serve, Triton Inference Server
Part X — Interview Playbook
114 The interview framework: clarify, estimate, design, drill, ops 115 Capacity planning math: the back-of-envelope kit 116 Design a chatbot for 1M users 117 Design a RAG system over 10TB of documents 118 Design a content moderation pipeline 119 Design a real-time recommendation system 120 Design a multi-tenant model serving platform 121 Design a multi-tenant fine-tuning service 122 The vocabulary that interviewers respect 123 Common mistakes and how to recover mid-interview 124 The coding interview: twenty ML systems algorithms 125 Design: the upstream platforms 126 Design: the production infrastructure 127 Design: the frontier scenarios 128 Behavioral interviews and the levels ladder 129 The day of the interview 130 Company-specific prep and mock interview transcripts

Don't skip: Appendix E — 530+ interview questions organized by topic, difficulty, and role.

Conventions

  • Code snippets are in Python or pseudocode unless the topic is language-specific.
  • Code examples are real, lifted from production patterns and adapted — names stripped, identifying paths removed.
  • Diagrams use inline SVG and Mermaid — they adapt to light/dark mode automatically.
Begin Chapter 1