Understanding Large Language Models from the Ground Up

This bilingual open textbook explains how language models work, from observable responses to the calculations behind them. Part I assumes no programming background. Later practical chapters introduce the Python, mathematics, and engineering needed to build models and applications.

All 57 chapters and both primers are available

All 57 chapters and the Python and HTTP primers are available in English and Chinese. Chapters 40–57 and the HTTP primer were released on September 15, 2026. Chapters 5–39 and the Python primer retain September 13, and Chapters 1–4 retain their September 10 first-edition date. Examples, diagrams, calculations and exercise solutions are directly readable on the page.

• Chapter 1: judge a response against the task and its evidence, and distinguish a model from the complete assistant.
• Chapter 2: follow how a response is generated and how training changes the model.
• Chapter 3: turn text into pieces, IDs, and vectors, then examine what those representations preserve.
• Chapter 4: build a small language model by counting, calculate its predictions, and test its limitations.

Continue through the published parts:

• Part II, Chapters 5–10: build the mathematical and programming foundations, then compare sparse and neural baselines. Read the Python primer after the core of Chapter 5.

• Part III, Chapters 11–14: follow sequence memory, recurrent models and attention.

• Part IV, Chapters 15–21: work through a Transformer and build, train and examine a Mini GPT.

• Part V, Chapters 22–33: connect model design and operation to measured evaluation and model selection.

• Part VI, Chapters 34–39: distinguish training objectives, runtime computation and parameter adaptation, then choose changes from failure evidence.

• Part VII, Chapters 40–46: connect context, structured answers, tools and retrieval into a cited knowledge assistant. Read the HTTP primer before Chapter 42’s practice.

• Part VIII, Chapters 47–53: follow workflow state, permissions and recovery, then evaluate and govern the complete system.

• Part IX, Chapters 54–57: turn the knowledge assistant into a local service, examine versions and failures, and evaluate a capstone evidence package.

The full roadmap links the complete core sequence. The optional multimodal extension remains planned and unpublished. Companion services are local demonstrations; publishing the book does not deploy them.

Help improve the next edition

Where did the explanation lose you? Which example helped, and where was a step missing? Share the page and passage through reader feedback. You do not need to know the correction to describe a difficulty.

For a specific error, use the errata and feedback page. Source text and accompanying materials are available on GitHub.