Foundations of Large Language Models

There are five serious certifications in LLM engineering, and between them they ask about

the same twenty or so topics. But each one cuts every topic off at a different depth. One

exam wants you to know that a tokeniser splits text into pieces. Another wants you to know

why that splitting makes the model bad at counting letters. A third wants to know what it

costs you per request. So the candidate ends up assembling each topic from five different

sources that use five different names for the same thing — and never quite knows whether

they have the whole picture or a fragment of it.

This book is organised the other way round. The chapter is the topic, not the exam. Each

chapter teaches one subject properly, once, from the ground up — then shows you exactly

where each certification stops, what it calls things, and what it will ask you. Chapter 10

maps every certification back to the chapters it needs, in the order to read them.

It goes about one and a half steps deeper than the exams ask, and stops one and a half

steps short of research. Take the attention mechanism: the exam wants to know what the

query, key and value vectors are for. This book also shows you why the scores are divided

by the square root of the head dimension, and what visibly happens to the output when you

remove that division — because knowing the mechanism is what lets you answer a question

phrased in a way you have never seen, which is what these exams are largely made of.

And everything in it was measured rather than asserted. A working language model is built

from counts in forty lines and then broken, so you can see the exact failure that

transformers were invented to fix. Byte-pair encoding is trained from scratch. Attention,

a full transformer block, and a small decoder are implemented in NumPy and trained — on a

single CPU core, so you can reproduce every number rather than take it on trust. The

measurements that did not cooperate were kept: attention that averaged instead of routing,

an ablation where removing a component changed nothing, a bigger model that generalised

worse. Each of those is more instructive than a clean result, and together they teach the

habit that matters most on these exams and in production: a measurement describes a regime.

What you get:

- 10 chapters covering tokenisation, embeddings, attention, the transformer block, model

families, context windows and the KV cache, decoding and sampling, and the training

lifecycle.

- 61 original practice questions, tagged by certification and by difficulty, where every

option is explained — not just why the right answer is right, but why each wrong answer

is wrong, because on these exams the distractors are where the teaching is.

- A reverse map from each certification to the chapters that serve it, in reading order.

- All the code and experiment scripts, so every table and figure in the book can be

regenerated.

Written to the published objectives of NVIDIA NCA-GENL, NVIDIA NCP-GENL, NVIDIA NCP

Agentic AI, AWS Certified Generative AI Developer – Professional, and Databricks Certified

Generative AI Engineer. Objectives were checked in August 2026; confirm the current

blueprint with the certifying body before you sit.

You need to be able to read Python. That is genuinely all — neural networks, transformers

and the necessary linear algebra are built up from scratch where they are needed.

Every question in this book is original, written from published exam objectives. Nothing is

reproduced from, or based on recollection of, any live examination.

This book was created through a process that combines careful human planning, content direction, and advanced AI technology, followed by thorough refinement and review to ensure a high-quality final work.