The LLM Engineering Series

There are five serious certifications in LLM engineering, and between them they ask about

the same twenty or so topics. But each one cuts every topic off at a different depth. One

exam wants you to know that a tokeniser splits text into pieces. Another wants you to know

why that splitting makes the model bad at counting letters. A third wants to know what it

costs you per request. So the candidate ends up assembling each topic from five different

sources that use five different names for the same thing — and never quite knows whether

they have the whole picture or a fragment of it.

This book is organised the other way round. The chapter is the topic, not the exam. Each

chapter teaches one subject properly, once, from the ground up — then shows you exactly

where each certification stops, what it calls things, and what it will ask you. Chapter 10

maps every certification back to the chapters it needs, in the order to read them.

It goes about one and a half steps deeper than the exams ask, and stops one and a half

steps short of research. Take the attention mechanism: the exam wants to know what the

query, key and value vectors are for. This book also shows you why the scores are divided

by the square root of the head dimension, and what visibly happens to the output when you

remove that division — because knowing the mechanism is what lets you answer a question

phrased in a way you have never seen, which is what these exams are largely made of.

And everything in it was measured rather than asserted. A working language model is built

from counts in forty lines and then broken, so you can see the exact failure that

transformers were invented to fix. Byte-pair encoding is trained from scratch. Attention,

a full transformer block, and a small decoder are implemented in NumPy and trained — on a

single CPU core, so you can reproduce every number rather than take it on trust. The

measurements that did not cooperate were kept: attention that averaged instead of routing,

an ablation where removing a component changed nothing, a bigger model that generalised

worse. Each of those is more instructive than a clean result, and together they teach the

habit that matters most on these exams and in production: a measurement describes a regime.

What you get:

- 10 chapters covering tokenisation, embeddings, attention, the transformer block, model

families, context windows and the KV cache, decoding and sampling, and the training

lifecycle.

- 61 original practice questions, tagged by certification and by difficulty, where every

option is explained — not just why the right answer is right, but why each wrong answer

is wrong, because on these exams the distractors are where the teaching is.

- A reverse map from each certification to the chapters that serve it, in reading order.

- All the code and experiment scripts, so every table and figure in the book can be

regenerated.

Written to the published objectives of NVIDIA NCA-GENL, NVIDIA NCP-GENL, NVIDIA NCP

Agentic AI, AWS Certified Generative AI Developer – Professional, and Databricks Certified

Generative AI Engineer. Objectives were checked in August 2026; confirm the current

blueprint with the certifying body before you sit.

You need to be able to read Python. That is genuinely all — neural networks, transformers

and the necessary linear algebra are built up from scratch where they are needed.

Every question in this book is original, written from published exam objectives. Nothing is

reproduced from, or based on recollection of, any live examination.

This book was created through a process that combines careful human planning, content direction, and advanced AI technology, followed by thorough refinement and review to ensure a high-quality final work.

Prompting is the part of this field with the worst ratio of advice to evidence. Almost

everyone has a list of techniques that work; almost nobody can tell you why a technique

works, when it stops working, or what it costs. So the same practices circulate as folklore

— say "think step by step", assign the model a role, offer a tip — and when one of them

fails there is nothing to reason from.

The certifications ask about this material in the way folklore cannot answer. They describe

a system that behaves oddly and ask what to change. They give you two prompt designs and ask

which costs more. They describe an output that fails to parse and ask for the fix that

guarantees the format rather than merely improving the odds.

This book is organised by topic rather than by exam. Each chapter explains one mechanism

from the ground up, then shows exactly where each certification stops and what it will ask

you. The final chapter maps every certification back to the chapters it needs, in the order

to read them.

And it measures rather than asserts. Prompting is a behaviour of large models, and the build

environment for this book has no access to one — so instead of assembling other people's

results, it finds out how much of the subject can be built small enough to measure. The

answer turned out to be most of it:

- **In-context learning.** A model reproduces a pattern from the prompt alone at 0.90

accuracy against a chance floor of 0.03 — on a task whose content is regenerated at random

for every example, so the answer provably cannot be in the weights. A model one layer

shallower manages 0.26, which locates the ability in the architecture rather than the data.

- **Chain of thought.** Two identical models, the same task, the same training budget,

differing only in whether they may write intermediate steps: 1.000 against 0.089. The one

that must answer directly never leaves chance at any difficulty.

- **Structured output.** The same trained model decoded two ways. Free decoding never

reaches perfect validity at any training budget tested — twenty-four times the training

moved it from 0.933 to 0.987 and stopped. Constraining the decoder reaches 1.000 always,

discarding about one per cent of the model's probability mass.

- **Hallucination.** A model that answers known questions perfectly answers questions about

subjects it has never seen correctly 2.4% of the time — while reporting 0.786 confidence

in those answers, and 0.781 confidence specifically on the wrong ones. Putting the fact in

the context, with no change to the model, moves it to 1.000.

- **Prompt injection.** One conflicting instruction placed among the data takes a model from

1.000 to as low as 0.06 — and the model that recognises instructions by content, the more

capable one, is the more damaged. Capability and exposure turn out to be the same property.

- **What prompts cost.** The same twenty records in six formats, tokenised with a tokeniser

trained here: a 1.70x spread between the cheapest and most expensive way of writing

identical data. Instructions are nearly free; demonstrations dominate.

There is also a chapter on evaluation, which measures the thing that invalidates most claims

about prompting: eight identical configurations differing only in random seed produced

accuracies from 0.821 to 1.000, and one unchanged model measured on 20 test items returned

anything from 0.650 to 1.000.

Where a result comes from published research rather than this book's experiments, it appears

in a grey box marked "Not measured here", and no chapter rests its argument on one. Two

experiments that failed are reported as failures rather than quietly dropped.

What you get:

- 9 chapters covering prompts as context, in-context learning, chain of thought, structured

output and constrained decoding, hallucination and grounding, prompt injection, prompt

cost, evaluation, and a final chapter of limits.

- 48 original practice questions, tagged by certification and by difficulty, where every

option is explained — not just why the right answer is right, but why each wrong answer is

wrong, because on these exams the distractors are where the teaching is.

- A reverse map from each certification to the chapters that serve it, in reading order.

- All the code and experiment scripts, so every table and figure can be regenerated.

Written to the published objectives of NVIDIA NCA-GENL, NVIDIA NCP-GENL, NVIDIA NCP Agentic

AI, AWS Certified Generative AI Developer – Professional, and Databricks Certified

Generative AI Engineer. Objectives were checked in August 2026; confirm the current

blueprint with the certifying body before you sit.

You need to be able to read Python. That is all — and nothing in this volume assumes you

have read any other book.

Every question is original, written from published exam objectives. Nothing is reproduced

from, or based on recollection of, any live examination.

This book was created through a process that combines careful human planning, content direction, and advanced AI technology, followed by thorough refinement and review to ensure a high-quality final work.

Vector search is the part of this field where the documentation is thickest and the

understanding is thinnest. Every vector database offers a choice of similarity metric, an

index type with three or four tuning parameters, a compression option, and a way to filter by

metadata — and almost none of them tell you what any of it costs, or when a setting that

works quietly stops working.

So the defaults get copied. Cosine similarity because everyone uses it, HNSW because it is

the default, whatever dimension the embedding model happens to produce, and a filter applied

after the search because that is the obvious place to put it. Each of those is right in some

regime and wrong in another, and this book is about knowing which regime you are in.

Books about search benchmark other people's implementations. This one builds them — and that

has a consequence that matters more than any other: **every recall figure in this book is

measured against exhaustive brute-force search over the same vectors.** Not sampled, not

approximated, not taken from a published benchmark. When this book says an index reached

0.9745 recall, that means it returned 9.745 of the true 10 nearest neighbours on average,

where "true" was established by comparing the query against every single vector.

That is possible because vector search is deterministic. There is no training, no random seed

that changes a conclusion, and no behaviour that only appears at scale. This volume needed

cited results less often than any other in the series — three boxes in the whole book.

What gets measured:

- **Why high-dimensional space is strange.** In two dimensions the farthest point in a

collection is 263 times farther away than the nearest. In 1,024 dimensions it is 1.15 times

farther. Every design decision in the rest of the book is a response to that.

- **Why cosine is the default.** Vector norms in a real embedding space span a factor of 15,

and the largest belong to function words — so searching by dot product for *love* returns

*our, my, for, so, thy*, with zero overlap against cosine's results on four of five queries.

After normalisation, cosine, dot product and Euclidean distance rank identically on 300 of

300 queries.

- **What exhaustive search actually costs.** 268 queries per second against 128,000 vectors on

a single core, exactly correct, with nothing to build or tune. It becomes untenable not

because it is slow but because it is unamortisable — which reframes when an index is worth

building at all.

- **What a proximity graph buys.** Recall 0.9745 while examining 590 vectors out of 8,000 —

13.5 times fewer comparisons. And recall 1.0000 while saving only 1.9x, which is the

measurement behind an uncomfortable conclusion: an index tuned for near-perfect recall has

surrendered most of its purpose.

- **What compression costs, and how it is recovered.** Compressing vectors 32-fold dropped

recall to 0.2795. Re-scoring 500 candidates exactly — 2.5% of the collection — brought it

back to 0.9380, with the index unchanged at 8 bytes per vector.

- **When filtering breaks.** Searching first and filtering afterwards held recall 0.9675 at

50% selectivity and collapsed to 0.2000 at 1% — returning fewer results than requested for

**every single query**, silently, with no error raised. At that selectivity, scanning the

matching subset exhaustively is both exact and twenty times cheaper.

What you get:

- 9 chapters covering embeddings, similarity metrics, exhaustive search and its true cost,

dimension, graph indexes, quantisation and re-ranking, filtered search, vector databases in

practice, and a final chapter of limits.

- 48 original practice questions, tagged by certification and by difficulty, where every

option is explained — not just why the right answer is right, but why each wrong answer is

wrong, because on these exams the distractors are where the teaching is.

- A reverse map from each certification to the chapters that serve it, in reading order.

- All the code — every index, every experiment — so each table and figure can be regenerated.

Written to the published objectives of Databricks Certified Generative AI Engineer, NVIDIA

NCA-GENL, NVIDIA NCP-GENL, NVIDIA NCP Agentic AI, and AWS Certified Generative AI Developer –

Professional. Objectives were checked in August 2026; confirm the current blueprint with the

certifying body before you sit.

You need to be able to read Python, and to remember that a dot product multiplies two lists of

numbers pairwise and adds them up. That is the whole prerequisite — and nothing in this volume

assumes you have read any other book.

Every question is original, written from published exam objectives. Nothing is reproduced

from, or based on recollection of, any live examination.

This book was created through a process that combines careful human planning, content direction, and advanced AI technology, followed by thorough refinement and review to ensure a high-quality final work.

Retrieval-augmented generation is the most widely deployed pattern in this field and the one

with the least measurement behind its defaults. Chunk at 500 tokens with 50 of overlap.

Retrieve five passages. Add a reranker. Use hybrid search. Each of those is repeated

everywhere and justified almost nowhere.

The result is a system with several tuning points, no instrumentation, and a single accuracy

number that cannot say which of its two stages is failing. When it works, nobody knows which

choice mattered. When it does not, the fixes are attempted in the order they were heard

rather than the order the evidence supports.

The obstacle to measuring retrieval is knowing what the right answer was — and human

relevance judgements are expensive, subjective and unavailable at the scale a book needs.

This book sidesteps that entirely by building corpora whose correct answers are true by

construction. Queries are drawn as spans from a text, so the chunk containing that span is

correct by definition. Documents describe invented entities, so no retriever can succeed by

prior knowledge, and each fact appears exactly once, so attribution can be checked exactly.

Nothing here is scored against a judgement. Every number is scored against a fact about how

the corpus was built.

Four of the results contradict advice that circulates as settled:

- **Hybrid search made things worse.** Combining a strong lexical retriever with a weaker

dense one lost on every metric in every query condition — and a fusion-weight sweep fell

monotonically with no interior optimum, which turns out to be a reusable diagnostic: a peak

in the middle means fusion is buying something, a monotonic curve means drop the weaker

retriever rather than tune the fusion.

- **Neither reranker beat the retriever** — and, more importantly, none of the differences

exceeded the noise floor, which this book computes before reporting any of them. The same

table, reported without the floor, would support two opposite headlines.

- **Attribution collapses long before retrieval does.** With near-duplicates in the corpus,

the answer was still retrieved 92% of the time while the fraction of claims traceable to a

single source fell from 1.000 to 0.170. A citation feature becomes arbitrary while every

retrieval metric looks healthy.

- **Retrieval scores are a usable abstention signal** — unlike a model's own confidence. On

50 queries about entities absent from the corpus, the highest scored 1.62 against a

threshold at 2.94, with nothing in between.

And two that confirm the advice, with the numbers behind it:

- **Chunk overlap is the cheapest fix in the book.** Ten words of overlap took the rate of

answers cut by a boundary from 10.0% to 0.5%, for a 9% larger index — while chunk size,

the parameter people reach for, costs prompt tokens on every request instead.

- **Retrieval depth trades signal for cost brutally.** Going from one chunk to twenty

multiplied the chance the answer was present by 2.4 and the token cost by 20, ending with

95% of the context irrelevant.

One boundary is stated plainly rather than hidden. A RAG pipeline has two stages and this

book can measure only one directly: retrieval is deterministic and is measured exactly;

generation requires a capable language model the build environment does not have. So

generation is either simulated with a controllable accuracy — which is enough to derive the

arithmetic of a two-stage pipeline exactly — or cited and marked in a visually distinct box.

No claim about what a model does with retrieved context is presented as if measured here.

What you get:

- 8 chapters covering the two-stage pipeline, chunking, lexical and dense and hybrid

retrieval, retrieval depth and reranking, duplication, evaluating two stages separately,

citation and abstention, and a final chapter of limits.

- 42 original practice questions, tagged by certification and by difficulty, where every

option is explained — not just why the right answer is right, but why each wrong answer is

wrong, because on these exams the distractors are where the teaching is.

- A reverse map from each certification to the chapters that serve it, in reading order.

- All the code — every retriever, every experiment — so each table and figure can be

regenerated.

Written to the published objectives of Databricks Certified Generative AI Engineer, NVIDIA

NCP-GENL, NVIDIA NCA-GENL, NVIDIA NCP Agentic AI, and AWS Certified Generative AI Developer –

Professional. Objectives were checked in August 2026; confirm the current blueprint with the

certifying body before you sit.

You need to be able to read Python. Embeddings, lexical scoring, ranking and evaluation are

built here from the ground up, and nothing in this volume assumes you have read any other

book.

Every question is original, written from published exam objectives. Nothing is reproduced

from, or based on recollection of, any live examination.

This book was created through a process that combines careful human planning, content direction, and advanced AI technology, followed by thorough refinement and review to ensure a high-quality final work.

Fine-tuning is the option teams reach for when prompting is not enough, and the one they

understand least. The advice around it is unusually confident and unusually unquantified.

So adaptation is attempted, the model gets better at the new task, and whether it got worse

at everything else is a question nobody measures — because measuring it means deliberately

evaluating a capability you were not trying to improve.

That is the design decision behind this book: **every experiment reports two numbers.** The

domain the model was adapted to, and the general text it started from. The second is the one

production teams do not collect, and it is where the entire cost of adaptation appears.

Nothing here is described rather than built. Low-rank adaptation is implemented from scratch —

frozen base weights, a low-rank pair on every attention projection, initialised so the adapted

model is exactly the base model before training begins. Direct preference optimisation is

implemented from scratch, with a frozen reference model. Quantised training, gradient

accumulation, loss masking and multi-task interference are all measured on the same base

model, so the comparisons differ only in the variable under test.

What gets measured:

- **The trade happens immediately, not gradually.** Probed every fifty steps, the first 8% of

training captured essentially the whole domain gain — perplexity from 19,952,720 to 8.18 —

and inflicted most of the forgetting, taking general perplexity from 256 to 5,111. There is

no stopping point at which the domain is adapted and the general ability is intact.

- **LoRA's reputation does not survive measurement — and neither did my first conclusion.**

The initial sweep showed full fine-tuning beating LoRA on both axes. That sweep tied the

scaling constant to the rank, so two variables moved together. Holding rank fixed and

varying only the scale reversed it: at a low scale LoRA retained general ability *better*

than full fine-tuning, while training 1.68% of the parameters. Both sweeps are in the book,

because the mistake is more instructive than the result.

- **Mixing 12.5% general data into each batch** cost 0.6% of the domain gain and returned

general perplexity from 6,245 to 236 — better than the base model's own 256. No method that

merely restricts the update can end above where it started.

- **Preference training costs a different order of magnitude from demonstration.** Teaching by

comparison reached perfect preference accuracy for 2.8–9.1% of general perplexity, where

supervised adaptation cost 2,339%.

- **A second adaptation erases the first.** A model at accuracy 1.000 on one task scored 0.000

after fifty steps on another — erasure rather than decline, because a discrete learned

behaviour has no partial version. Interleaving the two gave 1.000 on both, and better

general text than either order.

- **Data volume is the wrong question.** Eight examples — one per fact — reached 1.000, and

thirty-two times as much taught nothing further. A quarter of the labels could contradict

the rest with no measurable loss; at half, the failure was per-fact rather than uniform,

which spot-checking misses.

- **Quantisation costs the base model, not the adaptation.** Four-bit base weights cost 8.3%

of what the model already knew and left the adapted result unchanged, because the adapter is

trained afterwards in full precision and fits itself to the quantised base.

What you get:

- 10 chapters covering the adapt-or-retrieve decision, full fine-tuning and forgetting,

parameter-efficient methods, mitigations, instruction tuning, preference training, dataset

design, multi-task interference and evaluation, memory-constrained training, and a final

chapter of limits.

- 54 original practice questions, tagged by certification and by difficulty, where every

option is explained — not just why the right answer is right, but why each wrong answer is

wrong, because on these exams the distractors are where the teaching is.

- A reverse map from each certification to the chapters that serve it, in reading order.

- All the code — the transformer, LoRA, DPO, the quantiser — so every table can be regenerated.

Written to the published objectives of NVIDIA NCP-GENL, NVIDIA NCA-GENL, AWS Certified

Generative AI Developer – Professional, Databricks Certified Generative AI Engineer, and

NVIDIA NCP Agentic AI. Objectives were checked in August 2026; confirm the current blueprint

with the certifying body before you sit.

You need to be able to read Python. Transformers, gradients, low-rank factorisation and

perplexity are built up from the point where they are needed, and nothing in this volume

assumes you have read any other book.

Every question is original, written from published exam objectives. Nothing is reproduced

from, or based on recollection of, any live examination.

Inference optimisation is the part of this field where the techniques are most numerous and the

reasoning behind them least examined. Each one is real, and they are usually applied in the

order someone heard of them — producing a serving system with several optimisations, no

instrumentation, and no way to say which one is helping.

This book measures the underlying ratio first and then measures every technique against it. The

reference machine was clocked at 22.54 GB/s of memory bandwidth and 116.58 GFLOP/s of

arithmetic, giving a ridge point of 5.17 operations per byte. Generating one token performs

0.22 — twenty-three times below it. That single fact explains why decoding is slow, why

batching works, why the cache dominates long contexts, and why a second model can run almost

for free.

Several results are not what the technique's reputation suggests:

- **Unstructured pruning saves nothing.** Zeroing 60% of the weights cost 5% of perplexity and

not one byte of memory, because a dense matrix full of zeros is the same size as one full of

numbers. A sparse format costs positions as well as values and does not pay until roughly two

thirds sparsity.

- **Structured pruning destroyed the model — and that is not the verdict.** Perplexity went from

146.02 to 2569.46, and fine-tuning afterwards brought a model with a *quarter* of its

feed-forward channels back to 162.68, within 11% of the original. Pruning is

prune-then-recover, and the intermediate number should never be evaluated.

- **Batch 2 is slower than batch 1.** A real dispatch discontinuity, kept and explained rather

than smoothed away — the throughput curve is not smooth, and a batch size chosen by reasoning

rather than measurement can land in a hole.

- **An unrelated draft model accepts nothing.** Speculative decoding was verified as producing

output *identical* to greedy decoding, and then measured against a control: an ordinary model

of similar size accepted 0.000 of its proposals at every draft length. "A small fast model" is

an incomplete description of what a draft must be.

- **Tensor parallelism exchanges almost no data** — between 0.002% and 0.036% of the weight

bytes each device reads — and is still confined to one machine, because the all-reduce is a

synchronisation point hit once per block per token.

- **The largest saving is work not done.** A request sharing a 1,024-token prompt prefix with an

earlier one was 72.6x faster, superlinearly, because prefill attention is quadratic.

- **Memory and compute can be pulled apart.** A mixture-of-experts model with 128 experts and

one active per token holds 70.2x the parameters of its dense baseline and performs exactly the

same arithmetic — so "smaller model" and "less work per token" stop being the same statement.

And the ones that confirm their reputation come with the numbers: 8-bit quantisation costs 0.0%

of perplexity at 4x compression; granularity governs the cliff below that, cutting 4-bit damage

from 8.6% to 3.2%; batch 32 gives 3.9x the throughput for 8.2x the per-step latency; and at

131,072 tokens the KV cache is twenty-three times the size of the model.

**On quantisation, and a companion book.** Quantisation has an entire book of its own —

*LLM Quantization: From the Bits Up*, by the same author, 104 pages building the subject from

the representation of a single number upward. Chapter 2 of this volume covers quantisation to

the full depth these certifications require, measured, and refers the derivations, the named

algorithms and bit-level packing onward for readers who want them. You do not need that book to

pass the exams or to use this one; Chapter 2 stands alone.

What you get:

- 12 chapters covering where inference time goes, quantisation, pruning, distillation, the KV

cache and its remedies, speculative decoding, batching and continuous batching, throughput

against latency, multi-device parallelism, prefix caching and chunked prefill,

mixture-of-experts, and a final chapter of limits.

- 66 original practice questions, tagged by certification and by difficulty, where every option

is explained — not just why the right answer is right, but why each wrong answer is wrong,

because on these exams the distractors are where the teaching is.

- A reverse map from each certification to the chapters that serve it, in reading order.

- All the code and measurement scripts, so every table can be regenerated.

This volume covers the two heaviest domains on the NVIDIA NCP-GENL blueprint — Model

Optimization and GPU Acceleration — which together carry 31% of that exam.

Written to the published objectives of NVIDIA NCP-GENL, NVIDIA NCA-GENL, AWS Certified

Generative AI Developer – Professional, Databricks Certified Generative AI Engineer, and NVIDIA

NCP Agentic AI. Objectives were checked in August 2026; confirm the current blueprint with the

certifying body before you sit.

You need to be able to read Python. Attention, the KV cache and the arithmetic of memory traffic

are built up from the point where they are needed, and nothing in this volume assumes you have

read any other book.

Every question is original, written from published exam objectives. Nothing is reproduced from,

or based on recollection of, any live examination.

A deployed model is a model plus four other systems: a queue that decides how long requests

wait, a process lifecycle that decides what a restart costs, a configuration that decides what

the answer looks like, and an input distribution that changes while the model does not.

All four produce the same symptom — it used to work and now it does not — and the model absorbs

the blame because it is the most visible component and the least understood. A team that

responds by changing the model has spent its effort on the one thing that did not change.

This book measures the other four.

- **Waiting time is hyperbolic, not linear.** Raising utilisation from 0.50 to 0.95 — less than

doubling it — multiplied the mean wait by 19. Expressed against the service time: at half

capacity a request waits about as long as it is served, and at 0.95 it waits nineteen times

longer. The overwhelming majority of its life in the system is spent doing nothing.

- **A system under load and a system past capacity look identical at any instant.** Below

capacity the queue drains — at 0.90 utilisation the last quarter of requests waited 0.3x what

the first did. Past capacity it does not: at 1.20 utilisation they waited 4.1x. A dashboard

reporting nine seconds describes both. Only the direction distinguishes them, which inverts

the usual alerting rule: a high number that is falling is fine, and a moderate number that is

rising is an incident.

- **Irreproducibility is a choice.** Greedy decoding with a fixed seed agreed with itself 1.000

of the time. Changing temperature from 0.7 to 0.8 — a tenth of a point — dropped agreement to

0.164: five answers in six became different. "Same model, same prompt, different answer" is

almost always a configuration difference, and recording the seed rather than fixing it costs

a few bytes and makes any past request replayable.

- **Whether a good guardrail threshold exists depends on the signal, not the threshold.** With

well-separated populations one setting caught 93.5% of bad inputs while blocking 6.4% of good

ones. With a weaker signal, catching that same 93.5% required blocking 49.6% of legitimate

traffic. A team that keeps tuning a threshold and keeps being unhappy has diagnosed the wrong

component.

- **Drift detection is a sample-size question.** A shift of 0.1 standard deviations is invisible

in 100 samples and unmistakable in 1,000 — the same shift, the same test, a different amount

of evidence. Sensitivity and detection latency are one parameter seen from two ends.

- **Judging a release costs traffic, quadratically.** Detecting a five-point quality drop takes

685 requests per arm; a one-point drop takes 14,735. A null A/B result means nothing until the

smallest difference the test could have seen is computed.

- **A process is not ready when the file is open.** Loading a model was measured at roughly half

a millisecond per megabyte, of which only about 70% is reading — the rest is the first pass

that makes memory resident. A process can report itself loaded, pass its health check, and

charge a user for the pages nobody touched.

**On what this book measures and what it does not.** This is the most infrastructure-dependent

volume in the series, and it was written on a machine with one CPU core. That constraint was

faced before a single experiment was designed rather than after. Thirteen objectives were listed

from the published blueprints and then triaged: six are measured directly, two are simulated and

validated against closed-form results, and five are reasoned or cited. The ratio is printed on

the cover.

Where the book simulates, it says what that means: generating arrivals and service times from

known distributions and running the queue discipline exactly as a server would. Every simulated

result is checked against theory — the queue matches to three decimal places up to 0.95

utilisation, and where it stops matching, at 0.99, the book reports the discrepancy and explains

it rather than quietly presenting whichever number agrees. A simulation you have validated is

stronger evidence than a measurement you have not.

What you get:

- 9 chapters covering the four systems around a model, queueing and capacity, cold start and the

process lifecycle, configuration and reproducibility, drift, degradation dynamics, guardrails

and fallbacks and cost, releasing a change, and a final chapter of limits.

- 48 original practice questions, tagged by certification and by difficulty, where every option

is explained — not just why the right answer is right, but why each wrong answer is wrong,

because on these exams the distractors are where the teaching is.

- A reverse map from each certification to the chapters that serve it, in reading order.

- All the code and simulation scripts, so every table can be regenerated.

Written to the published objectives of NVIDIA NCP-GENL, NVIDIA NCA-GENL, AWS Certified

Generative AI Developer – Professional, Databricks Certified Generative AI Engineer, and NVIDIA

NCP Agentic AI. Objectives were checked in August 2026; confirm the current blueprint with the

certifying body before you sit.

You need to be able to read Python. Queueing, drift statistics and the arithmetic of thresholds

are built up from the point where they are needed, and nothing in this volume assumes you have

read any other book.

Every question is original, written from published exam objectives. Nothing is reproduced from,

or based on recollection of, any live examination.

Safety is the part of this field with the most confident language and the least visible

arithmetic. A model is described as safe, a filter as effective, an evaluation as thorough — and

almost none of those statements comes with a number attached, or with an account of what the

number would have to be to mean anything.

That is a problem for practitioners, because the claims are made about systems they are

responsible for. It is also what these exams actually test: not whether you believe a system is

safe, but what evidence would establish it, what a clean test result proves, and which failures

a given check cannot see.

This book supplies the measurable substrate. Not the ethics — which are contested and are not a

book's to settle — but the arithmetic underneath:

- **"We reviewed thirty and found no problems" permits a failure rate of one in eleven.**

Bounding a rate below one in a thousand takes 3,000 clean samples. The rule of three — the

bound is about 3/n — makes this arithmetic you can do while somebody is still describing their

testing.

- **A clean red-team run bounds almost nothing.** Three hundred tests against a thousand failure

modes cover 30% and leave 700 unexamined. Nobody knows the size of the space for a language

model, which is a real difficulty and does not excuse omitting it — state it as an assumption

so somebody can disagree.

- **Training has no fairness correction in it.** On a task where the true association is 50/50

by construction, a data gap of 0.80 produced an output gap of 0.804. Reproducing the imbalance

is the *optimum* of the training objective, not a failure to reach it — a model predicting

50/50 on 90/10 data would have higher loss.

- **Perfectly balanced data still produced a gap of 0.026.** That is the noise floor, and a

measured disparity of 0.03 in a real system means nothing until it is compared against one.

- **Ten repetitions is enough to memorise.** A unique seven-token string at 0.332% of a corpus

became fully recoverable from a two-token prompt. At three repetitions it was 60% recoverable

— a real leak that an exact-match test reports as a clean pass.

- **Misuse filtering is beaten by its base rate, not its quality.** A filter separating malicious

from benign requests by three standard deviations blocks 92 benign requests for every malicious

one it catches, when one request in a thousand is malicious. The same filter is fine when one

in ten is. Filters evaluated on balanced test sets overstate production performance by a factor

the test set does not contain.

- **"A human reviews the output" describes three different controls.** Full review prevents;

sampled review measures; escalated review inherits the flagging mechanism's blind spots. A

document that does not say which is describing a control nobody has characterised.

**On the constraint that shaped this book.** It would be possible to study refusal behaviour by

attempting to elicit harmful output, and bias by generating stereotyped text about real groups.

This book does neither. Every experiment measures a *mechanism* — how a known imbalance

propagates, how many repetitions make a string recoverable, what fraction of a space a test set

covers, what a review sample bounds — on synthetic, neutral tasks where the ground truth is exact

by construction.

That costs something and is stated openly: refusal behaviour is cited rather than measured. It

also buys something. Because the true answer is known by construction, "bias" has an exact

definition here rather than a contested one, and the noise floor of a perfectly fair setup can be

measured — which is the number every fairness claim should be compared against.

**On what is measured and what is not.** Ten objectives were listed from the published blueprints

and then triaged before any experiment: six are measured, two are reasoned, two are cited. The

ratio is printed on the cover. A book about safety could very easily imply more evidence than it

has, and on this subject in particular, overclaiming is itself a safety failure.

What you get:

- 8 chapters covering what a safety claim must contain, bias propagation and noise floors,

memorisation and privacy, red-team coverage, human oversight, the obligations that are not

measurements, misuse and base rates, and a final chapter of limits.

- 42 original practice questions, tagged by certification and by difficulty, where every option

is explained — not just why the right answer is right, but why each wrong answer is wrong.

- A reverse map from each certification to the chapters that serve it, in reading order.

- All the code, so every table can be regenerated.

Written to the published objectives of NVIDIA NCP-GENL, NVIDIA NCA-GENL, AWS Certified

Generative AI Developer – Professional, Databricks Certified Generative AI Engineer, and NVIDIA

NCP Agentic AI. Objectives were checked in August 2026; confirm the current blueprint with the

certifying body before you sit. Regulatory frameworks are described as of that date and this

book does not give legal advice.

You need to be able to read Python. The statistics — confidence bounds, coverage, noise floors —

are built up from the point where they are needed, and nothing in this volume assumes you have

read any other book.

Every question is original, written from published exam objectives. Nothing is reproduced from,

or based on recollection of, any live examination.

This volume exists to repay a design decision.

The eight volumes before it each teach one domain and none refers to any other, so a reader who

buys one gets a complete book rather than a fragment. The cost of that design is the connections

between domains — and the connections are exactly what the certifications test.

Three full-length practice exams, 150 questions, 150 explanations. Every question crosses at

least two domains by construction.

**What is checkable, and checked.** A practice exam whose domain mix does not match the

blueprint trains attention in the wrong proportion. That mix is arithmetic, so it is verified

here rather than claimed:

- **Exam One** — 60 questions to NVIDIA NCP-GENL weighting. Worst deviation from the published

blueprint: 1.0 percentage point.

- **Exam Two** — 50 questions to a blend of NVIDIA NCA-GENL and Databricks weighting. Worst

deviation: 0.0.

- **Exam Three** — 40 questions to a blend of AWS Certified Generative AI Developer and NVIDIA

NCP Agentic AI weighting, with the heaviest emphasis on agent behaviour. Worst deviation: 0.0.

Chapter 1 shows the allocation and the code that produces it, so a reader can rebuild the exams

against a revised blueprint rather than discarding the book when one changes.

**What is not checkable, and is said plainly.** Whether these questions resemble real ones in

phrasing and difficulty cannot be verified by anyone who has not sat the exam, and this book was

written entirely from published objectives. So it prints **no pass mark** — a pass mark would

imply a calibration nobody has established.

What the book does support is comparison within itself. Chapter 5 scores by domain rather than

by total, because a per-domain breakdown survives any mismatch in difficulty: it holds the

instrument constant. A total tells you whether you passed a practice exam; a breakdown tells you

which volume to reread, which is the only actionable output.

**What this book is not.** It is not a reprint — the 427 questions in the other volumes are not

repeated here, and that was verified mechanically against every one of them rather than assumed.

It is not new teaching: no mechanism is explained here for the first time, and where an

explanation needs one it names the volume that measures it. This is the one book in the series

that refers to the others, because connecting them is its entire function.

**Why the distractors are harder here.** On a single-domain question the wrong options are

misconceptions about that domain. On a cross-domain question they are frequently *correct

answers to the wrong domain's version of the question* — an accurate statement about chunking

offered for a problem that is a decode-side cost. The skill being tested is not knowing each

domain but knowing which domain a symptom belongs to, and every explanation names the domain

each wrong option belongs to.

What you get:

- 3 full-length exams: 60, 50 and 40 questions, each allocated to a verified blueprint mix.

- 150 explanations, each naming the volume that measures the mechanism and identifying which

domain every wrong option belongs to.

- Scoring sheets by domain for all three exams, and four result patterns with what each calls

for.

- A chapter on how the exams were constructed, including the allocation code.

You will get little from this book without the material behind it. The explanations assume the

mechanisms rather than teaching them, and a reader working these exams cold will score badly and

learn less than the score suggests.

Written to the published objectives of NVIDIA NCP-GENL, NVIDIA NCA-GENL, AWS Certified

Generative AI Developer – Professional, Databricks Certified Generative AI Engineer, and NVIDIA

NCP Agentic AI. Weightings were taken from the published guides in August 2026; confirm the

current blueprint with the certifying body before you sit.

Every question is original, written from published exam objectives. Nothing is reproduced from,

or based on recollection of, any live examination.