Build C++: Compilers, Databases & LLM Engines

Build an LLM Inference Engine in C++ — Through Challenges

You don't truly understand how large language models run

until you've built the engine yourself.

This book takes you from a blank C++ project to a complete,

working inference engine that loads a real Llama-family model

and generates text — one challenge at a time.

What you'll build:

- A strided tensor system with zero-copy views and arena allocation

- Math kernels: RMSNorm, SwiGLU, softmax, GEMM with SIMD

- A byte-level BPE tokenizer

- A full Transformer: RoPE, GQA, Flash Attention, KV Cache

- int8/int4 quantization with direct block multiplication

- GGUF model loading with mmap

- Sampling, streaming, speculative decoding, and continuous batching

- An optional CUDA capstone for the heaviest kernels

By Unit 14, the engine runs a real model on CPU. Every concept

earns its place right after you've built the thing it improves.

Companion Source Code

The complete source code for the book is available on GitHub:

https://github.com/Hatem-M-lab/llm-inference-engine

Development Methodology

This book was created through a process that combines careful human planning, content direction, and advanced AI technology, followed by thorough refinement and review to ensure a high-quality final work.

Who this is for:
C++20 developers comfortable with algorithms

and memory layout who want to understand what actually happens

inside an LLM runtime — not by reading, but by building.

Most compiler books start with a lexer, and nothing runs for chapters. This one starts with twelve bytes of machine code.

In the first hour you encode an x86-64 program by hand, wrap it in an ELF executable by hand, and run it. No assembler and no linker touches it. Then you climb, one layer per unit, until you have a complete compiler for a language called Slate: an assembler, relocatable object files the system linker accepts, a machine IR, two register allocators, SSA, an optimiser, a lexer, a parser, a type checker, a one-command driver, and DWARF line tables that let `gdb` step through Slate source one line at a time.

The order is the point. Because the back end comes first, every stage you write later has a real machine underneath it, and when the front end arrives in Unit 13 it already has somewhere to send its output.

This book was created through a process that combines careful human planning, content direction, and advanced AI technology, followed by thorough refinement and review to ensure a high-quality final work.

**Nothing is taken on trust.** The byte counts, addresses, sizes and exit statuses in the book come from running the code, and a five-layer verification suite stands behind them: hand-written encodings assembled independently by `nasm`, nearly every transcript re-executed, every "break it and watch it fail" experiment actually performed, every listing compared byte for byte with the file it came from, and random-program oracles that compile what nobody wrote. The repository's fault log lists 171 real mistakes found while the book was written, and what caught each one.

The last unit is the proof. A 270-line ray tracer is written twice, in Slate and in C. Your compiler builds one, `gcc` builds the other, and forty-one scenes must come out identical: the same picture to the byte, and the floating-point numbers behind every pixel the same to the bit.

What you get

- 19 units in five parts, more than 400 pages laid out for the screen

- More than 130 code listings, each with its real file path and line numbers, and more than 200 terminal transcripts

- About 22,000 lines of C++23 you can build, run and break, with the verification suite that checks them

- Linux on x86-64 and the usual toolchain (g++ 13+, nasm, binutils, gdb); no framework, no package manager, no dependencies

Build a working SQL database engine in C++20 -- from an empty directory to

a query processor that runs real SQL against data on disk and survives a

crash at any write -- through a single relentless method: nothing is

asserted; everything is demonstrated.

Every data structure in this book is built, compiled, and run. Every

performance claim is a table printed by a benchmark whose source code is on

the page in front of you. Every design decision is followed by the

measurement that justifies it -- and, where the design has a cost, by the

measurement that exposes that cost too. And once per unit, something fails

in front of you: a real bug, reproduced deterministically, diagnosed from

the evidence, fixed, and locked shut with a regression test.

This is not a survey of database theory. It is a lab manual. You will not

find hand-waving about how B+Trees are "generally logarithmic" -- you will

find the fan-out arithmetic, the page-count math, and a benchmark that

walks a tree of a million keys and prints the real number.

THE COMPLETE BOOK: fourteen units in four parts.

PART I -- STORAGE

1. Slotted Pages & the Pager -- self-describing 4 KiB pages, records with

stable slot ids, a pager with a free list

2. B+Tree: Insert & Search -- logarithmic lookup, proven against a

million-key tree

3. B+Tree: Delete & Range Scans -- rebalancing, merges, ordered range

queries

4. The Buffer Pool -- a real cache with clock eviction, measured 2.5x

faster with identical logical work

PART II -- FROM BYTES TO A QUERY

5. The Record Layer -- typed rows: a schema-aware codec and a table heap,

reached by key through the index

6. The Catalog -- persistent schemas and named tables: the engine's

self-knowledge

7. The Front End -- a SQL tokenizer and recursive-descent parser, with

compiler-quality caret-pointed errors

8. The Executor -- the milestone: a real executor that runs CREATE TABLE,

INSERT, and SELECT end to end, against a database of hundreds of

thousands of rows

PART III -- MAKING IT FAST & REAL

9. Joins -- the executor rebuilt as a pipeline of operators, JOIN ... ON

with aliases and qualified columns, and two join algorithms raced:

at 100,000 rows a side the hash join is 1,293x faster than the

nested loop, measured

10. Aggregation -- COUNT, SUM, AVG, MIN and MAX with SQL's NULL rules,

GROUP BY and HAVING through a hash table, ORDER BY and LIMIT; the

top 10 of a million rows costs one pass over them, and a GROUP BY's

cost per row, measured, grows tenfold from 10 groups to a million

11. Indexes & the Planner -- ranges of the key, CREATE INDEX on a B+Tree

that keeps duplicate keys, and a planner that chooses scans, indexes

and join algorithms by costs measured on its own machine; an index

loses, measured, past 55% of a cached table and 16% of one that is

not, and Unit 10's questions, replanned, run up to 12,000x faster

12. Vectorized Execution -- rows read where they lie in the buffer pool,

and every operator rebuilt to move them 1,024 at a time as columns: a

scanned row, measured, from 116 ns to 11, Unit 9's pipeline twelve

times faster, and an index's break-even moved from half a table to a

sixth

PART IV -- DURABILITY & CAPSTONE

13. Write-Ahead Log & Crash Recovery -- every page through a log of

checksummed page images, every statement a transaction, BEGIN, COMMIT

and ROLLBACK; the engine crashed at each of the 804 writes and syncs

of a workload, as a dying process and as a power cut, and by 300

kill -9s -- and every time it held exactly what it had acknowledged

14. Capstone -- the engine becomes a program; 20,000 generated statements

run in labdb and in PostgreSQL with not one different answer, and

eight bugs planted to test that comparison, all eight caught; then the

same work beside SQLite, in one process: faster on ten of thirteen

questions, and a file nearly three times the size, explained page by page

By the last page, the engine answers SQL it parsed from text, against rows

it stored on disk, through indexes it built, a cache it manages and a log

that brings it back after a crash -- and its answers are PostgreSQL's. Every

number in the book came from actually running that code.

Reference machines: g++ 13.3.0, Ubuntu 24.04, C++20, stdlib + POSIX only

(Unit 14 also uses PostgreSQL and SQLite, when installed, as the references

it compares against). Every benchmark ships with the exact command that

produced it, so you can run it yourself and get the same structural numbers.

Concurrency and MVCC are not in this book -- labdb is single-threaded by

design, and the book says so plainly rather than promising a chapter it has

no intention of writing in a hurry.

Development Methodology
This book was created through a process that combines careful human planning, content direction, and advanced AI technology, followed by thorough refinement and review to ensure a high-quality final work.

Source Code:

https://github.com/Hatem-M-lab/labdb