Modern LLMs

Modern LLMs is an engineering reader on the complete lifecycle of a language model—from data, tokenization, and architecture to pretraining, post-training, efficient inference, evaluation, and applied systems.

This is not a catalogue of model names. The goal is to provide a durable framework for reading model cards and technical reports, checking architectural arithmetic, estimating memory and compute costs, and separating reusable mechanisms from dated market snapshots.

The book covers:

• the modern LLM landscape, a Transformer reference route, tokenization, embeddings, and positional mechanisms including RoPE;

• MHA, MQA, GQA, and MLA attention, normalization, activations, block design, and Mixture of Experts;

• SSM/Mamba, hybrid and diffusion language models, data pipelines, scaling laws, optimizers, and pretraining;

• KV caches, prefill and decode, PagedAttention, speculative decoding, and quantization;

• SFT, RLHF, DPO, RLOO, GRPO, reasoning models, and test-time compute;

• multimodal LLMs, embeddings and retrieval, RAG, agents and tool use, evaluation, and safety.

Formulas are connected to architectural and numerical checks throughout the text. The emphasis is on mechanisms, assumptions, trade-offs, failure modes, and reproducibility—not on presenting educational implementations as production-ready systems.

The book is written for ML, AI, and LLM engineers, researchers, graduate students, and developers who already use language models and want to understand what happens below the API layer. Basic familiarity with machine learning, Python, tensors, and probability is helpful.

This Leanpub edition contains the open lecture texts and illustrations. The paid Stepik course adds a structured learning path, more than 500 auto-graded tests, exercises and self-check activities, detailed solutions, tested notebooks, and a capstone in which you build a compact MiniLLM from data and byte-level BPE through post-training, inference, and controlled ablations.

Modern LLMs is part of an evolving series of standalone courses and readers on ML and LLMs. Each volume is self-contained; cross-references provide optional depth.

Complete Stepik course: https://stepik.org/a/294814