vLLM & SGLang: From First Request to Production

Your model works. Now it has to serve real users without burning your GPU budget.

vLLM and SGLang are two of the most widely used open-source engines for serving open-weight language models. Both are fast. Both have dozens of settings. And much of the advice online is old, copied from version to version, or based on tests that are not fair.

This book gives you a better way: measure first, then tune, using your own numbers.

You start with what happens to one request inside the server. Then you build up, chapter by chapter, to a complete project: a written target, a fair benchmark of both engines on the same GPU, tuning, a container, a Kubernetes file, monitoring, and a cost per million tokens you can defend in a meeting.

What you will be able to do

* Explain prefill, decode, and the KV cache, and do the memory math before you rent a GPU

* Understand how PagedAttention (vLLM) and RadixAttention (SGLang) work, and when each one helps

* Serve a model with either engine: streaming, structured JSON output, tool calls and LoRA adapters

* Benchmark fairly with TTFT, TPOT, p99, throughput and goodput, and spot the mistakes that make a result look better than it is

* Tune one setting at a time and keep a log that proves what helped

* Use FP8 and 4-bit formats, multi-GPU and mixture-of-experts serving, and speculative decoding

* Package, deploy and monitor the server, and turn tokens per second into a price

Real numbers, not guesses

Chapter 13 includes a complete benchmark of both engines on the same GPU, with exact versions, settings, tables and a chart. It also shows the traps met on the way, such as a default setting that quietly limited one engine's throughput. You get the method and a repeatable script, so you can run the same test on your own hardware.

Made to be learned from

The book is written in plain English. Key terms are explained when they first appear, with simple diagrams and a kitchen analogy for beginners. "Going deeper" boxes give engineers the internals and the trade-offs. Every chapter has a summary, 5 exercises and a 5-question quiz, with answers at the back. You also get a one-page cheat sheet, a glossary and a code index.

Free code

All the code is open source (MIT) at github.com/salmansajidsattar/vllm-sglang-book: simulators that run on a laptop with no GPU, a benchmark harness for both engines, deploy files, and a one-file runner for a free Colab or Kaggle GPU.

Who it is for

* AI and ML engineers moving a model from a notebook to production

* Data and platform engineers who must run LLMs on GPUs and keep the cost under control

* Students and self-taught developers who want to understand how LLM serving really works

You need basic Python and the command line. The first four chapters and the simulators need no GPU.

An independent guide, not affiliated with the vLLM or SGLang projects. Benchmarks use vLLM 0.32.0 and SGLang 0.5.22 on a Tesla T4 (October 2026).