Leanpub Header

Skip to main content

Run Large Language Models on ROCm

Run Large Language Models on ROCm

Install, validate and tune ROCm for LLM inference on MI300X, MI355X and Radeon hardware — then scale it across GPUs with RCCL, vLLM, SGLang and llama.cpp.

Minimum price

$19.00

$29.00

You pay

Author earns

$

Also available for 1 book credit with a Reader Membership

PDF
About

About

About the Book

A working engineer's guide to running large language models on AMD GPUs with ROCm, aimed at ML engineers, platform engineers, data scientists and developers who already know Python and transformers.

The book opens with the hardware and software stack you are actually choosing between — Instinct MI300X, MI325X and MI355X accelerators, RDNA4 Radeon cards, Ryzen AI APUs — and the `gfx` targets (gfx942, gfx950, gfx1201, gfx1151) that decide kernel behaviour. From there it moves through five installation paths (package manager, runfile, containers, PyTorch pip wheels), with a three-layer verification that checks the `amdgpu` module, device nodes and group membership, plus a reproducible idempotent install script.

You then get PyTorch on ROCm: the `torch.cuda` device-string illusion, dtype rules for CDNA versus RDNA, automatic mixed precision, hipBLASLt, Triton on ROCm, RCCL and hipFile, wrapped in a vendor-neutral device layer you can drop into any project. Model loading covers safetensors, sharding, attention backends, GPU ordering, GGUF via llama.cpp, and cache layout.

The porting chapter uses hipify-perl and hipify-clang with CMake `hipcc` builds and a CUDA2HIP support report, and is explicit about what does not port. Optimisation covers the seven vLLM attention backends on ROCm, AITER switches, FP8 KV cache, AWQ and GPTQ quantization, with a systematic tuning procedure. Serving compares llama.cpp, vLLM and SGLang on real numbers; profiling uses rocprofv3, rocprof-compute, AMD SMI and the roofline model, including the RDNA performance-level requirement. Distributed inference covers XGMI topology, RCCL channels, TP/PP/DP+EP placement and scaling efficiency. Security and observability cover device-node permissions, container hardening, the AMD Kubernetes device plugin, Prometheus metrics and supply-chain pinning. A symptom-indexed troubleshooting ladder and three end-to-end walkthroughs close the book.

Every chapter has version-pinned commands, runnable code, pitfalls, exercises and a recap checklist.

Author

About the Author

SouthStone Publications

SouthStone Publications is a publisher dedicated to advancing practical knowledge across web development, IT engineering, programming, artificial intelligence, and cybersecurity. Working closely with a network of subject matter experts and industry practitioners, SouthStone produces books, guides, and training programs designed to translate complex technical concepts into clear, actionable knowledge. Its catalog has supported developers, engineers, analysts, and security professionals around the world in building more resilient, scalable, and well-architected systems.

Proud to be the first publisher in the Cybersecurity category on LeanPub, SouthStone Publications is committed to bridging the gap between deep technical expertise and accessible education, helping practitioners at every stage of their careers stay ahead in a rapidly evolving digital landscape.

The Leanpub 60 Day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

See full terms...

Earn $8 on a $10 Purchase, and $16 on a $20 Purchase

We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub