My Adventures with Large Language Models

Description

Welcome to the Leanpub Launch video for My Adventures with Large Language Models: Build foundational LLMs from Transformers to DeepSeek, from scratch, in PyTorch. https://leanpub.com/adventures-with-llms by Prathamesh S.! 00:00 Introduction to My Adventures with Large Language Models 00:17 Prathamesh's background in AI engineering and production systems 00:53 Why he wrote the book he wished he'd had when starting out 01:20 Who the book is for: understanding LLMs beyond APIs 01:37 From Attention Is All You Need to building transformers 01:48 Training GPT-2, Llama architectures, KV caching, and DeepSeek 02:09 Reproducible experiments on affordable GPU hardware 02:45 Learning LLM fundamentals by building models from scratch 02:58 Writing for beginners: explaining what matters most 03:29 Why understanding how LLMs work makes you a better AI practitioner 03:45 Final recommendation and closing remarks About the Book Most LLM tutorials stop at GPT-2. This book doesn't. My Adventures with Large Language Models walks you through building five real LLM architectures from scratch in PyTorch, starting from a vanilla encoder-decoder Transformer and ending at DeepSeek's Multi-Head Latent Attention and Mixture-of-Experts. Every chapter has runnable, end-to-end code. No pseudocode, no hand-waving. You type it, you run it, you understand it. What you'll build: Chapter 1: A vanilla encoder-decoder Transformer for English-to-Hindi translation. The fundamentals, implemented from the ground up. Chapter 2: GPT-2 (124M parameters) from scratch, then load real OpenAI pretrained weights to verify your implementation works. Chapter 3: Llama 3.2-3B by swapping exactly four components of your GPT-2. LayerNorm becomes RMSNorm. Learned positional encodings become RoPE. GELU becomes SwiGLU. Multi-Head Attention becomes Grouped-Query Attention. Then load Meta's pretrained weights. Chapter 4: KV cache, Multi-Query Attention, and Grouped-Query Attention for inference optimisation. Chapter 5: DeepSeek's full architecture. Multi-Head Latent Attention (with the absorption trick and decoupled RoPE), DeepSeekMoE (shared experts, fine-grained segmentation, auxiliary-loss-free load balancing), Multi-Token Prediction, and FP8 quantisation. The code repository is open source: https://github.com/S1LV3RJ1NX/mal-code This book is for ML engineers, researchers, and senior developers who know Python and PyTorch and want to understand modern LLMs at the level of code, not slides or blog posts. If you've read Raschka or watched Karpathy and want to go further, into Llama, GQA, MLA, and MoE, this is the book. About the Author Prathamesh is a Senior Forward Deployed Engineer at TrueFoundry, where he helps enterprises and startups solve real problems with LLMs and agents. He wrote this book because he wanted a resource that went past GPT-2 and into the architectures actually running in production. He is based in Bangalore, India. Portfolio: https://psaraf.pages.dev Follow the author here! https://x.com/S1LV3R_J1NX Thank you for watching, please like and leave a comment, we'd love to hear from you! Please Subscribe and Follow! YouTube: https://www.youtube.com/leanpub X: https://x.com/leanpub Instagram: https://www.instagram.com/leanpub Facebook: https://www.facebook.com/leanpub Create Your Own Leanpub Book! You can create your own book anytime here: https://leanpub.com/create/book Here's the tutorial showing how to write and publish a Leanpub book in your browser (it's free!): http://help.leanpub.com/en/articles/2932527-getting-started-writing-a-book-in-the-web-browser-writing-mode If you're a Leanpub author and you'd like to submit your own Launch video for us to publish, or if you'd like to record a Launch video with Len, please go here: https://leanpub.com/launch. #books #leanpublishing #selfpublishing #leanpub #writing #machinelearning #deeplearning #ai #python #computerscience #computerprogramming #datascience #neuralnetworks