Leanpub Header

Skip to main content
Go to Leanpub.com

Modern Compiler Engineering: From Source Code to Machine Code

Designing and Implementing a Production Compiler with Rust

Modern Compiler Engineering: From Source Code to Machine Code
This book is 100% completeLast updated on 2026-10-07
Steve Publications

Build a real compiler from the ground up with Rust. Follow the journey from source code to native x86-64 machine code as you design Lumina, explore optimization and register allocation and tackle the practical challenges that come with building a production-quality compiler.

Minimum price

$29.00

$39.00

You pay

Author earns

$

Also available for 1 book credit with a Reader Membership

PDF
EPUB
WEB
APP
273
Pages
About

About

About the Book

This book takes you through the complete process of designing and implementing a modern compiler from the ground up. You will build Lumina, a statically typed, compiled language with a full frontend, optimizing intermediate representation, register allocator and native code generator for x86-64. Along the way, you will learn the architectural decisions, engineering trade-offs and practical techniques used by production compilers like Clang, GCC, rustc and LLVM. Every chapter includes complete, working code examples in Rust that you can compile, extend and study. By the end, you will not only understand how compilers work, you will know how to build, optimize, test and evolve one in practice.

Share this book

Categories

CompilerComputers and ProgrammingSoftware EngineeringAlgorithmRust

Feedback

Contact the author

Author

About the Author

Steve Publications

Steve is a technology professional with over 20 years of experience in software development, server infrastructure, cybersecurity, vulnerability research and reverse engineering. He specializes in analyzing complex systems, identifying weaknesses and improving security.

Beyond his professional work, Steve collaborates with researchers and industry experts to create practical technical books covering software development, cybersecurity, cloud computing, DevOps, AI and enterprise technologies. His goal is to help IT professionals, students and enthusiasts develop practical skills through clear explanations, real-world examples and hands-on exercises.

At Steve Publications, we use AI to assist with translation, proofreading and editing but not to replace the research, technical expertise or hands-on experience behind our books. Some authors publish under pseudonyms for privacy or professional reasons.

Every book is written, reviewed and maintained by experienced professionals, supported by a private community of over 420 engineers and researchers. We prioritize technical accuracy, practical value and regularly updated content over mass-produced AI-generated text.

We welcome experienced professionals interested in writing or reviewing books, especially in specialized and emerging technology fields. If you'd like to contribute, contact us with your area of expertise.

We stand behind our work. If a book doesn't meet your expectations, Leanpub offers a 60-day money-back guarantee.

Contents

Table of Contents

Designing and Implementing a Production Compiler with Rust

Introduction

  1. From Machine Code to High-Level Languages
  2. The Compilation Pipeline in One Picture
  3. Compilers, Interpreters and JITs
  4. Why Build Your Own Compiler?
  5. How This Book Is Structured

Chapter 1: What Is a Compiler?

  1. The Compiler as Translation Engine
  2. The Modular Architecture
  3. A Concrete Example
  4. Compilers as Complex Systems
  5. The Compiler Landscape Today
  6. The Lumina Reference Compiler
  7. Prerequisites and Conventions
  8. The Journey Ahead

Chapter 2: Compiler Architecture

  1. The Need for Modular Design
  2. Phases and Representations
  3. The Lumina Pipeline Architecture
  4. Error Handling as a First-Class Concern
  5. Frontend-IR and Backend Separation
  6. The Lumina Build System and Project Layout
  7. Design Trade-offs in Compiler Architecture
  8. Summary

Chapter 3: Language Design for Implementation

  1. Language Design and Compiler Complexity
  2. Syntax Design Choices
  3. Type System Implications
  4. Memory Model and Ownership
  5. Evaluation Strategy and Semantics
  6. Standard Library and Runtime Expectations
  7. The Lumina Language Specification
  8. Implementation Language: Why Rust
  9. Summary

Chapter 4: Lexical Analysis

  1. The Role of the Lexer
  2. Token Types and Lexical Rules
  3. Building a Hand-Written Lexer
  4. Understanding the Lexer Code
  5. Error Handling in Lexical Analysis
  6. Lexer vs. Parser Responsibilities
  7. Lexer Engineering and Whitespace Handling
  8. Testing the Lexer
  9. Summary

Chapter 5: Parsing Fundamentals

  1. From Tokens to Structure
  2. Context-Free Grammars and Syntax
  3. LL vs LR Parsing
  4. Recursive Descent Parsing
  5. Parser Combinators vs Hand-Written
  6. Error Recovery in Parsing
  7. Implementing Lumina Parse with Recursive Descent
  8. Understanding the Parser Code
  9. Summary

Chapter 6: Abstract Syntax Trees

  1. Concrete Syntax Trees vs Abstract Syntax Trees
  2. AST Node Design and Representation
  3. Type Safety and Pattern Matching
  4. Source Location Tracking
  5. Desugaring and Pre-Transformation
  6. Implementing Lumina AST
  7. AST Traversal Patterns
  8. Summary

Chapter 7: Semantic Analysis and Symbol Tables

  1. What Semantic Analysis Does
  2. Symbol Tables and Scope
  3. Name Resolution Algorithms
  4. Declarations and Definitions
  5. Shadowing and Visibility Rules
  6. Semantic Errors and Diagnostics
  7. Implementing Lumina Semantic Analysis
  8. Understanding the Semantic Analyzer
  9. Summary

Chapter 8: Type Systems and Type Checking

  1. Static vs Dynamic Typing
  2. Type Checking Algorithms
  3. Type Inference vs Type Annotation
  4. Type Errors and Suggestions
  5. Implementing Lumina Type Checker
  6. Summary

Chapter 9: Intermediate Representations

  1. Why Intermediate Representations?
  2. Three-Address Code and SSA
  3. High-Level vs Low-Level IRs
  4. IR Design Principles
  5. Implementing Lumina IR
  6. Lowering AST to IR
  7. Summary

Chapter 10: Control Flow Graphs

  1. Basic Blocks and Control Flow
  2. Building Control Flow Graphs
  3. Dominators and Post-Dominators
  4. Natural Loops and Loop Headers
  5. CFG Optimization
  6. Implementing Lumina CFG
  7. Summary

Chapter 11: Static Single Assignment

  1. SSA and Its Benefits
  2. Phi Functions and Versioning
  3. SSA Construction Algorithms
  4. Sparse Conditional Constant Propagation
  5. Optimizing SSA Form
  6. Implementing Lumina SSA
  7. Summary

Chapter 12: Data Flow Analysis

  1. Data Flow Analysis Fundamentals
  2. Reaching Definitions and Available Expressions
  3. Live Variable Analysis
  4. Interprocedural Analysis
  5. Analysis Precision and Performance
  6. Implementing Lumina Data Flow
  7. Summary

Chapter 13: Core Optimizations

  1. The Optimization Framework
  2. Dead Code Elimination
  3. Constant Propagation and Folding
  4. Copy Propagation
  5. Common Subexpression Elimination
  6. Summary

Chapter 14: Loop Optimizations

  1. Loop Optimization Opportunities
  2. Loop-Invariant Code Motion
  3. Strength Reduction
  4. Induction Variable Elimination
  5. Loop Unrolling and Peeling
  6. Implementing Lumina Loop Optimizations
  7. Summary

Chapter 15: Advanced Optimizations

  1. Function Inlining Strategies
  2. Escape Analysis
  3. Tail Call Optimization
  4. Algebraic Simplification
  5. Profile-Guided Optimization
  6. Implementing Lumina Inliner
  7. Summary

Chapter 16: Instruction Selection

  1. The Instruction Selection Problem
  2. Tree-Pattern Matching
  3. Instruction Scheduling
  4. Peephole Optimization
  5. Lowering for x86-64
  6. Implementing Lumina Instruction Selection
  7. Summary

Chapter 17: Register Allocation

  1. Register Allocation Problems
  2. Graph Coloring Algorithms
  3. Linear Scan Allocation
  4. Spilling Strategies
  5. Register Pressure
  6. Implementing Lumina Register Allocator
  7. Summary

Chapter 18: Code Generation and Assembly

  1. Assembly Language and Encoding
  2. Machine Code Generation
  3. Relocations and Symbols
  4. ABI and Calling Conventions
  5. Emitting ELF Object Files
  6. Implementing Lumina Code Generator
  7. Summary

Chapter 19: Linking and Loading

  1. The Linker’s Job
  2. Static vs Dynamic Linking
  3. Symbol Resolution and Relocation
  4. Link-Time Optimization
  5. The Loader and Dynamic Linking
  6. Integrating Lumina with the Toolchain
  7. Summary

Chapter 20: Runtime Support

  1. What Compilers Need from Runtimes
  2. Initialization and Finalization
  3. Standard Library Design
  4. Error Handling and Panics
  5. Memory Management Patterns
  6. Building Lumina Runtime
  7. Summary

Chapter 21: Diagnostics and Debugging

  1. Error Message Design
  2. Source Maps and Location
  3. Diagnostic Suggestions
  4. Debug Information: DWARF
  5. Debugging the Compiler
  6. Implementing Lumina Diagnostics
  7. Summary

Chapter 22: JIT Compilation

  1. When and Why JIT?
  2. Compilation at Runtime
  3. Tiered Compilation
  4. Deoptimization
  5. LLVM ORC and Modern JIT Infrastructure
  6. JIT Design for Lumina
  7. Summary

Chapter 23: Incremental and Parallel Compilation

  1. The Incremental Compilation Problem
  2. Incremental Dependency Tracking
  3. Query-Based Incremental Systems
  4. Parallel Compilation
  5. Distributed Compilation
  6. Making Lumina Fast
  7. Summary

Chapter 24: Compiler Testing and Verification

  1. Compiler Testing Strategies
  2. Unit Testing Compiler Components
  3. End-to-End and Regression Tests
  4. Compiler Fuzzing
  5. Formal Verification of Compilers
  6. Testing Lumina
  7. Summary

Chapter 25: Performance Engineering

  1. Profiling Compiler Performance
  2. Compiler Memory Usage
  3. Compilation Time Optimizations
  4. Cache Efficiency in Compilers
  5. Benchmarking the Compiler
  6. Profiling Lumina Compilation
  7. Summary

Chapter 26: Compiler Frameworks and Ecosystems

  1. LLVM Architecture and Use Cases
  2. GCC: The Veteran Toolkit
  3. MLIR: Multilevel Intermediate Representation
  4. Cranelift for WebAssembly
  5. Parser Generators: ANTLR, LALRPOP, Tree-Sitter
  6. Choosing Your Compiler Stack
  7. Summary

Chapter 27: Hardware Awareness

  1. CPU Architectures and ISAs
  2. Instruction Latency and Throughput
  3. Memory Hierarchy Awareness
  4. Branch Prediction and Control Flow
  5. SIMD and Vectorization
  6. Hardware Features in Lumina
  7. Summary

Chapter 28: The Complete Lumina Compiler

  1. Architecture Overview
  2. Complete Project Structure
  3. The Compilation Driver
  4. Complete Lumina Program
  5. Extending Lumina
  6. Summary

Chapter 29: Extending and Evolving a Compiler

  1. Adding Language Features
  2. Supporting New Platforms
  3. Scaling the Codebase
  4. Maintaining Compiler Quality
  5. Compiler Versioning and Breaking Changes
  6. Community and Ecosystem
  7. Summary

Conclusion

References

Get the free sample chapters

Click the buttons to get the free sample in PDF or EPUB, or read the sample online here

Download Sample PDFDownload Sample EPUB

The Leanpub 60 Day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

See full terms...

Earn $8 on a $10 Purchase, and $16 on a $20 Purchase

We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub