The Art and Science of RAG

Understanding, Designing, and Deploying Retrieval-Augmented Generation Systems

Chapter 1: Why RAG? The Case for Retrieval-Augmented Generation

1. The Knowledge Problem with Language Models
2. A Simple Example: What Goes Wrong Without Retrieval
3. The Core Idea of RAG in One Sentence
4. What This Book Will Teach You

Chapter 2: Language Model Limitations and the Need for External Knowledge

1. How Pretrained Language Models Store Knowledge
2. Hallucinations: Causes and Manifestations
3. The Training Cutoff Problem
4. Parameter-Efficient Knowledge Updating
5. Why Finetuning Alone Is Not Enough
6. The Scalability Argument for Retrieval

Chapter 3: Information Retrieval Foundations

1. What Is Information Retrieval
2. The Vector Space Model and TF-IDF
3. The BM25 Algorithm
4. Inverted Indexes and Boolean Retrieval
5. Relevance Metrics: Precision, Recall, NDCG
6. The History That Led Us to RAG

Chapter 4: Embeddings and Vector Representations

1. From Words to Vectors: Word Embeddings to Sentence Embeddings
2. What an Embedding Actually Represents
3. Architecture of Modern Embedding Models
4. Cosine Similarity and Vector Distance Metrics
5. Evaluating Embedding Quality
6. Choosing an Embedding Model

Chapter 5: Similarity Search and Vector Databases

1. The Nearest Neighbor Problem at Scale
2. Brute-Force Search and Its Limits
3. Approximate Nearest Neighbor Algorithms: HNSW, IVF, PQ
4. Vector Database Architecture and Design
5. Choosing a Vector Database
6. Vector Index Maintenance

Chapter 6: Document Ingestion and Preprocessing

1. The Ingestion Pipeline End-to-End
2. Document Format Parsing: PDFs, HTML, Office Documents
3. Text Cleaning and Normalization
4. Handling Special Content: Tables, Code, Images, Math
5. Metadata Extraction and Structuring
6. Building a Robust Ingestion System

Chapter 7: Chunking Strategies

1. Why Chunking Matters
2. Fixed-Size and Sliding-Window Chunking
3. Semantic and Recursive Chunking
4. Chunking for Code, Legal Documents, and Technical Manuals
5. Parent-Child and Hierarchical Chunking
6. Measuring and Tuning Chunk Quality

Chapter 8: Index Construction

1. Embedding and Storing Chunks
2. Building Vector Indexes at Scale
3. Combining Dense and Sparse Indexes
4. Metadata Indexing and Filtering
5. Incremental Indexing and Data Freshness
6. Index Versioning and Rollbacks

Chapter 9: Basic Retrieval

1. The Retrieval Operation
2. Dense Retrieval Mechanics
3. Sparse Retrieval Mechanics
4. Combining Dense and Sparse: Hybrid Search
5. Metadata Filtering and Faceted Search
6. Retrieval Performance Tuning

Chapter 10: Query Transformation and Expansion

1. The Query Transformation Problem
2. Query Rewriting for Clarity and Completeness
3. Multi-Query Retrieval
4. Hypothetical Document Embeddings (HyDE)
5. Query Decomposition for Complex Questions
6. When Query Transformation Hurts

Chapter 11: Reranking

1. Why Reranking Is Necessary
2. Cross-Encoder Architecture
3. Reranker Training and Fine-Tuning
4. Late-Interaction and ColBERT Models
5. Practical Reranking: Latency vs. Quality Trade-Offs
6. Choosing and Configuring a Reranker

Chapter 12: Context Construction and Prompt Design

1. From Chunks to Context
2. Formatting Retrieved Context for the LLM
3. Context Compression and Summarization
4. Token Budget Management
5. Prompt Design for Grounded Generation
6. Structuring Multi-Chunk Contexts

Chapter 13: The Generation Step

1. Guiding the LLM with Retrieved Context
2. Temperature, Top-P, and Generation Control
3. Enforcing Faithfulness to Context
4. Handling Conflicting or Insufficient Evidence
5. Streaming Responses and Partial Answers
6. Generation Failure Modes

Chapter 14: Building a Naïve RAG Pipeline

1. Designing the Naïve RAG Architecture
2. Project Structure and Dependencies
3. Ingestion and Indexing Code
4. Retrieval Implementation
5. Generation Integration
6. End-to-End Execution and Testing

Chapter 15: Advanced RAG Patterns

1. Hybrid Search with Reranking: The Standard Advanced Pattern
2. Contextual Compression Retrieval
3. Self-Correction and Self-RAG
4. Adaptive Retrieval Strategies
5. Implementing Advanced Patterns

Chapter 16: Modular RAG Architectures

1. The Case for Modularity
2. Toolformer-Inspired Approaches
3. Modular Design Patterns
4. Routing and Orchestrating Modules
5. Implementing a Modular RAG System

Chapter 17: Hierarchical and Parent-Child Retrieval

1. The Granularity Problem
2. Parent-Child Retrieval
3. Multi-Level Hierarchical Indexes
4. Small-to-Big Retrieval
5. Implementing Hierarchical Retrieval

Chapter 18: Multi-Hop and Agentic RAG

1. Single-Hop vs. Multi-Hop Reasoning
2. Iterative Retrieval Strategies
3. Agentic RAG: Reasoning, Planning, Acting
4. Search and Retriever Tools for Agents
5. Controlling Agentic RAG Complexity
6. Implementing Multi-Hop Retrieval

Chapter 19: GraphRAG and Knowledge Graphs

1. Why Graphs for RAG
2. Knowledge Graph Construction from Documents
3. Graph-Based Retrieval
4. GraphRAG Architecture Patterns
5. Implementing GraphRAG

Chapter 20: Conversational RAG and Multimodal RAG

1. Conversational RAG: Maintaining Context Across Turns
2. Conversational Query Understanding
3. Conversation History in the Prompt
4. Multimodal Embeddings and Retrieval
5. Image, Audio, and Video in RAG
6. Implementing Conversational RAG

Chapter 21: Long-Context Approaches and Structured Data

1. Long-Context LLMs: RAG Competitors or Complements
2. Attention at Scale
3. When Long Context Is Preferable to RAG
4. Hybrid: RAG with Long-Context Models
5. Retrieval from Structured and Tabular Data
6. SQL Retrieval and Text-to-SQL
7. Retrieval from Structured and Tabular Data

Chapter 22: RAG Evaluation: Measuring What Matters

1. What to Evaluate: A Complete Taxonomy
2. Retrieval Quality Metrics: Recall, Precision, MRR, NDCG
3. Answer Quality and Faithfulness Metrics
4. Hallucination Detection
5. Automated Evaluation: RAGAS, TruLens, and Custom Metrics
6. Human Evaluation and Benchmark Design

Chapter 23: Debugging RAG Systems

1. Classifying RAG Failures
2. Retrieval Failures: Diagnosing and Fixing
3. Generation Failures: Diagnosing and Fixing
4. End-to-End Debugging Workflow
5. Logging, Tracing, and Observability
6. Continuous Improvement Loops

Chapter 24: Security and Access Control for RAG

1. Prompt Injection: Direct Attacks
2. Prompt Injection: Indirect Attacks Through Retrieved Documents
3. Authorization-Aware Retrieval
4. Data Leakage
5. Tenant Isolation in Multi-Tenant Systems
6. Malicious Documents and Content Poisoning
7. Red-Teaming and Security Testing

Chapter 25: Production Architecture and Deployment

1. Small-Scale Architecture: Single-Service Design
2. Enterprise Architecture: Modular and Scalable Design
3. Multi-Tenant Architecture
4. Deployment Options: Cloud, On-Premise, and Hybrid
5. Scalability Patterns
6. High Availability and Fault Tolerance

Chapter 26: Production Operations and Optimization

1. Monitoring and Observability
2. Latency Optimization
3. Cost Optimization
4. Data Freshness and Incremental Indexing
5. Versioning, Rollback, and Continuous Deployment

Chapter 27: Real-World Case Studies and Patterns

1. Case Study 1: Customer Support Bot for SaaS Company
2. Case Study 2: Enterprise Knowledge Management System
3. Case Study 3: Financial Research and Analysis
4. Case Study 4: Healthcare Information System
5. Case Study 5: Developer Documentation Assistant
6. Case Study 6: E-Commerce Product Information System
7. Lessons Across Case Studies

Chapter 28: Conclusion: The RAG Landscape and Future Directions

1. The Core Insight: Why RAG Endures
2. Key Principles for Effective RAG
3. The RAG Technology Landscape
4. Future Directions: Where RAG Is Heading
5. Getting Started: Practical Guidance
6. The Promise of RAG

References