Building Hallucination-Resistant RAG and AI Agents

A Complete Guide to Reliable, Evidence-Grounded AI Systems

Introduction: The Hallucination Problem Is Not a Prompting Problem

1. What Hallucination Means in Practice
2. Why Hallucinations Are Inevitable
3. What This Book Covers
4. How to Use This Book

Chapter 1: What Hallucinations Are and Why They Matter

1. Defining Hallucination: Unsupported Claims Versus Errors Versus Fabrication
2. Origins in Language Models: Distributional Generalization, Memorization, and Training Bias
3. Real-World Harm: Medical Misinformation, Legal Errors, Financial Decisions, Security Vulnerabilities
4. Why Prompt Engineering Alone Fails: The Fundamental Probabilistic Problem
5. Setting the Stage for Defense in Depth

Chapter 2: The RAG and Agent Pipeline as a Hallucination Attack Surface

1. The Standard RAG Pipeline: Ingestion, Indexing, Retrieval, Generation
2. Agentic RAG: Tools, Planning, Iteration, Multi-Step Reasoning
3. Hallucination Modes by Stage: Knowledge, Retrieval, Context, Generation, Verification
4. Error Propagation: How Upstream Failures Cascade Downstream
5. The Defense-in-Depth Philosophy: Prevention Versus Detection Versus Verification Versus Abstention
6. What Comes Next

Chapter 3: RAG Architecture Patterns and Their Hallucination Profiles

1. Standard RAG: Strengths and Hallucination Failure Modes
2. Hybrid RAG: Combining Lexical and Semantic Retrieval for Robustness
3. Corrective RAG: Detecting and Correcting Bad Retrievals
4. Self-Reflective and Self-RAG: Model Self-Critique and Conditional Generation
5. Adaptive RAG: Dynamic Strategy Selection Based on Query Complexity
6. Graph RAG: Structured Knowledge for Multi-Hop and Relation Accuracy
7. Agentic RAG: Iterative Retrieval and Tool Use with Hallucination Risks
8. Multi-Agent Systems: Cross-Agent Verification and Conflict Resolution
9. Choosing an Architecture for Your Risk Profile

Chapter 4: Selecting and Vetting Knowledge Sources

1. Source Trustworthiness Criteria: Authority, Accuracy, Recency, Completeness
2. Primary Versus Secondary Versus Tertiary Sources: When Each Is Appropriate
3. Domain-Specific Requirements: Medical, Legal, Financial, Technical
4. Detecting and Excluding Unreliable, Adversarial, or Misleading Sources
5. Source-Quality Scoring and Metadata Standards
6. Provenance Tracking from Origin to Production Retrieval
7. Building a Source-Governed Knowledge Base

Chapter 5: Document Ingestion, Parsing, and Cleaning

1. Format Challenges: PDFs, Office Documents, HTML, Code, Images, Tables
2. PDF Parsing Pitfalls: OCR Errors, Layout Confusion, Page Breaks Mid-Claim
3. HTML Extraction: Boilerplate Removal, Script Noise, Embedded Ads
4. Table and Figure Extraction: Structured Data Loss
5. Cleaning and Normalization: Encoding, Whitespace, Special Characters
6. Detecting Malformed or Corrupted Documents Before Indexing
7. The Cost of Bad Parsing

Chapter 6: Metadata, Deduplication, and Knowledge Validation

1. Metadata Extraction: Titles, Dates, Authors, Sections, Entities
2. Automatic Metadata Quality Validation
3. Semantic Deduplication: Collapsing Near-Identical Content
4. Conflict Detection: Identifying Contradictory Sources
5. Temporal Metadata: Handling Versioning and Superseded Information
6. Knowledge Quality Scoring: Flagging Low-Confidence Documents
7. Maintaining Knowledge Quality Over Time

Chapter 7: Chunking Strategies and Semantic Segmentation

1. Why Chunking Matters: Context Fragmentation Versus Noise Dilution
2. Fixed-Size Versus Semantic Chunking: Comparative Analysis
3. Recursive and Hierarchy-Aware Chunking
4. Entity-Aware and Claim-Aware Segmentation
5. Overlapping Chunks: Tradeoffs Between Continuity and Duplication
6. Adaptive Chunking: Size Selection Based on Content Type and Query Patterns
7. Choosing and Validating a Chunking Strategy

Chapter 8: Embeddings and Vector Search for Faithful Retrieval

1. Embedding Model Selection: Domain Fit, Language Coverage, Dimensionality
2. Embedding Drift and Degradation Over Time
3. Vector Search Metrics: Cosine Similarity, Dot Product, Hybrid Scoring
4. The False-Negative Problem: When Relevant Content Is Not Retrieved
5. The False-Positive Problem: When Irrelevant Content Misleads Generation
6. Embedding Evaluation for Retrieval Quality
7. Practical Recommendations

Chapter 9: Indexing Strategies and Knowledge Graphs

1. Vector Index Types: HNSW, IVF, Flat, and Their Recall-Accuracy Tradeoffs
2. Hybrid Indexes: Combining Vectors with Inverted Lexical Indexes
3. Knowledge Graph Construction from Documents
4. Graph-Based Retrieval: Multi-Hop Queries and Relation Grounding
5. Graph RAG Architectures: Structured Plus Unstructured for Reliability
6. Maintaining Graph Freshness and Consistency

Chapter 10: Query Understanding and Rewriting

1. Query Ambiguity and Vagueness: Causes of Hallucinated Best Guesses
2. Query Expansion: Synonyms, Related Terms, Controlled Vocabularies
3. Query Decomposition: Breaking Complex Questions Into Subqueries
4. Query Rewriting for Retrieval: Style Matching, Entity Normalization
5. Negative Prompting: Specifying What Not to Retrieve
6. Entity Linking and Disambiguation Before Retrieval

Chapter 11: Hybrid Search, Filtering, and Reranking

1. Lexical Versus Semantic Versus Hybrid Search: When Each Succeeds and Fails
2. Pre-Retrieval Filtering: Metadata Filters, Date Ranges, Source Types
3. Cross-Encoder Reranking: Precision Versus Recall Versus Latency
4. Query-Aware Filtering: Dynamic Filter Selection
5. Reranking Model Selection and Fine-Tuning for Domain Accuracy
6. Confidence Scoring of Retrieved Documents

Chapter 12: Iterative and Agentic Retrieval Patterns

1. Multi-Hop Retrieval: Chaining Queries for Complex Evidence Chains
2. Self-RAG Style Reflection: Did I Find Enough Evidence?
3. Iterative Deepening: Refining Search Based on Partial Results
4. Agentic Retrieval: When the Agent Decides What to Look For Next
5. Stop Conditions: Knowing When Further Retrieval Is Futile
6. Cost and Latency Tradeoffs of Iterative Retrieval

Chapter 13: Context Construction and Window Management

1. Context Assembly: Ordering, Grouping, and Labeling Retrieved Documents
2. Context Prioritization: Most Relevant Evidence First or Last
3. Context Window Limits: Truncation Strategies and Information Loss
4. Handling Contradictory Evidence in Context
5. Document Source Attribution in Context Prompts
6. Reducing Context-Induced Hallucination from Conflicting or Misleading Sources

Chapter 14: Prompt Engineering for Grounded Generation

1. System Instructions: Explicit Grounding Constraints
2. Few-Shot Prompting with Grounded Examples
3. Citation Instructions: Requiring Source References Per Claim
4. Abstention Patterns: Say You Do Not Know When Evidence Is Absent
5. Structured Response Formats: Reducing Free-Form Hallucination Space
6. Prompt Injection Risks from Untrusted Retrieved Content

Chapter 15: Constrained Decoding and Structured Outputs

1. Grammar-Constrained Decoding: Limiting to Expected Output Structures
2. Vocabulary Constraints: Restricting to Source-Derived Terms
3. JSON and Schema-Based Output Validation
4. Token-Level Constraints for Numerical and Entity Accuracy
5. Sampling Parameters: Temperature, Top_p, and Hallucination Rates
6. When Constraints Are Worth the Complexity

Chapter 16: Claim Extraction and Evidence Mapping

1. Claim Extraction from Generated Text
2. Entailment Testing: Does Evidence Entail the Claim?
3. Citation Correctness: Verifying Cited Sources Actually Support Claims
4. Citation Completeness: Detecting Unsupported Claims With No Citations
5. Numerical Verification: Checking Numbers Against Sources
6. Temporal Verification: Checking Date-Sensitive Claims

Chapter 17: Verification Models and Critic Systems

1. LLM-as-Judge Patterns: Strengths and Failure Modes
2. Cross-Model Verification: Different Models Checking Each Other
3. Dedicated Verifier Models: Trained for Entailment and Contradiction
4. Self-Consistency: Generating Multiple Answers and Comparing
5. Critic-Model Architectures: Separate Generation and Verification
6. Confidence Estimation and Uncertainty Quantification

Chapter 18: Rule-Based and Deterministic Verification

1. Schema Validation: Type Checking, Required Fields, Value Ranges
2. Deterministic Rules: Regex Patterns, Format Validation
3. Numerical Consistency: Arithmetic Verification
4. Contradiction Detection: Logical Consistency Checks
5. Citation Format and Existence Verification
6. When Deterministic Checks Are Superior to Model-Based Checks

Chapter 19: Multi-Layer Defense Architectures

1. Layered Architecture Design: Where Each Check Sits in the Pipeline
2. Pre-Generation Verification: Validating Retrieved Evidence
3. In-Generation Constraints: Real-Time Guidance
4. Post-Generation Verification: Independent Checking
5. Escalation Patterns: When to Re-Retrieve, Ask Human, or Abstain
6. Reference Architectures for Different Risk Profiles

Chapter 20: Hallucination Risks Specific to AI Agents

1. Planning Hallucination: Inventing Steps That Do Not Exist or Will Not Work
2. Tool-Selection Errors: Choosing Wrong Tools or Inventing Tools
3. Fabricated Tool Results: Hallucinating API Responses
4. Tool-Argument Errors: Wrong Parameters Leading to Wrong Data
5. Multi-Step Error Propagation: Compounding Failures Across Steps
6. Reasoning Failures: Logical Errors in Multi-Step Deduction

Chapter 21: Agent Memory and Conversation History Management

1. Memory Contamination: Old Facts Applied to New Situations
2. Conversation History Pruning: Keeping Relevant, Discarding Stale
3. Cross-Session Leakage: Hallucination From Unrelated Conversations
4. Summarization Hallucination: Errors Introduced When Compressing History
5. Memory Consistency: Ensuring Stored Facts Remain Accurate
6. Forgetting: When Agents Should Drop Outdated Information

Chapter 22: Multi-Agent Systems and Cross-Agent Verification

1. Specialized Agents: Reducing Hallucination Through Domain Focus
2. Debating Agents: Adversarial Refinement of Answers
3. Cross-Verification: Independent Agents Checking Each Other’s Work
4. Consensus Mechanisms: Aggregating Multiple Agent Perspectives
5. Conflict Resolution: When Agents Disagree
6. Cascading Failures: When One Agent’s Hallucination Infects Others

Chapter 23: Evaluation Methodologies and Metrics

1. Core Metrics: Faithfulness, Groundedness, Factual Correctness
2. Retrieval Metrics: Context Relevance, Retrieval Precision and Recall
3. Citation Metrics: Citation Correctness, Completeness, Hallucination Rate
4. End-to-End Metrics: Answer Relevance, Unsupported-Claim Rate
5. Calibration and Abstention Quality
6. Combining Metrics Into System-Level Reliability Scores
7. Practical Evaluation Workflow

Chapter 24: Test Data, Benchmarks, and Red-Teaming

1. Building Ground-Truth Evaluation Datasets
2. Synthetic Test Case Generation With Controlled Difficulty
3. Adversarial Tests: Queries Designed to Trigger Hallucination
4. Existing Benchmarks: QAFactEval, FaithDial, HaluEval, RAGTruth
5. Regression Testing: Preventing Hallucination Regressions
6. Red-Team Methodologies for Hallucination Discovery

Chapter 25: Production Monitoring and Continuous Improvement

1. Real-Time Monitoring: Latency, Confidence, Verification Results
2. User Feedback Loops: Thumbs Up/Down, Corrections, Escalations
3. Drift Detection: Detecting When Retrieval or Generation Quality Degrades
4. Incident Response: Handling Hallucination-Related Outages
5. Continuous Evaluation: Automated Testing in Production Pipelines
6. Documentation and Governance for Hallucination-Critical Systems

Conclusion: The Reality of Reliability

1. What Hallucination Rates Are Realistically Achievable
2. Fundamental Limits: Why Probabilistic Models Will Always Hallucinate
3. The Defense-in-Depth Mindset: Accepting Risk and Layering Safeguards
4. Cost Versus Reliability: Building Appropriate Safeguards for Your Risk Profile
5. Future Directions: What Research Might Change the Landscape
6. A Framework for Ongoing Improvement

References