Prompt Engineering & Security

Building Injection-Resistant AI Systems

1. About This Book

Introduction: The New Attack Surface

1. The Attack Surface Has Changed
2. Why This Book Exists
3. Who This Book Is For
4. The Central Thesis
5. How This Book Is Organized
6. A Note on Scope and Limitations
7. The Stakes Are Real

Chapter 1. How LLMs Read Instructions: A Primer on Model Mechanics

1. The Tokenization Layer: Where Text Becomes Numbers
2. Attention: How LLMs Contextualize Every Token
3. The Three Sources of Input in a Typical LLM Application
4. Why This Is Not a “Bug”: It’s a Feature of the Architecture
5. Historical Context: How We Got Here
6. Competing Perspectives: Is the Risk Overstated?
7. The “Blast Radius” Argument: Capability Determines Risk
8. The Human-in-the-Loop Argument
9. The “Models Are Getting Better” Argument
10. The “It’s Just Input Validation” Argument
11. Key Takeaway for Security Engineers

Chapter 2. The Principles of Prompt Engineering: Clarity, Reliability, and Security

1. The Core Techniques
2. The Security Dimension: Designing Secure Prompts
3. Design Patterns for Effective Prompts
4. Case Study: A Healthcare Chatbot That Hallucinated Because of Ambiguous Instructions
5. Trade-offs and Competing Perspectives
6. Quantitative Data: What Actually Works?
7. Trade-offs and Competing Perspectives
8. A Deeper Dive. The Role-Playing Prompt: Security Implications
9. The Feedback Loop Risk: Why Multi-Hop Systems Are Inherently More Vulnerable
10. Practical Walkthrough: Building a Secure RAG Prompt Template
11. Key Takeaway

Chapter 3. Anatomy of Prompt Injection: Direct Attacks

1. Direct Injection Mechanics
2. Real-World Examples: Documented Incidents
3. Prompt Leaking as Reconnaissance
4. Why Legacy Defenses Fail
5. The Fundamental Tension
6. The Remoteli.io Incident: A Deep Dive
7. The Bing/Sydney Leak: A Case Study in Reconnaissance
8. Prompt Leaking as Reconnaissance (Continued)
9. Why Legacy Defenses Fail (Expanded)
10. The Fundamental Tension (Expanded)
11. Key Takeaway

Chapter 4. The Indirect Injection Threat: RAG, Web Content, and Data Poisoning

1. The RAG Trust Paradox
2. Case Study: Slack AI (August 2024)
3. Case Study: ChatGPT Memory / SpAIware (September 2024)
4. PoisonedRAG: Five Documents, 90% Success Rate
5. Vector Database Vulnerabilities
6. Three-Layer Defense Architecture for RAG
7. The Slack AI Attack: A Full Technical Walkthrough
8. Competing Perspectives: Is RAG Really That Vulnerable?
9. Three-Layer Defense Architecture for RAG (Expanded with Implementation)
10. Key Takeaway

Chapter 5. Jailbreaks: Social Engineering for Machines

1. What Jailbreaking Is (and Isn’t)
2. The Taxonomy of Jailbreak Techniques
3. Characteristics of Jailbreak Prompts
4. Automated Jailbreak Discovery
5. The Consequences: Character.AI and Beyond
6. The GOAT Framework: How Automated Red Teaming Works
7. TAP: Tree of Attacks with Pruning
8. Defense Against Jailbreaks: A Practical Guide
9. The Character.AI Case: A Cautionary Tale
10. Key Takeaway

Chapter 6. AI Agents and Tool Use: When Prompts Become Shells

1. From Chatbots to Agents
2. The Semantic Kernel RCE Vulnerabilities (May 2026)
3. Documented Incidents
4. The Expanding Blast Radius
5. Framework-Agnostic Vulnerabilities
6. CVE-2026-26030: A Step-by-Step Code Analysis
7. The Expanding Blast Radius: A Framework for Analysis
8. A Deeper Look: The Semantic Kernel RCE Vulnerabilities (CVE-2026-26030 and CVE-2026-25592)
9. Framework-Agnostic Vulnerabilities: The Deeper Truth
10. Key Takeaway

Chapter 7. Defense-in-Depth Architecture: Layered Protections

1. The Three-Layer Model
2. Guardrails: The Runtime Enforcement Layer
3. The Trade-Off: Security vs. Usability
4. The Trade-Off: Security vs. Usability (Expanded with Calibration Examples)
5. Competing Perspectives: Is Defense-in-Depth Overkill?
6. Key Takeaway

Chapter 8: Instruction Hierarchy and Isolation Patterns

1. The Core Principle: Instruction Hierarchy
2. The Six Design Patterns for Injection-Resistant Agents
3. Architectural Isolation Patterns
4. Case Study: Applying Patterns to a Customer Service Chatbot
5. Pattern Selection: A Decision Framework
6. Case Study: Applying Patterns to a Healthcare Data Analysis Platform
7. Competing Perspectives: Are Design Patterns Enough?
8. Key Takeaway

Chapter 9. Enterprise Security Stacks: FIDES, LlamaFirewall, and Beyond

1. Microsoft FIDES: Deterministic Defense Through Information Flow Control
2. Meta LlamaFirewall: Open-Source Guardrail System
3. Comparing Enterprise Options
4. OWASP Top 10 for LLM Applications 2025
5. Choosing the Right Stack
6. FIDES Implementation: A Practical Walkthrough
7. LlamaFirewall Deployment Guide
8. LlamaFirewall’s Agent Alignment Check: A Deeper Dive
9. The TaskTracker Alternative: Detection Through Internal Activations
10. Comparing Enterprise Options (Expanded)
11. Competing Perspectives. Deterministic vs. Probabilistic: The Real Trade-Off
12. Key Takeaway

Chapter 10. Red-Teaming AI Systems: Practical Security Testing

1. Why Red-Teaming Is Different
2. Setting Up Automated Red Teams with Promptfoo
3. PyRIT and Garak: Alternative Frameworks
4. Building a Red-Team Playbook
5. The LLMail-Inject Challenge: Lessons from 370,000+ Test Prompts
6. CI/CD Integration
7. PyRIT: Multi-Turn Attack Walkthrough
8. Garak: Comprehensive Vulnerability Scanning
9. Garak: Comprehensive Vulnerability Scanning
10. CI/CD Integration: Making Red-Teaming Continuous
11. The LLMail-Inject Challenge: Lessons from 370,000+ Test Prompts (Expanded)
12. Practical Red-Team Exercise: A Complete Walkthrough

Chapter 11. Production Deployment: Monitoring, Incident Response, and Compliance

1. Runtime Monitoring and Anomaly Detection
2. Data Governance in AI Systems
3. Incident Response Playbooks
4. Compliance Frameworks
5. Runtime Monitoring: A Practical Implementation Guide
6. Incident Response Playbook (Expanded Template)
7. Compliance Walkthrough: NIST AI RMF
8. Compliance Walkthrough: EU AI Act
9. The Real-World Cost of Prompt Injection Incidents
10. Key Takeaway

Chapter 12. The Future of AI Security: Emerging Threats and Defenses

1. Multimodal Injection
2. Supply Chain Attacks in AI
3. Adaptive Attacks
4. Deterministic vs. Probabilistic Defenses
5. The Role of Regulation and Standardization
6. The Adaptive Arms Race: Attack and Defense Evolution
7. Multimodal Injection: The Next Frontier
8. The Supply Chain Attack Surface: Beyond Code
9. The Adaptive Arms Race: Attack and Defense Evolution
10. The Regulatory Landscape: A Practical Guide
11. Supply Chain Security: Beyond Code
12. The Regulatory Landscape: A Practical Guide
13. Final Synthesis: Building a Security-First Culture

Conclusion: Trust as a Design Principle

Glossary

References / Endnotes