Agentic QA

Preface

1. Who this book is for
2. What you will build
3. How this book is organized
4. How to use the code
5. Part 1 — Foundations

From Copilot to Crew: Why Agentic QA

1. Four ways to use AI in testing
2. Where agents help today, and where they hurt
3. You are the test lead now
4. Meet QA Crew
5. Setting up the companion project
6. Summary

Anatomy of a QA Agent

1. The five parts of an agent
2. A vendor-neutral model layer
3. The agent loop
4. Structured output: results you can test
5. Your first QA agent
6. Testing the agent loop
7. What you built
8. Summary

Tools and MCP: Giving Agents Hands

1. The problem with hand-written tools
2. What MCP is
3. Using an existing server: Playwright MCP
4. Building qa-tools
5. Connecting the agent to MCP
6. One server, every assistant
7. Tool safety
8. Testing the tools
9. What you built
10. Summary

Where QA Plugs into Agentic Development: SDD and ODD

1. Spec-Driven Development
2. Organic Driven Development
3. Choosing a workflow, and where QA fits
4. One list of requirements
5. The trace report
6. What you built
7. Summary
8. Part 2 — Building Single QA Agents

The Analyst: From Requirements to Risks

1. What the Analyst does, and what it does not
2. Grounding: giving the Analyst a memory
3. The risk map contract
4. The Analyst’s instructions
5. The model proposes, code checks, you decide
6. Running the Analyst
7. A real run
8. Testing the Analyst
9. What you built
10. Summary

The Designer: Generating Test Cases You Would Actually Keep

1. From risks to test cases
2. Techniques as instructions
3. Test data: ask code, not the model
4. The test plan contract
5. The Designer agent
6. Curation: code keeps the plan honest
7. Running the Designer
8. A real run
9. Testing the Designer
10. What you built
11. Summary

The Executor: Agents That Write and Run Playwright Tests

1. Conventions first: fixtures and page objects
2. Quality gates: rules a test must pass before it exists
3. A sandbox for generated tests
4. The Executor’s procedure
5. Trust, but verify
6. Running the Executor
7. A real run
8. Exploratory testing with a charter
9. Testing the Executor
10. The crew so far
11. What you built
12. Summary

The Triager: From Red Build to Actionable Bug

1. Five answers to one question
2. Evidence first: what code collects
3. The tools: the tracker, history and what we know
4. The Triager’s rules
5. Guards: when the answer and the evidence disagree
6. A red build to triage
7. Triaging the Executor’s findings
8. Testing the Triager
9. What you built
10. Summary
11. Part 3 — Multi-Agent Orchestration

Orchestration Patterns for QA Teams

1. Five ways to organize a crew
2. A run, on disk
3. The crew’s four stages
4. Gates: where the human decides
5. A budget for the whole run
6. The right model for each role
7. Running the crew
8. When things go wrong
9. Fan-out: should test cases run in parallel?
10. Testing the orchestrator
11. What you built
12. Summary

The Reviewer: Agents That Check Agents

1. What a reviewer is for
2. The rubric
3. Code first: facts and suspicions
4. The Reviewer
5. The evaluator loop
6. The crew catches TC-05
7. Who reviews the reviewer?
8. What a reviewer costs
9. Testing the Reviewer
10. What you built
11. Summary

QA Crew Inside Your IDE and Your Team

1. Three kinds of tools
2. Asking the person, not the assistant
3. Long work behind a quick tool
4. Files as resources, routines as prompts
5. Setting it up
6. A session in the editor
7. The bug, filed
8. The crew in your team
9. Testing the crew server
10. What you built
11. Summary
12. Part 4 — Production and Trust

Evaluating Your QA Agents

QA Crew in CI/CD

Testing Software That Has AI Inside

Appendix A — Setting Up Your Environment

Appendix B — Mapping QA Crew to Agent Frameworks

Appendix C — Prompt and Rubric Library