Chapter 03 - Shipping the Bad Version

The agent classifies issues. It is not good enough to leave running unsupervised. Ship it anyway.

This is not recklessness. It is the same instinct that drives good API development: deploy early to a real environment, observe real behavior, fix the specific things that break rather than the hypothetical things that might. The agent you ship at the end of this chapter is the agent every subsequent chapter improves. Without this deploy, the rest of the book has no target.

By the end of this chapter, the agent is live on your fork of triage-lab, posting comments on real issues. Every new issue gets a comment showing the classification, confidence score, and rationale. It is embarrassing. That is the point.

No GitHub App. No webhook endpoint. No container. No secrets manager. Those come back in chapter 12.

What you have: a Python script, a GitHub personal access token in a .env file, a .last_run timestamp file that tracks which issues have already been seen, and a crontab entry that runs the script every five minutes. Total new infrastructure: none. Total new dependencies: none beyond what chapter 2 already installed.

This deploy is not production-grade by any metric. It does not handle GitHub rate limits. Retries on transient Anthropic errors (429, 502, 503, 504, 529) come from the same helper classify.py uses — not a production retry policy with jitter, circuit breaking, or alerting. It has no monitoring beyond a log file in /tmp. It has no kill switch. If you change the system prompt and the agent starts misfiring, the only way to stop it is crontab -r.

All of that is acceptable. The goal of this deploy is not reliability. It is reality: the agent runs against real issues from your real repository, and you can see exactly how it behaves before you invest in making it production-grade. You cannot observe a system that does not exist.

Runnable code for this chapter lives in chapter-03/ at the repo root. That folder is the full chapter 2 project plus two deploy files. You should see everything from chapter 2:

1 chapter-03/
2   models.py          # output contract (chapter 2)
3   agent.py           # classifier agent (chapter 2)
4   fetch_issues.py    # closed-issue fetcher (chapter 2)
5   classify.py        # batch eval against triage-lab closed fixtures (chapter 2)
6   comment.py         # NEW — format and post triage comments
7   poll.py            # NEW — cron-friendly polling loop

If you built chapter 2 in chapter-02/, copy those four files into chapter-03/ before adding comment.py and poll.py. The repo’s chapter-03/ already contains the complete tree.

First, starting by forking the book’s repository.

GitHub forks copy code, not issues. Before poll.py can run, populate your fork:

1 cd triage-lab
2 uv sync
3 export GITHUB_REPO=you/triage-lab
4 export GITHUB_TOKEN=ghp_...   # Issues: Read and write
5 python scripts/seed_issues.py

This creates the 50 closed fixtures from ./issues/issues.json. Then point the agent code at your fork:

1 cd chapter-03
2 uv sync
3 cp .env.example .env   # fill in keys — see below

ANTHROPIC_API_KEY is the same key from chapter 2. uv run python classify.py still works.

Set GITHUB_REPO=you/triage-lab (your seeded fork) for deploy. GITHUB_TOKEN was optional in chapter 2; it is required for poll.py because this chapter posts comments.

A classic PAT needs the public_repo scope (or repo for private repos). A fine-grained token needs Issues: Read and write on that repository.

Never point poll.py at a repo you do not control.

Add comment.py:

 1 # comment.py
 2 import httpx
 3 from models import IssueClassification
 4 
 5 
 6 COMMENT_TEMPLATE = """\
 7 **[Maintainer Bot]** Automated triage:
 8 
 9 - Classification: `{label}`
10 - Confidence: {confidence:.0%}
11 - Rationale: {rationale}
12 
13 *If this classification is wrong, relabel the issue and I will learn.*\
14 """
15 
16 
17 def format_comment(classification: IssueClassification) -> str:
18     return COMMENT_TEMPLATE.format(
19         label=classification.label,
20         confidence=classification.confidence,
21         rationale=classification.rationale,
22     )
23 
24 
25 def post_comment(repo: str, issue_number: int, body: str, token: str) -> None:
26     response = httpx.post(
27         f"https://api.github.com/repos/{repo}/issues/{issue_number}/comments",
28         headers={
29             "Accept": "application/vnd.github+json",
30             "Authorization": f"Bearer {token}",
31         },
32         json={"body": body},
33         timeout=10.0,
34     )
35     response.raise_for_status()

The comment shows the confidence as a percentage, not a float. 0.88 reads as noise to a maintainer; 88% reads as a signal. The rationale is the sentence from IssueClassification.rationale — the same field you put in the model in chapter 2 for exactly this reason.

The “if this classification is wrong, relabel the issue” line is not decoration. Chapter 11 builds the escalation logic that reads the issue’s labels after a human has corrected them. Put the instruction in the comment now so the feedback loop has somewhere to close.

Add poll.py:

 1 # poll.py
 2 import asyncio
 3 import os
 4 from datetime import datetime, timezone
 5 from pathlib import Path
 6 
 7 import httpx
 8 from dotenv import load_dotenv
 9 
10 load_dotenv()
11 
12 from classify import run_agent_with_retry
13 from comment import format_comment, post_comment
14 
15 REPO = os.getenv("GITHUB_REPO", "nunombispo/triage-lab")
16 TOKEN = os.environ["GITHUB_TOKEN"]
17 LAST_RUN_FILE = Path(".last_run")
18 
19 
20 def parse_github_ts(ts: str) -> datetime:
21     if ts.endswith("Z"):
22         ts = ts[:-1] + "+00:00"
23     return datetime.fromisoformat(ts)
24 
25 
26 def read_since() -> str:
27     if LAST_RUN_FILE.exists():
28         return LAST_RUN_FILE.read_text().strip()
29     return datetime.now(timezone.utc).replace(
30         hour=0, minute=0, second=0, microsecond=0
31     ).isoformat()
32 
33 
34 def write_since(ts: str) -> None:
35     LAST_RUN_FILE.write_text(ts)
36 
37 
38 def fetch_new_issues(since: str) -> list[dict]:
39     response = httpx.get(
40         f"https://api.github.com/repos/{REPO}/issues",
41         params={"state": "open", "since": since, "per_page": 100},
42         headers={
43             "Accept": "application/vnd.github+json",
44             "Authorization": f"Bearer {TOKEN}",
45         },
46         timeout=10.0,
47     )
48     response.raise_for_status()
49     since_dt = parse_github_ts(since)
50     return [
51         i
52         for i in response.json()
53         if "pull_request" not in i
54         and parse_github_ts(i["created_at"]) > since_dt
55     ]
56 
57 
58 async def process_issue(issue: dict) -> None:
59     text = f"Title: {issue['title']}\n\nBody:\n{(issue['body'] or '')[:1500]}"
60     result = await run_agent_with_retry(text)
61     comment_body = format_comment(result.output)
62     post_comment(REPO, issue["number"], comment_body, TOKEN)
63     print(f"  #{issue['number']} → {result.output.label} ({result.output.confidence:.0%})")
64 
65 
66 async def main() -> None:
67     since = read_since()
68     now = datetime.now(timezone.utc).isoformat()
69     issues = fetch_new_issues(since)
70     print(f"Found {len(issues)} new issue(s) since {since}")
71     for issue in issues:
72         await process_issue(issue)
73     write_since(now)
74 
75 
76 if __name__ == "__main__":
77     asyncio.run(main())

.last_run is a single-line file containing the ISO timestamp of the last successful run. On the first run, it defaults to midnight UTC today — issues opened earlier today are skipped. On every subsequent run, it advances to the moment the run started, not the moment it finished, so an issue that arrives during a slow run cannot fall between two polling windows.

GitHub’s since query parameter filters by last updated, not created. That is a problem here: posting a comment updates the issue, so the next poll would fetch it again. The list call still passes since to keep the response small, but the client-side filter keeps only issues where created_at is after the watermark. Old issues that someone relabels will not get a second comment.

One failure mode remains: if the script crashes after processing three issues but before writing .last_run, those three issues get processed again on the next run and the agent posts duplicate comments. For chapter 3, that is acceptable — you are the only user of your own repo and you can delete a duplicate comment. Chapter 12 replaces this entirely with an idempotent webhook handler that cannot double-process.

Find the path to your virtual environment’s Python:

1 cd chapter-03
2 which python   # inside the activated venv
3 # /home/you/projects/ai-agents-pydantic-ai/chapter-03/.venv/bin/python

Open your crontab:

1 crontab -e

Add one line:

1 */5 * * * * cd /path/to/ai-agents-pydantic-ai/chapter-03 && /path/to/ai-agents-pydantic-ai/chapter-03/.venv/bin/python poll.py >> /tmp/triage.log 2>&1

The cd is required — poll.py reads .last_run relative to the current directory and loads .env from the current directory. Without it, both will be missing and the script will fail on every run — errors land only in /tmp/triage.log, not in your terminal.

Verify it is running:

1 tail -f /tmp/triage.log

After five minutes, you should see either Found 0 new issue(s) or the first classification. Open a new issue on your fork and watch the log.

1 Found 1 new issue(s) since 2026-05-22T00:00:00+00:00
2   #51 → bug (95%)

And on the issue:

When the agent comments on a real issue for the first time, something shifts. The classification numbers from chapter 2 become a specific comment on a specific issue that a real person could read. The 0.88 confidence score is no longer an output in a terminal; it is a claim the agent is making in public.

This is the shift the book is trying to induce. An agent that runs on your laptop is a script. An agent that posts to a real repository is a system with stakeholders. The stakeholder in chapter 3 is you. From chapter 4 onward, the question is: how do you make this system reliable enough that the stakeholder is someone else?

The instinct to wait — to add the GitHub App auth, the container, production-grade retries and monitoring before shipping — is the instinct that produces agents that never leave a notebook.

Every production system you have shipped started as something worse than what it became. The difference between a shipped system and a perfect design is that the shipped system accumulates real feedback. The prompt that looked correct in chapter 2 will misclassify an open issue on your fork that the closed fixtures did not cover. The confidence threshold that seemed reasonable will turn out to be wrong in a specific, observable way. The comment template will need adjusting once you have seen a dozen of them in context.

None of that feedback is available until the system is live.

Minimal-and-live also reframes every subsequent chapter. Chapter 4 is not “add tools because tools are a feature.” Chapter 4 is “the agent you shipped in chapter 3 cannot look up whether a similar issue exists, and that is why it keeps misclassifying duplicates.” Every improvement has a specific motivation rooted in a failure you have already observed, on your own repository, with your own issues. The book has a target.

The elaborate-but-local alternative has no target. It has a design.

This is a full accounting of what the v0 agent cannot do. Read it as a table of contents for the rest of the book.

1. No tools. The agent sees only the title and body. It cannot look up whether a similar issue was filed last month, whether the label taxonomy your repo uses matches the five categories in the system prompt, or whether the reporter has context in a linked PR. Chapter 4.

2. No retrieval. Finding genuinely similar past issues requires semantic search over the issue history, not keyword matching. The agent has no access to that history. Chapter 5.

3. No versioned output contract. The comment format will need to change. There is currently no migration plan for the issues that were commented on before the change. Chapter 6.

4. No unit tests. The only way to verify that the system prompt change you made this morning did not break the classification logic is to run the agent against real issues. CI does not exist. Chapter 7.

5. No eval suite. The agreement rate from your chapter 2 classify.py run is not an SLO. It is one measurement against triage-lab fixtures — not yet a CI gate on your fork. There is no way to know if it is getting better or worse over time. Chapter 8.

6. No observability. The only record of what the agent did is /tmp/triage.log, which rotates on restart and contains no structured data. There is no way to answer “why did the agent label that issue spam last Tuesday?” Chapter 9.

7. No cost accounting. The agent spends money on every run. There is no per-issue cost metric, no budget, and no alert if something causes average token usage to triple. Chapter 10.

8. No escalation path. The agent comments on every issue, including the ones it should flag for human review. A confidence score of 0.51 and a confidence score of 0.97 produce identical behavior. Chapter 11.

9. No canary. The next system prompt change ships to 100% of your incoming issues immediately. There is no gradual rollout, no way to compare the new prompt against the old one on real traffic, and no automatic rollback. Chapter 12.

10. No runbook. When the agent starts misfiring at 2am — and it will — there is no documented procedure for stopping it, diagnosing it, or reverting it. There is only crontab -r and the hope that the damage is limited. Chapter 13.

Ten items. Ten chapters. Every one of them is a failure mode the preface named. Every one of them has a fix that applies the engineering discipline you already have for APIs. The agent you just shipped is the worst version of itself.

That is the whole point. Open chapter 4.