An AI agent hands you a diff. It’s well-formatted, the variable names are sensible, the tests pass, and it reads like something a competent engineer wrote in five focused minutes. That confidence is exactly the problem — code that looks right is easy to approve without actually checking whether it is right, and AI-written code is unusually good at looking right.
A code review checklist for AI-written code needs to compensate for a specific failure mode: agents are fluent, not necessarily correct, and fluency is what human reviewers instinctively trust. Reviewing AI-written code well means checking for a different set of failure modes than you’d check for in human-written code — not because the bugs are more numerous, but because they’re differently shaped. This post walks through what to actually look for.
Why AI Code Needs a Different Review Lens
Human-written bugs tend to cluster around fatigue, unfamiliarity with a codebase, or rushed deadlines — a reviewer who knows the author can often predict where the weak spots are. AI-generated bugs cluster differently: plausible-sounding but wrong library usage, confidently invented APIs that don’t exist, logic that handles the example case perfectly but nothing else, and code that mirrors a common pattern from training data even when that pattern doesn’t fit this codebase.
None of these failure modes show up as obviously bad code. They show up as code that reads fine and is subtly wrong, which means the review has to actively hunt for specific risks rather than skimming for general quality. It’s a shift from “does this look like good code” to “what specifically would have to be true for this code to be wrong, and did I check for it” — a more deliberate, less intuitive kind of reading.
The Checklist
| Check | What you’re looking for |
|---|---|
| Does it do what was asked, not just something reasonable? | Agents sometimes solve an adjacent problem that’s easier than the one you posed. |
| Are external calls and APIs real? | Verify library functions, endpoints, and config keys actually exist and are used correctly — don’t assume fluency implies accuracy. |
| Does it handle the edge cases, not just the happy path? | Empty inputs, nulls, concurrent access, and off-by-one boundaries are common gaps. |
| Are tests actually testing behavior, or just passing? | An agent under pressure to make tests green can write a test that asserts the wrong thing but still passes. |
| Does it match the codebase’s existing conventions? | Naming, error handling style, and architecture patterns should match what’s already there, not what’s statistically common elsewhere. |
| Is anything deleted or changed outside the stated scope? | A wider diff than expected is worth a second look — agents can wander into unrelated files. |
| Would you be comfortable explaining this code to a teammate? | If you can’t explain why it works, you’re not ready to approve it. |
Read the Diff, Not the Summary
It’s tempting to read an agent’s plain-language summary of what it changed and approve based on that. Summaries are compressed and optimistic by nature — they describe intent, not necessarily the actual lines of code. Always open the diff itself; a summary saying “added input validation” can hide a validation check that only covers one of three input fields.
# Never trust the summary alone — always look at the actual diff
git diff main..agent/add-input-validation
# For a focused look at just the files that matter to this task
git diff main..agent/add-input-validation -- src/forms/
Reading the raw diff also surfaces the scope-creep problem faster than any summary will — you notice immediately if a “fix the validation bug” task also touched an unrelated config file.
Run It, Don’t Just Read It
Reading code catches a lot, but running it catches more. Before approving an AI-written change, actually execute the tests, and where possible, exercise the changed behavior manually — click the button, hit the endpoint, load the page. Static review is necessary but not sufficient for code that was optimized, whether intentionally or not, to look convincing on the page.
This matters more for AI-written code than human-written code specifically because the failure mode is plausibility rather than obvious sloppiness. A human-written bug often has a visible tell — a rushed comment, inconsistent formatting, an obviously missing case. An AI-written bug frequently has none of those tells, so the only reliable signal left is whether the thing actually works when you run it.
A concrete example makes this vivid: an agent asked to add pagination to an API endpoint might produce code that reads cleanly, passes the one test that checks the first page, and still be silently wrong for the last page of results because of an off-by-one in the offset calculation. Nothing in the code itself looks suspicious. The only way to catch it is to actually request the last page and look at what comes back — which static reading, however careful, won’t surface on its own.
Building the Checklist Into Your Habits, Not Just Your Head
A checklist you only remember to use when you’re already suspicious of a diff isn’t doing much work — the diffs that need it most are the ones that look fine at a glance. A few ways to make the checklist a default rather than an afterthought:
- Keep it somewhere you’ll actually see it, like a PR template comment or a pinned note next to your terminal, rather than trusting memory under time pressure.
- Apply it uniformly, even to “small” changes. A one-line diff can still invent an API that doesn’t exist or silently narrow an edge case; size isn’t a reliable proxy for risk with AI-generated code.
- Treat a fast approval as a warning sign, not a compliment to the agent. If you approved a nontrivial diff in under a minute, ask whether you actually ran through the checks or just skimmed for vibes.
- Adjust the checklist over time based on what you actually catch. If invented API calls keep showing up in your reviews, that’s the check to run first next time, not last.
None of this requires new tooling — it’s closer to a habit shift, the same one that made test-driven development stick once developers treated writing a test as a default step rather than something to do “when there’s time.”
Getting Started
- Before approving your next AI-generated diff, run through the checklist above line by line rather than skimming for general impression.
- Open the actual diff instead of relying on the agent’s written summary of its own changes.
- Run the tests and manually exercise the changed behavior, not just the automated suite.
- If you’re reviewing several agent sessions at once, Agents puts a right-side diff review panel next to every session so you can check each one against this checklist without leaving the sidebar — download Agents and try it free for 7 days.
Fluent code and correct code are not the same thing, and AI agents are much better at the first than reviewers instinctively assume. A checklist that specifically hunts for invented APIs, missing edge cases, and scope creep catches what a general “looks good to me” skim will miss. Code review has always been about catching what the author couldn’t see in their own work — AI-written code just changes which blind spots to expect. For the review-request half of this workflow, see prompts for AI coding agents — better prompts produce diffs that need less of this checklist in the first place, but never zero.