▶ YouTube Summaries

• YT :: https://www.youtube.com/watch?v=TFCDWrEBaFM
• Original title :: How Claude Fixes It's Own Code

Theo reads Anthropic's "Solving code review at Anthropic" post about how the Claude Code team reviews code when agents write almost all of it, and compares it with T3 Code, which gets up to 800 PRs a day with a three-person team. He agrees with most of it: layered automated checks, verification before a PR exists, risk-based human approval, fast rollback, and capping review rounds so agent author and agent reviewer don't loop forever. His main disagreement: when a reviewer keeps flagging the same thing, fix the codebase and static checks first, and only then add CLAUDE.md / REVIEW.md / skills prose.

Why code review has to change

• Anthropic's monorepo went from ~2,500 changes landing per day in May to 30,000+ in September; merged PRs to Claude Code more than doubled June-September while the revert share fell 40%+. Theo credits model improvements (Opus 5.5) as much as process - his own revert rate also fell sharply after Opus 5.5 and after banning Gemini models from landing code.
• Code review has two jobs: catch issues before shipping, and spread shared understanding of the codebase. Theo calls the second the most underrated.
• "Did a human read the code" is a poor proxy for "what could go wrong and which check - automated or human - should catch it". Theo: none of the regressions shipped in T3 Chat/T3 Code would have been caught by a human reading the diff; agents are now better reviewers than most humans, and if not, the codebase is the problem.
• Review is no longer a gate at the end but continuous, like spell-check. Theo: every PR (not "most") should be agent-reviewed before a human sees it.
• The scarce resource is human time and attention. Theo adds human energy - keep people off painful, draining work - and the wait time between PR, review, fix and re-review, which often dominates.
• Anthropic's goal: most Claude Code PRs merge with at most one human (possibly the author) without more reverts or incidents. Metrics: human approvals per PR, human comments, time waiting on a person; revert and incident rates are the guardrail.

The layers a change passes through

1. Verification before the PR: tests, build, lint, a fresh-context subagent review, internal skills for bug-hunting, simplifying the diff, end-to-end checks, design-guideline checks for UI.
2. Required CI: lints, types, cross-platform tests, build and packaging; red blocks merge.
3. Claude code review: parallel agents hunt bugs, verifier agents vote on each finding, survivors ranked by severity.
4. Separate security review.
5. Risk-based approval: a classifier picks zero, one or two humans.
6. Merge queue, internal feature flag, gradual release (latest channel before stable), monitoring, fix forward or roll back.

Theo likes the "safety net" framing but warns that every layer can also add code: a 50-line fix bloats to 800-2,000 lines when the agent addresses every comment from every layer. Most important layer is the last one: rollbacks must be cheap and the blast radius small.

Verification is the biggest lever

• Boris Cherny says giving Claude a way to verify its work makes results 2-3x better: a bash command, a test suite, a browser or phone simulator, or something much more complex.
• Theo's T3 OS setup: each agent thread gets its own macOS-workspace-like window, can spin up an Electron dev server of T3 Code, record a video of the change working and attach it to the PR. Dev builds have fake model selections replaying pre-baked events, so agents run full end-to-end tests without burning model tokens.
• Code that's hard for an agent to check is code that's hard for a new hire to work in.

Where to put fixes for repeated mistakes

• Anthropic: put recurring reviewer findings into a skill, a lint rule, CLAUDE.md or a code-owner-written REVIEW.md (short, says what failure looks like in that area, caps nits, lists paths to skip).
• Theo (citing Lauren's guidance) orders it differently - fix as close to the top as possible:

1. The codebase itself: better architecture, not comments everywhere.
2. Static analysis: linters, type checks, custom rules.
3. Only then prose: CLAUDE.md, AGENTS.md, REVIEW.md, skills.

• Mental model: a brilliant new hire who has seen thousands of codebases but not this one and tolerates any tedium; make it hard for them to make dumb mistakes. T3 Code's AGENTS.md changes about once a day and only when the repo itself changes, not to patch agent misbehaviour. Analogy: fix the holes in the maze rather than giving longer instructions at the door.

AI review on every PR

• Claude's review reads the diff against the whole repo and the PR's stated intent: does it solve the problem, fit the system, break anything; for bug fixes, find the root cause. After the research preview, internal PRs with substantive comments went from 16% to 54%; most 1,000+ line PRs get findings, most small ones come back clean. Quick/standard/deep levels.
• Theo doesn't recommend the paid Claude review product - too expensive at this shipping rate. He uses a "council" skill: three models from three families review on a real machine using subscriptions, and it can run the code.
• Rant to AI-review vendors: drop the "copy this prompt into your agent" suggestions. The author's agent should babysit the PR and decide which feedback is valid; humans shouldn't see bot-to-bot comments.
• Anthropic filters findings: multiple agents vote, style/cleanup-only findings are dropped (A/B tested), the rest are tagged important / nit / pre-existing. Every surviving finding must be fixed or resolved with a reason; often an agent fixes them overnight. Theo's PRs go through 8+ rounds in an hour or two.
• Security review findings also block merge until fixed or closed with a reason; later commits that fix an issue clear it automatically.

Risk tiers and approvals

• Low risk: no human; medium: one; high: two. The tier depends on risk, not on who authored it. Theo would prefer a more dynamic scale and is considering a PR-template blurb rating impact, users affected and risk 1-10.
• Bots should describe risk, not pick which human reviews.
• Cursor/Graphite friends say risk tiering is a losing battle in their experience.
• Some paths always need a person: owner-listed sensitive paths, CI workflow files (Theo: anything touching scroll areas in T3 Code).
• Only trusted authors get light paths; outsiders need a second approver. An agent's approval never counts as a human's. Theo wants GitHub to show "Theo's bot approved" separately from "Theo approved".
• Whoever prompted the change owns it. Approvals cover specific commits; agent pushes or hand-resolved merges reset them.
• Weekly on-call; incident review asks which layer should have caught it. No circuit breakers yet that tighten approval when revert/incident numbers rise - wanted before more zero-human merges. T3 Code already zero-human merges easy bug fixes (how Julius lands ~400 PRs/day).
• "Slowing down lowers your error rate mostly because you do less of everything" - get faster at noticing and fixing instead.

Improving the reviewer and making reviews converge

• Incidents become evals: past PRs that caused incidents (with fixes/reverts), confirmed by engineers who know the code, run against the reviewer daily.
• Noise is measured by what authors do with each comment (fixed, wrong, intentional, didn't matter) and by running the reviewer on known-clean PRs. Priority: catch real bugs first, cut review rounds second.
• Noise costs more with agent authors: a human waves off a nit, an agent fixes it, triggering another commit and review round.
• Non-convergence: one CLI PR went 50+ review/fix rounds overnight. Theo's own "iterate until the whole council passes" run went four days and grew the PR to ~80k lines. git blame showed ~60% of repeat findings were in code present since the first pass.
• Anthropic's fixes: cap rounds (after three, ask the author to reconsider the approach), diff-only re-review after the first passes, flag a bug class so all instances get fixed, never repeat posted findings, plus nightly review agents over existing code. Results: +13% PRs passing after one round; PRs needing 5+ rounds dropped from 9% to 1.5%; mean rounds to pass 2.08 → 1.32.
• Theo's fix: reviewers give a 1-10 score with a pass bar instead of counting comments (18 nits can still be a 9; one comment can be a 6). Jake (Graphite) argues for a boolean instead: a well-contextualised agent should decide "can this comment be ignored or not", like a paid engineer would.

Where humans stay in the loop

• Direction, taste, scope - "should we even do this?"; one team that solved its review bottleneck still kept human review for outside changes.
• Risky areas: billing, auto-updater, CI workflows.
• Architecture, because it sets patterns others copy.
• Reading the evidence: test output, commands and results, screenshots, review findings. If you can't say what the checks would miss, ask for a real review rather than a quick approval.
• Owning the outcome regardless of where the code came from.

Sharing context without line-by-line reading

• Nobody holds a full mental model of Claude Code, and nobody needs to. Theo: any codebase small enough for one person to understand changes too slowly to be a full-time job.
• Ask Claude for an explainer of exactly the slice you need, reflecting current code; some engineers have it build an interactive artifact. Theo pastes heavy PRs into T3 Code asking what changes, why, and where the risk is. Better use of attention than reading trivial diffs.
• Default to open Slack channels so people see work in progress.
• "You land it, you own it": small reversible changes, blast radius sets the bar, fast regression detection via telemetry and internal use; ownership means context, not blame.
• Agents make starting cheap, so plenty of work shouldn't land - about half of Theo's T3 Code PRs get closed. Raise a different direction early in the thread, not as a blocking comment or a competing PR.

Adoption checklist (from the post)

• Before the PR: runnable checks for the agent, local code-review and security-review skills, REVIEW.md, fix the code first.
• Every PR: required CI; PR template with how to test, risk and how to revert; AI reviewer tuned for a few weeks; automated security review; all threads resolved before merge; review depth matched to risk; capped rounds.
• Approval: written risk tiers based on the change, lighter paths for trusted authors, CODEOWNERS and branch protection for auth/billing/CI/release, bot approval never replaces a human's.
• Merge and after: merge queue, feature flags, internal use first, staged release, one-step rollback; turn incidents and reverts into evals; rescore the reviewer on prompt/tool/model changes; measure human time alongside revert and incident rates. Theo: anything you can measure, agents can hill-climb against.

Closing take

Theo disagrees with the post's conclusion that a missed review should lead to a CLAUDE.md, skill or check fix - fix the code first. He welcomes Anthropic publishing this much about its internal process (authors include Addy Osmani and Emma), complains about the post's LLM-ish prose, and notes the post invites feedback on which checks to automate.