▶ YouTube Summaries

• YT :: https://www.youtube.com/watch?v=PDNQsGJxmB4
• Original title :: AI Coding Is Making Bad Estimates Look Accurate

The video argues that a coding-speed gain from AI should not be turned into a shorter delivery date. AI does speed up bounded, well-specified tasks, but discovery, review, testing, integration, release and recovery do not automatically speed up with it. The operating rule is to estimate verified scope and validation capacity, not code volume, and to treat any speed gain as capacity upside until comparable completed work supports using it in a forecast. The author recommends a small bounded test on one class of work and a measured answer for leadership.

The problem with cutting dates

• A local gain gets converted into a commitment, e.g. "12 weeks is 8".
• Code is only part of getting something safely into production. Discovery, review, tests, integration and release decisions remain.
• Dashboards counting tickets moved, code generated or PRs opened show activity, not delivery confidence.

What the research shows

• GitHub Copilot controlled experiment (2023): developers finished a bounded JavaScript task 55.8% faster.
• METR study (2025): experienced open-source maintainers fixing real issues in repositories they knew took 19% longer with the AI tools of the time. The lesson is that context matters, not that AI makes developers slower.
• Google Research (2024): about an hour of active author work between sending a change for review and submitting it, and more reviewer comments meant more work. Review is real work that estimates tend to omit.
• DORA (2024): AI adoption was associated with some reported gains in code quality and review speed, and also with lower delivery throughput and stability. This is an association, but enough to reject a single productivity multiplier.

Review, tests and rework

• A change looks done when the code exists. Review, tests and production then each find further issues.
• Review capacity and trust do not scale with code output, nor does the ability to catch odd failures before customers do.
• Bigger or stranger generated changes can bring new unknowns and unchecked assumptions.

Estimating verified scope and validation capacity

• Classify the work: boilerplate, bounded bug fix, cross-cutting change, incident cleanup.
• Define what must be true to call it done: tested, reviewed, integrated, released, observed in production.
• Identify the bottleneck: who reviews, what they are not doing meanwhile, and what new unknowns appear.
• Ask what local evidence shows this class of work finishes faster end to end. Without it, there is no basis for cutting the date.
• Use the tool anyway, but call the gain capacity upside, not committed schedule capacity.

A small bounded test

• Pick one repeated class of work, e.g. bug fixes, and keep a baseline.
• Track end-to-end cycle time, review rounds, reviewer time, post-merge fixes, rollback or incident signals, and task context.
• Apply an AI planning assumption only to that cohort, and only if the whole path improves and quality guardrails do not worsen. Repeat before making it a habit.

What to tell leadership

• AI can speed up some bounded implementation work.
• It is not yet known to shorten the delivery path end to end.
• So the team will forecast validated scope, show the uncertainties, and measure this class of work before converting it into a promise.
• Use AI to remove typing friction, without turning a local gain into an organizational promise.