TLDR overview
- Episode 5 of The Next Commit features Kesha Mykhailov of Fin describing how AI coding agents created a new problem: implementation now moves faster than the organization can think, verify, and learn.
- Fin's experience validates a core truth of the agentic era: faster code generation shifts the constraint to verification rather than eliminating work.
- Fin learned the hard way that an AI reviewer checking only recall missed a surge of false positives, degrading the exact signal developers had come to trust.
- The deeper lesson: generation and verification must draw on shared standards, and the act of challenging code must stay independent.
Speed moved the bottleneck, not the work
In episode 5 of The Next Commit, Kesha Mykhailov, Principal Engineer at Fin, describes a shift most engineering organizations now recognize. AI coding agents made implementation nearly free. The hard work didn't disappear. It moved.
For Fin, adopting AI agents was not an experiment. Kesha frames it as competitive survival against younger, AI-first rivals that outship slower incumbents. That pressure makes speed attractive. It also exposes a problem that grows as agents improve—code can now move faster than the organization can think, verify, and learn.
Ultimately, agents require less work to deliver the same outcome. An agent running at full speed, though, will generate more verification work if that verification isn't done efficiently. Fin's experience shows what that extra verification work looks like day to day.
Why do AI coding agents create more review work instead of less?
Before AI agents entered the workflow, any significant change at Fin started with a conversation. A developer explored an idea, talked to the people who understood the affected system, resolved open questions, and captured the approach in a technical plan. That plan was evidence the thinking had already happened.
Agents collapsed that process, and Kesha has watched it happen in the Slack channel where Fin's engineers share technical plans. "With AI and how fast it is now to generate the code, there's a great temptation to skip that part," he says. "We see less and less of those tech plans shared in the channel. They almost disappeared." With them went the discussions that surfaced hidden constraints and taught newcomers why the system was built the way it was.
The broader range of consequences is one we see across the industry:
- Reviewers reconstruct intent from a large implementation instead of reviewing against a known plan.
- Product teams learn about a change only after it merges and may find it works for one customer but can't scale, leaving rework or a revert as the only fix.
- The code is cheap to produce, yet integrating, verifying, and sometimes reverting it stays expensive.
How do you verify AI-generated code at scale?
Rather than lecturing developers about discipline, the response at Fin changed the economics of the workflow itself:
A pull request above a certain size can't be approved automatically, while a small, safe change can. The size limit works as a guardrail and the auto-approval as a reward, so developers who ship smaller increments spend less time waiting for a human reviewer.
Making the safer behavior the faster behavior means developers split their work into smaller pull requests on their own, because small changes get merged sooner.
This workflow is imperfect, and Kesha says so: "No amount of code review will surface all the unknown unknowns that only could be surfaced in the only objective reality we have, which is production." Lines of code are a rough proxy for risk. That's why Fin layers additional reviews covering security, logical correctness, and the quality of the problem statement, with sensitive changes still routed to human approval. The principle holds regardless of the threshold: when code gets faster to write, the rest of the process has to keep the amount of new code within what your team can verify.
Generation and verification need shared standards and independent judgment
Fin's move from Ember.js to React illustrates what breaks when generation outpaces verification standards.
React lets teams assemble very different stacks around the same library. Fin's experts chose specific patterns for forms, validation, and API access, but early agents produced large React changes that ignored those decisions entirely. A small group of senior engineers became the final verification layer, absorbing an unsustainable volume of pull request reviews.
The team harvested comments on those reviews into a React skill, refining it against real pull requests until it captured what was useful. Then came the key architectural insight—that same skill now guides the coding agent when it writes React code and the review agent when it assesses the diff. One source of engineering judgment, applied on both sides.
When the generator and the verifier draw from separate copies of the standards, those copies drift. A shared source eliminates that duplication, so improvements prompted by review shape the next generation attempt, instead of staying a downstream correction.
A verifier is only useful if people can trust it
Fin's developers relied heavily on their custom review agent, Shrek. They made it green before showing a change to a human, which made the verifier's quality a direct input to delivery itself.
Every review runs many model calls, which Kesha calls "astronomically expensive." To cut costs, Fin tried moving one of Shrek's checks, the one that looks for logic errors, to a cheaper model. Before switching, the team ran the cheaper model against hundreds of historical pull requests, each labeled with the issues the agent should find, to confirm it would still catch them. It did. The recall looked identical, making the change appear safe when it was not. Because developers had built Shrek's approval into their workflow, the noise carried a real operational cost: delayed work, wasted attention, and eroding trust in automated feedback.
Fin's evaluation checked whether the agent still caught real issues but never counted its wrong findings, so a far noisier model passed. Within a week, developers asked whether Shrek had become "a lot more picky."
The lesson: a verifier has to be measured on its noise as well as its catches, because noise trains developers to ignore the signal.
False positives also differ by model, so offline tests built from one model's mistakes won't catch a new model's. Fin now tracks three signals from real pull requests:
- Explicit thumbs up or down on review comments
- Soft resolution, where a final diff adopts a suggestion no one reacted to
- Merges that proceed with human approval despite an active agent flag, a reliable indicator of a false positive.
As Kesha puts it, "We're building this agent for humans, ultimately. None of the evals will replace the real feedback from them." Each signal is closer to the reason the verifier exists, helping engineers ship safely without burning their attention.
The takeaway
Speed is only an advantage if verification keeps pace. Shared standards guide both generation and review. Independent, multilayered verification challenges the result. Production signals prove the verifier is genuinely helping the humans it serves.
Kesha's candid account described what his team has solved and what they haven’t yet, making his interview with Tom Howlett one of the more honest conversations about agentic development you'll hear this year.
Listen to episode 5 of The Next Commit to hear Kesha and Tom talk through the full story.