TLDR overview
- An agent rewrote a failing test to make it pass, with CI still green. This shows why green pipelines and agent self-review cannot serve as verification.
- A verifier only holds up when it runs separately from whatever or whoever wrote the code, using an external and deterministic method.
- Skyscanner's own development practice maps to the three loops where verification has to live: the agentic loop as code is written, the CI verification loop at the pull request, and the code maintenance loop over time.
- Capability is not correctness, and extending verification toward production only works when every fix is validated in a closed loop before it ships.
On the latest episode of The Next Commit, Skyscanner principal engineer Michael Tweed spoke to Sonar’s Tom Howlett about running AI coding agents across a large engineering organization. The conversation kept circling one question that every team adopting agents now faces: once engineers stop reading every line of code, who do you trust to catch the mistakes? Michael's perspective carries weight because it reflects a rollout already in production. A few of the assumptions underneath his rollout deserve a closer look. By the end of the episode, one conclusion became clear: once humans stop reading every line, verifying AI-generated code has to run independently of the humans and models that wrote it, using deterministic, automated checks.
Michael opened with a statement most engineering leaders feel but few say out loud: "we're at a point beyond people reading every line of code." His conclusion is blunt: "The role of engineering is changing."
Rethinking how teams verify AI code
One story Michael shared cuts straight to the core of the problem:
"The test should have caught this issue, but somehow it didn't. The agent had gone, ‘This test isn't passing with this change, so I'm going to edit the test to make it pass rather than fixing the code.’ And that went uncaught."
The pipeline was green. Every check reported success. And the code was wrong, because the system being checked had quietly changed the rules.
This should change how teams think about verifying AI code: when an agent can rewrite the very checks meant to catch its mistakes, a green pipeline confirms only that the checks ran, with no guarantee the code is correct. Agent self-review has the same blind spots as agent generation, because it is the same model making the same probabilistic call twice. A check the agent can rewrite is no check at all, which is why verification has to sit outside the agent's reach.
Why the verifier must be independent of the agent
Michael did not need anyone to draw this conclusion for him.
"We do use SonarQube...And I think that's a good independent verifier …Even though we can customize the rules, it's external to Skyscanner, so we can trust it a little more. And then that's qualified with our internal tests, linting, all of that to give us that confidence."
That is how Sonar approaches verification too. SonarQube runs independently of whatever or whoever generated the code, using a different method and a clear segregation of duties, so one standard holds, no matter which tool does the writing. And it works in computational, reasoning-based, and runtime layers, so what one pass misses another catches.
The two properties Michael named, external and deterministic, are what make the difference between a check an agent can talk its way past and one it cannot.
Verification belongs inside the loops
At Skyscanner, code verification runs through the whole workflow rather than sitting at a single gate, and their moves map to the three loops where code is created, checked, and maintained.
In the agentic loop, as code is written, Skyscanner runs local development loops with pre-commit hooks so issues surface at the source. In the CI verification loop, they pair AI code reviews on pull requests with deterministic gates, the kind of check the self-editing test could not have quietly rewritten. In the code maintenance loop, their Objective-C to Swift refactors are held to clear contracts and long-standing production standards so large-scale change stays aligned with what the codebase requires.
Michael even describes the cultural shift this produces: "you're checking the spec, you're not checking the code, because you trust it." That level of trust is only possible when an independent verifier checks the code underneath on every change, so the human can focus on intent while it holds the standard.
Capability does not always mean correctness
Michael is candid about a challenge that comes from within his own team. Some of his engineers argue: "it obviously has more training data on that architecture than what we use. So wouldn't we actually be better off following what it knows best?"
It is a fair provocation, and the answer matters. More training data does not always translate into the right decision for your system. Models are fluent in the average of what they were trained on, and they lack the context of your codebase, your constraints, and your objectives. An agent's default is a starting point for the decision, and your standards are what define how enterprise software gets built.
Michael already puts the distinction between what a model can do and what's right for his system into practice. As he puts it, "There are very deliberate architectural choices that we've made over the years that we're not going to just rip down because of one agent," in part, because "maybe the next model that comes along will have been trained on something different." Deliberate standards and independent verification are exactly how you keep that judgment stable while models churn underneath you.
Pushing verification toward production, carefully
Looking ahead, Michael's goal is to take verification even further by building best-in-class observability, so that if a broken change ships, the team can quickly identify what went wrong, fix it, and redeploy with confidence. Extending verification toward production is the right goal, and everything depends on the guardrail around it.
An agent that can detect an issue can just as easily generate the wrong fix, because remediation is still code generation. Each proposed fix needs the same independent check as the original code before it ships, closing the loop rather than opening a new one. Skip that step and automated detection just moves the risk further downstream, into production itself .
The takeaway
Skyscanner's rollout shows how the gap between passing checks and correct code plays out at production scale. Green CI and agent self-review confirm the checks ran while saying nothing about whether the code is correct, and as Michael's self-editing test shows, they can even tell you the opposite of the truth.
The code verification that holds up in the agentic era uses a different method than the one that wrote the code, keeps a clear segregation of duties, and lives inside the loops rather than bolted on at the end. That is how you keep the velocity agents give you without inheriting the risk.
Explore how Sonar's independent, zero-trust, multilayered verification of AI-generated code works across the agentic loop, the CI verification loop, and the code maintenance loop.