As teams move towards agentic SDLC, engineers are focusing on intent rather than writing code, which often means that they have a lower understanding of how the implementation actually works. As a consequence, code reviews risk becoming shallow or could be fully delegated to AI. Previously, issues missed during review were mostly edge cases in systems that the team usually understood in depth, but the nature of issues has changed and now agents can create both subtle implementation issues and a missed obvious user need.
Reviewing code you did not write in a system you don’t understand in depth is hard, and it requires more cognitive energy. Engineers facing this cognitive load might suffer automation bias and rely on the rest of safety layers (e.g. other reviewers, automated tools, or existing tests), reducing the overall effectiveness of those layers.
Engineering leaders must prevent that “the AI hallucinated” becomes the modern “human error”. As Sidney Dekker already explained in The Field Guide to Understanding ‘Human Error’, blaming the final human action (or, in this case, the AI) only describes the final visible event, but it does not explain why the change was accepted, with the risk of stopping the investigation where it should actually start.
Lessons from other industries
While AI-generated code is new, managing systems where humans don’t understand the exact actions a system is taking is something that multiple other industries have been working on for decades, with one clear lesson: do not attempt to build a perfect machine.
Manufacturing scaled by investing in sensors, guards, jigs and processes. The machines were still producing defects, but companies implemented independent inspection (human and automated) and measurable tolerances.
In aviation, as automation controlled more routine flying, the pilots became supervisors of automated systems. This improved safety but also introduced new risks like losing some of the skills they needed when the automation failed. The industry mitigated this through recurrent simulator training, active monitoring practices, and systematic learning from incidents.
These industries share a pattern for scaling automation: they started with manual inspection, then moved to automated verification, measuring defect rates and impact, monitoring drift, and using humans to focus on the cases that are new, exceptional or have a high consequence. Software engineering is going through this same journey: we automated tests and delivery pipelines but, until very recently, relied heavily on manual code review.
Containing the impact
Applying this pattern to software engineering means that we cannot trust the code to respect its own limits. The limits need to be enforced outside the implementation so neither the agent nor an engineer can bypass them accidentally. AI-built software needs constraints enforced at platform level rather than at engineer or agent level, using architectural boundaries and controls like least-privilege access.
The goal of these controls is to limit what the code can do by restricting where it can be used and containing the impact of a missed issue. It’s the equivalent of a physical guard where the machine can produce an incorrect result but it cannot go beyond its boundaries.
Senior leaders must ensure that the assumptions behind those controls are understood and owned by the engineers, and not consumed blindly. Engineers must understand why a limit was set, why a deployment stopped, and what the boundary protects. Those assumptions will eventually change as the system evolves, so they also need to understand when the original reason for that control does not apply any more.
Building these controls requires considerable engineering effort, so part of the efficiency savings provided by AI will need to be reinvested in the platform capabilities that make AI-generated code safer to release. This in turn requires political capital that engineering leaders must defend from stakeholders expecting immediate feature velocity.
Maintaining operational understanding
With agents producing more of the implementation, engineers won’t understand the codebase line by line any more. We need to move up a level by focusing on understanding the system behaviour, its boundaries and how it fails. This does not mean that we stop reviewing code. As Ian Cooper argues, we should review it more closely when certainty is low and blast radius is high. The challenge is figuring out how to trust a system when it’s not realistic any more to understand every part of the implementation.
As teams move to agentic SDLC, engineers need to shift left (product intent, discovery, architecture) and shift right (system monitoring, diagnosis, recovery). Automated tests cannot solve this entirely because they verify something that was previously defined, so new incidents often come from interactions that nobody included in the test suite.
To avoid becoming passive supervisors, engineers must maintain their operational understanding in a similar way to how pilots maintain their readiness: through examining not only incidents but also near misses, working on critical paths when appropriate to keep hands-on skills, and running chaos engineering exercises. Engineering leaders must redesign the engineer role so that there is less implementation and more active responsibility for how systems behave in production.
When an organisation generates code faster than it can safely operate the system, the benefit should not be called productivity. The important outcome is how ready the team is to contain a failure, diagnose it and recover from a complex incident that AI cannot solve, under high pressure.