Humans Hang On for Dear Life as AI Accelerates Software Delivery

PUBLISHER ARTICLE PREVIEW

From the original article

For most of the history of software development, people have been the control. A developer makes a change, another person reviews it, someone approves it, and eventually it gets deployed.

AI is accelerating that entire system while we’re still trying to keep people in the middle of it. Developers can now create code and changes in seconds. Agents can work across repositories, tools, infrastructure, and other systems with less human involvement.

Our instinct is to add humans back into the process. We review the pull request, approve the tool call, check the change, and confirm the deployment because we want to make sure the AI didn’t do something it wasn’t supposed to do. We’re hanging on for dear life. That instinct makes sense.

Human review has given us a way to maintain control as software moves toward production. But AI is starting to operate at a speed and volume where humans can’t remain the unit of scale for governance.

The first wave of generative AI in software development focused mostly on helping developers write code faster. That alone changes software delivery. More code means more application changes, infrastructure changes, and database changes moving through testing, security, review, deployment, and production.

The problem isn’t necessarily that AI creates worse changes. It creates more changes, faster. If the control for all of that new output is another person reviewing every change, eventually the math stops working.

We’re already seeing signs of it. Anthropic recently reported that Claude Code users approve about 93% of permission prompts. The company found that repeated prompts can create approval fatigue, with people paying less attention as the number of approvals increases. Anthropic is now using an automated classifier to evaluate actions and stop potentially dangerous ones rather than asking a person to approve everything.

Think about what that says about human oversight. If someone clicks approve 93% of the time, adding another approval doesn’t necessarily give you more control. At some point, the human becomes another step in the workflow.

We can use AI to create more software. We can’t respond by creating an equally large human review operation behind it.

Coding assistants gave AI a role in development. Agents give AI the ability to participate in much more of the SDLC. An agent can receive a goal, decide how to accomplish it, use tools, observe the results, and adjust what it does next.

In software engineering, that can mean modifying files, running commands, interacting with repositories, calling APIs, testing code, or working with infrastructure. People are also getting more comfortable letting agents work on their own. In a study of millions of human-agent interactions, Anthropic found that experienced Claude Code users used full auto-approval in more than 40% of sessions, roughly twice the rate of new users.

That doesn’t mean autonomous agents are running production environments everywhere today. They’re not. But software development gives us an early look at where this is heading.

Today, AI creates more change, and human review starts to strain. Next, AI participates across more of the SDLC. Eventually, agents will create, validate, deploy, observe, and remediate changes with much less human involvement.