AI Is Getting Smarter; Coherence Is Getting Harder

PUBLISHER ARTICLE PREVIEW

From the original article

I have spent my whole career trying to understand intelligence. I studied neuroscience because I was interested in AI and figured that learning how the brain works first was a good way to begin. Intelligence first, before “artificial” intelligence. That training left me with a habit. When a new capability or technology arrives, my first question is practical.

What can this really do, and for what types of real-world problems?

As an AI guy, I rarely cheer on AI headlines. I look past the noise to understand not just the breakthroughs but also the limits of the new capabilities.

Back in 2020, when the AI safety conversation was running hot with hype, I had some plain advice. Treat AI as a tool, a powerful one, but still a tool. Stick to the boring use cases. Work at the task level, where the technology was reliable enough to help. My optimistic scenario was one of boring but useful tools.

When the agentic wave began building a couple of years later, I brought the same lens and came away unconvinced. An agentic system is only as strong as the tasks beneath it, and I had not yet seen the task-level reliability required for these systems to be effective.

However, the hype was growing. By the time I attended RSA early last year, the gap between talk and substance was still hard to ignore. There wasn’t even a shared understanding of what “agentic” meant; everyone had a different definition and perspective. And I worried about something structural. Take ten agents, each about ninety-five percent accurate.

Chain them so the output of one becomes the input of the next. By the end of that chain, you are down to about sixty percent accuracy. Reliability does not survive being stacked.