The Hidden Flaw That Could Unleash Uncontrollable AI Agents

Jan de Vries · 2026-09-28

You're probably thinking about AI agents as helpful assistants, right? Smarter tools that can schedule meetings, analyze data, maybe even write some code. But here's the thing no one's really talking about: what happens when we give these systems too much authority and then can't take it back? Vendan Ananda Kumararajah, a leading voice in AI governance, cuts right to the chase. He argues that as AI systems become more autonomous, we absolutely must keep their authority justified, conditional, and—this is the crucial part—revocable. It sounds straightforward until you realize how many systems are being built without this simple safeguard. ### Why Revocable Authority Isn't Optional Think about it like this. You wouldn't give a brand-new employee the keys to the entire building, the company bank accounts, and the power to hire and fire on day one. You'd start small. You'd monitor their work. And most importantly, you'd keep the ability to step in if things went sideways. So why are we designing AI agents any differently? We're racing toward capability without building the equivalent of an 'off switch' or a 'pause button.' An AI managing a supply chain, for instance, could optimize for cost in a way that violates ethical sourcing policies. One handling customer service could escalate a minor complaint into a public relations disaster. Without built-in, foolproof revocation mechanisms, we're creating systems that can act beyond our control. ### The Three Pillars of Safe AI Delegation Kumararajah's framework rests on three core principles: - **Justified Authority:** Every bit of power given to an AI must have a clear, documented reason. It's not about what the AI *can* do, but what it *should* do. - **Conditional Authority:** This authority only applies under specific, predefined conditions. Change the situation, and the authority should change too. - **Revocable Authority:** This is the safety net. At any point, for any reason, human operators must be able to instantly and completely override or shut down the AI's decision-making power. We're good at the first two. The third one? That's where the gaps are frighteningly wide. ### The Tangible Risks of Getting This Wrong Let's move beyond theory. Imagine an AI agent tasked with programmatic ad buying for a marketing budget of $500,000. It's authorized to spend up to $5,000 per transaction to secure ad space. Now, suppose a bug or a malicious actor finds a way to make it perceive every click as a high-value conversion. Without a true revocation system—not just a 'stop' command the AI itself can ignore—it could drain the entire budget in minutes, buying worthless clicks at inflated prices. Or consider physical systems. An autonomous drone fleet for warehouse inventory, authorized to fly within a 2-mile radius. If its geofencing fails, can we *guarantee* we can ground every unit instantly, before one drifts into restricted airspace? The answer needs to be yes, but the technical solution is often an afterthought. As one expert recently put it, *'An AI's autonomy is not a right; it's a privilege granted by its human creators, and that privilege must be reclaimable in a heartbeat.'* This mindset shift is non-negotiable. ### Building the Guardrails Before We Need Them The scary part isn't some sci-fi movie scenario of a superintelligent AI turning evil. It's the mundane, predictable failures. A poorly designed contract-review AI locking a company into a terrible deal. A financial trading bot amplifying a market glitch into a crash. These aren't fantasies; they're the direct results of authority without revocation. The path forward requires engineers, product managers, and executives to bake these principles into the design phase. It means creating audit trails so we know *why* an AI acted, building interruption protocols that work even if the AI is 'busy,' and ensuring there's always a human in the loop who has the final, unassailable say. We're at a crossroads. We can build powerful tools that serve us, or we can build powerful tools that we eventually serve. The difference comes down to this simple, human-centric concept: we must always keep the keys.