The Hidden Question Behind 900 Billion AI Agents

Jan de Vries · 2026-09-30

Let's talk about something that feels like it's straight out of science fiction, but is becoming our daily reality. Vendan Ananda Kumararajah recently put a spotlight on a question that should keep every tech professional up at night. Who exactly is in charge when AI agents start making their own decisions? We're not talking about a few helper bots here. The scale is staggering—think 900 billion autonomous systems learning, coordinating, and potentially acting on their own. That number isn't just big. It's a fundamental challenge to how we think about authority and governance in the digital age. If you're working in European startups or following EU Inc developments, this isn't a distant philosophical debate. It's a practical business problem you'll need to solve, probably sooner than you think. ### When Machines Start Making Their Own Rules Here's the tricky part. These AI agents aren't static programs. They learn. They adapt. They coordinate with other agents in ways their original programmers might not have anticipated. So how do you ensure they retain what Kumararajah calls 'legitimate authority' as they evolve? It's like raising a child, but this child can process more data in a second than you could in a lifetime and might be connected to millions of other 'children' doing the same. The traditional model—where humans write the rules and machines follow them—starts to break down. When systems are learning in real-time from vast datasets, the rulebook is being rewritten constantly, often by the AI itself. This creates a governance gap that existing legal and corporate structures simply aren't designed to handle. ### The European Startup Perspective For founders and investors in the European tech scene, this governance question isn't abstract. It's about compliance, liability, and competitive advantage. The EU's regulatory approach, particularly with upcoming AI legislation, means startups building with autonomous systems need to bake governance into their architecture from day one. Consider this: If your startup's AI agent makes a decision that affects a user in another EU country, which jurisdiction's rules apply? What if the agent learned a behavior that violates regulations, but no human programmer instructed it to do so? These aren't hypotheticals anymore. They're design considerations that will separate successful companies from those facing regulatory shutdowns. - **Proactive Compliance**: Building audit trails for AI decisions - **Transparency Requirements**: Explaining agent behavior to regulators - **Liability Structures**: Determining responsibility for autonomous actions - **Cross-Border Coordination**: Managing agents operating across EU markets ### Building Legitimate Authority Into Autonomous Systems So how do we solve this? Kumararajah's examination suggests we need new frameworks. It's not enough to just program initial rules. We need systems where authority and governance mechanisms can evolve alongside the AI's capabilities. Think of it like constitutional design for digital entities. You establish core principles and processes for how rules can change, rather than trying to predict every possible future rule. The system needs mechanisms for accountability, oversight, and course-correction that work at machine speed and scale. As one expert recently noted, 'The most successful AI implementations won't be the most powerful ones, but the ones we can trust to govern themselves responsibly.' That's becoming the real competitive edge. ### What This Means For Your Business Today If you're incorporating a startup in Europe or scaling across EU markets, you can't treat AI governance as an afterthought. It needs to be part of your founding documents, your technical architecture, and your investor pitches. Regulators are watching, and the companies that figure this out first will have a significant advantage. Start asking questions now. How will your autonomous systems demonstrate their decisions are legitimate? What happens when they learn something unexpected? How do you maintain human oversight without crippling the very autonomy that makes AI valuable? These aren't just technical questions. They're business survival questions. And as those 900 billion agents continue to multiply, finding answers becomes more urgent every day. The companies that build governance into their DNA today will be the ones shaping Europe's digital economy tomorrow.