The Silent Question Behind 900 Billion AI Agents

Jan de Vries · 2026-09-30

Let's talk about something that's been quietly keeping me up at night. It's not the usual startup funding drama or regulatory headaches. It's a question posed by Vendan Ananda Kumararajah that hit me like a ton of bricks: who exactly is in charge when we have 900 billion AI agents running around? Think about that number for a second. That's not just a few clever chatbots. We're talking about an invisible workforce larger than the human population by over a hundred times. These aren't simple scripts—they're autonomous systems making decisions, learning from each other, and coordinating in ways we can't always predict. ### The Authority Problem No One's Talking About Here's the real kicker. When you build an AI agent, you program its initial rules. But as it learns and interacts with other agents, it evolves. The system you deployed isn't the system operating six months later. So here's my question: how does legitimate authority transfer as these systems change? It's like giving a teenager the keys to your car, then watching them modify the engine, change the suspension, and repaint it—all while driving 70 miles per hour on the freeway. At what point is it still "your" car? At what point do they need a new license? - Traditional governance models break down when systems self-modify - Legal frameworks assume static entities, not evolving networks - Accountability becomes blurred across millions of interactions ### The Coordination Challenge We're Underestimating Now imagine these agents aren't working in isolation. They're talking to each other. A supply chain agent negotiates with a logistics agent, which coordinates with a manufacturing agent, which interacts with a quality control agent. They're forming what I call "emergent organizations"—structures that weren't designed but emerged through interaction. As Kumararajah points out, this creates a legitimacy crisis. If a decision emerges from a thousand agents coordinating, who's responsible? The original programmers? The company that deployed them? The network itself? I was discussing this with a colleague over coffee last week, and they put it perfectly: "We're building skyscrapers without checking if the ground can support them." We're creating incredibly sophisticated AI ecosystems on top of governance foundations meant for much simpler times. ### What European Startups Need to Consider If you're building with AI agents in Europe or serving European markets, this isn't just theoretical. The EU's regulatory approach tends to be principles-based rather than rules-based. They care about outcomes, not just compliance checkboxes. That means you need to think about: - How your agents maintain audit trails of their learning - What "kill switches" or override mechanisms you've built in - How you explain agent decisions to regulators (and customers) - Whether your governance model scales as your agent network grows Remember when GDPR first hit and everyone scrambled? This could be that moment for AI governance, but on a much larger scale. The companies that figure this out now will have a massive competitive advantage. ### Where Do We Go From Here? Honestly, I don't have all the answers. Nobody does. But I do know we need to start having this conversation more openly. We need engineers talking to ethicists, startups talking to regulators, and technologists talking to the public. Kumararajah's examination gives us a starting point. We need to design systems that retain legitimacy even as they learn. We need governance models that are as adaptive as the agents they oversee. And we need to do this before we wake up one day with 900 billion autonomous systems operating in governance gray zones. What's your take? Have you encountered this in your work? I'd love to hear how different startups are approaching this challenge. Because one thing's for sure—this conversation is just getting started, and we all need to be part of shaping what comes next.