Why AI Governance Needs a Radical Rethink Before It's Too Late

Jan de Vries · 2026-10-09

### The Hidden Risk in AI Governance Vendan Ananda Kumararajah recently made a compelling argument: if we don't rethink AI governance, we risk an uncontrolled intelligence explosion. That's not just a catchy phrase; it's a warning about how flawed assumptions can multiply as increasingly capable systems develop successors. Think of it like a game of telephone, but each player is smarter than the last, and the message gets more distorted with every pass. Before we know it, the original intent is lost, and we're left with something we can't control. ### Why Independent Oversight Matters Kumararajah points out that AI governance needs independent oversight. Why? Because when the same group that builds the AI also sets the rules, there's a conflict of interest. It's like letting the fox guard the henhouse. Independent oversight ensures that decisions aren't just based on what's best for the company, but what's best for society. And as AI systems become more capable, the stakes get higher. We're not just talking about chatbots that can write poetry; we're talking about systems that could make decisions affecting millions of lives. ### The Problem with Flawed Assumptions Here's the kicker: flawed assumptions don't just stay flawed; they compound. If an AI system is built on a wrong premise, and then it creates a successor, that successor inherits the flaw and adds its own. It's like a snowball rolling downhill, getting bigger and more dangerous. Kumararajah argues that without independent oversight, these assumptions can go unchecked, leading to an intelligence explosion that no one can control. That's not science fiction; it's a real possibility if we don't act. > "The greatest risk isn't that AI will become evil, but that it will become competent without being wise." – This quote, often attributed to AI ethicists, perfectly captures the essence of Kumararajah's argument. ### What Can We Do? So, what's the solution? Kumararajah suggests a few key steps: - **Establish independent regulatory bodies** that include ethicists, technologists, and policymakers. These bodies should have the power to audit and approve AI systems before they're deployed. - **Promote transparency** in AI development. Companies should be required to disclose how their systems make decisions, especially in high-stakes areas like healthcare and criminal justice. - **Encourage international cooperation**. AI doesn't respect borders, so governance can't be limited to one country. We need global agreements on AI safety standards. - **Invest in AI safety research**. We need to understand how to align AI with human values before it's too late. ### The Clock Is Ticking We're at a crossroads. AI is advancing faster than our ability to govern it. Kumararajah's warning is a wake-up call. If we don't rethink AI governance now, we risk an intelligence explosion that could spiral out of control. But it's not too late. With independent oversight, transparency, and global cooperation, we can steer AI toward a future that benefits everyone. The question is: will we?