AI Governance Needs a Rethink Before It's Too Late

Jan de Vries · 2026-10-09

Vendan Ananda Kumararajah recently made a point that stopped me in my tracks. He argues that AI governance needs independent oversight to prevent flawed assumptions from multiplying as increasingly capable systems develop successors. That's a mouthful, I know. But let's break it down, because this isn't just academic—it affects all of us. ### Why AI Governance Is Stuck in the Stone Age Think about how we regulate other powerful technologies. Nuclear power? Independent watchdogs. Pharmaceuticals? Third-party trials. But AI? Most of it is self-policed by the same companies building it. That's like letting the fox guard the henhouse. Kumararajah's core worry is simple: if we base our rules on wrong assumptions, those mistakes don't just sit there. They compound. Each new generation of AI learns from the last, and if the last was built on shaky ground, the next one is even shakier. It's like a game of telephone, except the message is "how to control superintelligence" and the players are moving faster than we can correct them. ### The Intelligence Explosion: Not Sci-Fi Anymore You've probably heard the term "intelligence explosion." It sounds like something from a summer blockbuster. But it's a real concept: once AI can improve itself, the pace of progress could go from walking to warp speed in the blink of an eye. Here's the catch: we don't get to hit pause. If we're not careful, we could end up with systems that are smarter than us but not aligned with our values. And that's not a risk we can afford to ignore. > "The real danger isn't that AI will turn evil. It's that it will do exactly what we asked—just not what we meant." — a sentiment Kumararajah echoes. ### What Independent Oversight Would Actually Look Like So what does Kumararajah propose? Independent oversight. Not another government agency buried in red tape, but a genuinely separate body with the power to audit, question, and—yes—say no. - **Third-party audits**: Regular, unannounced checks on AI systems, especially those used in critical areas like healthcare, finance, and defense. - **Transparency requirements**: Companies must disclose how their models make decisions, not just the outcomes. - **Liability frameworks**: If an AI causes harm, someone needs to be accountable. "The algorithm did it" can't be a get-out-of-jail-free card. - **International cooperation**: AI doesn't respect borders, so neither should our rules. A patchwork of national laws won't cut it. ### The Clock Is Ticking We're at a crossroads. Keep letting AI companies mark their own homework, or build a system that actually protects people. Kumararajah's warning is blunt: if we don't rethink governance now, we risk an uncontrolled intelligence explosion. And once that genie's out of the bottle, there's no putting it back. I don't know about you, but I'd rather not find out what happens next. The time to act is now—before the next generation of AI makes the decision for us. What do you think? Is independent oversight possible, or are we too far gone? Let me know in the comments.