AI Governance Warning: Flawed Assumptions Could Spark Intelligence Explosion

Jan de Vries · 2026-10-09

Vendan Ananda Kumararajah isn't mincing words. He believes AI governance is on a dangerous path, and if we don't rethink it, we could be looking at an uncontrolled intelligence explosion. That's a fancy way of saying: machines get smarter than us, and we lose the ability to steer them. Sounds like sci-fi, right? But Kumararajah argues it's a real risk, and the problem starts with flawed assumptions baked into today's governance models. ### The Problem with Playing Catch-Up Right now, AI governance often feels like we're trying to put out fires after they've already started. Regulators draft rules, but by the time they're implemented, the technology has moved on. Kumararajah points out that this reactive approach is doomed because it assumes we can predict how AI will evolve. We can't. And when those predictions are wrong, they multiply as increasingly capable systems develop successors. Think of it like a game of telephone, but instead of whispers, it's algorithms making decisions based on bad data. Each generation of AI inherits the mistakes of the last, and before you know it, you've got a system that's not just smart—it's dangerously unpredictable. ### Why Independent Oversight Matters Kumararajah's solution? Independent oversight. Not just government agencies, but a mix of experts from tech, ethics, and civil society who can adapt as quickly as the technology does. The idea is to create a feedback loop that catches flawed assumptions before they become catastrophic. But here's the catch: independence is hard to come by. Many oversight bodies are funded by the very companies they're supposed to regulate. That's like asking a fox to guard the henhouse. Kumararajah insists we need truly independent entities—ones that aren't beholden to corporate interests or political pressures. ### The Stakes Are Higher Than You Think This isn't just about avoiding a Terminator scenario. It's about ensuring AI benefits everyone, not just a select few. If we get governance wrong, we risk widening inequality, eroding privacy, and even threatening democratic processes. The intelligence explosion Kumararajah warns about isn't just about machines outsmarting us—it's about losing control of the systems we've built to make life better. So, what can we do? Start by demanding transparency. Ask questions about who's writing the rules and whose interests they serve. Support independent research and advocacy groups that are working on these issues. And most importantly, don't assume that someone else has it handled. Kumararajah's message is clear: we need to rethink AI governance now, before it's too late. The alternative is a future where we're not just along for the ride—we're in the back seat, with no say in where we're going.