AI Governance Knows the Risks—But Who Decides When AI Is Fit to Act?

Jan de Vries · 2026-08-18

AI governance has come a long way. We've mapped the risks, cataloged the biases, and built frameworks to keep autonomous systems in check. But here's the question that keeps me up at night: who actually decides when an AI is fit to act? Not just technically fit—but legitimately authorized to make decisions that affect real people. Vendan Ananda Kumararajah, a sharp voice in this space, recently dug into this exact problem. And honestly, it's one of those questions that seems simple on the surface but gets messy the moment you scratch it. Because fitness isn't just about accuracy scores or test results. It's about trust, accountability, and who gets to pull the trigger. ### The Gap Between Risk Mapping and Real Authority We've spent years building risk maps for AI. We know the dangers—bias, hallucination, security holes, unintended consequences. Regulators have stepped in with guidelines, and companies have adopted best practices. But mapping the risks is one thing. Deciding when an AI system has the green light to operate autonomously? That's a whole different ballgame. Think of it like this: we know the dangers of driving a car, but that doesn't mean everyone gets a license. There's a test, a judgment call, and a human at the DMV who decides you're ready. With AI, that DMV doesn't exist yet. We're handing out licenses without ever checking if the driver can handle the road. ### Who Should Hold the Keys? Here's where it gets tricky. Should it be the developers who built the system? They know the code, sure, but they also have a vested interest in seeing it succeed. Maybe regulators? They have the authority, but they're often slower than the technology itself. Or should it be independent auditors—people with no skin in the game who can look at an AI system and say, "Yes, this is ready" or "No, not yet"? There's no easy answer. But here's what I think matters most: the decision can't happen in a vacuum. It needs to involve multiple stakeholders, each bringing a different lens. Engineers bring the technical reality. Ethicists bring the moral questions. End users bring the lived experience of interacting with the system. And regulators bring the legal framework that keeps everything honest. ### Fitness Isn't a One-Time Check Even if we figure out who decides, there's another layer: fitness changes. An AI system that's perfectly fit to handle customer service queries in 2024 might be completely unfit by 2026, when the data it was trained on is outdated or the world has shifted. So the decision to grant authority isn't a one-time event. It's an ongoing process of evaluation, re-evaluation, and sometimes revocation. - Continuous monitoring beats one-time certification - Real-world feedback loops should inform authority decisions - Sunset clauses could force regular reassessment - Transparency in the decision-making process builds public trust ### The Human Element We Can't Ignore Here's the thing that often gets lost in these conversations: AI doesn't exist in a bubble. It operates in a world shaped by human decisions, human biases, and human consequences. When an autonomous system makes a mistake, it's not the algorithm that gets blamed—it's the people who deployed it. So the question of fitness to act isn't just technical. It's deeply human. Kumararajah's point, as I read it, is that we need to move beyond the risk-mapping phase and start building real decision-making structures. Who has the authority to say "go"? And more importantly, who's accountable when things go wrong? Those aren't questions we can answer with better code alone. They require policy, governance, and a willingness to have hard conversations. ### What This Means for the Future We're at a crossroads. AI is becoming more powerful, more autonomous, and more embedded in our daily lives. The risk maps are drawn, but the authority structures are still blank. If we don't figure this out soon, we'll keep deploying systems that are technically capable but not truly fit to act—and that's a recipe for disaster. The answer won't come from a single expert or a single framework. It'll come from a collective effort to define what fitness really means, who decides it, and how we keep that decision honest over time. It's a big ask, but it's the only way forward if we want AI to be a tool we trust, not a risk we manage.