The Hidden Question Behind AI Governance That No One's Asking

Jan de Vries · 2026-08-27

We talk a lot about managing AI risks. You know the usual topics: bias in algorithms, job displacement, those sci-fi scenarios that keep tech ethicists up at night. And sure, that's important work. But here's something that's been nagging at me lately, especially watching the EU's recent proposals and the scramble to regulate this technology. We're so focused on governing the *technology* that we might be missing the bigger picture. Who's governing the *governors* themselves? ### The Real Challenge Isn't The Tech, It's The Watchdogs Think about it for a second. We're setting up institutions, committees, and regulatory frameworks at a breakneck pace. The goal is to create legitimate, accountable bodies that can keep AI in check. That's the promise. But institutions, like any system, can drift. They can become outdated, captured by special interests, or simply unfit for the exponential pace they're trying to regulate. Legitimacy isn't a one-time grant. It's a constant process. It requires public trust, transparency, and results that people can see and understand. When a regulatory body makes a decision that impacts millions of businesses and citizens, can we trace the logic? Do we know who advised them? Is there a clear path for appeal or challenge? ### What Does 'Accountable' Actually Mean in Practice? Accountability sounds great in a white paper. In reality, it's messy. It means having clear lines of responsibility when something goes wrong. If an AI system approved by a governing body causes harm, where does the buck stop? With the developer? The company that deployed it? Or the institution that gave it a regulatory thumbs-up? - **Transparency in Process:** We need to see how decisions are made, not just the final verdict. - **Accessible Redress:** There must be a clear, fair way for startups and citizens to challenge rulings without needing a million-dollar legal team. - **Performance Metrics:** How do we measure if these governors are doing a good job? It can't just be about the number of regulations written. This isn't just theoretical. For European startups looking to incorporate and innovate, unclear or unaccountable governance creates a fog of uncertainty. It makes planning difficult and raises the cost of compliance in a market that's already fragmented. ### The 'Fit' Test For a World Moving at AI Speed Then there's fitness. Is a traditional, slow-moving bureaucratic structure truly *fit* to govern a technology that evolves by the week? A regulatory process that takes 18 months to evaluate a system is dealing with a version of AI that's essentially a museum piece by the time the ruling comes out. We need governance that is as adaptive and iterative as the technology it seeks to oversee. That might mean sunset clauses on regulations, more sandbox environments for live testing, and governors who are themselves continuously learning. As one veteran tech founder told me over coffee last week, *"We can't use a horse-and-buggy rulebook to manage a hyperloop."* The metaphor stuck with me. So, where does this leave us? The conversation about AI governance is crucial, but it's only half the conversation. The next phase, the harder phase, is looking inward at the structures we're building. We must design them not just for the AI of today, but with the built-in flexibility, transparency, and accountability to remain legitimate for the AI of tomorrow. It's about building governors that are worthy of governing something this powerful. Because in the end, the strength of our rules depends entirely on the strength and integrity of the rule-makers.