AI Might Destroy Us in 10 Years. But What Happens on Tuesday?

There's been a ton of discussion lately about whether artificial intelligence could eventually pose an existential threat to humanity. Some of the people closest to the technology are warning that its capabilities are advancing faster than our safety mechanisms, regulation and oversight can keep up.

I don't dismiss that. When people who understand these systems best are worried about where they lead, it's worth paying attention.

But here's the thing I can't get out of my head. We're worried AI may someday advance faster than society's ability to control it. Inside a lot of companies, that has already happened.

Not in the science fiction sense. AI hasn't seized control of anything in your company.

Something more ordinary happened. People found useful technology and started using it faster than their organizations adapted the rules and safeguards around the work. Technology almost always outruns organizations. What's different now is the speed and the scale.

And it's already costing real money. In a recent EY survey of companies with over a billion dollars in revenue, 99% reported financial losses tied to AI risk, and 64% said those losses topped a million dollars. Those are the companies with the biggest budgets and the most mature IT organizations. So ask yourself the question. If it's happening to them, what makes you confident it isn't already happening to you? And would you even know?

Most of these incidents never make the news. They surface as rework, bad decisions, customer problems, compliance issues and uncomfortable conversations about how something was allowed to happen. And they're becoming more common, not less. IBM's latest breach report found that security incidents involving shadow AI, employees using tools nobody approved, more than doubled in a year, from 20% of breaches to 43%. Most of that risk is invisible to the people responsible for managing it. You can't control what you can't see.

That's what makes the current AI risk conversation so strange to me. We're pouring energy into risks the average executive can't touch. A CFO in Charlotte can't set the pace of frontier models. A CEO running a $500 million manufacturer isn't going to solve global AI safety.

But they have complete control over what happens inside their own companies. And that's exactly the part many of us haven't been looking at closely enough.

Governance sounds bigger than it is. At its core it's just management responsibility applied deliberately. What are we comfortable allowing? What aren't we? Where could this go materially wrong? Who owns that risk, and how will we know if our safeguards are working? Leadership teams have answered those same questions about new technology, financial controls and cybersecurity for decades. AI doesn't exempt us just because it moves fast.

And waiting won't help. Regulation will develop, safeguards will improve, standards will mature. That helps. None of it decides how AI gets used inside your own four walls. That part is yours.

That's the part I think we have backwards. Whether AI becomes an existential threat may be one of the most consequential questions of our lifetime. It's also almost entirely outside your control.

Whether your company takes an avoidable AI loss next quarter is a far more mundane question. But that one is yours.

You can decide what matters enough to govern. You can establish accountability. You can put sensible boundaries around the higher-risk decisions without burying every AI interaction in bureaucracy. You can treat incidents as something to learn from. Most of all, you can admit that widespread AI adoption has changed your organization's risk profile, and manage it accordingly.

None of that requires waiting for Washington or Silicon Valley. It doesn't require knowing what AI will do in 2036. It requires managing the company you actually have today.

Maybe the people warning us about the long-term dangers are right. I hope we take those risks seriously enough to never find out.

In the meantime, there's a risk sitting squarely inside your control. You should probably take that one seriously too.

About Root Idea

Root Idea helps CFOs protect the business from AI decision risk. We work directly alongside finance teams to map AI influence, establishes decision governance controls that hold up to board scrutiny, and delivers training and change management to make governance stick.

If your organization is scaling AI and governance hasn't kept pace that's exactly the conversation we're built for. Learn more at rootidea.ai.