Keep tabs on AI agents
Former CIO and longtime IT advisor Tim Crawford on who should own AI governance, and where agents require the shortest leash
Key takeaways
Many enterprises are failing to keep pace with AI adoption.
Customer-facing and HR uses require tight controls.
The business owns agent governance, and the CIO ties it all together.
AI agents are moving into the enterprise faster than IT teams can set up the governance tools to keep them in check. Anthropic has issued fresh warnings about the risk, and the Hugging Face hack showed how quickly a trusted tool can become vulnerable without the right guardrails in place. Yet the pressure across organizations to adopt AI isn’t easing.
Tim Crawford, a longtime IT advisor and host of the podcast “CIO in the Know,” says the first step is to start where governance issues could cause the most pain, and course-correct as you go. But the most successful efforts involve a CIO who truly understands the whole business and partners with the rest of the C-suite.
“The CIO understanding the rest of the business function is super critical today. It’s almost table stakes now—and it’s what separates the transformational CIO from the traditional one,” Crawford says.
The conversation has been edited lightly for clarity.
IT leaders are under pressure to adopt AI quickly and responsibly. How do they do both?
Tim Crawford: Very carefully. Full stop. I mean that somewhat tongue in cheek, because it's genuinely challenging. Every week we're handed another reason not to trust AI, yet there's a lot of FOMO, and we can see the opportunity.
The studies on ROI are telling. Something like 90% of the dollars going toward AI today are showing up as negative ROI, while the other 10% is wildly positive. That's the dichotomy. And the top reason CIOs hold back isn't just cost or value, it's governance and risk.
Read more from Freshworks’ Tyler Sloat: I'm a CFO and COO. Here's what IT leaders need to know.
I just spoke with a major manufacturer that rolled out AI across the enterprise because they wanted everyone using it. Now they're trying to figure out how to control it: how to keep data from leaking, how to keep agents from going rogue, and how to show what they got for the money.
Governance is more than regulatory and compliance requirements, and more than agents going rogue. Agents can also learn and drift over time. Agent evaluations and observability only catch so much, and it gets harder once companies string multiple agents into a workflow.
How McLaren, New Balance & more turn service performance into a competitive edge
Is some level of risk simply the price of moving fast?
Of course, but companies have to balance opportunity against risk. With AI, we've mostly looked at the opportunity and not the risk.
Take personal agentic tools like OpenClaw or Claude Cowork. People use them to get real work done, but they hand over their corporate credentials so the tool can act on their behalf. As a result, IT can lose track of what’s happening and often don't have the tooling to tell whether it's Tim logging in or an agent. That agent can reach whatever my credentials can reach. Once the horse is out of the barn, IT can't control it. The challenge is supporting innovation without taking on so much risk that it cancels out the benefit.
Read also: Don’t ban shadow AI. Bring it into the light.
Is that the same calculus as past technology shifts?
On the surface, yes: opportunity versus risk. Underneath, it's far more complicated than anything we've dealt with. Distributed computing, the internet, virtualization, and cloud were all relatively straightforward by comparison.
The challenge is supporting innovation without taking on so much risk that it cancels out the benefit.
Why is AI so much harder to govern?
Part of it is surface area. But the bigger issue is that AI is nondeterministic, and we don't fully understand how it does its work. That's a feature for creative tasks. If I want a novel image, great. But some things need the same response every time: certain customer interactions, supply chain, service management, anything with regulatory requirements. The challenge is knowing when variability is good and when you need consistency. Most software until now has been deterministic.
How can IT leaders govern something that behaves unpredictably?
Take a page from the human playbook. Say you hire somebody straight out of college and give that person tasks, the equivalent of a prompt, plus credentials. If the employee starts veering off course, you notice and correct them. If the employee finds something that works, you tell them to do more of it. We don't have that kind of management for agents.
Agents also lack judgment. For example, if I’m told to get a chair but somebody’s sitting in it, I’ll ask that person to move. An agent might just pull it out from under them. Teams can add guardrails for agents’ behavior, but that requires anticipating every variation of a scenario.
Where does governance matter most?
Wherever a mistake is costly. Service interactions with upset or frustrated customers, or net-new customers trying to transact, need tight governance. HR data should be buttoned up because of policy. But if an employee is brainstorming legal theories, a miss is fine. You can just ask again, and you haven't damaged anyone's relationship with the agent.
Who owns AI governance: IT, or the business?
The overall framework is typically cross-functional, such as an AI council led by the CEO or CIO, and in a few cases the chief legal or chief compliance officer. Governance of a specific agent usually falls to whoever owns that agent, within the broader framework.
IT typically won't set it, because it takes deep business context, and that has historically not been an IT function. IT will put the rules and authority structures in place for identity management and security. But the business unit knows its function and its risks.
That said, AI is pushing IT to understand the business far more intimately than before. As cross-functional workflows and multi-agent systems take shape, someone has to pull that red thread through the business, and that's IT.
In a few years, will we govern agents like software or like people?
A bit of both. There's a programmatic reality to how agents are built and operated, like software, and similarities to how humans are managed. It isn't an argument for IT versus the CHRO. But you have to think about agent behavior, not just their functional and operational components.
