What I Find When I Audit a Company's AI Stack (And Why Leaders Are Usually Surprised)

When I start an AI Systems Audit, I ask the senior leader one question before I look at anything: "What do you think the biggest AI problem in your organization is right now?"

They almost always get it wrong. Not completely, there's usually a real problem underneath the one they name. But the root cause and the symptom they're pointing at are rarely the same thing.

Here's what I actually find.

Finding 1: More Tools Than Anyone Knows About

Every organization I've audited has AI tools running that the leadership team didn't sanction. Individual contributors and team leads adopt tools independently, which is rational, they're trying to do their jobs better and the official procurement process takes six months.

The problem isn't the tools. It's the invisibility. When no one has a complete picture of what's running, you can't govern it, you can't assess it, and you can't integrate it into a coherent strategy.

The audit almost always starts with building the actual inventory, not the approved list, the real list.

Finding 2: Process Documentation Gaps Everywhere

The most consistent finding across every audit: organizations are trying to automate processes they haven't documented.

You cannot build a reliable AI workflow on top of an undocumented process. The AI will codify whatever ad-hoc behavior exists, including the errors, the workarounds, and the personal preferences that vary by employee.

Before any automation discussion, the process needs to exist in writing. Not a flowchart, a written procedure that someone unfamiliar with the work could follow and produce the same output.

This is where most AI adoption stalls. Leaders expect the AI to figure out the process. The AI executes whatever process it's given.

Finding 3: The Middle Layer Is Missing

Organizations tend to invest at the edges: tools that collect data at the front end, and dashboards that display outcomes at the back end. The middle layer, the decision logic that sits between data and action, is usually absent or fragile.

This is where agentic AI creates the most value and the most risk. Agents operate in the middle layer: they receive data, make decisions, and take actions. Without a well-designed middle layer, agents either do nothing useful or do something that shouldn't be automated.

Finding 4: Accountability Stops at Deployment

In most organizations, the conversation about an AI tool ends when it's deployed. "We launched it, it's running, we're done." No one owns its ongoing performance. No one is reviewing error rates. No one is checking whether the outputs six months in still match the outputs from day one.

This is how model drift, data drift, and quiet failures accumulate. The tool is technically running. It's just producing results that no one is looking at closely enough to notice have degraded.

Finding 5: The ROI Case Is Retrospective

Organizations invest in AI tools, then try to justify the investment afterward. The ROI case wasn't built before the decision, it's being built to defend a decision already made.

This produces a specific failure mode: because the success criteria weren't defined in advance, the tool is deemed "successful" if it's being used, regardless of whether it's producing business value. Usage is not value.

The Insight

The pattern across all five findings is the same: AI adoption is being treated as a technology problem when it is fundamentally an operations and management problem.

The technology works. The deployment, documentation, governance, and performance management, that's where organizations are failing.

The Takeaway

Run a 30-minute AI audit on yourself. List every AI tool you use. For each one, answer: do you know exactly what it does with the data you give it? Is there someone who reviews its outputs for quality? Could you describe the process it supports in writing?

If the answer to any of those is no, you have the beginning of a governance gap. That's fixable, but only if you can see it.