AI Governance in Regulated Industries: Where to Start

If you lead security at a regulated organization, you have probably already had the conversation: leadership wants to move faster on AI, and someone in the room asks who is making sure it is safe. The honest answer, at most companies, is no one in particular — not because people are careless, but because AI adoption has outrun the governance structures built for traditional software.

You do not fix that with a 60-page policy. You fix it by answering a few concrete questions in order. Here are the three I start with on every engagement.

1. What AI systems are already running?

“Shadow AI” — staff using consumer tools without IT visibility — is now the norm, not the exception. People paste contract language into a chatbot to summarize it, or wire a department spreadsheet into an AI add-on, long before any policy catches up. Before you can govern AI, you need an honest inventory of what is actually in use: sanctioned tools, embedded vendor features, and the unsanctioned ones too.

The gap between what an organization thinks is happening with AI and what is actually happening is the core governance problem. Close that gap first.

2. What data are those systems touching?

An AI tool working with de-identified, low-sensitivity data carries a fundamentally different risk profile than one with access to PHI, cardholder data, or proprietary research. The same model, pointed at different data, is a different decision.

Map the data flows: what goes in, where it is processed, whether it is used for training, and where the output lands. In regulated environments this is also where most of the real exposure hides — not in the model itself, but in the permissions and data paths around it. (A retrieval assistant does not create new access; it surfaces access that already existed, which is why an honest data-flow map so often turns up years of accumulated permission debt.)

3. Which regulatory frameworks actually apply?

The EU AI Act, the NIST AI Risk Management Framework, ISO/IEC 42001, and sector-specific guidance — OCC expectations for banks, OCR considerations for healthcare, PCI DSS where payments are involved — do not impose identical requirements. Knowing which apply to your use cases shapes everything downstream: which systems need formal review, what evidence you must retain, and what you can defend to a regulator or your board.

You do not need to implement all of them. You need to know which ones you are on the hook for, and be able to show your work.

Start small, scale deliberately

A governance program does not have to arrive fully formed. The organizations that succeed start with a limited-scope assessment of their highest-risk AI use cases, stand up a lightweight review path for new ones, and expand from there — rather than attempting a boil-the-ocean audit that stalls before it ships anything.

The goal is not to slow AI adoption down. It is to make adoption defensible, so the business can move quickly and still answer the question in the room: who is making sure this is safe?

If you would like to talk through where your organization stands on these three questions, book a discovery call — it is a 30-minute conversation, no obligation.