AI Agent Governance: Enterprise Two-Layer Framework Guide

AI agent governance is the discipline of controlling what enterprise AI agents can do, and critically, what they can know. Most frameworks address the first layer: identity, permissions, and output policies. The second layer, data governance, is almost entirely absent. Gartner finds that traditional AI governance misses 60-70% of agent-specific risk vectors. Without governing the data agents consume, behavioral controls alone cannot prevent systematic failure at scale.

What It Is	The set of controls that determine what AI agents can do, access, and consume in an enterprise environment
Why It Matters	71% of enterprises lack a formal governance framework for autonomous agents, even as 64% plan to increase agent autonomy within 12 months (Forrester, 2026)
Two Governance Layers	Agent-layer controls (identity, permissions, action limits, output policies) + Data-layer controls (what agents can know, trust, retrieve, and cite)
The Gap	Current frameworks govern agent behavior but not the data agents consume, the layer where most enterprise AI failures originate
Atlan’s Role	Governed data substrate that certifies data quality, tracks full lineage, and access-controls the context agents retrieve through MCP

What is AI agent governance?

AI agent governance is the set of policies, controls, and frameworks that define what autonomous AI agents can do, access, and act on within an enterprise. As agents move from assistants to decision-makers, governance determines whether they operate within sanctioned boundaries, and whether the decisions they make can be trusted, traced, and audited.

Traditional AI governance treated models as assets: register them, test for bias, monitor their outputs. AI agent governance is a broader discipline. It must cover identity (who is the agent?), authorization (what can it access?), action limits (what can it trigger?), output policies (what can it produce?), and auditability (can every decision be traced?). These controls apply equally to single agents, multi-agent pipelines, and hybrid human-agent workflows.

The urgency is not theoretical. Gartner projects that 40% of enterprise applications will feature task-specific AI agents by 2026 [10]. Yet only 9% of enterprises have mature AI governance despite 72% having AI in production [11], and governance has become the defining challenge of the agentic era [7]. And 78% of leaders admit AI adoption is outpacing their organization’s ability to manage the risks it creates [5]. The shift from AI as a tool to AI as an actor changes what governance must cover.

There is, however, a structural blind spot in the current governance conversation. The dominant frameworks focus on what agents do, their outputs, their actions, their autonomy. This page introduces a two-layer model that separates agent-layer controls (what the agent can do) from data-layer controls (what the agent can know). The second layer is where most enterprise AI failures actually originate, and it is the layer that no current governance framework addresses. For the broader case that context is the infrastructure problem, see our guide to context infrastructure for AI agents.

The two layers of AI agent governance

Complete AI agent governance requires two distinct control surfaces. Agent-layer governance controls what agents can do: their identity, permissions, and permitted actions. Data-layer governance controls what agents can know: whether the data they retrieve is certified, lineaged, access-controlled, and semantically consistent. Most enterprise frameworks deploy the first layer and leave the second entirely unaddressed.

Layer 1 — Agent-layer controls

The agent layer governs what an autonomous agent is and what it is permitted to do. Key controls at this layer include:

Identity management: Every agent must have a non-human identity with scoped credentials, not shared service accounts that make attribution impossible
Permission controls: Role-based access and least-privilege principles that define what actions each agent can initiate
Output policies: Guardrails on what agents can produce, publish, or trigger downstream
Action limits: Human-in-the-loop triggers for high-stakes decisions, plus a documented kill switch capability for runaway autonomy
Observability: Audit trails at the orchestration layer: what did the agent do, when, and in what sequence

Several vendors operate at this layer. Zenity provides agent identity management, shadow AI detection, and access governance [1]. Palo Alto Networks focuses on threat containment, prompt injection defense, and sandboxing. Microsoft’s Cloud Adoption Framework covers agent observability and agent security. As HBR documented in March 2026, AI agents acting with unchecked action authority can behave like malware, accessing systems, exfiltrating data, and triggering actions far beyond their intended scope [4].

But here is the structural limitation: agent-layer controls govern what happens after data is consumed. A perfectly governed agent running on ungoverned data still produces wrong, biased, or harmful outputs. The behavioral guardrail cannot compensate for stale, misclassified, or semantically ambiguous context at the retrieval layer.

Layer 2 — Data-layer controls

The data layer governs what an agent can know. This means controlling:

Whether the data agents retrieve has been certified as trusted and current
Whether lineage is available: where did this data come from, through which transformations?
Whether semantic consistency is enforced: does “revenue” mean the same thing to the agent as it does to the finance team?
Whether access controls apply equally to agent MCP calls as to human queries

Kiteworks named this gap directly: “The primary security control point is no longer the endpoint or the application, but rather the interaction layer, the communication fabric between agents, tools, models, and data, which today is largely ungoverned.” [2]

No current governance framework approaches this layer as a governed data infrastructure problem. The entire governance conversation has been captured by security vendors and model safety organizations, while the data infrastructure layer remains unaddressed as a first-class control surface. See how the MCP connected data catalog becomes the governance delivery mechanism when data-layer controls are in place.

Governance layer comparison

Governance dimension	Agent-layer controls	Data-layer controls
What it governs	Agent identity, permissions, actions, outputs	Data quality, lineage, certification, access
Primary vendors	Zenity, Palo Alto, Microsoft	Atlan (governed context layer)
Failure mode prevented	Unauthorized actions, prompt injection, runaway autonomy	Stale data, uncertified sources, semantic confusion
Current framework coverage	Well-addressed	Largely absent
Where failure originates	At the action/output layer	At the retrieval/context layer

Why ungoverned agent data is an enterprise risk

When agents retrieve ungoverned data, failures are not random. They are systematic. An agent consistently pulling from a stale metric, a misclassified field, or a semantically ambiguous term will produce consistently wrong outputs. No behavioral guardrail catches this. Governing agent behavior without governing the data underneath it means governing only half the problem [7]. Two documented cases illustrate what data-layer failure looks like at enterprise scale.

The healthcare example — systematic, not random

A healthcare company deployed AI achieving 87% accuracy in testing. In production, accuracy fell to 34%. The root cause was not model quality or agent behavior. Production data was missing 40% of the diagnostic codes the model had been trained on. This was a data-layer governance failure: the data the agent consumed in production was materially different from the data it learned on, and no governance control was in place to detect or flag that gap.

No agent-layer policy could have detected or prevented this. The distinction that matters: this was not a hallucination. It was a systematic, predictable degradation caused by a known gap in the data the agent was consuming at runtime. The agent was doing exactly what it was governed to do. It simply had no visibility into the fact that its data substrate had changed.

The MCP supply chain attack vector (Feb 2026)

In February 2026, 1,184 malicious skills were found poisoning an agent marketplace. Thousands of MCP servers were exposed without authentication. The attack surface was not the agent layer. It was the context infrastructure layer: the tools and data sources agents call at runtime.

This is the same layer that data-layer governance addresses. When agents retrieve context through ungoverned MCP connections, every tool call is a potential injection point. A governed MCP implementation, where the context source is certified, access-controlled, and lineaged, substantially reduces this surface. See what Atlan’s MCP server does differently to address this layer.

Systematic failure vs. random hallucination

Legal RAG implementations hallucinate citations 17-33% of the time. Stanford researchers found general-purpose LLMs hallucinated in 58-82% of legal queries [5]. These are not random errors. They are directly traceable to ungoverned retrieval: stale source documents, missing context, uncertified knowledge bases.

63% of breached organizations lacked AI governance policies at the time of their breach [6]. The pattern is consistent: governance gaps at the data layer produce systematic, not random, failures. Systematic failures require data-layer intervention, not model-layer tuning.

What AI agent governance frameworks get wrong

Most AI agent governance frameworks operate at the orchestration layer, governing what agents do after data has been consumed. They treat the underlying data as a given. This is the structural gap: frameworks govern outputs without governing inputs. The context an agent consumed when making a decision is not logged, certified, or traceable.

They govern outputs, not inputs

Every current major framework, including Zenity, Palo Alto Networks, Microsoft, and NIST AI RMF, focuses on what agents produce. NIST AI RMF includes data quality as a concern, but treats it as a pre-deployment checkpoint, not a runtime governance control [8]. There is no mechanism in any current framework to track what context an agent consumed when making a specific decision. Forrester’s AEGIS framework covers access governance but stops at the orchestration layer [9]. The record of what the agent knew at decision time does not exist.

They treat data as a given, not as governed

The implicit assumption embedded in every current framework is: if the agent has a policy, the data must be fine. Enterprise data estates make this assumption untenable. Stale metrics, duplicate definitions, deprecated datasets, and uncertified sources are the norm, not the exception. Agents inherit all of this, and behavioral governance at the agent layer cannot see it.

Only 43% of organizations have a centralized AI data gateway. The remaining 57% operate with fragmented controls or no dedicated AI data controls at all [2]. In these environments, agents are making consequential decisions based on data that is ungoverned by any layer of any framework.

They stop at the orchestration layer

Kiteworks explicitly names the gap: the interaction layer between agents, tools, models, and data is “largely ungoverned” [2]. The orchestration layer, where LangChain, agent frameworks, and MCP routing operate, is where most current observability tools focus their attention. But the data flowing through that layer is the actual risk surface. Governing the pipe without governing what flows through it is incomplete governance.

How to govern AI agents at both layers

Governing AI agents completely requires working both layers in parallel. Agent-layer controls establish who the agent is and what it can do. Data-layer controls certify what it can know. The sequence below establishes both layers and points to the gaps most enterprise governance programs currently skip.

Prerequisites before starting:

Inventory of all active agents and their data access paths
Data catalog or metadata management system capable of certification and lineage
Non-human identity framework for agent credentials
Observability tooling at the orchestration layer
Policy framework aligned to relevant regulations (EU AI Act, HIPAA, SOX, CMMC)

Step 1: Map what data your agents are consuming (and from where)

Before any governance can be applied, the data access paths must be visible. This means cataloging every dataset, API, and tool agents call at runtime, not just at deployment. Shadow agent access (agents querying data without catalog visibility) is the most common gap. An AI-ready data catalog is the infrastructure that makes this visibility possible.

Step 2: Certify and govern the data sources

Apply data certification to every source agents retrieve from. Is this dataset trusted, current, and owner-assigned? Establish lineage from source to agent: if an agent cites a metric, can you trace exactly where that metric came from? Tag sensitive data, PII, and regulated fields so agents cannot retrieve them without explicit authorization. Read more about business context for AI and why semantic disambiguation is as important as access control.

Step 3: Implement agent-layer identity and permission controls

Assign each agent a non-human identity with scoped, least-privilege credentials. Define what each agent can access, trigger, publish, and escalate. Deploy human-in-the-loop triggers for high-stakes or irreversible actions. Forrester’s AEGIS framework provides a useful starting checklist for this layer [9].

Step 4: Establish auditability: log what context agents consumed per decision

Every significant agent decision should be traceable: what data did it retrieve? From what source? At what time? This requires data-layer instrumentation, not just orchestration-layer logging. McKinsey’s AI Trust Maturity Model now includes agentic context traceability as a maturity indicator [3]. The orchestration log records what the agent did; the context log records what it knew. Both are required for a complete audit.

Step 5: Monitor data drift that may degrade agent behavior over time

Data changes after deployment. Metrics are deprecated, definitions shift, new fields appear. Governance must be continuous: set alerts when certified datasets are modified or deprecated. This is the mechanism that would have caught the healthcare 87% to 34% accuracy collapse before it reached production.

Common pitfalls to avoid:

Treating data governance as a one-time pre-deployment audit rather than a runtime control
Governing agent behavior at the model layer while ignoring retrieval-time data quality
Assigning shared service account credentials to multiple agents, making attribution impossible
Logging orchestration actions without logging the data context that drove each decision

How Atlan enables data-layer governance for AI agents

Most enterprises deploying AI agents today have some form of agent-layer controls: identity management, permission scoping, output guardrails. The data layer beneath those agents remains ungoverned: no certification, no lineage, no semantic disambiguation at the retrieval layer. As Metadata Weekly argued in their 2026 AI reality check, the model is not the problem; the foundations underneath are [10].

Atlan operates at the data-layer governance gap no current agent framework addresses. Here is what that looks like in practice:

Data certification: Atlan flags datasets as trusted, deprecated, or under review before agents can consume them. When an agent retrieves data through MCP, the certification status travels with the data. Agents retrieving deprecated or unverified data surface a governance flag in real time: the kind of signal that would have prevented the healthcare accuracy collapse described earlier.

Column-level lineage: Every metric an agent cites can be traced back through every transformation to its source system. When an agent makes a decision based on a revenue figure, the full lineage of that figure, from source pipeline through every join, filter, and aggregation, is available for audit. This is what makes agent decisions genuinely auditable, not just logged.

Business glossary: The business glossary disambiguates semantic terms across the enterprise. Agents use “revenue” the same way finance defines it, not the way an upstream pipeline labeled it. “Customer” means the same thing in HR, sales, and the agent’s context window. Semantic consistency is enforced at the retrieval layer, not assumed at the model layer. See how business context for AI addresses this disambiguation challenge across enterprise systems.

Access controls via MCP: The same policies that govern human queries apply to agent MCP calls. No data leakage occurs through AI interfaces because the access control layer is applied at the protocol level. An agent with read access to a dataset does not inherit write access, or access to adjacent sensitive fields, simply because it is an agent.

Active metadata: Quality scores, sensitivity labels, ownership, and certification status are pushed into the context layer agents use at query time. Governance is not a pre-deployment checkbox; it travels with the data at the moment of retrieval.

The result is agents that can only consume governed, certified, contextual data. Every agent decision traces back to a certified, lineaged data source. The MCP protocol teams are already wiring becomes the delivery mechanism for governance, not an additional layer to implement. Learn more about what Atlan’s MCP server enables at this layer.

Real stories from real customers: governing data that agents actually trust

"AI initiatives require more context than ever. Atlan's metadata lakehouse is configurable, intuitive, and able to scale to hundreds of millions of assets. As we're doing this, we're making life easier for data scientists and speeding up innovation."

— Andrew Reiskind, Chief Data Officer, Mastercard

Watch Now

"Context is the differentiator. Atlan gave our teams the shared vocabulary and lineage to move from reactive data management to proactive AI enablement across CME Group."

— Kiran Panja, Managing Director, Data & Analytics, CME Group

Watch Now

Both organizations illustrate the same pattern at enterprise scale: AI initiatives stall or fail not because of model quality, but because the data underneath lacks the certification, lineage, and shared vocabulary that agents require to produce trustworthy outputs. Governing the data layer is what enables the agent layer to actually work.

Governance begins at the data layer: what enterprises need to do now

AI agent governance is not a single-layer problem. The agent-layer controls that current frameworks provide, including identity, permissions, output policies, and observability, are necessary but insufficient. They govern what agents do after data has been consumed. They cannot prevent failures that originate in the data itself: stale metrics, deprecated datasets, semantically ambiguous terms, uncertified sources.

Complete governance requires a second layer: governing what agents can know. This means certifying data sources before agents retrieve them, maintaining lineage through every transformation, enforcing access controls on agent MCP calls, and making every decision traceable back to a governed data point. The context layer for enterprise AI is the infrastructure that makes this possible.

The two-layer framework is not aspirational. It is the minimum required for AI agents to operate safely at enterprise scale. 71% of enterprises currently lack a formal governance framework for autonomous agents [1]. The ones that build both layers now, before agent autonomy scales further, will be the ones that can audit, explain, and trust what their agents produce.

The governance conversation has been happening at the wrong layer. The data underneath your agents is the layer that matters most, and it is currently unprotected in most enterprise environments. Governing it is not a follow-up task. It is the foundation everything else depends on. See the CIO guide to context graphs for a practical framework for building this foundation.

Book a Demo

FAQs about AI agent governance

1. What is AI agent governance?

AI agent governance is the set of policies, controls, and frameworks that determine what autonomous AI agents can do, access, and act on within an enterprise. It covers identity management (who is the agent?), permission controls (what can it access?), action limits, output policies, auditability, and critically, the governance of the data the agent consumes to make decisions. Both the agent behavior layer and the data context layer must be governed for the framework to be complete.

2. How does AI agent governance differ from traditional AI governance?

Traditional AI governance focuses on models as assets: registering them, testing for bias, monitoring outputs at a point in time. AI agent governance is broader. It must also cover autonomous decision-making, non-human identity management, multi-agent trust chains, real-time action authorization, and the quality of data agents retrieve at runtime. Traditional frameworks govern a model at rest; agent governance must operate at the speed of the agent, continuously, across every retrieval event.

3. What are the risks of ungoverned AI agents?

Ungoverned agents produce two categories of risk. The first is behavioral: agents take unauthorized actions, leak sensitive data, or operate beyond their intended scope. The second, and less discussed, is epistemic: agents make decisions based on stale, uncertified, or semantically ambiguous data. The second category creates systematic failures that behavioral guardrails cannot detect or prevent. Both risks scale directly as agent autonomy increases and as agents are connected to more data sources.

4. What is the role of data governance in AI agent governance?

Data governance defines whether the information an agent retrieves is trustworthy. If an agent retrieves a deprecated dataset, a misclassified field, or a metric with no lineage, no behavioral policy at the agent layer can prevent the resulting bad decision. Data governance, including certification, lineage, access controls, and semantic disambiguation, is the foundation that makes agent-layer governance effective rather than cosmetic. An agent can only be as trustworthy as the data it is allowed to consume.

5. What frameworks exist for AI agent governance?

The main frameworks include NIST AI Risk Management Framework (covers risk identification, data quality as a pre-deployment concern), Forrester’s AEGIS framework (access governance, action authorization at the orchestration layer), and vendor-specific checklists from Zenity and Palo Alto Networks. Microsoft’s Cloud Adoption Framework lists data governance as a layer but treats it as a checkbox pointing to Microsoft Purview. No current framework fully addresses real-time data-layer governance for agents at runtime. The interaction between agents and data remains largely ungoverned across all published frameworks.

6. How can enterprises implement AI agent oversight?

Effective oversight requires two parallel tracks. At the agent layer: assign non-human identities with scoped credentials, log every action, set human-in-the-loop triggers for high-stakes decisions, and maintain a documented kill switch capability. At the data layer: certify every dataset agents can access, enforce lineage traceability from source to retrieval, apply access controls to agent queries at the protocol level, and monitor continuously for data drift that degrades agent accuracy over time.

7. What does the EU AI Act require for AI agents?

The EU AI Act classifies autonomous AI systems by risk level. High-risk systems, including those making consequential decisions in healthcare, finance, HR, and similar domains, require transparency, human oversight, data quality controls, and auditability of both training and runtime data. Agents operating in these contexts are likely to fall in the high-risk tier. Data-layer governance, particularly lineage documentation and certification of runtime data sources, directly supports these compliance requirements and provides the audit trail regulators expect.

8. How do you audit AI agent decisions?

Auditing agent decisions requires two distinct records: the orchestration log (what actions the agent took, in what sequence) and the context log (what data the agent retrieved when making each specific decision). Most current observability tools capture the first. The second requires data-layer instrumentation: every piece of context the agent consumed must be traceable to a certified, lineaged source with a timestamp. Without both logs, the audit is incomplete and cannot support regulatory review or root cause analysis when agent decisions go wrong.

Sources

Share this article

AI Agent Governance: Why Ungoverned Agents Are an Enterprise Risk

Key takeaways

What Is AI Agent Governance?

The two layers of complete AI agent governance

What is AI agent governance?