---
title: "AI-Ready Data Checklist: 12 Checks You Can Actually Fail"
url: "https://atlan.com/know/ai-agent/data-for-ai/ai-ready-data-checklist/"
description: "Most AI-ready data checklists cannot be failed. These 12 audit checks each have a pass condition and the artifact that proves it, scored per question."
author: "Emily Winks"
author_role: "Data Governance Expert"
published: "2026-09-23"
updated: "2026-09-23"
---

---

Almost every published AI-ready data checklist shares one flaw: you cannot fail it. "Know where your data lives" and "improve data quality" are instructions, not checks, and an organization that agrees with all of them has learned nothing about whether its agents will work. The twelve below each have a pass condition and an artifact behind it. Atlan is where most of those artifacts already live, because certified definitions, [lineage](https://atlan.com/know/ai-agent/data-for-ai/data-lineage-for-ai/), ownership, freshness and access policy sit in one [Enterprise Data Graph](https://atlan.com/know/enterprise-data-graph/) that can be queried rather than surveyed.

### Run the twelve checks on your own data

Give it the question your agent should answer. It marks each check pass, fail or unknown, and turns every unknown into something specific to go and fetch. [Read the skill](/skills/ai-ready-data-audit.md).

*Paste into a new chat*

```
Use the skill at https://atlan.com/skills/ai-ready-data-audit.md to audit our data against the twelve checks. Ask me for whatever it needs.
```

*Run once in a terminal*

```
curl -fsSL --create-dirs \
  -o ~/.agents/skills/ai-ready-data-audit/SKILL.md \
  https://atlan.com/skills/ai-ready-data-audit.md
```

*For an agent*

```
curl -fsSL https://atlan.com/skills/ai-ready-data-audit.md
```


    Human
    Agent


  ClaudeChatGPTGeminiCursorOther
  Copy

.skp{margin:1.75rem 0;padding:1rem 1.125rem 1.125rem}
.skp h3{margin:0 0 .375rem!important;font-size:20px!important;line-height:24px!important;
 font-family:var(--font-funnel-display)!important;font-weight:600!important;color:#2B2B39!important}
.skp > p{margin:0 0 .875rem!important;font-size:14px!important;line-height:22px!important;color:#555572!important}
.skp > p a{color:#2026D2;text-decoration:underline}
.skp-box{margin-bottom:.75rem}
.skp-p em{display:block;font-size:10px;font-weight:700;letter-spacing:.06em;text-transform:uppercase;
 color:#F34D77;font-style:normal;margin-bottom:.25rem}
.skp-p > p{margin:0!important}
#seo-layout article .skp .skp-p pre{margin:0!important;overflow-x:auto}
#seo-layout article .skp .skp-p pre code{color:#2B2B39!important;font-size:12px!important;line-height:19px;
 white-space:pre-wrap;overflow-wrap:anywhere;background:none;padding:0;
 font-family:ui-monospace,SFMono-Regular,Menlo,monospace}
.skp-p + .skp-p{margin-top:.625rem}
.skp-cs{display:flex;flex-wrap:wrap;align-items:center;gap:.375rem}
.skp-seg{display:inline-flex;flex:0 0 auto;border:1px solid #DDDDE3;border-radius:999px;overflow:hidden;background:#fff}
.skp-m{min-height:32px;padding:.25rem .75rem;border:0;background:transparent;font-family:inherit;color:#77778E;cursor:pointer}
.skp-m:hover{color:#2026D2}
.skp-m.on{background:#2026D2;color:#fff}
.skp-sep{flex:0 0 auto;width:1px;height:20px;background:#DDDDE3;margin:0 .25rem}
.skp-tools{display:inline-flex;flex-wrap:wrap;gap:.375rem;min-width:0}
.skp-cp{margin-left:auto;min-height:32px;padding:.25rem .75rem;border-radius:999px;background:#fff;
 font-family:inherit;color:#555572;cursor:pointer}
.skp-cp:hover{border-color:#2026D2!important;color:#2026D2}
.skp-cs .skp-c{display:inline-flex!important;width:auto;flex:0 0 auto;align-items:center;gap:.375rem;
 min-height:32px;padding:.25rem .625rem;border-radius:999px;background:#fff;font-family:inherit;
 color:#555572;cursor:pointer}
.skp-cs .skp-c img{display:block!important;width:14px!important;height:14px!important;
 max-width:14px!important;flex:0 0 auto;opacity:.55;margin:0}
.skp-c:hover img,.skp-c.on img{opacity:1}
.skp-cs .skp-c:hover{border-color:#797DE4!important;color:#2026D2}
.skp-cs .skp-c.on{border-color:#2026D2!important;background:#F4F4FD;color:#2026D2}
.skp-agent .skp-tools,.skp-agent .skp-sep{display:none}

(function(){
  var p=document.querySelector('.skp[data-skill="ai-ready-data-audit"]');if(!p)return;
  var panes=p.querySelectorAll('.skp-p');
  function show(id){panes.forEach(function(x){x.hidden=x.dataset.skp!==id;});}
  function cur(){var a=p.querySelector('.skp-p:not([hidden]) pre code');return a?a.textContent:'';}
  show('prompt');
  p.querySelectorAll('.skp-p em').forEach(function(e){
    var par=e.parentElement;
    ((par.tagName==='P'&&par.textContent.trim()===e.textContent.trim())?par:e).hidden=true;
  });
  p.querySelectorAll('.skp-c').forEach(function(b){
    b.addEventListener('click',function(){
      p.querySelectorAll('.skp-c').forEach(function(x){var on=x===b;x.classList.toggle('on',on);x.setAttribute('aria-pressed',on);});
      show(b.dataset.skp);
    });
  });
  p.querySelectorAll('.skp-m').forEach(function(m){
    m.addEventListener('click',function(){
      var agent=m.dataset.skpMode==='agent';
      p.classList.toggle('skp-agent',agent);
      p.querySelectorAll('.skp-m').forEach(function(x){var on=x===m;x.classList.toggle('on',on);x.setAttribute('aria-pressed',on);});
      if(agent){show('agent');}else{var s=p.querySelector('.skp-c.on');show(s?s.dataset.skp:'prompt');}
    });
  });
  var c=p.querySelector('[data-skp-copy]');
  c.addEventListener('click',function(){
    navigator.clipboard.writeText(cur()).then(function(){
      c.textContent='Copied';setTimeout(function(){c.textContent='Copy';},1600);
    });
  });
})();

---

| Area | How this audit differs |
| :---- | :---- |
| **Grammar** | "Show me" and "produce the record", not "ensure" and "establish" |
| **Scope** | One named business question, re-run per question |
| **Scoring** | Pass, fail, or not applicable. No maturity band |
| **What is new** | Determinism, inspectable logic and recorded procedure appear in no public checklist |
| **Shelf life** | Weeks. A pass expires when definitions, sources or agent tooling change |

---

## How to run this checklist

Pick a question the business already asks and somebody senior already chases by email. Run the twelve checks against that question rather than against the estate. Record pass, fail or not applicable, and for every pass, name the artifact.

Two rules keep the result honest. Score what operates today, never what is planned. And do not average: a question that passes on meaning and fails on reproducibility is not 60% ready, it is not ready, which is why the [readiness assessment](https://atlan.com/know/ai-agent/data-for-ai/ai-readiness-assessment/) version of this rubric gates rather than weights.

If you want the sequence for fixing what fails rather than the audit itself, [how to prepare enterprise data for AI agents](https://atlan.com/know/ai-agent/data-for-ai/how-to-prepare-enterprise-data-for-ai-agents/) is the build path.

---

## The checklist at a glance

| # | Check | Passes when | Evidence |
| :---- | :---- | :---- | :---- |
| 1 | Certified meaning | Every concept in the question maps to one signed-off definition | The definition, its certifier, its review date |
| 2 | Declared relationships | Join paths and entity relationships are modelled, not guessed | The approved join path for the question |
| 3 | Machine-readable docs | The dictionary resolves over an interface, not only in a wiki | An API response, not a screenshot |
| 4 | Freshness service level | Every asset in scope has a published SLA and an observed lag | The current lag against the stated SLA |
| 5 | Traceable lineage | One figure traces to source with no human filling a gap | The lineage path, end to end |
| 6 | Named quality thresholds | Critical fields have thresholds tested on a schedule | The last test run and its date |
| 7 | Entitlement-aware results | Two differently entitled users get correctly different results | Both result sets from one question |
| 8 | Agent audit identity | The agent appears in the log as itself | The audit entry naming the agent |
| 9 | Classified before connected | Sensitive fields are classified and masking demonstrably fires | A log line showing the control fired |
| 10 | Determinism | Ten runs on unchanged data give one distinct answer | The ten outputs |
| 11 | Inspectable logic | You can reconstruct how any past answer was computed | Question, definition, query, policy, data version |
| 12 | Recorded procedure | The method for the question is written down and owned | The procedure and its owner |

---

## Meaning: can the system tell what your data means?

### 1. Every concept in the question resolves to one certified definition

Nine of sixteen published checklists mention business context or a glossary. None asks whether a definition is certified, by whom, or whether two systems currently disagree about it, which is the only version of the check that can be failed.

**Passes when** each business concept in your question has one definition with a named human owner, not a team alias, and a review date. **Evidence:** the definition, the certifier, the date, and the same number returned by the BI tool and the agent. A [semantic layer](https://atlan.com/know/semantic-layer/) or a set of [data contracts for AI](https://atlan.com/know/ai-agent/data-for-ai/data-contracts-for-ai/) is what makes this retrievable instead of remembered, and [systems of semantics](https://atlan.com/know/ai-agent/data-for-ai/systems-of-semantics/) is the pattern underneath.

### 2. The relationships between entities are declared, not inferred

A question spanning two concepts depends on a join path, and if nobody has approved one the agent will invent it from column names. Researchers benchmarked GPT-4 at 16% accuracy answering questions directly against an enterprise SQL schema, and 54% over a knowledge-graph representation of the same data.

**Passes when** the join paths for the question are modelled and approved. **Evidence:** the approved path. This is where [ontology](https://atlan.com/know/what-is-ontology-in-ai/) does work a glossary cannot, and the [difference between ontology and a semantic layer](https://atlan.com/know/ontology-vs-semantic-layer/) decides which artifact you are looking for.

### 3. The documentation resolves over an interface, not only in a wiki

Machine-readability is the check almost nobody states. An open environmental-science checklist published in 2023 tests it explicitly, with separate items for machine-readable metadata and a machine-readable data dictionary, and the enterprise lists have not caught up.

**Passes when** an agent can retrieve the definition, owner and freshness of an asset programmatically. **Evidence:** the response, not a screenshot of a page. This is the practical reason [types of metadata for AI agents](https://atlan.com/know/ai-agent/data-for-ai/types-of-metadata-for-ai-agents/) and protocols like [MCP](https://atlan.com/know/what-is-model-context-protocol/) matter more than documentation volume.

---

## Trust: is the data fit to answer with?

### 4. Every asset in scope has a published freshness service level

Freshness appears on eleven of sixteen checklists and almost always without a threshold, which makes it unfailable.

**Passes when** each asset the question touches has a stated update frequency and you can read its current lag against that statement. **Evidence:** the SLA and the observed lag. [Context freshness](https://atlan.com/know/ai-agent/context-freshness/) is a readiness property rather than a hygiene task, because an answer computed from a stale source is wrong in a way no quality test catches.

### 5. One production figure traces to source with no gaps

Twelve of sixteen checklists ask for lineage. None states how to verify it.

**Passes when** you can pick one figure the question produces and follow it back to the originating systems without a human explaining a missing hop. **Evidence:** the trace. [AI-ready data lineage](https://atlan.com/know/ai-readiness/ai-ready-data-lineage/) is the difference between an answer you can defend in a review and one you can only repeat.

### 6. Critical fields have named thresholds, tested on a schedule

"We have quality tooling" is not a pass. A threshold, a schedule and a last-run result is.

**Passes when** the fields the question depends on have stated thresholds, a test cadence, and a retrievable last run. **Evidence:** the run and its date. This is where [data quality for AI agents](https://atlan.com/know/data-for-ai/data-quality-for-ai-agent/) and [data observability for AI pipelines](https://atlan.com/know/ai-agent/data-for-ai/data-observability-for-ai-pipelines/) stop being platforms and start being evidence.

---

## Boundaries: does policy hold when an agent asks?

### 7. Two differently entitled people get correctly different results

Fourteen of sixteen checklists test access control, and almost all of them mean human role-based access.

**Passes when** the identical question run as two principals with different entitlements returns correctly different results. **Evidence:** both result sets. If the answers match, the policy is being applied somewhere other than the query, which is the quiet failure [agent access control](https://atlan.com/know/ai-agent-access-control/) exists to catch.

### 8. The agent appears in the audit log under its own identity

An agent on a broadly entitled shared service account inherits access no individual user has, and the log records the account rather than the agent. Every answer it produces becomes an exfiltration path that governance reporting cannot see.

**Passes when** the agent's own identity appears in the audit entry for a query it ran. **Evidence:** the entry. [Giving agents access to enterprise data](https://atlan.com/know/ai-agent/how-to-give-ai-agents-access-to-enterprise-data/) is a governance decision before it is a connectivity one, and it belongs inside [AI agent governance](https://atlan.com/know/ai-agent-governance/) rather than beside it.

### 9. Sensitive fields are classified before they are connected, and the control fires

Classification is common on published checklists. Proof that the control fired is rare.

**Passes when** sensitive fields in scope are classified, and you can show a log line where masking or a usage control actually applied to an agent query. **Evidence:** that line. Existence of a policy is not evidence that it executed, which is the same gap [handling PII in AI pipelines](https://atlan.com/know/ai-agent/data-for-ai/how-to-handle-pii-in-ai-pipelines/) has to close.

---

## Reproducibility: does the same question return the same answer?

These three appear in no published AI-ready data checklist, and they are the ones agents fail on.

### 10. Ten runs on unchanged data return one distinct answer

Count the distinct answers. The pass mark is one.

Part of the instability is the model layer: Thinking Machines Lab ran 1,000 identical completions of an open model at temperature zero and got 80 distinct outputs, all agreeing through token 102 and diverging at token 103. The larger part is usually the data layer, where the same entity exists twice under two definitions and the agent picks differently on different runs. **Evidence:** the ten outputs, plus a set of questions with known-correct answers and the date it was last run. Retrieval-side practice already has this shape in [RAG evaluation](https://atlan.com/know/how-to-evaluate-rag-systems-explained/) and [agent evaluation](https://atlan.com/know/ai-agent-evaluation-benchmarks-and-metrics/).

### 11. You can reconstruct how any past answer was computed

When a model writes the logic inline, you get the number and lose the reasoning. Gartner's public guidance gets closest to this with "controlled inference and derivation," and attaches no test to it.

**Passes when**, for an answer given last month, you can produce the question, the definition it resolved to, the query that ran, the policies that applied, and the data version. **Evidence:** those five. Logic that lives in a versioned, reviewable analytical asset survives this check; logic improvised inside a model call does not, which is what makes [decision traces](https://atlan.com/know/what-are-decision-traces-for-ai-agents/) and [context observability](https://atlan.com/know/ai-agent/ai-agent-observability/context-observability-for-ai-agents/) worth the setup. Where the answer runs through [text-to-SQL](https://atlan.com/know/ai-agent/data-for-ai/text-to-sql-for-enterprise/), the generated query is part of the record.

### 12. The procedure for the question is written down and owned

Zero of sixteen published checklists ask this. Teaching an agent a skill is not the same as giving it the procedure: the ordered steps, the decision points, the checks, and what happens when a source is late.

**Passes when** the method for this recurring question exists somewhere other than in a person's head, with its retries and an owner. **Evidence:** the document and the name. Until it does, the workflow depends on [tribal knowledge](https://atlan.com/know/data-for-ai/tribal-knowledge/), and a [context layer for AI agents](https://atlan.com/know/context-layer-for-ai-agents/) is where that knowledge becomes something a system can execute rather than something a person remembers.

---

## What the published checklists are actually testing

Read twenty of them and a pattern shows up: they verify that a capability exists, not that it works. A catalog is in place. Governance is established. Lineage is captured. All true, all unfalsifiable, and none of it tells you whether last Tuesday's number was right.

The evidence grammar matters more than the item count. "Show me the same question asked by two users with different entitlements" is a check. "Proof that access, masking and usage controls fired" is a check. "Establish governance and clear ownership" is a sentence. Where the published lists do use evidence grammar, they borrow it from audit practice rather than from data practice, which is probably where it should have come from all along.

The measured consequence of the softer version is visible. A March 2026 survey of senior data and AI executives found 61% reporting silent failures, where monitored metrics looked normal throughout a critical issue, and 62% naming human review before release as their primary verification method. Reviewing outputs by hand is what an organization does when it has no test it trusts.

---

## What to do with the items you failed

A failed check is a work item with an owner, and most of them land outside the team that ran the audit. Definitions go back to the domain that owns the metric. Identity and entitlements go to platform. Procedures go to whoever runs the workflow today and has never written it down.

Order the work by what the question needs rather than by check number. Meaning failures block everything downstream, because a question that resolves ambiguously cannot be made reproducible by better plumbing. Boundary failures are the ones to fix before an agent goes anywhere near production data, since they are the only category on this list where the failure mode is a disclosure rather than a wrong number. Reproducibility failures usually resolve themselves once meaning is fixed, and the ones that survive are the interesting ones, because they point at two systems that have been disagreeing quietly for years.

Then re-run it. A pass is perishable: [what makes data AI-ready](https://atlan.com/know/ai-agent/data-for-ai/what-makes-data-ai-ready/) is a state that decays as definitions drift and sources change, and published measurements of accuracy decay run in weeks rather than years. The organizations that keep agents working treat this as a standing check on a small number of questions rather than a programme with an end date, which is also the honest reason [context engineering](https://atlan.com/know/what-is-context-engineering/) is a job and not a project.

---

## FAQs about the AI-ready data checklist

### 1. How many items should an AI-ready data audit have?

Fewer than you think, as long as each one can be failed. Published enterprise lists run from five aspirational imperatives to thirty open discussion prompts, and the length is not what makes them useful. Twelve checks scored against one named business question produce a shorter and more actionable result than thirty checkboxes scored against the warehouse.

### 2. If our data quality is good, do we still need this?

Yes. Gartner states that high-quality data, judged by traditional data quality standards, does not equate to AI-ready data. Quality checks test the contents of a table. These checks test whether an agent can resolve what the table means, whether policy holds when it asks, and whether the answer reproduces.

### 3. What counts as evidence that a check passes?

An artifact you can produce on request, not an assertion. A certified definition with a named owner and a review date. A lineage trace with no human-filled gaps. Two differently scoped results from the same question. Ten runs of the same question with one distinct answer. The written procedure, with its owner.

### 4. Why does asking the same question twice matter?

Because an unstable answer means something underneath is ambiguous, usually a concept that exists in two systems under two definitions. There is also a model-layer contribution: Thinking Machines Lab ran 1,000 identical completions of an open model at temperature zero and got 80 distinct outputs. No published AI-ready data checklist asks for this test.

### 5. Should agents share a service account?

No. An agent running on a broadly entitled shared service account inherits access no individual user has, and the audit log records the service account rather than the agent. Every answer it produces is then a potential exfiltration path that governance reporting cannot see.

### 6. How often should the checklist be re-run?

Treat a pass as perishable. Re-run when definitions change, when a source system is replaced, when the agent gains a new tool or data source, and on a fixed cadence in between. Gartner describes AI-ready data as a practice rather than a one-and-done project, and published measurements of accuracy decay run in weeks.

---

## Sources

1. [What Is AI-Ready Data? And How to Get Yours There, Gartner](https://www.gartner.com/en/articles/ai-ready-data)

2. [Lack of AI-Ready Data Puts AI Projects at Risk, Gartner (26 February 2025)](https://www.gartner.com/en/newsroom/press-releases/2025-02-26-lack-of-ai-ready-data-puts-ai-projects-at-risk)

3. [Defeating Nondeterminism in LLM Inference, Thinking Machines Lab (10 September 2025)](https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/)

4. [The State of AI Reliability, Monte Carlo with CDO Magazine (March 2026)](https://montecarlo.ai/state-of-ai-reliability)

5. [A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databases, Sequeda, Allemang and Jacob, arXiv:2311.07509](https://arxiv.org/abs/2311.07509)

6. [Checklist to Examine AI-Readiness for Open Environmental Datasets v1.0, ESIP Data Readiness Cluster](https://github.com/ESIPFed/data-readiness/blob/main/checklist-published/ai-ready-data-checklist-v.1.0.md)

7. [Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows, ICLR 2025](https://proceedings.iclr.cc/paper_files/paper/2025/file/46c10f6c8ea5aa6f267bcdabcb123f97-Paper-Conference.pdf)

8. [How Anthropic Enables Self-Service Data Analytics With Claude, Anthropic (3 June 2026)](https://claude.com/blog/how-anthropic-enables-self-service-data-analytics-with-claude)