--- title: "How Does AI-Ready Data Support Talk-to-Data Analytics?" url: "https://atlan.com/know/ai-agent/ai-ready-data/ai-ready-data-for-talk-to-data/" description: "How does AI-ready data support talk-to-data and natural language analytics? Test each business question, read accuracy evidence, and evaluate before rollout." author: "Emily Winks" author_role: "Data Governance Expert" published: "2026-11-06" updated: "2026-11-06" --- ## How does AI-ready data support talk-to-data and natural language analytics? AI-ready data supports talk-to-data when each business question reaches the agent with its metric definition, trusted join path, owner, lineage and access rule. Readiness is set per question, so test one question at a time, take its weakest dimension as the score, and prove it with repeated runs graded on the answer. Atlan's context layer assembles that context for one use case at a time. What one question needs to be ready: - **Meaning:** one signed-off definition for the metric, its terms and its rules - **Provenance:** certified tables, a known join path and lineage to source - **Boundaries:** the asker's permissions applied at query time - **Proof:** repeated runs and phrasings, graded on the answer ## Key takeaways - The unit of talk-to-data readiness is one business question, in the words people use. - LinkedIn's 133-question ablation: schema alone scored 9% correct or close; with descriptions and example queries, 49%. - Run each question three to five times, in more than one phrasing, and grade the rows it returns against the accepted answer. - Score a question by its weakest dimension, then prove it with repeated, answer-graded runs. --- AI-ready data supports talk-to-data when each business question reaches the agent with its metric definition, join path, owner, lineage and access rule, and a repeated evaluation proves the answer. Atlan's context layer assembles that context per use case in a [context repository (Atlan docs)](https://docs.atlan.com/product/capabilities/governance/context-engineering-studio/concepts/what-is-context-engineering-studio). According to [LinkedIn's ablation of 133 internal questions](https://arxiv.org/abs/2507.14372), schema alone scored 9% correct or close; adding table descriptions, clusters and example queries lifted it to 49%. ### Score One Business Question Before Your Agent Answers It Give it one business question people ask in plain words and what you have on its definition, lineage, and access rules. It scores five dimensions, gates on the weakest, and returns the gap that blocks the answer. [Read the skill](/skills/ai-ready-question-score.md). *Paste into a new chat* ``` Use the skill at https://atlan.com/skills/ai-ready-question-score.md to score whether one business question is ready for a talk-to-data agent. Ask me for whatever it needs. ``` *Run once in a terminal* ``` curl -fsSL --create-dirs \ -o ~/.agents/skills/ai-ready-question-score/SKILL.md \ https://atlan.com/skills/ai-ready-question-score.md ``` *For an agent* ``` curl -fsSL https://atlan.com/skills/ai-ready-question-score.md ``` Human Agent ClaudeChatGPTGeminiCursorOther Copy --- The same warehouse can be ready for "net revenue by region last quarter" and unready for "active customers added this fiscal quarter," because the second rests on a rule nobody wrote down. The [talk-to-data agent blueprint](https://atlan.com/know/ai-agent/talk-to-data-agent-blueprint/) covers the architecture that reads those rules. | Field | Talk-to-data | |---|---| | Use case | People ask questions of structured data in plain words; the agent writes and runs SQL | | Unit of readiness | One business question, in the words people use | | Primary stakeholders | The AI leader, analytics engineers, metric owners, data stewards | | Typical data challenges | Competing metric definitions; unwritten joins; coded values; fiscal rules; access that depends on who asks | | What proves readiness | Repeated runs and phrasings, graded on the answer, refusals counted apart | --- ## Why is talk-to-data readiness decided one question at a time? Talk-to-data readiness is decided per question because each question needs its own definition, join path and access rule, with no analyst beside the agent to fill gaps. Gartner's Rita Sallam, Distinguished VP Analyst, states the principle in [Gartner's AI-ready data guidance](https://www.gartner.com/en/articles/ai-ready-data): "There is no way to make data AI-ready in general or in advance. The readiness of data for AI depends on how the data will be used." In analytics, an analyst knew which revenue column finance trusts and when the fiscal year starts. That [tribal knowledge](https://atlan.com/know/data-for-ai/tribal-knowledge/) is the gap between [analytics-ready data and AI-ready data](https://atlan.com/know/ai-agent/ai-ready-data/ai-ready-data-vs-analytics-ready-data/), because talk-to-data takes the analyst out of each answer. Per question, three things break: - A metric with several definitions, where the agent picks one and returns a clean, wrong number. - Rules that live in someone's head: the fiscal calendar, test accounts to exclude, which stages count. - An answer that should change with who is asking, and does not. A warehouse that passes the checks in [what makes data AI-ready](https://atlan.com/know/ai-agent/data-for-ai/what-makes-data-ai-ready/) can still fail your CFO's Monday question, because the [business context for AI](https://atlan.com/know/business-context-for-ai/) behind it was never written down. --- ## How much does each layer of business context improve talk-to-data accuracy? Published tests add different context to different data and models, and together they show that the right context for a question beats loading everything. | Study (date) | Setting | Context added | Result | Read it as | |---|---|---|---|---| | [LinkedIn (Jul 2025)](https://arxiv.org/abs/2507.14372) | 133 internal questions, ablation | Schema only; plus table clusters and descriptions; plus example queries | 9%; 24%; 49% correct or close (LLM-judged) | Example queries were the largest step in this order | | [Atlan AI Labs (Jan 2026)](https://atlan.com/know/enhanced-metadata-improves-query-accuracy/) | 174 questions, each run 3 times, 13-table Formula One dataset | Glossary, SQL usage patterns, domain hints | 16.1% to 22.2% win rate, a 38% relative gain | Complex queries stayed at 1.00x; a benchmark, not a customer result | | [MotherDuck (Aug 2025)](https://motherduck.com/research/query-log-informed-schema-descriptions-text-to-sql/) | Own warehouse, 2,730+ columns | Descriptions mined from query logs | One model 36% to 52%; another 18% to 8% | The same context helped one model and hurt another | | [Atlan playbook benchmark (undated)](https://analytics-playbook.atlan.com/#results) | 107 business questions, one model | Whole metrics layer in every prompt vs retrieved per question | 52.3% vs 83.2%, under half the cost per right answer | Baseline is a whole-layer prompt; an Atlan measurement, not independently replicated | According to [Atlan's analytics agents playbook](https://analytics-playbook.atlan.com/#ch-retrieval), "a long prompt buries the decisive item among near-variants, and the agent picks the closest-looking one." Serving one question's context at the moment it is asked is the work of [context engineering](https://atlan.com/know/what-is-context-engineering/), and the daily job of a [context-engineered AI analyst](https://atlan.com/know/context-engineering-for-ai-analyst/). --- ## Why don't public text-to-SQL leaderboard scores predict accuracy on your data? Public leaderboard scores on prepared subsets now sit far higher than scores on private warehouses and business rules, and those private questions are the ones your agent gets. | Benchmark (date) | What it tests | Reported figure | What it tells you | |---|---|---|---| | [Spider 2.0 paper (Nov 2024)](https://arxiv.org/abs/2411.07763) | 632 enterprise workflow tasks, databases often over 1,000 columns | 21.3%, a code agent framework | The 2024 starting point | | [Spider 2.0 leaderboard](https://spider2-sql.github.io/) | Spider 2.0-Snow: 547 cloud-warehouse tasks, metadata and documentation provided | Top score 96.70 | Engineered agents on a prepared subset | | [BEAVER (v3, May 2026)](https://arxiv.org/abs/2409.02038) | 9,128 question-SQL pairs, 812 tables | 10.8%, state-of-the-art agentic frameworks | Private-warehouse questions stay hard | | [EntSQL (Jun 2026)](https://arxiv.org/abs/2606.03363) | 1,066 examples resting on internal metrics, reporting conventions and organizational rules | 15.9%, best system with long-form documents | Business rules are the gap | According to the [Spider 2.0 authors](https://arxiv.org/abs/2411.07763), the framework that solved 21.3% of Spider 2.0 tasks scored 91.2% on Spider 1.0. The [BEAVER authors](https://arxiv.org/abs/2409.02038) report that "SOTA agentic frameworks using the advanced model GPT-5.2 achieve only 10.8% accuracy" on private-warehouse questions. Tasks, data and systems differ, so these figures are not directly comparable. BEAVER shows warehouse-scale queries stay hard; EntSQL isolates business rules, the gap behind the metric drift in [text-to-SQL for enterprise](https://atlan.com/know/ai-agent/data-for-ai/text-to-sql-for-enterprise/), where fluent wrong numbers behave like [AI agent hallucination](https://atlan.com/know/ai-agent-hallucination/). Only your golden set measures your definitions. --- ## Which talk-to-data questions are ready first, and which are not? Questions built on one defined metric and a standard dimension are usually the first to pass; questions that depend on unwritten rules or cross domains are the last. ### A defined metric by a standard dimension "What was net revenue by region last quarter?" fails when several revenue columns exist and the agent writes valid SQL against the wrong one. It is ready when `net_revenue` has one approved definition as code, a certified table and a verified question-and-SQL pair. If your query tools store verified question-and-SQL pairs, the open question across several tools is which definition each pair follows. In [LinkedIn's ablation](https://arxiv.org/abs/2507.14372), example queries were the largest step in its order, 24% to 49%, and in [Atlan AI Labs' Formula One benchmark (Atlan, 2026)](https://atlan.com/know/enhanced-metadata-improves-query-accuracy/) medium-complexity queries gained 2.15x and simple ones 1.15x. ### A question that depends on a business rule "How many active customers did we add this fiscal quarter?" fails when "active" and `fiscal_quarter` live in a spreadsheet. It is ready when the rule is modeled and a named owner signs off, which is where [KPI definitions in a business glossary](https://atlan.com/know/ai-agent/how-to-manage-kpi-definitions-and-a-business-glossary-in-a-data-catalog/) come in. ### A cross-domain or judgment question "Which accounts are at churn risk?" has no single definition and spans domains. It is ready once an agreed definition or an approved risk score exists, with an owner; until then, the agent should refuse or hand off. According to the [EntSQL authors](https://arxiv.org/abs/2606.03363), EntSQL's best system reached 15.9% on rule-heavy questions, and complex queries in [Atlan AI Labs' benchmark](https://atlan.com/know/enhanced-metadata-improves-query-accuracy/) stayed at 1.00x. Whether a rule is written at all matters more than the [semantic layer vs metrics layer](https://atlan.com/know/ai-agent/semantic-layer/semantic-layer-vs-metrics-layer/) choice. A [semantic layer](https://atlan.com/know/semantic-layer/) answers what was modeled; an agent that says "not ready yet" on the rest is behaving correctly. --- ## How do you test whether one business question is ready for talk-to-data? Test one question at a time across five dimensions, ask for the artifact behind every pass, and take the weakest dimension as the question's score. Start from a question a named team asks, with an answer they accept. | Dimension | Ready when | Not-ready signal | Evidence to show | |---|---|---|---| | Meaning | Metric, terms and rules resolve to one definition a named person signed off | The agent writes its own formula; "Florida" misses `FL` | The definition, certifier and review date | | Provenance | Tables and join path are certified; lineage reaches the source; data is fresh | Wrong table, a duplicating join, a stale source | The lineage trace and each hop's freshness | | Boundaries | The asker's permissions apply at query time | Rows shown or hidden wrongly | One question run as two users with different access | | Reproducibility | Repeats and rephrasings return the accepted answer | One passing run | Results across repeats and phrasings, on one data snapshot | | Procedure | Tables, filters, checks and an escalation contact are written | It works while one person remembers it | The written method and its owner | The gate is the lowest dimension, never the average. A question strong on four dimensions and failing reproducibility returns different numbers from the same data. Keep the five scores, their artifacts and the run date next to the question in your golden set, and hold back any question whose lowest score fails. Provenance leans on [data lineage for AI](https://atlan.com/know/ai-agent/data-for-ai/data-lineage-for-ai/), boundaries on [role-based access control in the context layer](https://atlan.com/know/ai-agent/context-layer/context-layer-role-based-access-control/), and meaning on [owners and stewards for AI-ready data](https://atlan.com/know/ai-agent/ai-ready-data/ai-ready-data-owners-and-stewards/). An Atlan certificate takes a message, so name the use case it was certified for, the core of [trusted data for AI](https://atlan.com/know/ai-readiness/what-is-trusted-data-for-ai-and-how-do-you-get-it/). When a check fails, the table work behind each fix lives in [how to prepare data for natural language analytics](https://atlan.com/know/ai-readiness/how-to-prepare-data-for-natural-language-analytics/). --- ## How do you evaluate a talk-to-data agent before rollout? Evaluate a talk-to-data agent with a golden set of real questions and known answers, run each several times in several phrasings, and grade the returned rows against the accepted answer. 1. **Build the golden set first.** Atlan's [analytics agents playbook](https://analytics-playbook.atlan.com/#ch-evals) states the rule: "Build the golden set before any context." Without sealed answers, a fix and a lucky run look the same. 2. **Repeat runs and phrasings.** The playbook runs each question three to five times. Ask each one the different ways people phrase it. 3. **Grade the result set.** Compare the rows the agent returns with the accepted answer, and decide per question whether sort order, rounding or an extra column counts as a miss. 4. **Count refusals apart from wrong answers.** A refusal outside the model is correct behavior; a wrong number is not. If your warehouse-native or BI-native natural-language query tool has its own evaluation feature, check before you rely on it that it holds each question with an expected answer, compares executed results rather than SQL text, flags questions that passed before and now fail, and lets you repeat runs. That feature still tests definitions inside one tool, as the guides to [lakehouse agent context requirements](https://atlan.com/know/ai-agent/databricks/databricks-genie-context-requirements/), [conversational BI context](https://atlan.com/know/ai-agent/microsoft/power-bi-conversational-analytics-context/) and [warehouse-native natural-language query vs text-to-SQL](https://atlan.com/know/snowflake/cortex-analyst-vs-text-to-sql/) describe. Run more than one, and the same definition and owner must match in each, whatever [AI agent evaluation metrics](https://atlan.com/know/ai-agent-evaluation-benchmarks-and-metrics/) or [AI agent accuracy](https://atlan.com/know/ai-agent/ai-agent-accuracy/) target you set. [Context testing for AI agents](https://atlan.com/know/ai-agent/ai-agent-evals/what-is-context-testing-for-ai-agents/) checks what the agent was given, unlike row checks ([context evals vs data quality testing](https://atlan.com/know/ai-agent/ai-agent-evals/context-evals-vs-data-quality-testing/)). Re-run the whole set when a definition, schema or policy changes, because [context drift](https://atlan.com/know/context-drift-ai-agents/) breaks answers without an error. --- ## What goes wrong when teams judge talk-to-data readiness? Most readiness mistakes measure the wrong thing: a public score, a whole-layer prompt, a refusal scored as an error, or a single test. | Mistake | What happens | Do this instead | |---|---|---| | Trusting a public leaderboard | Private-rule questions score far lower ([BEAVER, 10.8%](https://arxiv.org/abs/2409.02038)) | Test on your own golden set | | Loading the whole context layer into every prompt | [52.3% vs 83.2% retrieved per question (Atlan)](https://analytics-playbook.atlan.com/#results) | Retrieve the question's context | | Treating a refusal like a wrong number | Teams tune the agent to guess | Score refusals separately | | Testing once | A changed definition breaks answers silently | Re-run on every definition, schema or policy change | A [context layer that controls hallucination](https://atlan.com/know/ai-agent/context-layer/how-context-layer-controls-hallucination/) helps only when someone owns each fix; [context layer ownership between data and AI teams](https://atlan.com/know/context-layer-ownership-data-vs-ai-teams/) settles who. --- ## How Atlan makes data ready for talk-to-data, one question at a time Atlan's context layer assembles the definitions, verified queries, owners and lineage one talk-to-data use case needs, and serves them to the tools that answer the question. - **A context repository per use case.** According to [Atlan's Context Engineering Studio docs](https://docs.atlan.com/product/capabilities/governance/context-engineering-studio/concepts/what-is-context-engineering-studio), a repository holds a semantic model of "dimensions, facts, relationships, derived metrics, filters, and verified queries," rendered in the semantic formats that warehouse, lakehouse and transformation tools read. - **Certificates and owners.** People set [certificates (Atlan docs)](https://docs.atlan.com/product/capabilities/discovery/how-tos/add-certificates) such as Verified, Draft or Deprecated, and [asset owners (Atlan docs)](https://docs.atlan.com/product/capabilities/discovery/how-tos/add-owners) are the primary contact for questions about status and quality. - **Lineage.** Atlan captures [column-level lineage (Atlan docs)](https://docs.atlan.com/product/capabilities/lineage/concepts/what-is-column-level-lineage) automatically across warehouses, pipelines and BI tools. - **Permissions on context.** [Atlan MCP (Atlan docs)](https://docs.atlan.com/product/capabilities/atlan-ai/how-tos/atlan-mcp-overview) is "a hosted server, available on every Atlan tenant with nothing to install," and "every action respects the permissions you've already set in Atlan." - **Evaluation.** The Context Engineering Studio docs point teams to Atlan Trident, [in private preview (Atlan docs)](https://docs.atlan.com/get-started/concepts/atlan-products), to test a talk-to-data agent on real questions. Starting from a catalog, [extend it for AI readiness](https://atlan.com/know/ai-agent/ai-ready-data/extend-data-catalog-for-ai-readiness/); for delivery, read [what is Atlan MCP](https://atlan.com/know/what-is-atlan-mcp/) and [AI-ready data over MCP](https://atlan.com/know/ai-agent/ai-ready-data/ai-ready-data-mcp/); for the rollout, [how to implement an enterprise context layer for AI](https://atlan.com/know/how-to-implement-enterprise-context-layer-for-ai/). --- ## Readiness is a verdict on one question, and it expires A question is ready when its weakest dimension passes and a repeated, answer-graded evaluation agrees. The verdict lasts until the next definition, schema or policy change. --- ## FAQs about AI-ready data for talk-to-data ### 1. How does AI-ready data improve talk-to-data accuracy? It gives the agent what an analyst used to supply: the approved definition, verified example queries, the join path and the business rules. In LinkedIn's 133-question ablation, schema alone scored 9% correct or close; adding descriptions and example queries, 49%. ### 2. How accurate is text-to-SQL on enterprise data? It depends on whose data and rules. On private-warehouse questions, BEAVER reports 10.8% for state-of-the-art agentic frameworks, and EntSQL's best system reached 15.9%. Spider 2.0-Snow shows a top score of 96.70 on the public leaderboard, on a prepared subset, a setup that does not compare directly. ### 3. Does a semantic layer make data ready for talk-to-data? For the questions it models, it helps, because the agent queries a defined metric instead of writing its own formula. Questions outside the model wait for modeling, and ownership, access and repeated evaluation need their own process. ### 4. How many questions should a talk-to-data test set have? Start with the real questions one named team asks, each with an accepted answer and several phrasings. Atlan's playbook benchmark used 107. Cover every metric family and access role the team relies on; realism and sealed answers matter more than size. ### 5. Which questions should a talk-to-data agent answer first? Questions built on one defined metric and a standard dimension, such as revenue by region last quarter. In Atlan AI Labs' Formula One benchmark, medium-complexity queries gained 2.15x from added context while complex queries stayed at 1.00x. Hold cross-domain questions back until a definition and an owner exist. ### 6. What does Atlan provide to make a question ready for talk-to-data? A context repository for one use case, with a semantic model of metrics, filters and verified queries rendered for the tools that answer questions. Certificates and owners mark which assets are fit to use, and column-level lineage traces each column to source. Atlan Trident, in private preview, tests agents on real questions. --- ## Sources 1. [Text-to-SQL for Enterprise Data Analytics, arXiv (2025)](https://arxiv.org/abs/2507.14372) 2. [How We Proved Metadata Delivers 38% Better AI Accuracy, Atlan (2026)](https://atlan.com/know/enhanced-metadata-improves-query-accuracy/) 3. [Query-Log-Informed Schema Descriptions and their Impact on Text-to-SQL, MotherDuck (2025)](https://motherduck.com/research/query-log-informed-schema-descriptions-text-to-sql/) 4. [The FDE Playbook for Analytics Agents, Atlan](https://analytics-playbook.atlan.com/) 5. [Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows, arXiv (2024)](https://arxiv.org/abs/2411.07763) 6. [Spider 2.0 leaderboard](https://spider2-sql.github.io/) 7. [BEAVER: An Enterprise Benchmark for Text-to-SQL, arXiv (2026)](https://arxiv.org/abs/2409.02038) 8. [EntSQL: A Benchmark for Grounding Text-to-SQL in Long-Context Enterprise Knowledge, arXiv (2026)](https://arxiv.org/abs/2606.03363) 9. [AI-Ready Data Essentials to Capture AI Value, Gartner](https://www.gartner.com/en/articles/ai-ready-data) 10. [What is Context Engineering Studio?, Atlan Documentation](https://docs.atlan.com/product/capabilities/governance/context-engineering-studio/concepts/what-is-context-engineering-studio) 11. [Add certificates, Atlan Documentation](https://docs.atlan.com/product/capabilities/discovery/how-tos/add-certificates) 12. [Add owners, Atlan Documentation](https://docs.atlan.com/product/capabilities/discovery/how-tos/add-owners) 13. [What is column-level lineage?, Atlan Documentation](https://docs.atlan.com/product/capabilities/lineage/concepts/what-is-column-level-lineage) 14. [Atlan MCP, Atlan Documentation](https://docs.atlan.com/product/capabilities/atlan-ai/how-tos/atlan-mcp-overview) 15. [Atlan products, Atlan Documentation](https://docs.atlan.com/get-started/concepts/atlan-products) ## Next steps - [See how Atlan's context layer works in recorded demos](https://atlan.com/context-layer-demo/) - [Book a demo or talk to Atlan's sales team](https://atlan.com/forms/talk-to-sales-contact/)