Talk-to-Data Agent Blueprint and the Context Layer

---

A talk-to-data agent answers business questions in natural language by writing and running SQL, and the hardest part of building one is not the SQL. Modern agents already reach about 91.2% execution accuracy on the academic Spider 1.0 benchmark, yet drop to 21.3% on enterprise-realistic Spider 2.0 tasks (Spider 2.0 paper, 2025). The missing variable is governed context, not model horsepower, and this blueprint treats that context as a first-class architectural tier. The bottleneck is meaning, not generation: agents write valid SQL but pick the wrong column or join, and the accuracy lever is governed context, delivered read-only at inference, not a bigger model.

---

Why does text-to-SQL fail on enterprise schemas?

Permalink to “Why does text-to-SQL fail on enterprise schemas?” #

Enterprise text-to-SQL collapses not on SQL syntax but on schema and semantic meaning. According to the Spider 2.0 paper (2025), an o1-preview agent framework solves only 21.3% of enterprise-realistic tasks, compared with 91.2% on the smaller Spider 1.0 schemas. The model writes valid SQL; it cannot reliably know which table, column, or join carries the business meaning the question intends.

The failure data points to one cause. According to Omni’s “Why text-to-SQL fails,” 81.2% of failures are schema-level errors such as wrong column selection, not SQL syntax, and Omni reports GPT-5 completed only 29% of enterprise tasks, so a bigger model does not close a meaning gap.

The clearest evidence for the lever sits in a controlled study (arXiv:2311.07509): GPT-4 answered only 16% of questions correctly over a raw SQL schema, but 54% of the same questions when given a knowledge-graph representation with ontology and mappings. The model did not change; only the context did. A knowledge graph is one form of governed context, and the 38-point lift shows that context, not a larger model, moves enterprise accuracy.

Accuracy-gap evidence (verified benchmarks only):

Benchmark / setting	Accuracy	Source
Spider 1.0 (academic, small schemas), o1-preview agent	91.2%	Spider 2.0 paper, ICLR 2025 (arXiv:2411.07763)
Spider 2.0 (enterprise-realistic), o1-preview agent	21.3%	Spider 2.0 paper, ICLR 2025 (arXiv:2411.07763)
GPT-4 zero-shot over raw SQL schema	16%	arXiv:2311.07509
Same questions over knowledge-graph context	54%	arXiv:2311.07509
Share of failures that are schema/semantic-level	81.2%	Omni, “Why text-to-SQL fails”
GPT-5 on enterprise tasks	~29%	Omni, “Why text-to-SQL fails”

Practitioners reach the same conclusion without the benchmarks: the recurring line in r/dataengineering threads is that text-to-SQL cannot function without a semantic layer. The investment that moves accuracy is the governed context layer for enterprise AI.

---

The talk-to-data agent reference architecture (six tiers)

Permalink to “The talk-to-data agent reference architecture (six tiers)” #

The talk-to-data agent reference architecture is six tiers, and only one is load-bearing for accuracy: the governed context layer that feeds the agent business meaning at inference time. The interface, model, and framework tiers are interchangeable execution machinery; the context layer is the proprietary, compounding spine that decides whether a valid query is also semantically correct. Wire the agent before deciding where governed context lives, and tiers 4 and 5 sit on a foundation that guesses at meaning.

Six-tier reference architecture:

Tier	What it does	Why it matters for accuracy	Atlan component
1. Interface	Chat or natural-language question intake	Captures intent; nothing accuracy-critical yet	n/a
2. Model + framework (substrate)	LLM writes SQL; framework orchestrates steps	Interchangeable and commoditizing; not the differentiator	model-agnostic
3. Governed context layer (the spine)	Supplies glossary definitions, certified joins, lineage, quality signals, and access context at inference time	The named accuracy driver; the 16% to 54% lift comes from context, not a bigger model	Context Lakehouse, Enterprise Data Graph, Context Agents
4. SQL generation + validation	Generates SQL constrained to certified joins; dry-runs and validates; read-only by default	Stops hallucinated joins and unsafe writes	certified joins from the Enterprise Data Graph
5. Execution + result verification	Runs the query; checks results against quality signals; surfaces uncertainty	Prevents confident wrong answers	data quality and trust signals
6. Transport (MCP)	Delivers governed context to any agent over MCP; filters by access policy	MCP is the wire, not the meaning	Atlan MCP Server

The design rationale is one equation: performance is a function of intelligence and context, and as model intelligence commoditizes, context becomes the compounding investment. Tier 2 is rented capability you can swap in a quarter; tier 3 is the asset that gets more valuable the more your business meaning accumulates in it.

---

What context does a talk-to-data agent need to be accurate?

Permalink to “What context does a talk-to-data agent need to be accurate?” #

A talk-to-data agent needs five governed inputs to be accurate, and each closes a specific failure mode. Omit any one and a predictable class of wrong answers returns. This is the substance of tier 3, the context layer for AI.

• Business glossary definitions so the agent knows what a metric means before it writes SQL. A glossary is how “active customer” resolves to the same logic every time, an input a semantic layer alone does not fully cover.
• Certified join paths from the Enterprise Data Graph so the agent stops inventing relationships between tables. This is the direct fix for hallucinated joins.
• Lineage and provenance so every answer carries a verifiable where-from a reviewer can trace to its source.
• Data quality and trust signals so the agent flags uncertainty on a low-quality source instead of returning garbage with full confidence.
• Policy and access context so row-level security is applied at query time and the agent returns only what the asking user is allowed to see.

One honest objection is circular: if I already had a clean, organized data model, I would not need an AI assistant. But a governed context layer does not require rebuilding every pipeline; it reuses the definitions, lineage, and join logic you already have, organizes them once, and keeps them current, so the agent never reasons over a definition that quietly changed.

Text-to-SQL solves query syntax; a governed context layer solves query meaning, which is why teams investing in context engineering reach reliable accuracy faster than teams that swap models.

---

How to build a governed talk-to-data agent: the six-step sequence

Permalink to “How to build a governed talk-to-data agent: the six-step sequence” #

Building a governed talk-to-data agent is a six-step sequence that moves from question scope to drift monitoring, with governed context established before any SQL is generated.

Step 1: Define the question scope and trusted data domain

Permalink to “Step 1: Define the question scope and trusted data domain” #

Start by bounding the problem: declare which schemas and certified assets the agent may reason over, so it never wanders into ungoverned or stale tables.

agent_scope:
  domain: revenue_analytics
  certified_assets_only: true
  schemas:
    - finance.certified
    - sales.certified
  default_access: read_only

Validation: the domain resolves to a finite set of certified assets, with all uncertified tables excluded.

Step 2: Establish governed context (definitions, certified joins, ownership, quality, access policy)

Permalink to “Step 2: Establish governed context (definitions, certified joins, ownership, quality, access policy)” #

Now build the spine: glossary definitions, certified join paths, ownership, quality signals, and access policy. Context Agents can accelerate this by drafting candidates from existing SQL history, lineage, and glossary entries, with a human certifying joins and definitions before they go live. An empty or stale tier 3 guarantees the 81.2% schema-level failure mode.

{
  "metric": "active_customer",
  "definition": "A customer with at least one paid transaction in the trailing 90 days",
  "certified_join": {
    "from": "sales.certified.orders",
    "to": "finance.certified.customers",
    "on": "orders.customer_id = customers.id",
    "status": "certified"
  },
  "owner": "revenue-data-team",
  "quality_signal": "freshness_verified"
}

Validation: the certified join is queryable in the Enterprise Data Graph and the definition resolves to a single canonical logic.

Step 3: Wire the context layer to the agent via MCP

Permalink to “Step 3: Wire the context layer to the agent via MCP” #

Connect the agent to the context layer over MCP so governed context is retrieved at inference, filtered by the asking user’s access policy. You are configuring transport, not meaning.

{
  "mcpServers": {
    "atlan": {
      "command": "atlan-mcp-server",
      "context": ["glossary", "lineage", "certified_joins", "quality", "access_policy"],
      "access_filter": "per_user"
    }
  }
}

Validation: the agent retrieves the glossary definition and lineage for an asset at query time, scoped to the user’s permissions.

Pro tip: Pointing the agent at the MCP server before you populate tier 3 returns an empty context payload, and the agent falls straight back to guessing.

Step 4: Generate SQL constrained to certified joins, validate before execution

Permalink to “Step 4: Generate SQL constrained to certified joins, validate before execution” #

Generate the SQL, but constrain it to the certified joins from tier 3 and validate it before it touches the warehouse. Enforce read-only and reject any uncertified relationship: this is the fix for hallucinated joins.

-- read-only, constrained to certified joins
SELECT c.region, COUNT(DISTINCT o.customer_id) AS active_customers
FROM sales.certified.orders o
JOIN finance.certified.customers c
  ON o.customer_id = c.id           -- certified join from the graph
WHERE o.paid_at >= CURRENT_DATE - INTERVAL '90 days'
GROUP BY c.region;
-- dry-run validate: reject any write or uncertified join

Validation: the query uses only certified joins, and any write attempt or uncertified join is rejected before execution.

Step 5: Execute, verify against quality signals, return with provenance and uncertainty

Permalink to “Step 5: Execute, verify against quality signals, return with provenance and uncertainty” #

Run the validated query, then check the result against the quality signals from tier 3. Attach provenance and an uncertainty flag so the user can trust or question the answer: a low-quality source should raise a flag, not a confident number.

result = warehouse.execute(validated_query)        # read-only
trust = context.quality_check(result.source_assets)
return {
    "rows": result.rows,
    "provenance": result.lineage,
    "uncertainty": None if trust == "high" else "low_quality_source"
}

Validation: a low-quality source raises an uncertainty signal rather than returning a confident answer.

Step 6: Monitor for context drift, keep definitions current

Permalink to “Step 6: Monitor for context drift, keep definitions current” #

Finally, watch for context drift. Configure alerts so the agent is told when a definition or join changes, instead of reasoning over a stale one. Drift is the worst failure mode: no error, no alert, just a confident wrong answer.

drift_monitoring:
  watch:
    - glossary_definitions
    - certified_joins
  on_change:
    alert: revenue-data-team
    flag_dependent_agents: true

Validation: changing a certified definition triggers the alert and flags the dependent agent so it never serves an answer built on the old logic.

---

Cortex Analyst vs Databricks Genie: which execution substrate?

Permalink to “Cortex Analyst vs Databricks Genie: which execution substrate?” #

Snowflake Cortex Analyst and Databricks Genie are both strong single-platform execution substrates, and the choice between them matters less than the tier above either. Each generates SQL against its own semantic model and stops at its own platform boundary; the portable, governed context layer is the part neither one provides.

Execution substrate comparison:

	Snowflake Cortex Analyst	Databricks Genie
NL substrate	SQL over a Snowflake semantic model	NL and SQL over the Lakehouse
Semantic source	Snowflake Semantic Views	Genie plus ontology vocabulary
Portability	Single-platform	Single-platform
What is still missing	Cross-platform, model-agnostic governed context	The same: context tied to one platform

Each is the sensible default for a team that lives entirely in one ecosystem, and even Databricks Genie now adopts ontology and context vocabulary, which tells you the governed-context claim is live and contested rather than fringe.

The deciding question is portability. If your data lives in one platform, the native substrate may be enough; if it spans clouds and engines, you need a context layer that travels, stored in an open, Iceberg-native Context Lakehouse above either substrate. For the underlying concepts, see conversational analytics and the context layer vs semantic layer breakdown.

---

How Atlan delivers the governed context layer

Permalink to “How Atlan delivers the governed context layer” #

Atlan automates the governed-context tier this blueprint requires and delivers it to any agent over MCP. Tiers 1, 2, 4, and 5 are implementable on any stack you already run; the hard part is tier 3. Assembling glossary definitions, certified joins, lineage, quality, and access policy by hand, then keeping it current as the business changes, is where most talk-to-data projects stall.

Atlan changes the economics of that work. Context Agents generate and maintain context continuously across the data estate; Context Engineering Studio is where teams bootstrap and ship the business understanding the agent uses; the Context Lakehouse stores it in an open, Iceberg-native substrate; the Enterprise Data Graph resolves canonical assets and certified joins; and the Atlan MCP Server delivers all of it to the agent at inference, filtered by access policy.

The customer evidence tracks the architecture: Workday and DigiKey both describe governed context delivered to AI through Atlan’s MCP server, tier 3 wired to tier 6. To go deeper, see how the Atlan MCP server builds context for AI tools and the context layer for AI agents.

---

Real stories from real customers: Governed context delivered over MCP

Permalink to “Real stories from real customers: Governed context delivered over MCP” #

---

Why governed context is the spine of the blueprint

Permalink to “Why governed context is the spine of the blueprint” #

The six-tier model holds up under the evidence: the model and framework are rented execution, and the governed context layer is the owned, compounding asset that decides accuracy. The bottleneck is meaning, not generation, which is why benchmark accuracy collapses from 91.2% to 21.3% on real enterprise schemas and recovers from 16% to 54% when knowledge-graph context is added.

That is the honest case and its honest limit. Governed context raises both the accuracy floor and the ceiling; it does not promise 100%, and any blueprint that claims it does is selling a demo. Build the six tiers, establish context before you generate SQL, and the agent will be as accurate as the context you feed it.

See how this fits the broader AI agent stack.

---

FAQs about building a talk-to-data agent

Permalink to “FAQs about building a talk-to-data agent” #

1. Why does text-to-SQL fail on enterprise schemas?
   The bottleneck is meaning, not syntax: Omni reports 81.2% of failures are schema-level errors where the model writes valid SQL but misses the business meaning.

2. How do you stop an AI data agent from hallucinating joins?
   Constrain SQL generation to certified join paths from a governed context layer and validate every query, so the agent draws relationships from the Enterprise Data Graph instead of guessing.

3. Do you need a semantic layer for accurate text-to-SQL?
   A semantic layer is one input; accurate text-to-SQL also needs certified joins, lineage, quality signals, and policy context, which a governed context layer combines and keeps current.

4. How accurate is text-to-SQL in the real world?
   An o1-preview agent solves only 21.3% of enterprise Spider 2.0 tasks versus 91.2% on Spider 1.0; governed context raises the floor and ceiling, but no approach promises 100%.

5. What context does a talk-to-data agent need to be accurate?
   Five governed inputs: glossary definitions, certified joins, lineage, quality signals, and policy and row-level access context, each closing a specific failure mode.

6. Snowflake Cortex Analyst vs Databricks Genie, which should I use?
   Both are strong single-platform substrates; a portable, model-agnostic governed context layer sits above either one and travels across both.

7. How do you build a governed, read-only talk-to-data agent?
   Enforce read-only access, constrain generation to certified joins, and return results with provenance and an uncertainty signal after validating every query.

8. Is MCP enough to make a data agent accurate?
   No. MCP delivers governed context at inference but does not create it, so accuracy comes from a populated governed context tier.

---

Sources

Permalink to “Sources” #

1. Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows, ICLR 2025 (arXiv:2411.07763). https://arxiv.org/abs/2411.07763
2. A Benchmark to Understand the Role of Knowledge Graphs on LLM Accuracy for Question Answering on Enterprise SQL Databases (arXiv:2311.07509). https://arxiv.org/abs/2311.07509
3. Omni, “Why text-to-SQL fails.” https://omni.co/blog/why-text-to-sql-fails
4. Orchestrating Agents and Data for Enterprise: A Blueprint Architecture for Compound AI Systems (arXiv:2504.08148). https://arxiv.org/html/2504.08148v1
5. dbt Semantic Layer / MetricFlow documentation. https://docs.getdbt.com/docs/build/about-metricflow
6. Vanna.ai, open-source RAG text-to-SQL agent (GitHub). https://github.com/vanna-ai/vanna
7. Snowflake Cortex Analyst documentation. https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-analyst
8. Databricks Genie documentation (vendor reference for the single-platform Lakehouse approach).