Your data science team spent a quarter cleaning a warehouse for AI agents. The nulls are gone, the pipeline runs clean, and the agent still gives wrong answers, because cleaning values was never the job that made agents work. According to DigitalApplied’s analysis of 120+ enterprise deployments, 88% of agent pilots never reach production, and the blockers are governance and evaluation gaps, not model quality. Atlan’s context layer implements the six-step sequence below as governed infrastructure rather than a one-time project, certified definitions, column-level lineage, and access policies the agent retrieves at inference time, plus the known-answer evaluation suite that validates every change before it ships; organizations following this sequence on a structured platform report 60-to-90-day deployment timelines against the 8-to-24-week baseline teams hit building it from scratch.
What prerequisites do you need before step one?
Permalink to “What prerequisites do you need before step one?”Three categories of readiness must be in place before you start. Skipping the readiness check is why 90-day plans turn into 24-week ones.
| Category | Requirement |
|---|---|
| Organizational | A pilot agent use case with a defined business question; a stakeholder with authority to approve definitions; an executive sponsor who has accepted the timeline in writing |
| Technical | A data warehouse or lakehouse with working schema-level lineage; a list of agent-relevant data sources scoped to what the pilot will query, not the entire estate |
| Team | One data engineer, one domain owner per source, one governance lead, one AI engineer; expect 0.5 FTE for 60 to 90 days with a structured platform |
The data infrastructure is already present in most enterprises. The gap is organizing and exposing it, the same memory layer gap that shows up when teams choose an agent memory architecture without governing what feeds it. Do not commit until the executive sponsor has signed off on the timeline in writing.
Why is preparing enterprise data for AI agents different from preparing it for BI?
Permalink to “Why is preparing enterprise data for AI agents different from preparing it for BI?”Preparing data for BI is about cleaning values. Preparing data for AI agents is about making meaning machine-readable. This is the inversion that separates teams that deploy working agents from teams that deploy failing ones.
When a BI team prepares data, the work is cleanup: nulls, formats, data types. Humans reading the dashboards add context: they know what ARR means, which tables matter, and how to handle ambiguity through conversation. AI agents have no context. An agent querying a perfectly clean warehouse still fails if that warehouse has no semantic layer.
Two cold-start problems exist. Frameworks easily solve the session cold start, preventing agents from forgetting recent prompts. The actual threat is the organizational cold start: deploying an agent that has never been introduced to your data estate. It lacks your company’s definition of customer churn, cannot identify the canonical revenue table, and doesn’t know who owns the customer-status field. Without this context, it defaults to whichever source ranks highest in similarity search. The preparation work is the layer above the data, and most organizations haven’t built it yet.
The data stack is shifting under AI
See the 7 shifts reshaping data infrastructure for an AI-first world, including why data prep for agents inverts BI prep.
Download the 2026 ReportStep 1: Inventory and prioritize agent-relevant data sources
Permalink to “Step 1: Inventory and prioritize agent-relevant data sources”Takes 5 to 7 days. A typical enterprise has hundreds of tables and dozens of document repositories; the pilot agent will touch fewer than 10% of them. Governing every source before deploying stalls projects for 18 months.
Walk the agent’s planned query patterns with the AI engineer, list the tables, dashboards, and documents it will retrieve, and rank by query frequency. Start with the agent’s questions, not the warehouse.
Example: A clinical data agent’s team defines six sources: Epic’s patient_encounter and diagnosis tables, the claims warehouse’s claim_line table, two clinical data warehouse views, lab results, and the clinical glossary. Six sources, not sixty.
Common mistake: Treating inventory as data discovery. It’s a query-pattern exercise; teams that skip this fail by trying to govern their entire estate before deploying anything.
Step 2: Assign ownership for every source
Permalink to “Step 2: Assign ownership for every source”Takes 7 to 10 days. The work is organizational coordination: identifying the right people before building the pipeline. An agent answering with conflicting definitions is an ownership problem dressed up as a data problem.
For each source from Step 1, identify who can sign off on a definition and enforce freshness, and get them to accept ownership in writing.
Common mistake: Letting data engineering own everything by default. Engineers build pipelines; they don’t own business definitions. Unclear ownership is a leading cause of context-layer failures in production AI systems.
Step 3: Build and certify canonical definitions
Permalink to “Step 3: Build and certify canonical definitions”Takes 14 to 21 days. This is where the missing translation layer lives. Every term the agent reads needs a definition that travels with the data; without it, the model guesses inconsistently across runs. AI agent hallucination in enterprise settings is almost always a definition problem, not a model problem.
Start with the highest-frequency terms from Step 1. For each, write a definition with the canonical formula, source-of-truth field, exceptions, and the named owner from Step 2. The definition is infrastructure, not documentation.
Common mistake: Treating glossary work as documentation rather than infrastructure. A glossary in Confluence that humans read is not a context layer an agent can consume.
Step 4: Map lineage from raw source to agent-facing view
Permalink to “Step 4: Map lineage from raw source to agent-facing view”Takes 10 to 14 days. When an agent gives a wrong answer, you need to debug the data pipeline, not the model. Lineage is the diagnostic system; without it, “the agent got it wrong” is a black box.
For each source, document the transformation chain: source table to ETL job to staging table to certified view to agent endpoint, capturing column-level lineage where the agent uses calculated metrics.
Common mistake: Settling for source-to-target lineage at table grain when the agent reasons at the column or metric grain. The context architecture requires column-level fidelity.
Step 5: Apply access policies for users and agents
Permalink to “Step 5: Apply access policies for users and agents”Takes 7 to 14 days. Agents inherit the access of whoever invoked them unless constrained explicitly. If the boundary isn’t enforced at the data layer, an unauthorized user can ask the agent to retrieve data they should never see. AI agent memory governance makes those boundaries explicit before the first production query.
For each source, define which users access at full grain, which at aggregated grain, and which have none, then apply equivalent policies to the agent itself via a service principal with explicit scope.
Common mistake: Granting the agent broad read access temporarily and intending to restrict it later. Broad access granted temporarily rarely gets restricted before a breach surfaces the gap.
Step 6: Build a known-answer evaluation suite
Permalink to “Step 6: Build a known-answer evaluation suite”Takes 14 to 21 days. Evaluation gaps are a primary blocker for stalled pilots. Teams that skip business-specific evaluation fall into what context engineering research calls the proof-of-concept testing hell pattern: perpetual pilot, no production.
Pull existing trusted dashboards, convert each chart into a question-answer pair, and cover every term in the Step 3 glossary. Run the suite on a schedule and on every change.
Common mistake: Using generic model benchmarks as a substitute for business-question evaluation. They measure the model, not your agent.
Where's your agent readiness gap?
Run the Context Gap Calculator to see which of the six steps your data estate is missing.
Calculate Your GapWhat are the most common implementation pitfalls?
Permalink to “What are the most common implementation pitfalls?”When the six-step sequence breaks down in practice, the failure modes cluster in three places, and none is fixed by switching frameworks or models.
Why does cleaning data first not produce a usable agent?
Permalink to “Why does cleaning data first not produce a usable agent?”Clean data without governed meaning still fails. A perfectly null-free revenue column doesn’t tell the agent what counts as revenue, which fiscal calendar applies, or whether deferred revenue is included. The cleaning is necessary; the meaning, delivered through a context-aware layer above the warehouse, determines whether the agent answers correctly.
Why do agents querying the same warehouse give different answers to the same question?
Permalink to “Why do agents querying the same warehouse give different answers to the same question?”Because there’s no single canonical definition. One team built a revenue view that excludes refunds; another built one that includes them. The agent retrieves whichever ranked highest in similarity search rather than the certified one. Without a governed semantic layer, agents operating on the same warehouse produce different answers to the same question across different runs, the same data discovery problem an undocumented warehouse creates for human analysts.
Why does starting with the whole data estate stall the project?
Permalink to “Why does starting with the whole data estate stall the project?”The math defeats you: a typical enterprise has thousands of tables and millions of files, and trying to govern everything before deploying stalls projects for 18 months. The fix is Step 1’s discipline: govern only what the deployed agent will touch. Organizational cold start is solved through scoping, not comprehensive coverage.
What are the best practices for preparing enterprise data for AI agents?
Permalink to “What are the best practices for preparing enterprise data for AI agents?”Three practices distinguish teams that ship from teams that stall.
Bootstrap context from existing metadata signals. SQL query history, BI dashboard usage, lineage graphs, and informal glossary entries already encode much of what you need. Use them as the first-draft input, not a blank page; the principle holds regardless of platform.
Version definitions and tests as code artifacts. A definition change should ship through the same review process as a code change. An evaluation suite belongs in version control, so every iteration of the context layer is auditable and reversible, the same discipline that keeps episodic memory traceable in agent systems. Without version control, a definition rollback after a production failure requires a meeting rather than a git revert.
Plan for multiple agents from the start. Building for one agent in isolation is the most expensive way to build for ten. Each new agent reuses certified definitions, lineage maps, and access policies from the first, avoiding the multi-agent memory silos that fragment external memory across teams that never built a shared enterprise context layer to begin with. Context infrastructure compounds in value with each deployment.
How does Atlan streamline each step of enterprise data preparation for AI agents?
Permalink to “How does Atlan streamline each step of enterprise data preparation for AI agents?”The challenge
Permalink to “The challenge”Executing the six-step sequence from scratch takes 8 to 24 weeks; most teams underestimate Step 3 (definitions) and Step 6 (evaluation) by a factor of four. Definitions sit scattered across Confluence and tribal knowledge; evaluation suites don’t exist because dashboards live in BI tools, not a portable test format.
The approach
Permalink to “The approach”Context Engineering Studio bootstraps a first-draft context model from existing signals, SQL query history, BI dashboard usage, lineage graphs, and glossary entries, in days rather than weeks. Context Repos package definitions, access policies, and evaluation examples as versioned, portable bundles, and simulation converts trusted dashboards into known-answer test suites automatically.
The Atlan MCP server delivers prepared context to any agent via one standard endpoint, regardless of framework. Context agents are purpose-built with this governed context already in place, bypassing the organizational cold start.
The outcome
Permalink to “The outcome”Timeline: 60 to 90 days to initial production deployment with Atlan’s context layer versus 8 to 24 weeks from scratch. Compounded across the second, third, and tenth agent, the gap becomes a roadmap problem no model upgrade closes.
Real stories from real customers: closing the data preparation gap for production agents
Permalink to “Real stories from real customers: closing the data preparation gap for production agents”"We built a revenue analysis agent and it couldn't answer one question. We started to realize we were missing this translation layer. All of the work that we did to get to a shared language amongst people at Workday can be leveraged by AI via Atlan's MCP server."
Joe DosSantos, VP Enterprise Data & Analytics, Workday
"Engineering and governance teams now work side by side to ensure meaning, quality, and lineage travel with every dataset, from the factory floor to the AI models shaping the future of mobility."
Sherri Adame, Enterprise Data Governance Leader, General Motors
Workday’s revenue analysis agent failed to answer the first question put to it: the team realized the missing piece was the translation layer between data and agent, precisely the organizational cold start Step 3 addresses. GM extends the same governance discipline to every dataset feeding the AI models shaping its mobility roadmap, the approach Steps 2 and 4 enforce at the individual agent level.
What's the ROI of getting this right the first time?
Run the Context Layer ROI Calculator to size the cost of a stalled pilot versus a governed six-step rollout.
Calculate the ROIWhy the context layer is the make-or-break step for AI agents
Permalink to “Why the context layer is the make-or-break step for AI agents”Data preparation is not the unglamorous prelude to the AI work; it is the AI work. The model is largely a commodity, and the orchestration frameworks are largely interchangeable. What isn’t interchangeable is whether your agent has a governed, owned, traceable, evaluated context layer underneath it.
60 to 90 days versus 8 to 24 weeks is the difference between an agent that ships this quarter and a pilot that stalls into the next. The right question isn’t which model or framework to use; it’s whether you’ve solved the organizational cold start.
FAQs about preparing enterprise data for AI agents
Permalink to “FAQs about preparing enterprise data for AI agents”1. What is the difference between data preparation for BI and data preparation for AI agents?
Permalink to “1. What is the difference between data preparation for BI and data preparation for AI agents?”BI preparation cleans values, fixes nulls, and standardizes formats so dashboards display correctly. Agent preparation is inverted: the work is making meaning machine-readable. Definitions, ownership, lineage, and access policies determine whether the agent answers correctly.
2. What is the organizational cold start problem?
Permalink to “2. What is the organizational cold start problem?”Organizational cold start is an AI agent having no introduction to your data estate when first deployed. It doesn’t know what a term like ARR means, which tables are canonical, or who owns which metric. Every framework solves session cold start; organizational cold start is yours to solve.
3. How long does it take to prepare enterprise data for an AI agent?
Permalink to “3. How long does it take to prepare enterprise data for an AI agent?”8 to 24 weeks from scratch without a structured context platform; 60 to 90 days with one. Underestimating the definitions step and the evaluation step is the most common scheduling error, typically by a factor of four.
4. Do I need to clean every data source before deploying an AI agent?
Permalink to “4. Do I need to clean every data source before deploying an AI agent?”No. Start with the agent’s planned query patterns and govern only the sources it will touch, typically fewer than 10% of the estate. Trying to govern everything before deploying stalls preparation projects without producing a working agent.
5. Can I bootstrap context from what I already have?
Permalink to “5. Can I bootstrap context from what I already have?”Yes, and you almost always should. SQL query history reveals which tables analysts actually use, BI dashboard usage shows which metrics matter, and existing glossaries capture organizational definitions. A structured context platform turns these signals into a first-draft context model in days.
6. What should I evaluate first when validating my prepared data?
Permalink to “6. What should I evaluate first when validating my prepared data?”Start with the agent’s highest-stakes question, pull the corresponding number from a trusted dashboard, and compare. A mismatch means a definition, lineage, or access policy is wrong.
7. What is the difference between ownership and governance?
Permalink to “7. What is the difference between ownership and governance?”Ownership is naming who is accountable for a definition or data source. Governance is enforcing that ownership. Ownership without governance is a name on a wiki page; governance without ownership is rules with no one enforcing them.
