--- title: "Agent Harness Engineering: Pipeline Trust vs Governance Trust" url: "https://atlan.com/know/harness-engineering/" description: "Agent harness engineering requires both pipeline trust and governance trust. Compare dbt+Fivetran vs Atlan for enterprise AI agent deployments in 2026." author: "Emily Winks" author_role: "Data Governance Expert" published: "2026-06-02" updated: "2026-06-02T00:00:00.000Z" --- --- Agent harness engineering is the practice of building the execution layer that wraps an AI model, governing its tools, context, memory, and behavior. When dbt Labs and Fivetran completed their merger in June 2026, they claimed to provide "data infrastructure for trusted AI agents." But trust in an agent harness has two distinct layers: pipeline trust, which ensures data is moved and transformed consistently, and governance trust, which ensures data is certified, policy-controlled, and auditable. According to Gartner, 52% of organizations cite data quality as the biggest blocker to agent deployment, not model capability. Data quality failures are, at root, governance failures: data enters agent context without certification, ownership checks, or policy clearance. This guide breaks down the difference, when you need each, and why production-grade enterprise AI agents require both. | Dimension | Pipeline Trust (dbt+Fivetran) | Governance Trust (Atlan) | |-----------|-------------------------------|--------------------------| | What it is | Consistent ingestion and transformation | Certified, policy-controlled, auditable metadata | | What it does | Moves and tests data reliably | Certifies data assets and enforces governance policies | | Who owns it | Analytics and data engineers | CDO, data stewards, governance teams | | Key mechanism | dbt contracts, lineage, Agents Schema | Certification workflows, policy engine, MCP server | | Trust signal | "Data is consistently transformed" | "Data is certified for this agent, policy, and user" | | Questions it answers | Is this data available and tested? | Is this data approved, owned, and safe for agents? | | Limitation | Scoped to dbt graph; no cross-stack certification | Requires catalog investment and governance workflows | | Best for | Pipeline reliability and analytics freshness | Enterprise AI agents requiring regulatory accountability | Is your data estate AI-agent ready? Assess Your Readiness --- ## What is pipeline trust in agent harness engineering? Pipeline trust is the claim that consistently moving and transforming data—using tested, contracted models and governed semantics—makes that data trustworthy for AI agents. dbt Labs and Fivetran built their [merger narrative](https://www.businesswire.com/news/home/20260601/fivetran-dbt-labs-complete-merger) around this: Fivetran ensures agents operate on complete, continuously synced data; dbt ensures data is defined, tested, and governed through business logic. The most concrete artifact is the [Agents Schema](https://www.fivetran.com/fivetran-dbt-labs-merger), an open-source standard designating a single SQL schema named `AGENTS` in the warehouse or lake as the shared context layer for AI agents. It stores metric definitions, semantic models, dbt lineage, and business documentation as plain SQL tables, queryable by any SQL-capable agent within existing security infrastructure. As [Addy Osmani](https://addyosmani.com/blog/agent-harness-engineering/) described agent harness engineering: "A harness is every piece of code, configuration, and execution logic that isn't the model itself." For analytics-centric agents operating within the dbt ecosystem, pipeline trust provides a strong foundation. Pipeline trust answers one question: "Is this data available and correctly transformed?" It does not answer: "Is this data certified for this agent, under this policy, for this use case?" That is governance trust, and it becomes the gap when agents need to operate across the full enterprise data estate or when regulatory accountability is required. ### Core components of pipeline trust - **Data ingestion and sync** (Fivetran): continuous, complete, reliable movement of source data to warehouse or lake, guaranteeing agents have fresh raw material - **Transformation contracts** (dbt): version-controlled SQL transformations with documented expectations, tests, and lineage tracking - **Semantic definitions** (dbt metrics): standardized metric definitions that agents can query to ground their reasoning in consistent business logic - **Agents Schema** (open-source): the `AGENTS` SQL schema containing lineage, metrics, and documentation as plain SQL tables, a dbt-graph-scoped context store --- ## What is governance trust in agent harness engineering? Governance trust is the property that a data asset is certified, policy-controlled, and auditable: an AI agent can act on it with accountability. Enterprise AI governance reviews stall agent deployments over data trust, not model performance—and it is the layer that pipeline tools structurally cannot provide. Enterprise governance teams reviewing agent deployments do not ask "Is the data transformed?" They ask: "Is this dataset certified for this use case under GDPR? Who owns this data product? Is there an active PII flag on these records?" These are governance questions—and the Gartner finding that 52% of organizations cite data quality as their biggest agent blocker reflects them: data quality failures in agent context are governance failures, with ungoverned data entering reasoning loops without certification or policy clearance. Governance trust separates an agent that runs from one that is accountable. Only 21% of organizations have a mature governance model for autonomous AI agents (Gartner 2026), meaning most enterprises deploy on pipeline trust alone. Atlan's [Context Engineering Studio](https://atlan.com/know/context-layer-harness-engineering/) compresses the governance loop into a repeatable workflow: bootstrap context from the enterprise data graph, simulate against test scenarios, deploy to the harness, and observe drift over time. Organizations using this workflow report moving from 10% to 90% AI accuracy on governed data tasks. Governance trust is one of the five standard harness components. It does not replace runtime reliability, tool-use safety, or observability; those are parallel harness engineering concerns. But it is the component that addresses the data layer specifically, and it is the component most often missing from current harness engineering practice. ### Core components of governance trust - **Certification workflows**: "certified," "draft," "deprecated" status agents query at runtime before acting on a dataset; the first gate in governed agent decisions - **Policy enforcement**: PII flags, domain policies, and sensitivity labels at the asset level, determining what agents see and what actions they can take - **Ownership chain**: data product owners, stewards, and domain owners: humans accountable for agent actions against their data - **Semantic definitions**: business glossary, canonical metric definitions, and certified values; the single source of truth for agent reasoning - **Human-on-the-loop governance**: approval gates for high-stakes agent decisions where autonomous agent action requires human sign-off --- ## Pipeline trust vs governance trust: head-to-head comparison Pipeline trust and governance trust diverge most sharply on three dimensions: what they certify, who is accountable when agents make mistakes, and what happens when agents operate across the full enterprise data estate beyond the dbt graph. The sharpest divide is scope. dbt+Fivetran's Agents Schema is explicitly a dbt-graph-scoped context store. It covers the datasets, metrics, and lineage that dbt has modeled. An enterprise data estate includes 100+ additional data sources: Salesforce CRM data, Power BI reports, Databricks notebooks, Snowflake raw tables, mainframe extracts, SaaS application data. For agents that need to reason across this full estate, the Agents Schema is a starting point, not a complete context layer. Tristan Handy, co-founder of dbt Labs, acknowledged as much: the Agents Schema is "a first step" and that a more graph-like, real-time semantic layer will be needed for the full picture. | Dimension | Pipeline Trust (dbt+Fivetran) | Governance Trust (Atlan) | |-----------|-------------------------------|--------------------------| | Primary focus | Data is reliably ingested and correctly transformed | Data is certified, owned, and policy-compliant | | Trust mechanism | dbt contracts, lineage, Agents Schema SQL tables | Certification workflows, policy engine, governed catalog | | Scope | dbt graph (dbt-modeled datasets and metrics) | Full enterprise estate (100+ connectors) | | Certification authority | Code passing tests and contracts | Human governance review plus automated policy rules | | What agents query | AGENTS SQL schema | Governance-gated MCP server (certification, access policy, semantic definitions) | | Handles PII policy | Indirectly via dbt masking rules | Directly via PII sensitivity flags at asset level | | Cross-stack coverage | No; scoped to dbt-modeled assets | Yes: Salesforce, Databricks, Power BI, SaaS, mainframes | | Regulatory accountability | Partial: lineage documented, no approval chain | Full: ownership, certification history, access audit logs | | Failure mode | Agent acts on data without certification checks | Governance overhead can slow initial deployment velocity | | Maturity indicator | dbt contracts in place; Agents Schema populated | Certified assets active; governance workflows running; MCP deployed | **Example: A financial services compliance agent** A financial services firm runs an AI agent that generates regulatory capital reports. Fivetran plus dbt ensures the agent has fresh, correctly transformed balance sheet data. That is pipeline trust, and it is valuable. But when the audit committee asks: "Was the Q3 capital figure your agent used certified by the Finance domain team? Was the customer PII in that dataset approved for regulatory reporting? Who signed off on the semantic definition of 'tier-1 capital' your agent used?" These questions require governance trust. The governed catalog answers them. The pipeline does not. --- ## How do pipeline and governance layers work together? Pipeline trust and governance trust are not competing approaches but different layers of the same harness engineering stack. The pipeline delivers reliable data; governance certifies it for autonomous agent use. The clearest way to see this is the [four-layer architecture](https://atlan.com/know/context-layer-harness-engineering/) that emerges from production AI agent deployments: ### The four-layer architecture **Model** (LLM reasoning engine) / **Harness** (tools, memory, orchestration, lifecycle) / **Context Layer** (Atlan: certified, governed metadata) / **Data Pipeline** (Fivetran+dbt: ingestion, transformation, Agents Schema) The pipeline feeds Atlan. Atlan certifies the data the pipeline delivers. The harness queries Atlan's MCP server for certified context at runtime. The model reasons on governed inputs. Each layer is distinct and necessary. ### When the agent queries a dataset When an agent needs to act on a dataset, the harness calls Atlan's governance-gated MCP server: "Is `revenue_q3` certified? Who owns it? Is it approved for regulatory reporting?" Atlan responds: "Certified by Finance team on 2026-05-14. Approved for regulatory use. No active PII flags." The agent proceeds. If the data is policy-restricted, the harness routes to a human approval gate. ### When to lead with pipeline trust vs governance trust Lead with pipeline trust when: data reliability is the primary failure mode; agent accuracy is suffering from stale or incorrectly transformed data; the use case is contained within dbt-modeled datasets; regulatory accountability is not required. Lead with governance trust when: enterprise accountability is required; agents operate on PII or regulated data; multi-source data spans beyond the dbt graph; regulatory audit is a requirement for the deployed agent. Invest in both simultaneously when: you are building a greenfield AI agent platform; integrating data from an M&A event; or deploying any enterprise agent that will face compliance review. The [how-to-build-ai-agent-harness](https://atlan.com/know/how-to-build-ai-agent-harness/) guide names this as Step 0: certify the data layer before writing any harness code. --- ## How Atlan approaches governance trust for AI agent harnesses Enterprise data teams building production AI agents consistently hit the same wall: who verifies that the data the harness is acting on is approved for that agent, under that policy? According to the [LangChain State of AI Agents report](https://www.langchain.com/stateofaiagents), quality is the top barrier to production for enterprises, with security and governance immediately behind it. The question "did the agent use the certified version of this metric?" is answered by the governance layer, not by dbt contracts. Atlan is the context trust layer the harness depends on. Using a [governance-gated MCP server](https://atlan.com/know/what-is-an-agent-harness/), any MCP-compatible harness can query Atlan at runtime for certification status, active lineage, access-controlled assets, and semantic definitions. Atlan covers 100+ connectors: BI tools, Salesforce, Databricks, SaaS platforms, mainframes, delivering governed context for the full enterprise estate, not just the dbt-modeled subset. 75+ enterprises use Atlan's MCP server today, generating 14 billion API calls per month. The production gap is not closed by better prompting or more capable models alone; it is closed by the governance layer. Gartner 2026 recognizes that Atlan "stands out in AI-native governance through agentic stewardship and orchestration of enterprise agentic systems." Book a demo --- See how Atlan builds governance trust for AI agent harnesses Watch Context Layer Live --- ## Real stories from real customers: AI agent governance in production Workday: governing AI agent context across a shared enterprise language "All the work we did to get to a shared language among people at Workday can be leveraged by AI via Atlan's MCP server." VP, Data Products Workday Watch Now → Workday's data team spent years building a shared semantic language across business units. With Atlan's MCP server, that institutional knowledge becomes accessible to AI agents at runtime as live, certified, policy-controlled context. The pipeline delivers fresh data; Atlan certifies which data is approved for which purpose. DigiKey: a context operating system for AI agents across 10M+ components "I'd rather build it inside of you guys because you see all of it... it's not just Snowflake, it's BigQuery, Qlik, MicroStrategy." Data Platform Lead DigiKey Watch Now → DigiKey's data estate spans Snowflake, BigQuery, Qlik, MicroStrategy, and additional sources. Their AI agents need policy-controlled access to product, pricing, and inventory data across a 10M+ component catalog. Atlan covers the full multi-cloud, multi-tool estate with a consistent certified context layer—not a feature added on top, but the operating system the agents run on. --- ## Why governance is the missing layer in agent harness engineering The dbt+Fivetran merger completed on June 1, 2026, with a clear claim: "data infrastructure for trusted AI agents." That claim is half right. Pipeline trust is real and necessary, and for analytics-centric agents within the dbt ecosystem, it may be sufficient. But the enterprise AI governance reviews that stall agent deployments are not asking "was this data transformed?" They are asking: "Is this data certified? Who owns it? Is it approved for this agent?" Those are governance questions. According to reports citing Gartner, more than 40% of agentic AI projects are at risk of cancellation by 2027 due to cost, unclear value, and risk—and the risk issues are governance failures, not pipeline failures. Runtime reliability, tool-use safety, and observability are also essential harness components; this comparison does not cover them. But data governance is the component most consistently absent from current harness engineering practice and the one that pipeline tools structurally cannot provide. The [governance lane in agent harness engineering](https://atlan.com/know/context-layer-harness-engineering/) is the lane nobody has built yet. [Atlan's Step 0 framework](https://atlan.com/know/how-to-build-ai-agent-harness/)—certify the data layer before writing any harness code—is the only published guide that starts there. The context layer is the trust layer the harness depends on, delivering governed context (certification status, active lineage, access policies, semantic definitions) to any MCP-compatible harness at runtime. Not the pipeline. The governance layer. --- ## FAQs about agent harness engineering ### 1. What is the difference between pipeline trust and governance trust? Pipeline trust means data is reliably ingested and correctly transformed: the claim dbt+Fivetran's merger makes. Governance trust means data is certified, policy-controlled, owned, and auditable, the property enterprise AI agents require for accountable autonomous action. Pipeline trust answers whether data is available and tested. Governance trust answers whether data is approved for this agent, under this policy, for this use case. Both are required for enterprise production agents. ### 2. What is agent harness engineering and why does it matter? Agent harness engineering is the discipline of building the execution layer that wraps an AI model, governing its tools, context, memory, safety, and lifecycle. As Martin Fowler defined it: "the term harness has emerged as a shorthand to mean everything in an AI agent except the model itself." Harness engineering matters because models alone are not agents; the harness determines whether agents are reliable, observable, and governable at production scale. ### 3. Does dbt+Fivetran's Agents Schema replace a data governance catalog? No. The Agents Schema is a dbt-graph-scoped context store; it covers datasets, metrics, and lineage that dbt has modeled. A full enterprise data estate includes Salesforce, Power BI, Databricks, SaaS platforms, and mainframe data outside the dbt graph. For agents requiring certified, policy-controlled context across the full estate, a governed catalog is required. dbt Labs co-founder acknowledged Agents Schema is "a first step" toward a more graph-like, real-time semantic layer. ### 4. How does Atlan's MCP server deliver governance trust to AI agent harnesses? Atlan's governance-gated MCP server allows any MCP-compatible harness to query the current governance state of any data asset at runtime. The harness asks whether a dataset is certified, who owns it, and whether it is approved for the current use case. Atlan responds with certification status, active lineage, access policy, and semantic definitions. If data is policy-restricted, the harness routes to a human approval gate. 75+ enterprises use this server, generating 14 billion API calls per month. ### 5. What is the most common reason AI agent projects fail to reach production? According to reports citing Gartner 2026, 52% of organizations cite data quality as the biggest blocker to agent deployment. Quality is the top barrier for enterprises, with security and governance immediately behind it. Most data quality failures in agent context are governance failures: ungoverned, uncertified data enters the agent reasoning loop without policy checks, ownership verification, or certification status. ### 6. What does "Step 0" mean in Atlan's agent harness engineering framework? Atlan's agent harness engineering guide names the prerequisite that precedes all harness code: certify your data layer before writing any harness logic. Step 0 is the governance check, ensuring the data assets the harness will operate on are certified, ownership-assigned, and policy-cleared. Without Step 0, the harness is built on an ungoverned foundation. ### 7. Can you have pipeline trust without governance trust? Yes, and most organizations do. Pipeline trust is achievable without a governance catalog. For analytics-centric agents on dbt-modeled data, it may be sufficient. But for enterprise AI agents requiring regulatory accountability, PII policy enforcement, or cross-stack certification, pipeline trust alone is insufficient. The boundary where governance becomes required is when agents must operate on data outside the dbt graph or when regulatory audit is expected. ### 8. What is the four-layer architecture for trusted AI agents? The four-layer architecture for trusted AI agents is: Model (LLM reasoning engine) / Harness (tools, memory, orchestration, lifecycle) / Context Layer (Atlan: certified, governed metadata delivered at runtime) / Data Pipeline (Fivetran+dbt: ingestion, transformation, Agents Schema). The pipeline delivers data. The context layer certifies it. The harness executes using certified context. The model reasons on trusted inputs. --- ## Sources 1. Fivetran, dbt Labs Complete Merger to Create Data Infrastructure for Trusted AI Agents. BusinessWire, June 2026. https://www.businesswire.com/news/home/20260601/fivetran-dbt-labs-complete-merger 2. Agent Harness Engineering. Addy Osmani, April 2026. https://addyosmani.com/blog/agent-harness-engineering/ 3. Harness engineering for coding agent users. Martin Fowler. https://martinfowler.com/articles/harness-engineering.html 4. The Anatomy of an Agent Harness. LangChain. https://www.langchain.com/blog/the-anatomy-of-an-agent-harness 5. Fivetran + dbt Labs Agents Schema specification. Fivetran. https://www.fivetran.com/fivetran-dbt-labs-merger 6. LangChain State of AI Agents report. LangChain. https://www.langchain.com/stateofaiagents 7. Gartner Predicts Over 40% of Agentic AI Projects at Risk of Cancellation. RCR Wireless, 2026. https://rcrwireless.com/20261001/analyst-angle/gartner-40-percent-agentic-ai-fail-2027 8. Code as Agent Harness. arXiv 2605.18747. https://arxiv.org/abs/2605.18747 9. Data Quality and AI Agent Harnesses: 3 Production Failures. Atlan. https://atlan.com/know/data-quality-ai-agent-harnesses/ 10. Context Layer Harness Engineering. Atlan. https://atlan.com/know/context-layer-harness-engineering/ 11. How to Build an AI Agent Harness. Atlan. https://atlan.com/know/how-to-build-ai-agent-harness/ 12. Agent Harness Failures and Anti-Patterns. Atlan. https://atlan.com/know/agent-harness-failures-anti-patterns/