Four criteria decide a semantic layer: consumer coverage, query identity, accuracy on your own questions and contested-metric ownership. Choose a semantic layer in three passes: read official docs, cut candidates with knockout questions, then test survivors on your own questions and permissions. Atlan, the AI Context Platform, sits alongside the layer you choose as the context layer that records each metric’s owner and certification status.
The engine you pick computes the number. Someone still has to say which definition is certified, who owns it and where it came from, and that record is what Atlan’s enterprise context layer holds next to the definitions a semantic layer stores. If your analytics or platform team picks the layer on a datasheet, the cost shows up later as a confident, wrong number served to someone who has no reason to doubt it. The stakes are rising: according to Forrester (2026), “agentic AI simply stalls when enterprise data lacks a consistent, shared semantic foundation.”
| Item | Detail |
|---|---|
| Decision covered | Which semantic layer, after you know you need one |
| Method | Three passes: official docs, knockout questions, tests on your own questions |
| Recorded from official docs | Interfaces, documented identity model, license tier, export format, feature status |
| Settled only by a test | Accuracy on your questions; behavior under another user’s permissions |
| Who should be in the room | The analytics lead, the data platform lead, and the owners of your contested metrics |
| Step before | Whether you need a semantic layer at all |
| Cost side | The build vs buy decision |
Is choosing a semantic layer the same decision as buying one?
No. Choosing a semantic layer is the middle step: after you know you need one, before you price building against buying. It asks which layer gives your consumers the right answer under the right permissions.
The basics of what a semantic layer is come first, and a ranked shortlist of semantic layer tools supplies the candidates. Ownership, certification and access context for agents sit in the layer above the engine and have their own context layer evaluation criteria. Keeping the two apart stops a team from scoring an engine on a job its context layer is meant to do.
How do you order the evidence when you choose a semantic layer?
Order the evidence by who can settle it: a datasheet records interfaces, license terms and the documented identity model, while only a test you run shows accuracy and permission behavior. Mixing the two is how a candidate scores well on paper and then serves every agent request with no per-user context.
The three passes run in a fixed order:
| Pass | What it settles | Source of truth | Output |
|---|---|---|---|
| 1. Docs | Interfaces, documented identity model, license tier, export format, feature status | The vendor’s official documentation | A one-line answer per candidate, with the page cited |
| 2. Knockouts | Yes or no requirements you set before scoring | Your own requirements | A shorter list |
| 3. Tests | Accuracy, refusals, wrong-but-plausible answers, permission behavior | Your questions and your users | A tally per candidate that you repeat after every change |
Vendors measure accuracy too
The vendors themselves treat accuracy as something you measure. One cloud warehouse documents evaluations that run the SQL its analyst generates against verified queries as ground truth. A lakehouse platform documents benchmark questions that can carry a SQL answer, with accuracy assessed by comparing that result set with the one its agent generates (Test and monitor a Genie Agent, lakehouse platform documentation, November 5, 2026).
A docs-first order can feel slower than a demo, but what you read first decides which demo is worth an afternoon. The passes apply to whichever component computes the metric, wherever you draw the line between a semantic layer and a data model, or how a context layer, a data catalog and a semantic layer split the work.

Three passes for choosing a semantic layer. Source: Atlan
What can official docs settle before you run a single demo?
Official documentation records which interfaces a layer exposes, which identity options it offers, what license tier you need, how definitions export and whether a feature is still moving. Each row states only what a vendor’s own docs say, with the page linked, and names what to carry into your test.
Interfaces, identity and access rules
| Question to answer from docs | What vendor documentation states | What to carry into the test |
|---|---|---|
| Which interfaces does it expose? | A universal semantic layer documents SQL, REST (JSON) and GraphQL interfaces. A transformation tool’s semantic layer documents that AI tools connect through its MCP server (Model Context Protocol, the standard agents use to call tools). A lakehouse platform documents metric views queried from SQL editors, notebooks, dashboards, its agents and alerts, and from external BI tools (Unity Catalog metric views, October 29, 2026). A BI tool documents a natural-language route that queries a modeled dataset | Run your own consumer through each interface you plan to use |
| Which identity runs a query? | A semantic layer FAQ documents two options: service tokens map to data platform credentials, while personal access tokens run queries with the user’s own credentials. A semantic layer MCP server intersects deployment access with the user’s role-based permissions. A warehouse-managed MCP server grants permission per tool and recommends OAuth over hardcoded tokens. A BI suite’s data agent reads a semantic model under Read permission, with row-level and column-level security still applied | The identity test, run with the option your agent will actually use |
| Where do access rules live? | One layer’s access policies declare group-scoped member-level access, row-level filters and data masking in the model. On a lakehouse platform, users with at least SELECT can query a metric view (Manage metric views, October 29, 2026) | A policy test with a restricted user |
License, portability and feature status
| Question to answer from docs | What vendor documentation states | What to carry into the test |
|---|---|---|
| What license or tier do you need? | One semantic layer requires a Starter or Enterprise-tier account. Apache Ossie, an incubating Apache project, is a vendor-neutral YAML specification under the Apache 2.0 license | Confirm your tier and the license of anything you embed |
| Can definitions leave? | The Apache Ossie ecosystem page lists converters, including ones from lakehouse metric views to Ossie and back, and the project home page says its compliance tests check that query results stay consistent, beyond matching model structure | Export one real definition and re-query it |
| Is the feature still moving? | As of November 17, 2026, one warehouse’s release notes list semantic view materializations as generally available on November 11, 2026, with many-to-many relationships (November 13) and level of detail expressions (November 2) in Preview. A lakehouse platform labels sharing metric views to other metastores or accounts as Beta (Manage metric views, October 29, 2026) | Record the status of each feature you need, dated, on the day you evaluate |
Identity is two questions
Read the identity row in the first table as two questions: which credential reaches the warehouse, and where per-user policy is enforced. A shared credential with per-user filters in the layer can pass; a shared credential with none answers every user the same way. That is why the MCP connection between agents and a semantic layer gets its own scorecard line.
Interfaces differ by architecture: a code-first universal semantic layer, a transformation-tool semantic layer, a lakehouse catalog’s built-in semantic layer and warehouse semantic views each expose metrics differently, and open source semantic layer options add their own license questions. The status row ages fastest, so date every entry.
Which knockout questions should cut a candidate before you test?
A knockout question is a yes or no requirement you write before scoring, so a candidate that fails it leaves the list instead of losing a few points. Write the knockouts with the platform lead and the analytics lead in the same room, because each side tends to forget the other’s consumers.
| Knockout | Fails if | Settled by |
|---|---|---|
| 1. Every consumer you run today (BI tool, notebook, app, agent) can query a definition through a documented interface | Any consumer needs its own copy of the definition to work | Docs, then one run per consumer |
| 2. A query runs as the asking user, or on a shared credential with per-user policy enforced in the layer, or you accept one shared credential on purpose and record why | The answer to “whose credential, and where is policy enforced?” is “it depends” | The identity test |
| 3. Definitions you already hold in transformation code, a BI model or the warehouse are read by the layer, or migrated once through a documented import or converter | The layer needs a second, hand-maintained copy that lives on after the migration | Docs, then an export test |
| 4. A definition change is reviewed, versioned and shows what depends on it | A change goes live with no review or no list of dependants | Docs, then a trial change |
| 5. Your license tier and the license of any embedded component fit your use | The tier or license you need is outside what you can buy or ship | Docs |
Where knockouts trip teams
Each knockout is pass or fail. Weighted criteria for cost and fit belong to the build vs buy step, after the list is short.
Consumer coverage is easy to get wrong, because a semantic layer built for BI and one built for AI agents expose different routes. The existing-definitions knockout can surface a quieter split between a semantic layer and a metrics layer already in your stack.
How do you test a semantic layer on your own questions?
Test each surviving candidate on questions you already know the answer to, ask them as real users with different permissions, and tally correct answers, refusals and wrong-but-plausible answers separately. The test takes the claims from the docs pass and checks them on your data.
Build the question set
Start from questions with known answers, each carrying the SQL that answers it. One warehouse calls this a Verified Query Repository: a collection of questions and the SQL queries that answer them. The same shape works for any layer. Give each entry the question, the expected SQL, the expected result, a rounding tolerance, the metric owner and the permission profile it runs under. Keep the set in version control and run it against a frozen data snapshot so expected results do not drift. Pick hard questions on purpose:
- Contested metrics, where two teams compute “active customer” or “net revenue” differently today.
- Questions that join tables at different grains, where fan-out mistakes hide.
- Questions your dashboards already answer, so you can compare to the decimal.
Teams building text-to-SQL with AI use the same kind of set, and it doubles as a fair comparison of text-to-SQL against a semantic layer for self-serve analytics.
Hold the agent constant
A natural-language answer depends on the model, prompt and agent as much as on the layer. Point one agent, with the same model and prompt, at each candidate through its API or MCP server, score any vendor’s own agent as a separate row, and ask each question several times, because model answers vary between runs.
Add one check with no model in it: request the metric through the layer’s API and compare it with the dashboard. That separates a wrong definition from a wrong interpretation. Record latency and warehouse compute per run for the build vs buy step.
Tally correct, refused and wrong-but-plausible separately
A refusal costs a person a retry. A wrong-but-plausible answer costs a decision, because nobody knows to retry. One accuracy percentage folds both into one number and hides the second.
Expect wrong answers even on a well-prepared system. According to Chen et al. (2025), expert review rated 53% of responses from LinkedIn’s internal text-to-SQL chatbot, which has over 300 weekly users, as correct or close to correct on an internal benchmark. Even that figure merges correct with close to correct, and it says nothing about which layer type does better. It shows an enterprise system still has to be measured on its own questions.
Whether a layer reduces the errors common in enterprise text-to-SQL is something your tally shows; running one question set through a warehouse’s native analyst and through custom text-to-SQL gives that comparison on your data.
Run the identity test
Ask the same question as two users with different permissions, then as one user with no access. The answers, and the refusal, should differ the way your access policy says they should.
If all three get the same answer, the layer is running on a shared credential with no per-user policy applied. One layer’s FAQ documents both routes: service tokens map to data platform credentials that admins share with a team, while personal access tokens run as the user. Test the route your agent will use; if it is shared, enforce policy inside the layer or accept the shared credential on purpose and write that down.
Where the agent queries as the asking user on the platform that holds the rules, those rules carry through, as with SELECT on a lakehouse metric view or Read permission on a BI semantic model, so the test is short. Fold it into the same run you use for an AI-ready data checklist or an AI readiness assessment, since both ask who may see what.
Semantic layer evaluation scorecard: what to record for each candidate
One scorecard row per candidate lets a buying group compare passes instead of impressions. Fill it in as you go; leave a cell blank rather than guess.
| Measure | Candidate A | Candidate B | Candidate C |
|---|---|---|---|
| Docs pass: interfaces covered (of your consumers) | |||
| Docs pass: identity options and policy location recorded | |||
| Docs pass: dated status for each feature you need | |||
| Knockouts passed (list) | |||
| Agent, model and prompt held constant (name them) | |||
| Questions asked, runs per question | |||
| Correct | |||
| Refused | |||
| Wrong-but-plausible | |||
| Direct API check matches the dashboard | |||
| Identity test (user 1, user 2, no access) | |||
| Latency and warehouse compute per run | |||
| Re-test after a change: result |
Rank by the wrong-but-plausible rate first, then the correct rate. A wrong-but-plausible answer on a contested metric drops a candidate until the cause is found, because its errors look like answers. The scorecard carries no weights, so a second reviewer can check every cell.
Who owns a contested metric definition before you pick a layer?
A semantic layer encodes whichever definition you give it, so a disagreement nobody settled becomes two answers served with the same confidence. Settle ownership before the pilot starts.
The ownership check is short. For each metric in your question set, name one owner and one certified definition. Two teams can keep different versions of “active customer” if each has its own name and owner; what fails is one name with two formulas. If you cannot name the owner, settle that first. The people who make that call are the owners and stewards of AI-ready data, and a glossary is one place to record it.
Atlan’s glossary guidance gives a concrete pattern: designate one certified core glossary as the source of truth for critical metrics, and expose only certified KPIs through a read-only “certified metrics” persona. That pattern is how teams manage KPI definitions and a business glossary side by side, and it shows why a semantic layer and a business glossary do different jobs: one computes, the other records who agreed.
Certification also needs a trail. Data lineage for AI shows which tables and transformations feed a certified metric, so a reviewer can see what a change upstream will break. An engine can store an owner field on a definition; recording who certified it, across every layer in the stack, in a form an agent checks before answering, is the context layer’s job.
Which layer owns a metric when your stack runs two?
When a warehouse-native layer and a BI model both define a metric, name one as the owner and treat the other as a consumer, or the two will drift. The question comes up for every metric two layers share.
| Situation | Name as owner | Check before you decide |
|---|---|---|
| Most consumers query one warehouse | The warehouse-native layer | Every BI consumer can read it through a documented interface |
| Analysts live in one BI tool and agents are secondary | The BI model | Agent access follows a documented route, for example a BI suite’s data agent reading the semantic model under Read permission with row-level and column-level security applied |
| Several warehouses, BI tools and agents share metrics | A layer that sits across them | It reads your existing definitions or migrates them once, instead of keeping copies |
| Both layers define the same metric today | Pick one, convert the other to a consumer | Export one definition and re-query it in the other layer |
| The metric is contested | The metric owner decides, not the tool | One named owner and one certified definition exist |
Portability between layers
Portability helps, with limits. Converters listed on the Apache Ossie ecosystem page translate lakehouse metric view definitions into the open specification and back. A translated copy is still a copy: convert once, retire the source and test one real definition before relying on the converter.
The trade-offs behind each row run deeper: warehouse-native vs universal semantic layers, three semantic layer architectures compared, a lakehouse catalog versus a cross-platform semantic layer, a BI semantic model versus a dedicated semantic layer, and a BI modeling language versus a universal semantic layer. Whichever row fits, the owner is a person first and a layer second.
When should you re-run the evaluation after you choose?
The first evaluation only proves the day you ran it, so re-run the same question set after any change to a definition, a layer, a connected consumer or a vendor feature. With the question set and expected results in version control, the re-run is a single command.
Re-test when:
- a metric definition changes, even a filter or a grain;
- a new consumer or agent connects to the layer;
- the vendor ships or changes a feature you depend on, as one warehouse’s November 2026 release notes show with semantic view features moving between Preview and general availability;
- a layer is swapped, or a second one is added.
When a definition changes on purpose, update its expected result in the same review, so the re-run separates intended changes from regressions, and alert on any question that used to pass and now fails. Context engineering and the semantic layer meet here, because the question set, the expected answers and the owners are context you maintain long after the purchase.
What red flags show up when teams choose a semantic layer?
Most of these five red flags come from reading evidence in the wrong order, scoring from a datasheet what only a test can answer. Each has a check that catches it early.
| Mistake | What it looks like | Check that catches it |
|---|---|---|
| Scoring identity or accuracy from a datasheet | “Supports MCP” and “enforces permissions” ticked before any run | Move both to the test pass; score them only from the identity test and the tally |
| Demoing on the vendor’s data | A clean demo on a sample schema with no contested metrics | Run the demo on your own question set |
| One accuracy percentage | A single figure in the report, with refusals and wrong answers mixed together | Record correct, refused and wrong-but-plausible as three columns |
| Comparing agents instead of layers | Each candidate tested through a different agent or model | Hold the agent constant and add a direct API check |
| Two layers each defining the same metric | The BI model and the warehouse layer disagree by a small, stable margin | Name one owner and convert the other to a consumer |
How Atlan approaches the semantic layer you choose
Atlan works next to the semantic layer you pick as the context layer around it. It reads the definitions the layer holds and adds the owner, the certification status, the lineage and the access context an agent needs before it answers.
Atlan’s documentation lists semantic offerings by connector, with lakehouse metric views and BI semantic models marked as supported sources. A certified core glossary with a read-only certified-metrics persona then gives those definitions an owner and a status. Austin Kronz, Atlan, describes the order he recommends to customers building analytics agents: “you should always start with certified metrics where you have them.” Where no certified metric exists, the agent browses the AI-ready data and tells the user which route it took.
Your tests decide which engine computes the number; Atlan holds the record of who certified it and which consumers read it. The full sequence is in how to implement an enterprise context layer for AI, and the discipline behind it is context engineering. A context layer and a semantic layer answer different questions, and natural-language tools need both, as the case for a context layer for natural-language analytics lays out. If your next step is designing the layer, semantic layers for analytics start where the evaluation does: the questions and the owners, before the tables.
FAQs about choosing a semantic layer
1. What is the best semantic layer?
The best semantic layer is the one that passes your knockouts and gives the fewest wrong-but-plausible answers on your own questions, under your users’ permissions. Rankings only supply candidates.
2. What is an example of a semantic layer?
A semantic layer defines a metric such as net revenue once and lets several consumers query it. One lakehouse platform’s metric views, for example, are queried from notebooks, dashboards, agents and external BI tools, all reading the same definition.
3. How to set up a semantic layer?
Settle the owner and certified definition of each metric first, then connect one consumer, then run a test on questions you already know the answers to. Add consumers one at a time and re-run the same test after each.
4. Is a data platform’s built-in metric layer a semantic layer?
Yes. Warehouse and lakehouse platforms document semantic objects, such as metric views and semantic views, that define metrics once for several consumers. Evaluate them with the same three passes.
5. Is a warehouse-native semantic layer enough on its own?
A warehouse-native layer is enough when most consumers query that one warehouse and its permissions already apply to them. When several warehouses, BI tools and agents share the same metrics, name one owning layer and treat the others as consumers.
6. What is the difference between a semantic model and a semantic layer?
A semantic model is a set of definitions for one tool or subject area. A semantic layer is the shared place that holds definitions and serves them to the consumers that query it.
7. How to design a semantic layer?
Start with the questions and the owners, then the tables. List the questions your consumers ask, name an owner and certified definition for each metric behind them, then model only what those questions need.
8. What is semantic layer vs ontology?
A semantic layer defines how to calculate metrics and dimensions over your tables. An ontology describes business concepts and the relationships between them. The two overlap in practice: the ontology explains what things are, and the semantic layer computes the numbers.
Sources
- Make Data AI Ready Via Semantic Layer Platforms (summary), Forrester, 2026
- Text-to-SQL for Enterprise Data Analytics, arXiv, 2025
- Evaluations for a warehouse natural-language analyst, vendor documentation
- Verified Query Repository, vendor documentation
- Warehouse-managed MCP server, vendor documentation
- Warehouse release notes and feature updates, vendor documentation, 2026
- Semantic layer overview, vendor documentation, 2026
- Semantic layer FAQs, vendor documentation, 2026
- Semantic layer documentation and API reference, vendor documentation, 2026
- Access policies, vendor documentation, 2026
- Semantic layer MCP server, vendor documentation, 2026
- Data agent creation, vendor documentation, 2026
- Query a modeled dataset in natural language, vendor documentation, 2026
- Apache Ossie (incubating), Apache Software Foundation
- Apache Ossie ecosystem, Apache Software Foundation
- Structure your glossary, taxonomy, and metrics catalog, Atlan Documentation
- Semantic offerings by connector, Atlan Documentation
- Unity Catalog metric views, lakehouse platform documentation, last updated October 29, 2026 (plain-text citation)
- Manage metric views, lakehouse platform documentation, October 29, 2026 (plain-text citation)
- Test and monitor a Genie Agent, lakehouse platform documentation, last updated November 5, 2026 (plain-text citation)