---
title: "Where Should the Graph Live?"
url: "https://atlan.com/context-and-chaos/issue/where-should-the-graph-live/"
description: "Every serious AI system needs relationships. Whether it needs a graph database is a different question, and it has a different answer for every workload."
keywords: "Graph Databases, AI Agents, Enterprise AI, Context Engineering"
---

> Atlan is hosting Context Conference, bringing together the leaders and builders at the frontier of giving AI the context it needs to understand their business. It runs online on October 28, 2026, from 11:00 AM to 2:00 PM ET. Atlan co-founder Prukalpa Sankar opens and closes the day. Leaders from AstraZeneca, BNY and Verizon share why they invest in context and what they get from it. Registrants get early access to The AI Context Gap, a new study from MIT Technology Review Insights. Register: https://atlan.com/context-conference/

A Context & Chaos deep dive (Atlan's practitioner newsletter) by **Austin Kronz, Field CDAO at Atlan** (former Gartner Research Director for analytics and data science; host of the WTF is the Context Layer series). Published September 17, 2026, 21 min read. Every serious AI system needs relationships; whether it needs a graph database is a different question, with a different answer for every workload. Built on an hour-long discussion with **Emil Eifrem (CEO and co-founder, Neo4j)** and **Prukalpa Sankar (chief editor of Context & Chaos)**: [watch the recording](https://atlan.com/wtf-context-layer/how-do-graph-databases-and-the-context-layer-fit-together-recording/).

Key points:

- A model of your business and the place you physically keep it are two different decisions. The same drawing can live in memory, on disk, in the warehouse you already run, or in a database built for connected data.
- Take one customer and raise the stakes three times. A known question against a stable schema does not need a graph database. A live network crossed under load, where the break shapes the query and the path is unpredictable, is exactly what one is for.
- Most real work sits between those rungs, and nobody can tell you your answer. What settles it is empirical: accuracy on your own evals, hop count, latency, volatility, and the real cost of the second copy.

## The drawing vs. where it lives

- Ask business experts to draw how the business works and they draw circles joined by lines, never tables. Eifrem built Neo4j on that: "We would just ask them to just draw your world on the whiteboard, and almost never did they end up drawing tables." He calls the gap between the drawing and the relational schema an impedance mismatch.
- The model (concepts and connections) and the physical store (memory, files in cloud storage, warehouse tables, a graph database) are separate choices. Terms like ontology, knowledge graph and context graph are contested; Eifrem calls it "ontology-washing". See the earlier issue [sorting through those terms](https://atlan.com/context-and-chaos/issue/ontologies-context-graphs-and-semantic-layers-what-ai-needs-in-2026/).
- Eifrem's own position: **having a graph and running a graph database are not the same commitment.** A knowledge graph can live in memory, on disk, or in object storage.
- There is no single answer for where the graph should live, only an answer per workload.

## Three rungs, one customer

Running example: Dana asks whether her $40 roaming charge is refundable because her service was down. (Diagram: the question broken into what an agent must know — who Dana is, what a roaming charge is, whether there was an outage on her line, what makes a charge refundable, how a refund is issued.)

**Rung one: a known lookup.** The agent needs who Dana is, her plan, the current refund policy and known outages on her line. The business still has to be modeled (relationships, meaning, rules; not loose documents and markdown files), but the queries are consistent: the same four or five joins against a schema that is not moving. A relational database is probably fine. **Likely does not need a graph database.**

**Rung two: the denial was wrong.** The agent says no: plan ended on the 14th, outage logged on the 15th, no refund outside an active plan. But the outage actually began on the 13th (the network event log and billing system disagree on outage start), and her plan end date was migrated incorrectly in a platform upgrade 18 months ago. A human overrides it. The valuable output is the finding: two systems disagree on outage windows, this plan class carries a bad end date, and the override is correct here. In a ticket comment, the next agent re-litigates it; worked back into context, every agent inherits the judgment. The question shifts from "what is true" to "what did we decide, why, and what does that change", and the relationships are worth more than the objects.

That is a statement about the model, not the store. Rung two is genuinely **"it depends"**: it could be a graph database (the idea of [context graphs](https://atlan.com/context-and-chaos/issue/context-graphs-are-a-trillion-dollar-opportunity-but-who-captures-it/)) or the relational store and warehouse you already run. Both ship in production. Decide empirically:

- **Accuracy on your own evals** — the only number that settles anything.
- **Maintainability by hop count** — two or three joins is routine; six is a conversation; past some depth only the author can safely change the query.
- **Latency** — if a couple of seconds is fine, the door is much wider than diagrams imply.
- **Volatility** — fixed, known queries reward the existing store; open-ended traversal with unpredictable paths is where a second store starts to make sense.
- **The cost of the copy** — pipeline, drift, two definitions of customer, on-call for both.

Rung two is a hill-climbing problem, not an architecture problem: instrument the agent, run evals, find the wall, move when you hit it.

**Rung three: the outage is happening again, to many customers.** Equipment fails or a fiber line is cut. Monitoring says what broke in seconds; the hard question is who is affected — which towers went quiet, which customers are on them, which are businesses with contracts that start owing money when service drops, which are already calling in. Three properties decide the architecture:

- You do not know how far to look: four steps or fifteen, depending on the break and how traffic was rerouted twenty minutes ago. No fixed query can be written in advance.
- The break shapes the question: every failure is different and the path changes with it.
- It has to answer while the phones ring.

At rung one the network was a fact looked up; at rung three it is a live graph crossed under load. **This is what a graph database is for.** The core rule: when the relationships between things become more valuable than the objects, you need the graph; when those relationships must be traversed across many hops, fast, under load, while someone waits, you need the graph database. Rung two is the first; rung three is both.

## Why this decision is live only now

- Graph databases and ontologies worked for decades but stayed niche because of cost. Eifrem: early wins were where graph was "truly antibiotics" — fraud rings, network topology, supply chains, recommendation engines.
- Prukalpa: ontology "has existed for decades as a business problem... the challenge with it is it's always been really hard to build and maintain," so it was done for one-off or high-value use cases. Palantir deploys application by application for the same reason.
- Eifrem named the two costs: "How do I model my data as a graph?" and "How do I query?"
  - **Query cost is largely gone:** natural language replaces specialist Cypher/GQL skills.
  - **Modeling cost moved further:** ontologies can be bootstrapped bottom-up from Postgres, Snowflake or Databricks schemas. Eifrem: "it'll probably never get to 100% for all use cases," but "it gets you out of this blank sheet of paper." The business layer bootstraps via end-to-end lineage (systems of record into warehouse into business apps and BI), reverse-constructing semantics and then ontology. Eifrem's phrase: "bottom-up produced by AI meets top-down."
- Prukalpa's warning: throw tables into Claude Code and ask for an ontology, and "it's going to create something that looks beautiful but it's not going to be accurate." Getting to accurate means applying data quality principles to context, in a pipeline, and owning and maintaining the model as definitions drift and acquisitions arrive.
- Demand changed too. A dashboard user brings years of context (which report is real, last quarter's caveat, who to ask); an agent has none of it. Dashboards answer anticipated questions via pre-joined tables; agents get unanticipated questions and must assemble answers across things nobody pre-joined.

## Where graph shows up in enterprise AI architecture

**GraphRAG** is where graph comes up most, by Eifrem's count. Traditional RAG chunks and embeds content and retrieves by similarity; Eifrem: vector search only says two things "scored 0.7 alike," never why. GraphRAG extracts entities and relationships into a concepts layer linked to chunks: retrieval starts with similarity to find an entry point, then follows links outward, giving multi-hop reach and an inspectable path. Prukalpa: "skills retrieval is an open problem. Should you retrieve via graph, should you retrieve via keyword" — she expects it all to change within a year.

(Diagram: a contact-center reference architecture — an agent harness drawing governed context from an AI context platform of AI-ready data, semantics and ontology, agent skills, long-term memory and an active context graph; one path to an operational store for real-time traversal; the analytics path to the warehouse marked "no graph required".)

Three distinct jobs, at different stages of settledness:

1. **The graph as the operational store.** System of record for entities and connections: customer 360, network topology, fraud rings, supply chain. Deep traversal, answers while people wait, the break sets the question. This is rung three, and it is settled; what changed is how many workloads qualify.
2. **The graph as agent memory.** Semantic, procedural, long-term memory. Eifrem: "Memory is an intrinsically graph-centric workload." The initial MCP spec's memory implementation was a graph ("it's a toy, it's 500 lines of Python code, but it's a graph"), and many agent-memory startups converged on graphs independently. In production, memory ranges from none, markdown wikis, table rows, vectors beside documents, to graphs; most systems mix two or three.
3. **The graph as part of the context platform.** Eifrem: "context graph" would more accurately be a "decision trace graph." Traces and episodic memory are table stakes but are just logs unless a system converts them into skills and longer-term procedural and semantic memory. The platform distills traces into reusable skills (the common pattern over the last six to nine months, per Eifrem); linking memories and traces to skills gives provenance, so a new trace forces a decision to supersede, retire or write a new skill. A skill with no link to what produced it is a snapshot with no expiry date.

All three jobs have to get done — operational state, agent memory, and the record of what was decided and why — regardless of which store holds each or how the agent retrieves it.

## Start at the whiteboard

- Draw the hardest, highest-consequence decision one of your agents makes. Ask: how far does the agent travel to decide; what must be explainable afterward (data, policy, approvals); what are the real latency and scale limits; what will change underneath it so a definition, skill or relationship can go stale unnoticed?
- Run two ladders. The first is the business decision (look up, assemble evidence, cross a live network on a clock): it tells you where operational data belongs. The second is the agent's own: what has it learned, where does that live, how will you know it expired? Dana's override is rung two of the first ladder and all of the second. Teams that skip the second get agents that cannot explain why they answered differently last quarter.
- Retrieve trusted facts and apply known rules: you may already have what you need. Go deep and decide fast against data that resists being moved: graph infrastructure earns its complexity. Rung two: go find out.
- Prukalpa's advice: "build yourself personally." The only way to develop judgment is to instrument something, watch it fail, and say precisely why you changed it — because the agent got more answers right or decided faster, not because it was on the diagram.

Closing cartoon: a cat on stage shouts "You get a graph database!" under "Every business is a graph" at an audience holding "3 fixed joins", "FAQ lookup" and "daily report"; caption: "That wasn't the question."

## About Context & Chaos

A community newsletter where practitioners, builders and thinkers share stories and lessons on context engineering, governance, architecture, discovery, and the human side of data and AI work. [Browse all issues](https://atlan.com/context-and-chaos/) or [contribute](https://atlan.com/context-and-chaos/contribute/).

Related reads:

- [Your Agent Doesn't Need to Walk the Graph](https://atlan.com/context-and-chaos/issue/your-agent-doesnt-need-to-walk-the-graph/) (August 2026)
- [Context Graphs Are a Trillion-Dollar Opportunity. But Who Captures It?](https://atlan.com/context-and-chaos/issue/context-graphs-are-a-trillion-dollar-opportunity-but-who-captures-it/) (February 2026)
- [Ontologies, Context Graphs, and Semantic Layers: What AI Actually Needs in 2026](https://atlan.com/context-and-chaos/issue/ontologies-context-graphs-and-semantic-layers-what-ai-needs-in-2026/) (January 2026)
- [Enterprise AI's 200-Millisecond Problem](https://atlan.com/context-and-chaos/issue/enterprise-ais-200-millisecond-problem/) (August 2026)