A Fortune 500 insurer I worked with earlier this year had been handed a reference architecture by every cloud vendor in their stack. Diagrams on diagrams. Somewhere in that pile they had picked up a conviction that their agent had to be wired into a managed knowledge-graph service before it could do its job. And with the number of terms flying around, I understood how they got there.
The conviction had not come from anything about their workload. It had come from the diagrams. Nobody had counted the questions the agent was going to be asked. Nobody had described out loud the path the agent would take across that graph to answer one of them. The graph was a box on a slide with an arrow running into it.
Their actual questions were things like: can this rep sell this product in this state? What is the status of this policy? Bounded questions, with a join path anyone on the data team could have drawn on a napkin. They were already running a governed metadata layer that held most of what those questions needed.
They did not need a graph database. More precisely, and this is the part that took longer to land, they did not need their agent going anywhere near a graph. They needed it to know what their columns meant. The gap between those two sentences is where a lot of the 2026 architecture budget is about to go, so it is worth being precise about which side of it you are on.
Store it in a graph if you like. Just don’t wire it to the agent.
Permalink to “Store it in a graph if you like. Just don’t wire it to the agent.”
Where context lives is a separate decision from what the agent queries. | Source: Context and Chaos
Storing your context in a graph is fine. It can be the right call when that context serves a lot of agents rather than one, because a graph is a good place to model and govern how things relate. That is a storage decision, and I have no quarrel with it.
I would go further, because the case for a graph here is better than “fine.” An ontology is the one part of this that is genuinely graph-shaped. Its content is the relationships, what subsumes what and which properties apply where, so a graph is faithful to its shape and not just a place to keep it. It is also small and slow-changing. People govern it, not pipelines. None of that is true of the instance data underneath. Model it there and govern it there. Then distil from it and let the agent read the distillation.
That is an argument about where you author, not about what the agent queries, and the two get run together constantly. It is the same slip behind “our data has relationships, so we need a graph.” Every schema has relationships. A foreign key is an edge, and a JOIN is a one-hop traversal. (Jessica Talisman untangled the vocabulary here in January; I am taking her definitions as read.) What you almost certainly do not need is that graph connected to the agent at query time. To be precise about what I am objecting to: a service that walks a graph and hands the agent a result is fine, and often the right build. What I am against is the model doing the walking, deciding hop by hop where to go next and then still having to write the query. The data lives in relational systems in the large majority of cases and will still live there after you have built the graph. The agent’s job is to structure a query against it correctly. A graph does not tell it how to do that. Curated context does.
There is a cost to getting this wrong that never shows up on the architecture diagram. If the agent has to reason its way across a graph and then reason its way to the SQL, you are running two rounds of inference to produce one answer: more tokens, more latency, and one more place where two runs of the same question come back different. Strip either problem down to what the agent genuinely has to decide, and it is almost always just how to shape the query. Anything I hand it as context instead of making it derive is variance removed before it happens, and that context is unglamorous: query patterns that are known to work, a note on how a table is meant to be used, the rule that governs a field. The agent reads it once and writes the query.
Paths or lookups. That is the whole sort.
Permalink to “Paths or lookups. That is the whole sort.”Ask what shape the agent’s questions are.
Lookups
Can this rep sell this product in this state. What is the status of this policy. How many claims were filed in this region last quarter. These are bounded questions: the join path is knowable at design time, and the answer is a value or a set. What the agent needs here is definitions, join keys, grain and the governing rule. Give it that and it queries the data you already have. No second database, and nothing traverses anything.
**Paths **
The answer is the shape of the connection itself. Shortest or all routes between two points. Fraud rings, which are cycles. Centrality, meaning which node matters structurally. Who is within N hops of this entity and how are they linked. The route is the answer, not a byproduct of finding it.
That split holds up better than the criterion most teams reach for, which is depth. Depth is the wrong term to worry about anyway: a recursive frontier grows at roughly fan-out to the power of depth, so a deep narrow acyclic graph is cheap to walk as far as you like; a shallow dense cyclic one is ruinous by depth three. Enterprise metadata is overwhelmingly the first shape. For every agent that truly needs traversal, some huge number are just talking to data.
If you know the path, it is a lookup. If the path is the answer, it is a graph problem. | Source: Context and Chaos
Lookups and most analytics are reachability questions: what is the set, is it connected, what is upstream. Recursive common table expressions make SQL Turing-complete, so the expressiveness argument was settled years ago and largely went unnoticed. Expressiveness is not speed, though, and I should be honest about where this runs out. One team walking a 335,000-node tree found their recursive CTE taking 47 seconds, and got it to 227 milliseconds by moving the traversal into a C extension. Note what they did not do, which is buy a graph database; they kept the traversal beside the data, in the engine they already ran. Another team published where it stops working: fine at a hundred thousand nodes and depth four, degrading at five hundred thousand and depth six.
Pathfinding is the other class, and here recursive SQL stops being merely slow and becomes a bad idea. Expressing betweenness centrality or community detection in it is an act of violence against whoever reads it next, and graph engines have decades of specialised algorithm work behind theirs. A path, a cycle, a cluster or a ranking of structural importance is a real signal and a clean one. Cycles are the sharpest case: a general cyclic graph requires explicit cycle handling that a DAG traversal never needs.
Density, not depth. The frontier explodes when connections multiply. | Source: Context and Chaos
Three conditions earn a second database. None of them put the agent on it.
Permalink to “Three conditions earn a second database. None of them put the agent on it.”These are query-pattern tests, not modelling ones: the ontology argument above is about where you author; this is about what the engine is asked to do.
The deliverable is a path or a topology result. Shortest or all paths, cycle detection, community identification, centrality ranking.
The frontier actually explodes. High fan-out, many back-edges, nothing useful to prune on. Measure it rather than assuming it.
You need single-digit-millisecond per-entity traversal under concurrent writes. Index-free adjacency makes per-hop cost independent of graph size, which matters for operational serving and not at all for analytical work.
Now read them again with the runtime question in mind. That second read is the point, not the list itself. Not one requires an agent to touch the graph. Each is satisfied by a system that runs the traversal and hands back a result. Meet condition one and buy the engine. That still does not mean you wire it to the model.
Aggregations, rollups and temporal windows are the standing exclusion, because graph query languages are conspicuously weak at aggregation and columnar engines exist for precisely that.
The failure I actually spend my time on is earlier than any of that, and it is not technical. It is a second store proposed when the governed metadata layer the organisation already runs would answer the question, with the conviction inherited from a reference architecture rather than from the workload. Nobody has measured their frontier. Nobody has sorted their questions. The diagram had a graph in it.
The teams that get past that meet the next one, the stale replica. The proposed graph is an ETL’d copy of an authoritative relational system of record, so you are buying staleness, which destroys trust fastest in exactly the compliance and executive settings where the graph got pitched as the trust layer, and paying twice to maintain both. One engineering team published their numbers after moving hierarchy, dependency and ownership questions off a graph database into recursive SQL in the warehouse they already ran: roughly 94% less cost, about $800 a month down to about $50 a year, one less system to operate. They kept the graph engine for centrality, community detection and unbounded pathfinding, and nothing else.
Read what the graph papers actually measured
Permalink to “Read what the graph papers actually measured”
The graph helped by selecting context, not by making the model traverse it. | Source: Context and Chaos
The reason this argument keeps getting made anyway is that two papers are read as saying something they do not say.
The most-cited evidence for graphs-and-LLMs is the clearest case. Sequeda and colleagues reported question-answering accuracy over an enterprise SQL schema rising from 16.7% to 54.2% when a knowledge graph was introduced. The number is real, and the paper is careful about something the citations drop: the knowledge graph there is an ontology plus mappings, functioning as a context layer the model reads, not a graph the agent traverses at query time. Three years of operational graph business cases rest on one word in its title.
GraphRAG deserves the same care, and the systematic evaluation of it is not the paper the enthusiasm suggests. Han and colleagues ran plain retrieval against four families of GraphRAG across single-hop, multi-hop and detail-oriented benchmarks. The variant that does what most people picture when they say GraphRAG, which is extract a knowledge graph from the corpus and retrieve over it, loses to plain retrieval on every question-answering benchmark in the study. On MultiHop-RAG, the benchmark built for exactly the reasoning that graphs are supposed to be good at, it scores 48.5% against plain retrieval’s 67.0%. On HotpotQA, 42.6 F1 against 60.0. Building its index takes 7,702 seconds where plain retrieval takes 135. Retrieval takes 14,434 seconds against 1,724. Fifty-seven times the preprocessing and eight times the query cost, to be worse at the thing it was built for.
Some GraphRAG variants do win, by about three points on multi-hop, and how they win is the whole argument in miniature. The strongest of them uses the graph to decide which text chunks to retrieve. The model never sees the graph. It sits upstream of the context as a routing structure, and it stops helping the moment the model has to reason across it. GraphRAG is a retrieval strategy over text, and it was never an argument for running an operational graph database.
One more number from that study, because it is the one I would put in front of anyone proposing a graph in a regulated setting. On questions the corpus cannot answer, where the correct behaviour is to decline, plain retrieval declines 96.0% of the time. The community-summarisation variant declines 19.3%. The rest of the time it has assembled a relationship out of what it found and answered anyway.
Two numbers on the same benchmark
Permalink to “Two numbers on the same benchmark”The obvious objection is that this is rearranging deck chairs, because a governed semantic layer is already a solved problem with near-perfect published results. Look at what those results measure. Benchmarks reporting accuracy in the high nineties are scoring questions inside the layer’s modelled scope, where a human has already defined everything. The boundary is drawn around the questions that were already understood, so near-perfect tells you almost nothing.Now measure the whole schema, including everything nobody has modelled yet. When Spider 2.0, the enterprise-grade text-to-SQL benchmark, arrived, frontier models scored in the low teens, against 86.6% for the same model on the older Spider 1.0 the field had spent years celebrating. That gap has since closed, and how it closed is the part worth reading, because the leaderboard happens to run the experiment you would want. Hold the model fixed at DeepSeek-R1 and change only what is built around it: 13.7% under the benchmark’s own reference scaffolding, 30.5% under one agent framework, 52.3% under another. Same model, same task, thirty-nine points. The same pattern shows up wherever a model repeats down the table. Nothing about the store changed in any of those runs, and nothing about the model changed either. What changed is what the system tells the model about the schema.
The controlled experiment this publication ran in February isolates that variable directly: 174 natural language questions over a thirteen-table schema, 522 runs, changing only the context layer. Accuracy moved from 16.1% on a bare schema to 22.2% with high-signal context, significant at p < 0.0001. A small absolute number on a small schema, and still the only lever in the experiment that moved at all. Verbose documentation-style context did worse than concise high-signal context, dropping 13.8% while costing 52% more, which should worry anyone planning to fix agent accuracy by pointing a model at a wiki, and for the same reason anyone planning to fix it by pointing a model at a graph.
The in-scope and whole-schema numbers are not a contradiction, and the vendor benchmarks are not dishonest. They measure different things, and only one of them resembles the request an agent gets from a user who has no idea the boundary exists.
The honest summary is less romantic than either camp wants. Enterprise text-to-SQL is not unsolved and it is not solved. It responds, reliably, to one thing: how much of the schema somebody has actually described, and how well. Which means the work is not choosing a cleverer retrieval architecture, and it is certainly not adding a traversal step. It is enlarging the modelled scope, and then keeping it current.
It also changes the failure mode in a way the percentage does not capture. A governed layer lets the system refuse a question it cannot answer instead of confidently returning a wrong number that ends up in a board deck.
The agent should never know the graph was there
Permalink to “The agent should never know the graph was there”I should be straight about a tension with this publication’s own archive. C&C has argued that semantic layers failed and context graphs are next, hedged with “unless we get it right.” That hedge is carrying the entire argument, so I’ll say what getting it right would have to mean.
Semantic layers did not fail because the model of them was wrong. They failed because they were built as documentation projects, maintained by hand, and went stale. A context graph maintained the same way inherits that failure exactly, with better branding and a larger bill.
That is the smaller of the two problems. The larger one is that even a perfectly fresh context graph is the wrong thing to hand an agent. Distil it instead into what the agent actually consumes: the definitions, the join keys, the grain, the governing rules, read in a single pass. The agent should never need to know the graph exists.
The distillation has to be generated, versioned, and close enough to the systems of record that nobody has to ask how old it is. Maintain it by hand and you have reproduced the staleness problem one level down, arrived at by a longer and more expensive route.
Mark twenty questions. Count the paths.
Permalink to “Mark twenty questions. Count the paths.”Take the last twenty questions your agent was asked, or the twenty you expect it to get, and mark each one three times.
First: path or lookup? Count the paths. If that number is at or near zero, no graph database will help you, regardless of how connected your data is.
Second: inside or outside the scope anyone has actually modelled? Count the outside ones. That number is your real accuracy ceiling, and it is the one the vendor benchmarks are not measuring.
Third, and this is the one that decides the architecture: to answer it, does the agent have to know the graph exists? Not your platform, not the pipeline that built the context. The agent. That column comes back empty even for teams who legitimately need a graph engine somewhere in the stack.
Most teams I do this with find almost no paths, a great many out-of-scope questions, and nothing at all in the third column. That combination is a context problem, and it is the one to fund.
What I ended up telling the insurer was this. A graph is for when your questions are shaped like paths. Yours look like lookups. Let’s not stand up a second database to answer a SQL question, and let’s not put the model in front of one either.
Your data was always a graph. That was never the question. The question is whether anyone has written down what it means, in a form the agent can read without walking anything.
The Cats of Context & Chaos
Permalink to “The Cats of Context & Chaos”
Why are you the one walking it? | Source: Context and Chaos
About Context & Chaos
Permalink to “About Context & Chaos”Context & Chaos isn’t just a newsletter. It’s shared community space where practitioners, builders, and thinkers come together to share stories, lessons, and ideas about what truly matters in the world of data and AI: context engineering, governance, architecture, discovery, and the human side of doing meaningful work.
Our goal is simple, to create a space that cuts through the noise and celebrates the people behind the amazing things that are happening in the data & AI domain.
Whether you’re solving messy problems, experimenting with AI, or figuring out how to make data more human, Context & Chaos is your place to learn, reflect, and connect.
Got something on your mind? We’d love to hear from you.
Share this article