For most of my career, the ontology problem I saw ran in the opposite direction: the model came first, and reconciling it with real data came later, if at all.
I came to data from library and information science, and what I kept meeting were models built carefully in the abstract that nobody could reconcile with a real database. You built the model. Then you reached the physical tables underneath and found the two did not connect, and the model sat there being correct and useless. An ontology describes how a business is supposed to work. The data records how it actually ran.
That is part of why, when we began exploring whether a business ontology could be induced from a company’s own data, we chose to work strictly bottom-up. No hand-authored schema, no seeded set of business concepts. Everything had to come from systems the business already used. The bigger reason was that we were designing for agents to consume the output first and people second. Nobody consulted a standard or brought in a specialist on the way in.
We over-corrected. Bottom-up answered the question of where definitions should come from. We treated it as an answer to how they should be represented too.
The design records where every business definition came from and how strongly the evidence supports it. It has no way to record when that definition was valid.
What follows is a review of a design, not an account of a working system.
The failure we were trying to correct
Permalink to “The failure we were trying to correct”The bottom-up choice makes more sense once you see what the system was built to prevent.
Ask an agent what a team’s win rate was last quarter and it produces one answer with complete confidence. Inside a company there is usually no single definition to retrieve. One team computes win rate as closed-won over all closed opportunities. Another excludes deals that never reached qualification. A third counts only new-business pipeline. The definitions sit in SQL analysts wrote, in dashboard formulas, and in a deal-desk playbook, and they disagree without any of them being obviously wrong. A model that cannot see the disagreement picks one reading and states it as fact.
So the goal was narrow. Mine the definitions already in use, keep each one attached to its source, score the evidence behind it, and where sources disagree, show the disagreement instead of quietly reconciling it. Contested cases go to a person who can approve, change, split, or reject them. Lineage at the column level fills in what a mined query cannot tell you on its own, which is the specific columns a formula runs on.
What the experiment actually produced
Permalink to “What the experiment actually produced”That was the design we set out to build. Some of it runs today, and the parts that were supposed to close the loop do not.
We pointed it at four of our own systems: a cloud warehouse, a CRM, a BI tool, and our own metadata catalog. The exercise yielded 32,237 claims, reconciled into 14 entities, 79 metrics, 61 properties, and 35 relationships.
The output lands in two separate places, not one. There is an exploration tool, where the candidate ontology takes shape: entities, their relationships, and the metrics tied to each, browsable as a graph. And there is a plainer destination, where results drop into the same catalog the claims were mined from, and anyone can edit them there like any other entry, adding what is missing or removing what is wrong. Those two are not the same thing yet. The exploration tool is closer to what we want the destination to feel like eventually, and getting there means learning from real use what people actually remove, edit, and find useful, before we build a polished review surface around it.
It has not been implemented or tested as a system, and we are still well inside the design phase, the very squiggly part of it. We are still experimenting with what we can do, and actively trying to break our own initial hypothesis. It is early enough to throw any of this out.
Three old arguments already in the design
Permalink to “Three old arguments already in the design”Ontology engineering has spent decades on a set of questions that are still unsettled. The design was never checked against them. Reading it back against them afterwards, four turned out to be relevant, and the mapping is uncomfortable. Kurt Cagle recently made the broader case in this newsletter that AI teams are rediscovering these fault lines without knowing the history behind them.
Open and closed worlds. Open-world systems let knowledge stay incomplete and let multiple descriptions coexist. Closed-world systems work inside a bounded model for a particular task. Ours keeps competing claims during discovery and then narrows what can be served through human review, which is not a formal implementation of either. What it does is walk into the governance question underneath both: when several definitions are defensible, who decides which one an agent may use? The design settles that question without drawing on the literature behind it.
Reification. Reification means treating a claim as an entity in its own right, so you can make further statements about it. The easy mistake is to describe what reification is useful for, knowing who asserted something and how much to trust it, and call that reification.
Our claim records are closer in spirit to the RDF-star work now reflected in RDF 1.2, where a reifier lets further statements be attached to a triple. The standard’s own illustration of what you would attach to a statement is its source, its confidence level, and a timestamp. We arrived at a similar mechanism without using the standard, and we took the source and the confidence.
These are two separate mechanisms in the design, not two descriptions of one. Open and closed worlds arrived as a governance problem: which of several defensible definitions may be served. Reification arrived through the need to attach a source and a confidence score to a claim.
Partitioning and authority. Multi-tenancy, access, provenance, and authority all need boundaries. Our design says what is in scope and logs what falls outside it. That is a boundary, and it carries no source authority, no access rules, and no query-time visibility. None of those exist. It resembles part of what a named graph does rather than amounting to one. Named graphs are one part of the approach we are now considering. They can give addressable boundaries between sets of assertions, but we would still have to define what those boundaries mean for authority and access.
Each of those problems had at least a rough counterpart in the design. The fourth did not.
The missing dimension was time
Permalink to “The missing dimension was time”Every claim carries a source. None carries a validity period.
Knowledge graphs often describe either one moment or a set of assertions treated as permanently true, and retrieval systems inherit the same weakness. The business changes while the retrieved definition goes on looking current.
Our design has no validity intervals, no effective dates, no supersession. A definition of win rate is mined, scored, reviewed, and served, and nothing anywhere records that it applied during a particular period.
Detecting a change is not the same as describing the period before it. | Source: Context and Chaos
Two things are easy to confuse here. On a later run the system may notice that a source has changed. What it cannot do is represent the interval during which the previous definition was the right one. Detecting a change is not the same as describing the period before it. Without that, the system cannot answer a question about last quarter without silently assuming that today’s definition also held then.
Reviewer feedback could make it worse. The design adjusts scoring weights from earlier decisions, so if an approved definition can expire but its approval cannot, judgments about a past state of the business would keep steering the current one.
The fix may be smaller than it sounds. Our claims already carry evidence and confidence, and getting them to carry a validity date too may not need a new mechanism, just two dates and a rule for who sets them.
What we do not know yet is whether we have built a machine that knows when to stop trusting its own past judgments. We had not asked our own design that question until now.
Did we build an ontology at all?
Permalink to “Did we build an ontology at all?”Time was the clearest omission and not the only one. Once we set the output beside what ontology standards actually require, a more basic question surfaced: had we built an ontology, or a well-documented model of our own data?
After extraction, the system pulls variant names together under one canonical entity. In library and information science, that resembles part of authority control, a discipline concerned with established names, their variants, and the evidence behind those choices. An authority record holds the established form of a name, the variants pointing to it, and the source and date that established it. We have the first two. The third, which is the part carrying provenance and time, is missing.
The standards sitting unused on our roadmap come from the same tradition. SKOS handles concepts, hierarchies and labels, and its distinction between a preferred label and an alternate label resembles part of the merge logic we derived. BARTOC is a registry of thesauri, ontologies and classifications that catalogers have used for years, not a discovery of ours. Dublin Core is a general vocabulary for describing resources rather than a temporal model, but creator and date are among its original fifteen elements. Provenance and time were not new requirements waiting for us to notice them.
The harder criticism is one we should make ourselves. An ontology, in the sense the field means it, carries axioms and constraints: statements about what must hold, which a reasoner can check and which fail when violated instead of passing quietly. OWL exists to express them. SHACL exists to validate against them. Our output has neither. It has entities, metrics, properties and relationships, each documented and traceable to a source, and nothing that enforces a single rule about any of them.
Nothing in the output enforces a single rule about any of it. | Source: Context and Chaos
So the honest description of what we produced may be something smaller: a careful account of our tables, the joins between them, and the metrics computed on top. That is a genuinely useful artifact, and a lesser claim than the word ontology makes. Whether the gap matters depends on what you meant to do with the thing. If the intention was for agents to reason over it, it matters a great deal.
We have shown the output to people outside the team. Some said it was a reasonable place to start, something you would edit and build from. One ontologist told me flatly that we should not be calling it an ontology. I did not disagree.
The question underneath is whether people want an ontology because they have something to do with it, or because everybody has decided they should have one. An artifact built for the second reason will not survive the first.
What should bottom-up inherit?
Permalink to “What should bottom-up inherit?”The bottom-up commitment still was not the error. No inherited ontology was going to tell us how a particular deal desk computes win rate, and that meaning has to come from the organization. The error was letting the first answer settle the second.
Calling the output something smaller does not make the bottom-up choice wrong. The hardest part is not whether to inherit form. It is how much to inherit before the inherited model starts deciding what the system is allowed to find.
The question I get asked most often, by people who have done this work for years, is how you know when something should be an entity and when it should be a metric or a property. It sounds like a modeling detail and it is the inheritance problem in miniature: adopt an industry model that treats win rate as a metric hanging off an opportunity, and you have decided something about a business that may treat it as an object in its own right, with an owner and a revision history. Inheriting the form quietly inherits that judgment too.
Discover meaning locally. Do not reinvent how meaning is represented. | Source: Context and Chaos
So, how should a system keep local business meaning while adopting established ways to represent time, authority, provenance and change? Which parts can be safely inherited, and which have to emerge from an organization’s own evidence?
If you have built or maintained an ontology in practice, I would like to know where this distinction breaks down. I would rather find the next omission in a design than after an agent has relied on it.
The Cats of Context & Chaos
Permalink to “The Cats of Context & Chaos”
Great. When was this true? | Source: Context and Chaos
About Context & Chaos
Permalink to “About Context & Chaos”Context & Chaos isn’t just a newsletter. It’s shared community space where practitioners, builders, and thinkers come together to share stories, lessons, and ideas about what truly matters in the world of data and AI: context engineering, governance, architecture, discovery, and the human side of doing meaningful work.
Our goal is simple, to create a space that cuts through the noise and celebrates the people behind the amazing things that are happening in the data & AI domain.
Whether you’re solving messy problems, experimenting with AI, or figuring out how to make data more human, Context & Chaos is your place to learn, reflect, and connect.
Got something on your mind? We’d love to hear from you.
Share this article