I opened the Context Conference by saying that context is under-hyped. I know that is a slightly ridiculous thing to say in a year when the word shows up in every AI conversation. I still believe it. Most of that conversation stays at the level of the word, but the questions that enterprise teams bring us have moved well past it.
When we put the keynote together, we collected eighteen of those questions. Here are four:
- Who owns the context in my company?
- How do I version a skill?
- Will it work with the next model?
- How do I trace a wrong answer back to its context?
The questions teams are asking about context | Source: Context Conference keynote.
Those questions get harder when the same context has to serve people across different tools. Every CIO I meet tells me some version of the same story.
“I know I will have dozens of platforms and hundreds of agents,” the CIO of a global top-10 pharma company said.
A head of data and AI at another pharma company had already deployed four agent platforms, and they “now all speak slightly different versions of the truth.”
Enterprise AI is multi-harness, multi-model, and multiplayer. You can already see the first two. People move between Claude, ChatGPT, Copilot, and the AI inside their business apps, and companies pick different models for different jobs. A harness is everything around the model: its instructions, its tools, and the machinery it works through.
Multiplayer is the part that deserves more attention. An assistant can learn a lot about one person without any of it reaching the people working alongside them. My assistant’s memory is still my memory. That works for one person. But companies run on teams, so what my assistant learns has to become context the whole team can build on.
Enterprise AI across tools, models, and teams | Source: Context Conference keynote.
And that team doesn’t work in one tool or on one model. For everyone to build on the same context, it has to be portable. It has to work in every tool a team uses, stay current for everyone who relies on it, and outlast whichever platform a company picks this year. The CIO of a global automaker put it plainly: “I don’t want my business IP locked into any one platform.”
Data had to solve portability too, and it started with open file formats. Context has reached that same first step.
Context already has open formats
In the last year, context has picked up its own open formats. Agent Skills packages procedures. AGENTS.md briefs coding agents. Google’s Open Knowledge Format covers knowledge, and Apache Ossie covers semantics. Each one lets more than one tool read a particular kind of context.
Formats let many tools read the same files. They don’t tell those tools which files belong together or which version is current. Data hit that same wall, and Iceberg is how it got past it.
What Iceberg did for data
Before Iceberg, running analytics on data you had processed elsewhere usually meant copying it. You could run ETL with Spark on files in object storage, but reporting meant loading that data into a warehouse, where it sat in proprietary storage no other engine could reach.
Parquet and ORC gave us open file formats that different engines could read. But a folder of readable files can’t tell you which files make up a table, what its schema is, or which snapshot to read.
Apache Iceberg’s table spec answers those questions with shared metadata, including versions and snapshots. Its REST catalog spec gives every engine one way to see which tables exist and how to read and write them safely. Spark, DuckDB, and Snowflake can work on the same copy of the data, in storage the company owns. That is what made the open lakehouse possible.
The layer context is missing
We have the open files for context. We don’t have the layer above them.
What Iceberg added for data, and what context still lacks | Source: Context Conference keynote.
Ask what a tool would need to know about a skill. Which version is current? Who owns it? Has it been retired? How does every harness on the team get the new one? The raw formats don’t answer that in any standard way.
Here is how that plays out. Many teams already keep their skills in a Git repo, which gives them one source. It doesn’t get the skill into everyone’s tools. When someone updates the team’s PR review skill, they post in Slack, and everyone downloads the new version into each harness they use. Imagine two colleagues install it a week apart. One makes a local edit. Another tool still has the copy from months ago. Each of them would say they have the review skill, and they’re running different material.
This setup is common. Ben Hudson, who leads product for our agentic interfaces and open source work, found that none of the teams his group talked to had solved distribution. One team keeps its skills in SharePoint and rolls out updates by email and Slack.
In the keynote I split this gap into two problems, and we’ve seen both before.
The first is distribution. Context has to reach every teammate and every agent, in every tool they use. npm solved this for code. You publish once, everyone can install the exact same version, and a bad release can be pulled.
The second is interoperability. Context has to work across harnesses, models, and agent frameworks, and tools need one way to read any registry that holds it. That is what Iceberg did for data: one open spec, on storage you own, that any engine can read and write.
Two problems, two precedents | Source: Ben Hudson's lightning talk, Context Conference.
Context needs both.
An open spec for registries
In the keynote, I announced a first draft of an open spec for that layer: ai-registry, under Apache 2.0. A registry under the spec is just files on storage a company already has, such as a Git repo or an object storage bucket. The spec sets how those files are laid out and adds what the raw formats leave out: versions, digests, and status.
The procedure keeps its own format. A SKILL.md file still holds instructions. To borrow the data comparison, skill files are the Parquet files, and the spec plays Iceberg’s role: it lets every tool agree on them. A tool builds one integration and can read any registry that follows the spec, then fetch and verify a specific version.
One open format and file layout for every registry | Source: Ben Hudson's lightning talk, Context Conference.
Back to the PR review skill. With the repo set up as a registry, an update is a merged pull request. Tools that read the registry can pick up the new version without anyone posting in Slack.
It is early. The spec covers skills and agents today, with MCP servers and metrics through Apache Ossie planned. Ben’s session walks through it.
What a spec can’t do
The analogy has a limit. A spec can’t approve a procedure or decide whether an exception becomes policy.
What a version does give you is a place to start. When an agent starts acting differently after a model upgrade, the first question is whether the context changed too. Without an identifiable version, that comparison gets much harder.
Getting context to every agent is half the job. The other half is knowing whether agents use it well, whether a change was right, and what a user’s correction means. That takes three practices: checking how agents use context, controlling changes, and using feedback to improve it.
Check that agents use the context
You can write the right skill, review it, publish it where the agent can reach it, and the agent may still never open it.
SkillsBench 1.1 tracked whether agents invoked the task-specific skills available to them. Across 18 combinations of model and harness, invocation ranged from 46% to 99%.
Liu and colleagues looked at selection more closely. With the curated skills available, Claude Opus 4.6 loaded all of them in 49% of runs. In the other half, at least one stayed unopened. Add distracting skills, and that fell to 31%. Task pass rates fell from 51.2% to 43.5%. The right skills were still there. Picking among them was part of what broke.
Then Vercel tested delivery for its Next.js docs. Offered as a skill the agent could call on demand, the docs produced a 53% task pass rate, the same as the baseline with no docs. Telling the agent to use the skill raised it to 79%. A compressed docs index embedded in AGENTS.md reached 100% on that evaluation.
The first two rows measure skill invocation or loading; the Vercel row shows task pass rates | Sources: SkillsBench 1.1, Liu et al., Vercel.
Think about what a company does before an agent even gets the chance to skip a skill. Someone sits with the person who knows a procedure, checks the rules, and writes the instructions down. You paid to get that knowledge out of someone’s head. Few teams check whether it reaches the work.
These studies measure different things, so I won’t pretend they add up to a rule about formats. They do give us three questions to ask of any run: did the agent load the right material, did it follow it, and did it finish the task? If you only check the last one, you can’t tell where a run went wrong.
How you package context matters too. A skill describes how to do a piece of work. A certified metric pins down meaning and calculation. An ontology describes relationships. Each should take the shape of what the agent has to do with it.
Test the artifact where it will run. A skill can pass careful human review and still fail, because its description doesn’t tell the agent when to reach for it, or because the rule that matters is buried among unrelated instructions.
Include cases that fail when the important instruction is ignored, and inspect whether the agent used the artifact. Rerun them when the model or harness changes. Otherwise a plausible answer can hide the fact that the agent never used the artifact.
When the artifact needs changing, that fix has to reach users without breaking the work that depends on it.
Change context the way you change code
A registry settles which release people get. What goes into a release is still a review decision. I’ve argued before that context needs to be managed like code. For the person maintaining an artifact, that creates a few responsibilities. They need an identifiable release and a review process. They need to know which work a change might affect, and a way to pull a release that causes problems.
Ownership, versions, dependencies, testing, and release lifecycle | Source: Context Conference keynote.
Say finance revises a metric definition. The change should have a named owner and reviewers who understand the calculation. Tests should cover representative questions, including the ones where the new definition changes the answer. Consumers need to know which release they’re on.
A text diff shows reviewers the edit. They also need to see what it means. Did a calculation change? Did a supported breakdown disappear? Does an exception now apply more widely?
Finance and operations may measure revenue differently, and both measures can be valid. A procedure may apply to one region only. Sharing context across a company has to keep those differences and help the agent pick the one that applies.
Match the machinery to the stakes. A small team can start with plain files, owners, and tests. Once context crosses teams and drives autonomous actions, an unreviewed change gets much harder to contain.
Review controls what changes. Everyday use tells you what needs changing, often through a user telling an agent it got something wrong.
Route corrections to the owner
Suppose a user asks an agent for Q3 net revenue retention. The agent says 112%, using version three of the metric definition. The user replies, “I wanted it by segment.”
That correction needs interpreting. Maybe the metric has no segment dimension. Maybe it does and the agent ignored it. Or maybe this user wanted a view for one analysis only.
Picture yourself as the metric owner in RevOps. Version three has no segment dimension, and similar requests show up in other runs. You check whether a segment-level calculation is valid, test a revision, and approve version four for the agents that need it.
If the agent had ignored a dimension that already existed, you’d have ended up somewhere else. Adding more to the metric wouldn’t have fixed that.
To tell those cases apart, the owner has to see three things together: who did the work, which artifacts they relied on, and what actually happened. I think of these as actors, artifacts, and activity, and the value is in connecting them.
Actors, artifacts, and activity connected | Source: Context Conference keynote.
The trace shows what the agent used. The user supplies the feedback. The owner decides whether it deserves a shared change and checks the release. A pile of artifacts tells you what knowledge exists. Connecting it to activity tells the owner how that knowledge is being used.
One person’s correction is a signal. Whether it becomes company guidance is the owner’s call. An exception that worked for one customer may need to stay with that customer.
This has a cost too. Send every conversation to every owner and you’ve built another backlog. Owners need patterns tied to their artifacts, with the evidence underneath when they go looking.
The payoff comes when a reviewed fix reaches the next person doing that work. That is where the registry comes back in. Version four ships as a release, and tools that read the registry can pick it up.
What owning context takes
Registries, tests, reviews, and feedback loops are the machinery. What they protect is how your company does business: its definitions, its procedures, its exceptions, and its judgment about its customers.
That knowledge needs a life beyond any one agent. When you replace a model or a platform, you should keep the artifacts and their owners, the release history, and the evidence of how they were used. Keep the evaluations that tell you whether they still work in the new setup, and a way to pull a bad release.
That puts a lot on the people who maintain context. Whoever owns the definitions, catalogs, ontologies, and playbooks is shaping what agents rely on when they act. Someone has to judge what belongs there and stay accountable for keeping it current.
You can start with one recurring agent task. Pick a recent run and follow it back to the context it used. Which artifact versions did it touch? Did it load the material that mattered, and did the answer follow it? Then ask who would approve a fix, and how a teammate on another tool would get it.
Models will keep getting better. You and your biggest competitor can rent the same one tomorrow, at the same price. What you can’t rent is what makes your company yours.
Rent the intelligence. Own the context.
Rent the intelligence. Own the context. | Source: Context Conference keynote.
The keynote deck
Here is the Context Conference deck, which you can take into your own organization. Swipe through the 32 slides below, or download the PDF.
Download the deck (PDF, 6.9 MB)
Watch the full Context Conference keynote.
The Cats of Context & Chaos
Of course. All three of them. | Source: Context and Chaos
About Context & Chaos
Context & Chaos isn’t just a newsletter. It’s shared community space where practitioners, builders, and thinkers come together to share stories, lessons, and ideas about what truly matters in the world of data and AI: context.
Our goal is simple, to create a space that cuts through the noise and celebrates the people behind the amazing things that are happening in the data & AI domain.
Whether you’re solving messy problems, experimenting with AI, or figuring out how to make data more human, Context & Chaos is your place to learn, reflect, and connect.
Got something on your mind? We’d love to hear from you.
Share this article