--- title: "How to Reduce Tribal Knowledge in Engineering Teams" url: "https://atlan.com/know/ai-agent/reduce-tribal-knowledge-engineering-org/" description: "Reduce tribal knowledge in engineering teams by mining lineage, SQL history, and usage patterns into context that new engineers and AI agents can both use." author: "Karthik Pasupathy" author_role: "Contributing Writer — AI Context & Agents" published: "2026-08-31" updated: "2026-08-31T00:00:00.000Z" --- --- Atlan turns that evidence into a working answer instead of another wiki page nobody updates. **Context Agents** mine lineage, SQL history, and usage patterns to draft the definitions and READMEs no one wrote, domain experts certify what the agents get wrong, and the result ships to new engineers and AI agents through the same [Enterprise Data Graph](https://atlan.com/know/enterprise-data-graph/) that backs the wider [enterprise context layer](https://atlan.com/know/context-layer-enterprise-ai/). That loop, not a one-time interview with your most senior engineer, is what actually closes the gap. --- ## Why is tribal knowledge a context problem, not a training problem? Every enterprise has tribal knowledge scattered across its processes and systems. Some of it is documented in schemas, tickets, and process guides. Some is encoded in SQL, lineage, usage patterns, and incident histories. The rest lives in people's heads, including the latest definition, the exception to a standard process, or the reason an earlier technical decision was made. Automation reduces the amount of knowledge people must carry, but it does not eliminate these hidden pockets. When an experienced engineer leaves or changes teams, the organization may retain the code and documentation while losing the reasoning required to use them correctly. This is how [tribal knowledge](https://atlan.com/know/data-for-ai/tribal-knowledge/) becomes [institutional knowledge loss](https://atlan.com/know/data-for-ai/institutional-knowledge-loss/). AI agents face the same problem. An agent may be trained to read schemas or follow documented procedures, but it cannot use context it cannot access. Training can improve how a model interprets the information available to it. It cannot recover a business rule that was never captured, determine which of several definitions is current, or infer an exception known only to one engineer. Atlan treats this as a context problem. A [context layer for data engineering teams](https://atlan.com/know/context-layer-for-data-engineering-teams/) connects definitions, relationships, ownership, rules, and history to the systems they describe. Teams can mine signals already present in their systems, then ask domain experts to resolve ambiguities and approve the resulting context, rather than reconstructing everything from memory. The same foundation supports [context layers for the software development lifecycle](https://atlan.com/know/ai-agent/context-layer-for-sdlc/) and [for AI agents](https://atlan.com/know/agent-context-layer/). In practice, [a context layer](https://atlan.com/know/what-is-context-layer/) should help an engineer or agent understand an asset, its dependencies, decisions, and exceptions without needing to find the original author. That missing context does not create only one kind of failure. Across engineering teams, it usually becomes visible through undocumented systems, dependence on a single expert, and slow onboarding. --- ## How does tribal knowledge show up in engineering teams? Tribal knowledge becomes visible when everyday engineering work slows down: a schema is hard to change, a routine decision waits for one person, or a new hire finds the documentation but still doesn't know which definition or exception to follow. It shows up in three recurring patterns. | Pattern | What it looks like | Context that is missing | | :---- | :---- | :---- | | Undocumented schemas | Engineers can see the fields but disagree about their meaning, safe joins, or whether a metric is still current | Definitions, lineage, ownership, change history, and approved usage | | "Ask the one person" routing | Incidents, reviews, migrations, and production changes wait for the same expert | Decision traces, dependencies, exceptions, and escalation paths | | Onboarding drag | New engineers can find the code and documentation but repeat questions or avoid unfamiliar systems | Searchable examples, process context, current guidance, and a trusted owner | These patterns reinforce one another. When a schema doesn't carry enough context, engineers route questions to the person who understands it. If the answer stays in a private conversation or meeting, the system stays undocumented, and the next person asks the same question. When answers stay with one expert, new hires can't learn independently: they wait for that person to explain systems, confirm exceptions, and review decisions, which slows onboarding and creates operational risk when the expert is unavailable. Once this cycle becomes routine, its cost shows up in delayed work, repeated investigation, fragile systems, and knowledge that leaves with the people who hold it. --- ## What does tribal knowledge actually cost? Tribal knowledge costs a team whether an expert is available or absent. Engineers lose time searching, waiting, and repeating investigations. If the expert leaves, the team has to reconstruct decisions and exceptions from incomplete evidence. Workforce movement makes that dependency hard to sustain. According to USAFacts (2024), [median employee tenure fell](https://usafacts.org/articles/how-long-do-americans-stay-at-their-jobs/) from 4.6 years in 2014 to 3.9 years in 2024, a 15% drop. According to Deloitte Insights, [92% of surveyed organizations](https://www.deloitte.com/us/en/insights/topics/talent/knowledge-management-plan.html) do not consistently capture knowledge from employees who are soon to retire. Neither figure predicts a particular departure, but together they explain why teams can't depend on permanent access to the same experts. The cost also shows up in everyday work. According to a 2025 Harris Poll study for GitLab, [inefficient processes consumed 7 hours per week](https://about.gitlab.com/press/releases/2025-11-10-gitlab-survey-reveals-the-ai-paradox/) for surveyed DevSecOps professionals. Limited knowledge sharing was one of several named barriers alongside fragmented toolchains, so tribal knowledge shouldn't be blamed for all seven hours. Manav Khurana, GitLab's Chief Product and Marketing Officer, summarized the wider problem: "coding is faster than ever, yet the lack of quality, security, and speed across the software lifecycle is causing friction." Technical debt is another symptom. According to Stack Overflow's 2024 Developer Survey, [62% of respondents selected](https://stackoverflow.blog/2025/01/01/developers-want-more-more-more-the-2024-results-from-stack-overflow-s-annual-developer-survey/) technical debt as a top workplace frustration, roughly twice the rate of the next most common complaint. When [systems of record, data, and knowledge](https://atlan.com/know/ai-agent/data-for-ai/systems-of-record-data-knowledge/) each hold a different part of the answer, teams repeatedly rediscover why workarounds and constraints exist. This recurring cost often leads to a documentation sprint, but writing after the fact doesn't guarantee that hidden knowledge surfaces or stays current. --- ## Why don't documentation sprints solve the tribal knowledge problem? Documentation gives engineers a shared place to find system details, processes, and decisions without asking an expert every time. It can cut repeated questions, support onboarding, and preserve knowledge that might otherwise be lost. But documentation has limits. It can't capture every unwritten rule or keep itself current as systems change. According to [DORA's documentation-quality research](https://dora.dev/capabilities/documentation-quality/), strong documentation doesn't just help on its own, it amplifies the return on every other technical capability the research studied, from trunk-based development to continuous integration. The limitation is the sprint model, not documentation itself. A one-off sprint asks experts to recall decisions, assumptions, and exceptions after the work is done. It produces a snapshot that can quickly drift from the system as it changes, leaving four gaps behind: * **Recall gaps:** Experts omit habits and exceptions that feel obvious to them. * **Evidence gaps:** A document records a conclusion without the lineage, SQL, or usage history that supports it. * **Freshness gaps:** Code, schemas, and processes keep changing after the documentation is published. * **Retrieval gaps:** An [internal knowledge assistant](https://atlan.com/know/ai-agent/data-for-ai/what-is-an-internal-knowledge-assistant/) can only retrieve what has actually been entered into its corpus. The better goal is to treat documents as one output of a broader [knowledge architecture for AI agents](https://atlan.com/know/ai-agent/data-for-ai/knowledge-architecture-for-ai-agents/) and people, part of the wider discipline of [context engineering](https://atlan.com/know/what-is-context-engineering/). [A knowledge base for AI agents](https://atlan.com/know/ai-agent/data-for-ai/how-to-build-knowledge-base-for-ai-agents/) can make approved answers searchable, but each answer still needs supporting evidence, a clear owner, and a way to stay current. A [business context layer](https://atlan.com/know/business-context-layer/) can also capture the nontechnical rules that explain why an engineering system behaves the way it does. --- ## How can teams mine context from systems that already have it? Teams can start with evidence generated during everyday engineering work instead of asking experts to reconstruct everything from memory. SQL history exposes transformations, [data lineage for AI](https://atlan.com/know/ai-agent/data-for-ai/data-lineage-for-ai/) traces dependencies, and usage, incidents, tickets, and change records reveal important assets, failures, and exceptions. Together, these signals create decision traces showing what changed, why it changed, and who approved it. Teams can turn that evidence into shared context in four steps. ### 1. Prioritize recurring questions Start with questions that require an expert and delay production work, incident response, or onboarding, rather than trying to document everything at once. ### 2. Collect the evidence Review lineage, SQL, usage, incidents, and change records for dependencies, transformations, and exceptions. ### 3. Draft the context Convert the evidence into definitions, rules, dependencies, exceptions, and decision traces. A [knowledge graph for AI agents](https://atlan.com/know/ai-agent/knowledge-graph/how-to-build-a-knowledge-graph-for-ai-agents/) can connect them across systems. ### 4. Validate and maintain it Ask domain experts to correct and approve the result, then assign an owner who keeps it current. This is [context mining](https://atlan.com/know/ai-agent/context-engineering/context-mining-for-ai/): collating organizational context from existing signals instead of starting from a blank page. Human review stays necessary because system evidence shows what happens, but not always why a business rule exists. A [semantic layer for AI agents](https://atlan.com/know/ai-agent/semantic-layer-for-ai-agents/) standardizes metrics and business terms, while a context layer adds lineage, ownership, exceptions, and decision history on top. This work is [context engineering for AI agents](https://atlan.com/know/context-engineering-for-ai-agents/), which is different from prompt engineering: [changing a prompt cannot supply missing context](https://atlan.com/know/context-engineering-vs-prompt-engineering/), while [context engineers](https://atlan.com/know/what-does-context-engineer-do/) coordinate evidence gathering, expert review, and maintenance. Technical signals explain systems, but teams also have to capture how work moves between people, decisions, and exceptions. --- ## How do teams capture how work gets done? System signals capture technical context, but teams also need process knowledge: the steps, decisions, exceptions, and approvals that shape everyday work. Teams can document this knowledge for people and package it as [enterprise skills](https://atlan.com/know/what-are-enterprise-skills/) for AI agents. The enterprise skill should capture three kinds of knowledge: * **Incidents and support:** Record diagnostic steps, when to involve another team, and recovery exceptions. * **Key process decisions:** Preserve accepted and rejected options, exceptions, and limitations. * **Repeated workflows and corrections:** Capture required checks, decision points, and rules revealed through corrections. Each procedure needs an owner, supporting evidence, approval history, and version history. This isn't a one-time write-up: the [context development lifecycle](https://atlan.com/know/ai-agent/context-engineering/context-development-lifecycle-for-ai-data/) keeps it current as processes change, and [context drift](https://atlan.com/know/context-drift-detection/) is what happens when nobody owns that upkeep, creating a shared source engineers and AI agents can both trust. --- ## How does Atlan turn team knowledge into shared context? Atlan turns tribal knowledge into shared context by bringing together lineage, query history, usage, BI activity, and existing business definitions. Context Agents use those signals to generate missing descriptions, READMEs, and SQL intelligence, in a first pass Atlan calls [context bootstrapping](https://atlan.com/know/context-bootstrapping/), while domain experts review and refine the output. The result is reviewed, versioned context that teams can update as systems and processes change. The table maps each engineering need to the Atlan capability that addresses it. | What engineering teams need | Atlan capability | Practical effect | | :---- | :---- | :---- | | One connected view of schemas, lineage, usage, definitions, ownership, and dependencies | [Enterprise Data Graph](https://atlan.com/know/enterprise-data-graph/) and Data Lineage | Engineers can trace where an asset came from, who owns it, and what a change may affect without reconstructing the answer across several tools | | A first draft of missing definitions and documentation based on existing system evidence | [Context Agents](https://atlan.com/know/context-agents/), including Scout, Scribe, Lexis, and Doc | Scout identifies important assets, while Scribe, Lexis, and Doc use available context to draft descriptions, glossary terms, and READMEs for expert review | | A way for experts to resolve ambiguity, test answers, approve changes, and preserve previous versions | **[Context Engineering Studio](https://atlan.com/context-engineering-studio/)** and Context Repos | Experts can refine definitions and exceptions, test them against real questions, and store approved context in a bounded, versioned repository | | Reusable representations of domains, relationships, business logic, and operating procedures | Atlas and Orion Context Agents, plus Enterprise Skills | Teams can organize business meaning, and package required checks, exceptions, decision paths, and approval steps as reusable context | Together, the Enterprise Data Graph supplies evidence, Context Agents draft context, and experts refine it in Context Engineering Studio. Context Repos preserve approved versions for reuse. This is how [the enterprise context layer](https://atlan.com/know/what-is-the-enterprise-context-layer/) cuts reliance on individual experts without removing their judgment. Experts can stop repeating established answers and focus on resolving ambiguity, approving exceptions, and maintaining knowledge. --- ## How does shared context help new engineers and AI agents? Shared context helps new engineers and AI agents act without reconstructing meaning from scattered systems and private conversations. It gives both access to current definitions, relationships, decision traces, and exceptions, but the benefit shows up differently for each. New engineers can trace where data came from, understand why a system works the way it does, and identify the correct owner or exception without waiting for an expert to explain the basics. AI agents can use approved definitions, relationships, and operating rules instead of inferring meaning from names or incomplete documentation; when the required context is missing, the workflow can route the question to a human rather than letting the agent make an unsupported assumption. Shared context doesn't mean giving every person and agent identical access. It means serving the same approved meaning within the permissions and requirements of each task. Atlan's Enterprise Data Graph makes this concrete: the same shared definitions and lineage that help a new engineer trace an asset also become the vocabulary an AI agent reads through Atlan's **MCP server**, instead of guessing at names it has never seen. If every team or agent maintains a separate version instead, the organization just replaces human tribal knowledge with [context silos across AI teams](https://atlan.com/know/enterprise-context-silos-ai-teams/). Reducing tribal knowledge can improve agent accuracy by removing ambiguity about which asset, definition, or rule applies. It doesn't guarantee a correct result: agents still need reliable retrieval, appropriate permissions, clear instructions, and human review for high-risk actions. New engineers will still need experienced colleagues, but those experts can focus on new or ambiguous problems instead of repeatedly explaining established knowledge. --- ## Where should your team start? Start by treating tribal knowledge as missing context, not a documentation backlog. Identify the systems, decisions, and workflows that depend on one or two experts, then collect evidence from lineage, SQL, usage patterns, incidents, and change history instead of asking those experts to reconstruct everything from memory. A focused pilot brings these steps together around four checks: * **Recurring questions:** Select ten questions that currently require an expert or repeatedly delay engineering work. * **Supporting evidence:** Gather the lineage, SQL, usage, incident, and change records that help answer them. * **Expert review:** Assign domain experts to correct assumptions and approve definitions, exceptions, and operating rules. * **Measurable outcomes:** Track fewer expert escalations, faster answers, easier onboarding, and safer AI agent outputs. Store the approved context with clear ownership and version history, then make it available to the engineers and AI agents that need it. Once the pilot produces answers the team trusts, expand the same process to the next system or workflow with a high dependency on individual knowledge, following the same rollout [how to implement an enterprise context layer for AI](https://atlan.com/know/how-to-implement-enterprise-context-layer-for-ai/) walks through in more detail. Book a Demo --- ## FAQs about reducing tribal knowledge in engineering teams ### 1. What is tribal knowledge, and why does it accumulate specifically in engineering teams? Tribal knowledge is the undocumented context engineers need to operate systems safely, including exceptions, decision history, and practical workarounds. It accumulates because engineering systems change continuously while explanations stay scattered across tickets, meetings, reviews, and private conversations. Experienced engineers learn these rules through repeated work, but that knowledge may never enter a shared system. ### 2. What does tribal knowledge actually cost when an engineer leaves or is out? When an expert is unavailable, the cost shows up as slower incident response, repeated investigation, delayed changes, and interruptions for other engineers. A temporary absence exposes the dependency, while a departure can turn it into a longer-term gap the team has to reconstruct. Measure blocked work, repeated questions, and expert escalations rather than relying on a generic replacement-cost formula. ### 3. How is tribal knowledge different from a documentation gap or technical debt? A documentation gap means information is missing from written material. Tribal knowledge is broader because the missing context may never have been clearly articulated, even by the expert who uses it. Technical debt concerns choices that make systems harder to change, and undocumented decisions and workarounds can make that debt harder to understand. ### 4. Can you capture tribal knowledge without a dedicated documentation sprint? Yes. Teams can use lineage, SQL history, usage patterns, incidents, code reviews, and workflow records as evidence, then ask experts to correct and approve the resulting context. This does not eliminate documentation, but it makes documentation a maintained output of everyday work instead of a separate exercise based entirely on memory. ### 5. How do AI agents get affected by the same undocumented knowledge that slows new engineers? An AI agent can use only the schemas, tools, instructions, and retrieved context available to it. Without organizational definitions and exceptions, it may choose the wrong table, filter, join, or operating rule, yet still produce a technically plausible answer. Reviewed context gives the agent access to meaning a new engineer might otherwise learn by asking an experienced colleague. ### 6. How does data lineage reduce dependency on "the one person who knows"? Data lineage shows how data assets are produced, transformed, and used across connected systems. It helps another engineer identify what feeds a table, which downstream assets depend on it, and what a proposed change may affect. Lineage does not explain every business decision or exception, but it makes expert review faster and more focused. ### 7. How long does new-engineer onboarding typically take, and what actually causes the delay? There is no universal onboarding duration because roles, systems, permissions, and team complexity vary. Common delays include environment setup, access approvals, fragmented documentation, complex processes, limited system visibility, and waiting for experts to explain local context. ### 8. Does reducing tribal knowledge make AI agents more accurate, not just faster onboarding? It can improve accuracy when missing or ambiguous context is causing the agent to select the wrong asset, definition, or rule. Better context does not guarantee a correct result because retrieval, permissions, instructions, tools, and model behavior also affect the output. Evaluate representative tasks before and after a context change, then test whether the agent actually retrieved and applied that context. --- ## Sources 1. [How Long Do Americans Stay at Their Jobs?, USAFacts](https://usafacts.org/articles/how-long-do-americans-stay-at-their-jobs/) 2. [Capturing Institutional Knowledge, Deloitte Insights](https://www.deloitte.com/us/en/insights/topics/talent/knowledge-management-plan.html) 3. [GitLab Survey Reveals the AI Paradox, GitLab and The Harris Poll](https://about.gitlab.com/press/releases/2025-11-10-gitlab-survey-reveals-the-ai-paradox/) 4. [How to Accelerate Developer Onboarding, GitLab](https://about.gitlab.com/the-source/platform/how-to-accelerate-developer-onboarding-and-why-it-matters/) 5. [Documentation Quality, DORA](https://dora.dev/capabilities/documentation-quality/) 6. [2024 Developer Survey Summary, Stack Overflow](https://stackoverflow.blog/2025/01/01/developers-want-more-more-more-the-2024-results-from-stack-overflow-s-annual-developer-survey/)