---
title: "Your AI Dashboard Is Hiding Its Worst Failures"
url: "https://atlan.com/context-and-chaos/issue/your-ai-dashboard-is-hiding-its-worst-failures/"
description: "The metrics that look green on the dashboard are the ones hiding where your system actually fails. The average conceals the experiences that matter most."
keywords: "AI Evaluation, Enterprise AI, AI Agents, Responsible AI"
---

> Atlan is hosting Context Conference, bringing together the leaders and builders at the frontier of giving AI the context it needs to understand their business. It runs online on October 28, 2026, from 11:00 AM to 2:00 PM ET. Atlan co-founder Prukalpa Sankar opens and closes the day. Leaders from AstraZeneca, BNY and Verizon share why they invest in context and what they get from it. Registrants get early access to The AI Context Gap, a new study from MIT Technology Review Insights. Register: https://atlan.com/context-conference/

A Context & Chaos contributor issue (Atlan's practitioner newsletter) by **Swati Tyagi, Applied AI/ML expert** working on responsible AI, LLMs and bias in production systems (knowledge graphs and explainable AI across healthcare and finance; earlier bias research at the University of Delaware). Published July 9, 2026, 8 min read. The metrics that look green on the dashboard are the ones hiding where your system actually fails; the average conceals the experiences that matter most.

Key points:

- A 92% success rate and a 61% success rate can describe the same AI system in the same week. Aggregate metrics compress millions of interactions into one number, and that compression is where failures hide.
- The most expensive failures live in segments nobody defined. Hidden stratification means a system can pass every declared breakdown yet consistently fail an intersectional cohort, like non-English queries about policy exceptions from first-time users.
- The competitive question is shifting from whose model is better to whose team knows where its systems fail, and who they fail. Treat subgroup evaluation as a first-class requirement.

## The green dashboard trap

The dashboard shows 92%; the customer hitting an exception experiences 61%. Accuracy, task completion and response times are all on target, yet complaints arrive: a failed policy exception, an incorrect escalation, a stuck workflow. The model is doing its job; the measurement system is not.

## The comfort of averages

Accuracy, precision, recall, F1, and newer groundedness, faithfulness, relevance and task-completion metrics compress millions of interactions into trackable numbers, and every compression discards information. Breaking down the 92%:

| Workflow type | Success rate |
|---|---|
| Simple workflows | 96% |
| Moderately complex workflows | 90% |
| Policy exceptions, human approvals, regulatory constraints, unusual customer situations | 61% |
| Dashboard average | 92% |

Both numbers are technically correct; only one reflects reality for the customer in the 61% segment.

## From predictive models to autonomous systems

- Subgroup failure is not new; fairness research made measuring performance separately across subgroups a core task.
- **Gender Shades (Buolamwini and Gebru, 2018):** three commercial facial-analysis systems looked strong overall, but error rates were at or below 0.8% for lighter-skinned men and up to 34.7% for darker-skinned women; in two systems the darkest-skinned women were misclassified close to half the time. Disaggregation made the failure visible.
- In enterprise systems the failing subpopulation is usually operational (a workflow type, a document class): a RAG system failing whenever one document type appears; an assistant breaking when information spans repositories; an agent failing when tools must coordinate, approval is required, or an external system returns something unexpected.
- Generative failure surfaces span document types, knowledge domains, languages, query complexity and user expertise; agentic ones span workflow types, exception paths, approval chains and tool orchestration. Greater capability produces greater opacity.

## When the dashboard becomes the problem

The instinct is to improve the model: swap foundation models, rewrite prompts, upgrade embeddings, widen context, add tools. Often that fails because the cause sits upstream: data drift, incomplete knowledge representation, weak retrieval orchestration, entity relationships never formally modeled, or business processes more complex than any benchmark. The dashboard becomes the blind spot.

## The segment you didn't define

- Segment breakdowns catch suspected failures; the expensive ones live in undefined segments. **Hidden stratification** (Oakden-Rayner et al., 2020): strong aggregate models consistently failing an unlabeled subset; the study found relative performance gaps of more than 20% on clinically important subsets.
- The subset is usually a combination: a model can do well on non-English queries and on policy exceptions, yet fail non-English policy-exception queries from first-time users. Only an intersectional view or an automated search for unhypothesized underperforming groups surfaces it.
- In high-stakes decisions the worst-performing segment tends to be the population the safeguard was built for (the unusual claim, the non-standard applicant), carrying the most legal and reputational weight.

(Illustration: an iceberg with known segments above the waterline - geography, product line, language, customer type - and hidden combinations below: non-English queries, policy exceptions, first-time users, complex workflows, high-risk approvals, edge-case customers.)

## What good evaluation looks like

- Treat subgroup evaluation as a first-class requirement. Break every headline metric down by risk-carrying dimensions: customer segment, geography, language, product line, user expertise, workflow type, query complexity, document category, business function.
- Declared slices are the start; pair them with a search for underperforming cohorts nobody defined.
- Don't slice too finely: a 61% success rate from nineteen cases tells you almost nothing. Slice by segments large enough to measure and important enough to matter; treat thin, high-stakes segments as a reason to gather more data.
- Ask "Accurate for whom?" instead of "How accurate is the model?" The worst-performing segment may say more about risk than the overall score. Make subgroup performance visible by default in monitoring.

## Where this is heading

As frontier models converge, the competitive question shifts from whose model is better to whose team knows where its systems fail, and who they fail. That comes from measurements built to show what the average hides. The most dangerous AI failures are the ones a dashboard averages away.

Contributor note (as on the page): views are the contributor's; submissions are vetted; Context and Chaos is information-first with no promotions, paid or otherwise.

Closing cartoon: an orange cat celebrates an all-green 92% dashboard ("Everything's green!") while a grey cat pulls back a curtain on failing segments (policy exceptions at 61%, first-time users, non-English queries, complex claims): "The average isn't where the failures live."

## References

- Buolamwini, J., and Gebru, T. (2018). Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. Proceedings of the 1st Conference on Fairness, Accountability and Transparency, PMLR 81, 77-91. https://proceedings.mlr.press/v81/buolamwini18a.html
- Oakden-Rayner, L., Dunnmon, J., Carneiro, G., and Re, C. (2020). Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging. Proceedings of the ACM Conference on Health, Inference, and Learning, 151-159. https://arxiv.org/abs/1909.12475

## About Context & Chaos

A community newsletter where practitioners, builders and thinkers share stories and lessons on context engineering, governance, architecture, discovery, and the human side of data and AI work. [Browse all issues](https://atlan.com/context-and-chaos/) or [contribute](https://atlan.com/context-and-chaos/contribute/).

Related reads:

- [Context Graphs as AI Evaluation Infrastructure](https://atlan.com/context-and-chaos/issue/context-graphs-as-ai-evaluation-infrastructure/) (April 2026)
- [Your AI Context Layer Is Being Built on Stale Metadata](https://atlan.com/context-and-chaos/issue/your-ai-context-layer-is-being-built-on-stale-metadata/) (May 2026)
- [The Onboarding Gap Is Killing Your AI Agents](https://atlan.com/context-and-chaos/issue/the-onboarding-gap-is-killing-your-ai-agents/) (May 2026)
- [BI-Ready Is Not AI-Ready](https://atlan.com/context-and-chaos/issue/bi-ready-is-not-ai-ready/) (March 2026)