Arize is an AI observability and evaluation company built around two products: Arize Phoenix, open source and OpenTelemetry-based, and Arize AX, the managed enterprise platform on the same foundation. Teams use Arize to trace what an LLM app or agent did, score whether its output was good, and catch drift in its inputs before it becomes an incident. Atlan, LangSmith, Langfuse, and Monte Carlo answer different pieces of “can we trust what this agent just did.”
Most 2026 coverage of Arize either explains Phoenix’s feature set or drops it into a comparison table against LangSmith and Langfuse. This page does both, then closes with the one question tracing and evals were never built to answer.
| Field | Value |
|---|---|
| What it is | AI observability and evaluation company, two products on one OpenTelemetry foundation |
| Founded | 2020, by Jason Lopatecki (CEO) and Aparna Dhinakaran (Chief Product Officer) |
| Funding | $70M Series C, led by Adams Street Partners |
| Open-source product | Arize Phoenix, Elastic License 2.0, self-hostable |
| Managed product | Arize AX, proprietary datastore, production scale |
| Scale (AX) | 1 trillion spans and 1 billion evaluations processed monthly |
| Analyst coverage | Named in Gartner’s first Market Guide for AI Evaluation and Observability Platforms (Feb 2026) |
What is Arize AI used for in LLM observability?
Permalink to “What is Arize AI used for in LLM observability?”Arize was founded in 2020 by Jason Lopatecki, CEO and previously at TubeMogul, and Aparna Dhinakaran, Chief Product Officer and previously at Apple and Uber. It splits into two products: tracing what an LLM app or agent did, evaluating whether its output held up, and watching for drift in its inputs over time.
Tracing captures the prompts, retrievals, tool calls, and outputs an agent produced, so an engineer can reconstruct what happened. Evaluation scores that output against defined criteria, usually with another LLM acting as judge. Drift detection watches whether the statistical shape of an agent’s inputs or outputs is shifting, before that shift shows up as a customer complaint. Arize AX runs this at real scale: 1 trillion spans and 1 billion evaluations a month.
Lopatecki and Dhinakaran frame it this way: “Production agents often fail in two ways: obvious failures you can alert on (timeouts, tool errors, 5xxs) and nuanced agent failures that aren’t deterministic and require intelligent examination of the agent trajectory (wrong retrieval, bad tool args, skipped steps, plausible answers built on bad reasoning).” Arize AX turns that second category into one workflow: the trace shows what happened, evals turn behavior into signal, managed agents move it to a fix.
Where “observability” fits for Arize specifically
Permalink to “Where “observability” fits for Arize specifically”Arize predates the current LLM-observability wave. It started in classic ML monitoring, watching drift and bias in traditional ML systems, and only later extended that instinct to LLM apps and agents. That’s why Arize’s drift detection feels more statistically rigorous than most LLM-native tools built after 2022: it’s a second act, not a first product.
What’s the difference between Arize AX and Arize Phoenix?
Permalink to “What’s the difference between Arize AX and Arize Phoenix?”Phoenix is open source and self-hosted, built for prototyping and dev-time evaluation. AX is the managed platform on the same OpenTelemetry foundation, adding online evals, drift and bias monitoring, and continual-improvement workflows at production scale.
Phoenix ships under the Elastic License 2.0, free to run locally, in Docker, on Kubernetes, or on one of two free Phoenix Cloud instances. It covers tracing, evals, datasets, experiments, and a prompt playground, what a team needs to prototype a RAG pipeline or agent before shipping. AX adds a proprietary managed datastore Arize calls “adb,” continuous online evaluations against live traffic, drift and bias monitoring, and the collaboration surface a multi-team deployment needs.
The typical adoption path, by Arize’s own account, is Phoenix first, AX later, once a team needs production scale and shared visibility across the org.
AX vs. Phoenix at a glance:
| Arize Phoenix | Arize AX | |
|---|---|---|
| License / cost | Open source (Elastic License 2.0), free | Managed, enterprise pricing |
| Deployment | Self-hosted (local, Docker, K8s) or Phoenix Cloud (2 free instances) | Arize-managed cloud |
| Best for | Prototyping, dev-time evaluation, individual engineers | Production scale, multi-team collaboration |
| Datastore | Local / self-managed | Proprietary managed datastore (“adb”) |
| Adds | Tracing, evals, datasets, prompt playground | + online evals, drift/bias monitoring, continual-improvement workflows |
Inside Atlan AI Labs: The 5x Accuracy Factor
How Atlan's AI Labs team measures and improves agent accuracy, the same evaluation discipline Arize's evals apply from the model side, applied instead to the context an agent reasons over.
Get the 5x Accuracy EbookHow does Arize work?
Permalink to “How does Arize work?”Arize’s feature set breaks into four capabilities: tracing shows what happened, evaluation scores whether it was good, drift detection watches for pattern shifts, and agent graph visualization turns raw spans into something readable.
Tracing (OpenTelemetry + OpenInference)
Permalink to “Tracing (OpenTelemetry + OpenInference)”Arize’s tracing runs on OpenTelemetry, with OpenInference layered on top, Arize’s own semantic-convention layer adding AI-specific span kinds (LLM, Tool, Agent, Retriever) and attributes like llm.token_count.total. That’s why tracing works across frameworks instead of locking into one vendor’s log format; Phoenix’s auto-instrumentors track agent-specific metadata across LangGraph, AutoGen, CrewAI, the OpenAI Agents SDK, and Agno, across 35+ Python and 10+ JavaScript packages.
Evaluation (LLM-as-judge)
Permalink to “Evaluation (LLM-as-judge)”Evals score an agent’s output against defined criteria using another LLM as judge, the same pattern behind most modern LLM evaluation frameworks. Datasets and experiments let a team benchmark a prompt or model change before shipping it, catching a regression before a customer does.
Drift detection
Permalink to “Drift detection”Arize’s drift-detection mechanism computes the Euclidean distance between embedding centroids across time windows to flag semantic shift, a deliberately simple approach found more stable in production than more complex alternatives. It’s the same instinct behind context drift detection: catching a slow, quiet shift before it becomes a hard failure, though it watches inputs and outputs statistically, not the business definitions behind them.
Agent graph visualization
Permalink to “Agent graph visualization”For multi-agent systems, Phoenix abstracts raw spans into a node-based graph mapping agent-to-agent and agent-to-tool flow, the feature most cited for multi-agent debugging. One practitioner writeup reports the shift from manual JSON-trace reading to visual inspection cut root-cause time from roughly four hours to about 30 seconds, a vendor-reported anecdote, directional but in line with how decision traces and context management across multiple agents are meant to work.
Who uses Arize, and what do they use it for?
Permalink to “Who uses Arize, and what do they use it for?”Three use-case clusters cover most of how teams deploy Arize: debugging a single agent’s execution path, evaluating retrieval quality at scale, and untangling coordination failures across a multi-agent system.
LLM app debugging is the narrowest and most common case: an engineer traces a single-agent or RAG pipeline’s execution path to find exactly where an output went wrong, whether that’s a bad retrieval, a malformed tool call, or a prompt that drifted from its intended instructions. RAG evaluation scales that same instinct across a whole test corpus, scoring retrieval quality and answer faithfulness before and after a change to the retrieval strategy, the same territory RAG accuracy problems and context quality testing cover in more depth. Multi-agent observability is the deepest case, and it’s covered at length on Atlan’s multi-agent debugging guide: agent graph visualization for coordination failures across LangGraph, AutoGen, CrewAI, and OpenAI Agents SDK pipelines, where a readable graph replaces hours of manual log-reading.
Logos named on Arize’s own site, including Reddit, DoorDash, Uber, Spotify, Booking.com, PagerDuty, and Atlassian, are scale evidence rather than case-study depth; Arize doesn’t publish the kind of per-customer outcome metrics the way Atlan’s own customer stories do, but the roster is a reasonable signal that Arize is deployed well past the demo stage. Teams running multiple frameworks at once, weighing LangGraph against CrewAI or an agent interoperability protocol to connect them, are exactly the profile Arize’s framework-agnostic tracing was built for.
How does Arize compare to LangSmith, Langfuse, and other LLM observability tools?
Permalink to “How does Arize compare to LangSmith, Langfuse, and other LLM observability tools?”Arize’s one-line differentiator is production evaluation depth combined with statistical drift detection, best suited to teams running multiple models in high-stakes domains rather than a single LangChain pipeline.
Gartner’s first Market Guide for AI Evaluation and Observability Platforms, published in February 2026, groups Arize with LangSmith, Langfuse, and Braintrust as the pure-play specialists in this category, separate from the platform-native bundlers building observability into a broader cloud offering. Third-party 2026 comparisons converge on where each tool wins: Langfuse on open-source purity, self-hosting, and transparent pricing; LangSmith on native LangChain/LangGraph integration, with the tightest framework lock-in of the three; Arize on production evaluation depth and statistical drift detection, fitting teams running multiple models in regulated or high-stakes domains.
Arize vs. LangSmith vs. Langfuse:
| Arize (AX/Phoenix) | LangSmith | Langfuse | |
|---|---|---|---|
| Best for | Multi-model production evals + drift detection | LangChain/LangGraph-native teams | Data sovereignty, self-hosting |
| Framework lock-in | Low (OpenTelemetry-based) | High (LangChain/LangGraph) | Low |
| Drift detection | Native, statistical (embedding drift) | Not a core feature | Not a core feature |
| Open-source option | Yes (Phoenix) | No | Yes (fully) |
Gartner projects LLM observability investment reaching 50% of GenAI deployment spend by 2028, up from roughly 15% in early 2026. A market-report aggregator, not a named analyst house, estimates the platform market at $2.69B in 2026, growing to $9.26B by 2030, a scale signal rather than a budget forecast.
What does Arize’s tracing and evals leave uncovered?
Permalink to “What does Arize’s tracing and evals leave uncovered?”Tracing answers what happened. Evals answer whether the output was good. Neither one, by design, answers whether the material an agent retrieved was itself the current, complete, and authorized version of a fact.
Arize itself is moving this way. Its own blog, “From observability to context: what’s next for Arize Phoenix,” argues agents acting as operators need “usable context” beyond a human-readable trace, evidence the gap is real. Atlan’s context observability work covers the distinction: a trace can show an agent faithfully retrieved a real document, and an eval can score the output faithful, while the document itself was cached, outdated, or wrongly scoped. Neither tracing nor evals check that; both assume whatever got retrieved was right to retrieve.
What Arize shows vs. what it doesn’t:
| Arize shows | Arize doesn’t check |
|---|---|
| That a tool call happened, and its latency/cost | Whether the data behind the call was current |
| That an output was faithful to what was retrieved | Whether what was retrieved was itself authorized for that agent |
| Drift in embeddings/model performance over time | Whether a single retrieval, at the moment it happened, was complete |
Atlan’s comparison of context observability against data and LLM observability works through this boundary tool by tool: a clean trace and a passing eval are necessary for trusting an agent’s output, not sufficient. Context freshness, LLM knowledge base freshness scoring, and AI agent hallucination each cover a different slice of what goes wrong once tracing and evals report green.
Context Gap Calculator
Score how much of your agent stack, tracing and evals included, is reasoning over governed context versus context nobody's checking for freshness or scope.
Calculate Your Context GapHow Atlan approaches the gap Arize’s observability leaves open
Permalink to “How Atlan approaches the gap Arize’s observability leaves open”Atlan doesn’t compete with Arize’s tracing or evals; it sits beneath them, as the source Arize would be tracing calls against in the first place.
Arize, and LangSmith, Langfuse, and Monte Carlo alongside it, can all report a clean trace and a passing eval while the agent underneath still acted on stale or unauthorized context, the same silent-failure pattern Atlan’s context observability work documents. A tool watching model calls has no way to see that the document behind a faithful answer was three weeks out of date.
Atlan maps that gap to specific capabilities instead of a general claim. Context Repos and Bounded Context Spaces enforce scope-correctness, which repo a given agent is even allowed to read from. Context Drift Detection covers freshness and staleness at the definition level, ahead of whatever an agent eventually retrieves. Column-level lineage and the Enterprise Data Graph cover completeness and provenance, tracing an answer back to the exact source column that produced it. The Atlan MCP Server is the delivery surface an Arize-style trace would be tracing calls to, not a competing product.
None of this replaces what Arize does well. It answers a different question, one layer beneath: was the material the agent reasoned over correct in the first place. Teams building that layer start with what context engineering actually does and how to implement an enterprise context layer for AI, rather than assuming a passing eval already covers it.
What Arize proves, and what it was never built to check
Permalink to “What Arize proves, and what it was never built to check”Arize is mature, well-adopted infrastructure for a real and largely solved problem: tracing, evaluating, and monitoring drift in LLM apps and agents at real production scale. AX and Phoenix split that problem cleanly, a free and open tier for prototyping, a managed enterprise tier for production, both built on the same OpenTelemetry foundation rather than two unrelated products wearing one brand.
What Arize isn’t built to check, and by its own public direction is reaching toward, is whether the material an agent acted on was itself current and authorized at the moment it acted. That’s a different layer underneath observability, not a missing feature inside it, and it’s the layer Atlan’s context layer work exists to cover.
AI Agent Context Readiness Checklist
A practical checklist for whether your agents are reasoning over context that's current, complete, and authorized, the layer tracing and evals don't check.
Take the Readiness CheckFAQs about Arize AI
Permalink to “FAQs about Arize AI”1. What is Arize Phoenix used for?
Permalink to “1. What is Arize Phoenix used for?”Arize Phoenix is used for tracing, evaluating, and debugging LLM applications and agents. It’s open source, built on OpenTelemetry, and self-hostable, a common first stop before a team decides whether it needs Arize’s managed tier, AX.
2. Is Arize Phoenix free?
Permalink to “2. Is Arize Phoenix free?”Yes. Phoenix is open source under the Elastic License 2.0, free to self-host locally, in Docker, or on Kubernetes, with no feature gates. Arize AX is the paid, managed tier that adds online evaluations, drift monitoring, and production-scale collaboration.
3. What is the difference between Arize AX and Arize Phoenix?
Permalink to “3. What is the difference between Arize AX and Arize Phoenix?”Phoenix is open source and self-hosted, for prototyping and dev-time evaluation. AX is the managed platform on the same foundation, adding online evals, drift/bias monitoring, and continual-improvement workflows at production scale. Most teams start on Phoenix and move to AX as usage grows.
4. Who founded Arize AI, and when?
Permalink to “4. Who founded Arize AI, and when?”Arize was founded in 2020 by Jason Lopatecki, CEO and formerly of TubeMogul, and Aparna Dhinakaran, Chief Product Officer and formerly of Apple and Uber. It has since raised a $70M Series C led by Adams Street Partners, with participation from M12, Microsoft’s venture fund, Datadog, and PagerDuty.
5. What is OpenInference, and how does it relate to OpenTelemetry?
Permalink to “5. What is OpenInference, and how does it relate to OpenTelemetry?”OpenInference is Arize’s open semantic-convention layer on top of OpenTelemetry. It adds AI-specific span kinds, like LLM, Tool, Agent, and Retriever, and attributes such as token counts, so tracing works across frameworks without vendor lock-in.
6. How does Arize detect drift?
Permalink to “6. How does Arize detect drift?”Arize computes the Euclidean distance between embedding centroids across time windows to flag semantic shift in a model’s inputs or outputs. It’s a deliberately simple statistical method; Arize’s engineering team found it more stable in production than more complex alternatives.
7. Does Arize tell you if an AI agent’s context was correct?
Permalink to “7. Does Arize tell you if an AI agent’s context was correct?”No. Arize’s tracing and evals check what an agent did and whether its output was faithful to what it retrieved, not whether the retrieved material was itself current or authorized. That’s a distinct, complementary check sitting one layer beneath observability.
Sources
Permalink to “Sources”- Phoenix product page, Arize
- Arize homepage
- Arize-ai/phoenix repository, GitHub
- “From observability to context: What’s next for Arize Phoenix,” Arize blog
- “Arize AI Raises $70M Series C to Build the Gold Standard for AI Evaluation & Observability,” Arize blog
- “Arize AI Secures $70M Series C to Fix AI’s Biggest Problem,” PRNewswire
- “TubeMogul, Uber alums launch Arize AI for AI observability,” TechCrunch
- Gartner, Market Guide for AI Evaluation and Observability Platforms, February 2026
- Gartner, “By 2028, Explainable AI Will Drive LLM Observability Investments to 50%,” Gartner newsroom
- “Monitoring Embedding/Vector Drift Using Euclidean Distance,” Arize engineering blog
- “Debugging Agent Loops: Bridging the Observability Gap with Arize Phoenix,” Medium, 2026
- “LLMOps Observability: LangSmith vs Arize vs Langfuse vs W&B,” Medium/Kanerika, 2026
- Large Language Model (LLM) Observability Platform Market Report 2026, ResearchAndMarkets
- “Building the AI Factory: Self-Improving Agents with Arize AX,” Arize blog
