Context Studio
/
finance-revenue
/
Eval Suite
Generated from
📊 Q4 Revenue Dashboard
📊 Churn Analysis Report
⌗ revenue_by_region.sql
⌗ arr_cohort_analysis.sql
→
47 tests generated
Test cases
41 pass
/
6 fail
✓
What was Q4 revenue by region?
📊 dashboard
✓ Passed
✗
Which enterprise accounts churned last quarter?
📊 dashboard
✗ Mismatch
✓
What is ARR for cohort Q3-2025?
⌗ sql
✓ Passed
✗
Show revenue excluding intercompany for Q4.
⌗ sql
✗ Mismatch
✓
Top 5 regions by net new ARR this quarter?
📊 dashboard
✓ Passed
✓
What is MRR for active enterprise contracts?
⌗ sql
✓ Passed
✗
Revenue trend — last 4 quarters vs prior year.
📊 dashboard
✗ Mismatch
✓
Gross margin by product line, Q4.
⌗ sql
✓ Passed
Result - Test 2
Question
Which enterprise accounts churned last quarter?
Expected answer
Expected
14 accounts, $840K ARR lost
Agent generated
✗ Mismatch
23 accounts churned (includes SMB tier)
Context used
churn.definition
90+ days no login — MISSING tier filter
churn.tier
⚠ not defined in context model
Suite passing
0%
41 / 47 tests