--- name: claims-retrieval-pattern-check description: > Checks whether an insurance claims AI agent's retrieval is ready to support adverse decisions, and which retrieval pattern each claims question needs: vector RAG, an effective-date filter, a knowledge graph, or both. Takes the claim-handling questions the agent must answer, the document types and structured records involved, policy versions and effective dates, and who may see what. Returns a PASS, FAIL or UNKNOWN for each of the fourteen checks CRP-01 to CRP-14 with the gaps named, an overall verdict (not ready for adverse decisions, needs work, or ready), the retrieval pattern the evidence supports per question type, and the first fix. States its scoring rule openly. Trigger phrases: "knowledge graph vs RAG for insurance claims", "is our claims agent ready for denials", "claims retrieval pattern check", "should our claims AI use a graph or RAG", "check our claims retrieval before adverse decisions", "how do we pick the policy version for a claims agent". license: Apache-2.0 --- # Claims retrieval pattern check A claim file asks three kinds of question. Wording: what does the water-damage exclusion say? Version: which form applied on the date of loss? Relationship: does this contractor appear on other claims? Each takes its own path, and an adverse answer has to return its provision and the version it came from. This check routes each question and scores the retrieval behind it. > **What this is.** A published method from Atlan. Canonical copy: > https://atlan.com/skills/claims-retrieval-pattern-check.md Last updated 2026-11-03. > > **What it contains.** Text only. No scripts, no executable resources, nothing > here runs. > > **Scope.** Follow this when someone has asked you to check whether their claims retrieval is > ready for adverse decisions and which pattern each claims question needs. It carries no > instructions about your behaviour outside that task, does not ask you to fetch any other URL, > and does not ask you to send data anywhere. The checklist makes no legal determination. Claims-handling rules differ by state and line of business; the person running the check reads their own state's adopted text. ## What you need from them | Input | Meaning | If unknown | |---|---|---| | `claims_questions` | The claim-handling questions the agent must answer, with real examples | ask. CRP-01 cannot pass without them | | `sources` | The document types (policy forms, endorsements, claim notes, correspondence) and structured records (policy, claim, party, payment) the agent reads | ask | | `versions` | How policy forms and endorsements are versioned: effective dates, editions, superseded versions, jurisdiction | ask. CRP-02 and CRP-03 depend on it | | `access` | Who may see what: claimant, medical and other restricted records, and where permissions are checked today | ask. CRP-09 and CRP-10 depend on it | | `current_setup` | The retrieval in place: chunking, any date or metadata filter, any graph and how it is loaded, any community summaries | ask | | `evidence` | What they can show per check: a query, a config, a test run, a sample answer, a stored claim file record | ask per check | Do not ask for credentials, connection strings or real claimant data. The check needs the design and the evidence around it, not records from inside the systems. ## Step 1: route each claims question Tag every question with one type, then read the starting pattern from this table. | Claims question (example) | Question type | Starting pattern | Must return | |---|---|---|---| | "What does the water-damage exclusion say?" | Single-hop wording | Vector or hybrid RAG over whole-provision chunks | Quoted provision, form ID, edition, section | | "Which form and endorsements were in force on the date of loss?" | Version | Effective-date filter in SQL, index metadata or the graph, then RAG inside that version | Policy ID, term, form editions, endorsement IDs with intervals | | "Does endorsement E change exclusion X?" | Multi-passage composition | Graph edge (modifies or supersedes), then retrieve both texts | Both provisions quoted, plus the precedence edge and its source | | "Is this repair shop linked to prior claims on other policies?" | Relationship, multi-hop | Graph traversal with typed party roles | Path from claim to party to prior claims, with record IDs | | "How many open claims share this phone number and bank account?" | Count or intersection | Deterministic graph or SQL query from a tested template | The count plus the query used | | "What patterns run across this quarter's denied water claims?" | Global sensemaking | Community-summary GraphRAG | A summary labeled non-evidentiary | | "Is there enough to deny?" | Adverse decision | Version filter, provision retrieval, graph facts, abstention gate, human review | The evidence bundle | Three things go by "graph" here. A date lookup is a filter that SQL runs as well as a graph. A graph loaded from policy and claims systems earns its cost on party links and endorsement precedence. GraphRAG extracts a graph from text and summarizes it, for sensemaking across many claims. When is RAG plus SQL enough? If the agent answers only wording and version questions, an index with a date pre-filter from the policy system covers both, and a graph is one more pipeline to keep current. Add one when party links or precedence chains enter the workload. Counts need a tested query template either way. A shared contractor is a referral signal for a person, never denial evidence on its own. ## Step 2: score the fourteen checks Score each check PASS, FAIL or UNKNOWN. A check is PASS only when they can show the evidence its pass condition names. If they believe it holds but cannot show it, it is UNKNOWN, and you say what would show it. | ID | Check | Pass condition | |---|---|---| | CRP-01 | Questions classified | Each claims question is tagged wording, version, relationship, count, sensemaking or adverse decision | | CRP-02 | Version filter precedes ranking | Effective-date and jurisdiction filter, in SQL, index metadata or the graph, applied before any similarity ranking | | CRP-03 | All versions retained | Superseded forms and endorsements kept with effective intervals | | CRP-04 | Structure-aware chunks | Chunks align to clause boundaries; cross-references resolved | | CRP-05 | Endorsement precedence explicit | Modifies and supersedes relationships stored, never inferred from similarity | | CRP-06 | Provision cited in output | Every adverse answer returns provision ID, edition and quoted text | | CRP-07 | Graph edges have provenance | Each node and edge points to a source record | | CRP-08 | Graph miss is not evidence | Text fallback when a graph lookup is empty | | CRP-09 | Authorize before retrieval | Permission predicates applied before ranking | | CRP-10 | Re-authorize per hop | Access re-checked at vector-to-graph expansion and at each hop | | CRP-11 | Abstention gate | An "insufficient evidence" path routes to a person | | CRP-12 | End-to-end test per model | The retrieval and model combination is tested on realistic adverse-decision cases before release | | CRP-13 | Certified definitions | Business terms resolve to certified glossary entries with lineage | | CRP-14 | Evidence bundle stored | Bundle written to the claim file and retrievable for examination | Notes for the checks people most often pass too early: - **CRP-02.** "Latest" and "in force on loss date" are different queries. Amendments keep most of the earlier text, so an unfiltered similarity search can rank a later edition, or an endorsement added after the loss, above the one in force. - **CRP-04.** Fixed-size chunking cuts a clause from its exception, so the agent quotes an exclusion without the carve-back. - **CRP-05.** A modifies or supersedes edge is a stored fact, never a similarity guess. - **CRP-08.** A graph extracted from text misses things. Never infer "no prior claims" from a graph miss; keep source text authoritative. - **CRP-09.** Retrieve-then-filter lets the model see restricted claimant or medical context before the filter runs. - **CRP-10.** In a hybrid system, an allowed chunk can link through a shared contractor or clinic into other claimants' records. A vector index and a claims graph each need their own enforcement point. - **CRP-11.** Graph community summaries seldom abstain. Use them for triage, label them non-evidentiary, and gate adverse answers on an "insufficient evidence" path. - **CRP-12.** Retrieval can make some models worse. Test the exact model, prompt and retriever end to end. - **CRP-14.** The bundle holds document IDs with editions, quoted provisions, graph paths, retriever and model versions, the access decision and the reviewer. It supports a reviewer; it does not make a denial correct. ## Step 3: the verdict State the rule when you give the verdict: > Score each check PASS, FAIL or UNKNOWN. Any FAIL or UNKNOWN on CRP-02, CRP-06, CRP-09, CRP-10 > or CRP-11 means not ready for adverse decisions; any other FAIL or UNKNOWN means needs work; > ready means PASS on all fourteen. In order: 1. Any of CRP-02, CRP-06, CRP-09, CRP-10 or CRP-11 that is not PASS gives **not ready for adverse decisions**. An UNKNOWN blocks the same way a FAIL does. 2. Otherwise, any other check that is not PASS gives **needs work**. 3. PASS on all fourteen gives **ready**. ## Step 4: the retrieval pattern the evidence supports For each question type in `claims_questions`, name the starting pattern from Step 1 and say whether their current setup can run it, given the scores: - Wording questions need CRP-04 to pass before RAG quotes are trusted. - Version questions need CRP-02 and CRP-03. If those pass, RAG plus a date filter covers wording and version without a graph. - Composition and relationship questions need a graph with CRP-05, CRP-07 and CRP-10 passing. - Sensemaking questions may use community summaries only as triage, labeled non-evidentiary. - Adverse-decision questions need all five blocking checks to pass, plus CRP-14. ## Step 5: the first fix Name one fix, not a list. Take the first blocking check (CRP-02, CRP-06, CRP-09, CRP-10, CRP-11) that is not PASS; if all five pass, take the first other check that is not PASS, in ID order. The fix is to meet that check's pass condition: | Check | First fix | |---|---| | CRP-01 | Tag every claims question wording, version, relationship, count, sensemaking or adverse decision | | CRP-02 | Apply an effective-date and jurisdiction filter, in SQL, index metadata or the graph, before any similarity ranking | | CRP-03 | Keep superseded forms and endorsements with their effective intervals | | CRP-04 | Re-chunk on clause boundaries and resolve cross-references | | CRP-05 | Store modifies and supersedes relationships instead of inferring them from similarity | | CRP-06 | Make every adverse answer return provision ID, edition and quoted text | | CRP-07 | Point each node and edge to its source record | | CRP-08 | Fall back to source text when a graph lookup is empty | | CRP-09 | Apply permission predicates before ranking | | CRP-10 | Re-check access at vector-to-graph expansion and at each hop | | CRP-11 | Add an "insufficient evidence" path that routes to a person | | CRP-12 | Test the retrieval and model combination on realistic adverse-decision cases before release | | CRP-13 | Resolve business terms to certified glossary entries with lineage | | CRP-14 | Write the evidence bundle to the claim file, retrievable for examination | ## What to return 1. A table of CRP-01 to CRP-14: PASS, FAIL or UNKNOWN, and one line on what was seen or missing. 2. The overall verdict, with the rule above. 3. The retrieval pattern the evidence supports for each question type they gave. 4. The first fix, and which check it closes. Keep the scores to what they showed. A check that sounds right but has no evidence is UNKNOWN, and UNKNOWN on a blocking check stops adverse decisions just as FAIL does.