Inside Mastercard's Context Agents Rollout

Zhenni Hu

0:00 All right. Hi, everyone. I'm Zhenni Hu, and from the AI and data governance team at Mastercard. A quick word on the scale. Mastercard operates in more than two hundred and twenty markets with billions of credentials and around hundreds of billions of the card transactions. So all of that generates data across various domains and spread across dozens of the platforms built over the decades. So those data only helps if people can find it, understand it, and trust it, and that's what we would want to talk about today, how we use AI to catalog our data.

0:34 Before I get into it, I'm curious where this group sits. So two quick questions. First, have you used the AI at work this week? And give me a thumbs up. I will put in the quick questions to the chat. Anything counts. It doesn't have to be anything fancy. Okay. Cool. I'm seeing the number has been increasing. Plenty of you, about what I expected. And the second one will be a little bit harder. Would you be comfortable letting AI to making the decisions that you're getting held accountable for? Yep.

1:24 Not surprisingly, I didn't see any thumbs up comments yet. Awesome. So, right. Noticeably, no surprise that the second number will be much smaller, and most people are comfortable using AI now, but far fewer are comfortable trusting it, and that's the gap we are trying to close. So here's the dynamic we are dealing with, and I think most of you will recognize it. Most of the enterprise governance team's capacity stays roughly flat. The data estate from the other side keeps growing and expanding. Every migrations, every acquisitions, every new platform onboarding or even just a refresh you do adds onto it.

2:06 And as a data steward, the legacy way of manually cataloging the data assets is always a tedious work. It is slow, and it gets stalled the moment you finish. And I suspect most of people here have run an initiative that started strong but quietly lost the momentum. And whoever builds those tables understood them perfectly well, while once they changing the roles or moving to another teams, that understanding went with them. And today, the stakes have risen. Cataloging is not only just critical to the compliance and regulations. Forbes says context is the oil of the AI economy. Documentation and metadata used to help people find the data. Now, it also helps AI understand the data.

2:53 When a person cannot find an answer, they reach out to a coworker, but when an agent cannot find an answer, it will often produce one anyway, which would cause the hallucination. So today, the quality of your metadata sets a ceiling on the quality of your AI.

Zhenni Hu

3:11 So next slide. What did we do? Three decisions, and honestly, the thinking behind it matters more than the specifics of our setup. The first one is the scope. Once you have the AI-generating descriptions, the temptation is to run it across everything, because why not? It's just one single click of a button.

3:31 We held back deliberately because generating hundreds and thousands of descriptions overnight, you build a review queue that your team will never be able to clear. And the first time a steward opens a weak one, that doubt spread across the whole batch. The goal was not just generating about volume, it was more generating something people could realistically review and trust. So we intentionally picked a bounded set of the critical assets small enough to review properly, visible enough that success would register with the business. And second, the evidence. This was probably the one of the most important learning for us like Prukalpa say, because it determines whether any of those is usable.

4:18 If the AI only see the column names, you just get a polished restatement of the schema. That's usually not what the steward really need. What made ours workable is that it reasons over the real signals how the data actually gets curated, either by the human or the systems, the real usage signals, and what it connects to the upstream and downstreams, and our existing enrichment information like the existing business glossary, classifications, data products, and et cetera. That's what made a human review realistic. Correcting a draft that's already close takes minutes, but rewriting something too generic takes as long as writing it yourself, which is the job that everyone was already too busy to do.

5:03 And third, the certifications. A subject matter expert signs off before anything goes live for the consumption. That was part of the design from day one, and I will be direct about the first question we put here, what does that mean for my job? If you roll something like this out and nobody asks you that, it doesn't mean they are not wondering. At Mastercard, we believe AI is here to augment people, not replace them, and I will come back to what that looks like in practice soon. So here's what we learned. Control scopes makes it realistic, evidence makes the draft looks good, and people make it trustworthy.

Zhenni Hu

5:45 Now let's talk about the results, because eventually, every AI conversation gets to the same questions. What's the value? In a two-week pilot, we enriched approximately thirty thousand assets across roughly seven hundred tables and views and estimated about six thousand two hundred hours of the steward effort saved. From the speed perspective, two weeks means we can repeat... Oh, can we go back to the previous slide? Thank you. So yeah, it means we can repeat this. Traditionally, the manual cataloging takes months. So think about that. A twelve months effort gives you the results that you can only celebrate once. While a two weeks means we could scale the same approach more broadly now.

6:33 We can run it again next quarter on a different domain without the need for a whole steering committee or a task force dedicating all of their time. And when the leadership asks about return of investment question about AI, and they will, these are the numbers that can answer it. A fixed teams now covers a growing estate, and the capacity that came back went to the work that generally needs a person.

Zhenni Hu

7:03 So next slide, please. Here is the one point that worth emphasizing. The data stewardship role didn't disappear. It's got upgraded. Think about what writing a description actually involves.

7:16 Much of it is mechanical, restating the structure, describing what a field holds, and keeping phrasing consistent. It's real work, but it's less valuable thing a steward can do. Their value shows up in knowing when something is subtly off, recognizing that a term means one thing to finance and something slightly different to risk, knowing which of the three near identical tables is the authoritative one, and that judgment takes years to build, and it stays with the person. So the work moved up, less drafting, more judging, and the coverage the same team maintains went up with it because the reviews moves much faster than the authorship. And there's also a second effect worth naming.

8:03 When the steward certifies something, they are standing behind it. That gives whomever consume it downstream, either a person or an agent, a much stronger signals than an unverified model output. That's where the steward creates more value. And next slide, please. One last thought before I wrap up, and briefly on the direction. The order here is the message. So if the catalog is empty underneath, search won't be able to give you a reliable answer. AI increases the value and importance for having a good foundation. Some of the organizations start with the agents or the search, the last two boxes, because that's the exciting part, and it's what the leadership is asking about.

8:48 That's usually where it starts. Handing organizational context to an autonomous agent where you haven't verified that context yourself is a fast way to lose the trust in the whole program. So we started with the foundations. What we did here built on the years of the inventory, cataloging, and lineage work, which was genuinely hard and slow at the time. Now, we have the magic fuel to accelerate and scale it. How well we build a base sets the ceiling for the rest. With that, it concludes my sharing today. Thank you.