The Rise of the Agentic Data Catalog

Nandini Tyagi

0:03 All right, we see folks coming in. Welcome, welcome, everyone. Good morning, good afternoon, evening, depending on which part of the world we are joining in from. As everyone is coming in, would love for you all to quickly share your name, the country you're joining from, and your favorite drink. If you have joined enough Atlan events, you know we love hearing everyone's favorite drinks. So would love to hear what we all like. Let's see. I'm seeing... Oh, is there a bad echo? Is that for everyone? Is everyone able to hear me okay? Yes?

0:53 Okay, perfect. Amazing. Thank you, thank you. Always need to do a, a tech check first. Amazing. Ooh, old fashioned coming in here. Kombucha, cold coffee. G&T, love that. We are in full weekend vibe, I can tell. Matcha. Glass of water, no ice. I'm also no ice club, by the way. As folks do that please keep sharing this. You'll make new friends here like you always do. For those of you who are meeting for the first time, I'm Nandini. I'm part of the founding team here at Atlan.

1:35 So excited to welcome you all to our session today on everything related to Rise of Agentic Data Catalog, and we've got an action-packed agenda. I see Prukalpa here is, has also joined us. I'll quickly give you a peek at the agenda for today. We'll kick off with Prukalpa, actually, co-founder here at Atlan, sharing learnings on building an agentic catalog. Then, and this is such an honor, we get to hear directly from our customers. We have Zhenni from Mastercard, who will walk us through how her team has been using Atlan's agentic catalog with a deep dive on context bootstrapping and what they have learned along the way.

2:13 And then closing us out, we'll do a roundtable with some incredible leaders across our customer community. So grateful that we get to have them here, and we'll of course dig into some burning questions. Awesome. With that, I'll hand it over to you, Prukalpa. Of course, you need to share your favorite drink first as we kick this off. Over to you.

Prukalpa Sankar

2:31 All right. Well, I am, you know, old school coffee and just give me my caffeine and I'm gonna be, I'm gonna be your biggest fan. All right. Well I'm just gonna start sharing my screen. Is that okay, Nandini?

Nandini Tyagi

2:47 You should have slide controls. Otherwise I can take over.

Prukalpa Sankar

2:51 I don't, I don't seem to, so I'm just gonna...

Nandini Tyagi

2:54 Go for it. Yeah.

Prukalpa Sankar

2:55 All right. Perfect. So as I was putting this together it felt like we were on a precipice of something. And it, it kind of took me back to my my very first article on, on data catalogs. My journey with data catalog started about a decade ago when I was a data practitioner myself and faced a lot of these challenges of trusting data and, you know, analyst quit my team and I... You know, they'd left with all the knowledge about their, the, the data with them and, you know, all of these challenges personally and failed a couple of times over before, before we actually built Atlan for ourselves, or the first version of Atlan for ourselves.

3:42 And just going back in memory lane this was actually our first this was the first slide of our first community conference. And I remember back then, and this continues to hold true till today, I remember saying there are three reasons why traditional data catalogs fail. And this was a survey we had done at the time in the, in the ecosystem. You know, first, they fail because of metadata supply. People say it's too manual, it's too hard to update. You know, it's too hard to curate the metadata so they never go live. Then on the other hand, you know, people will say, "Hey, it, you know, metadata was not consumed.

4:22 You know, it was not part of business workflows. Business didn't adopt. It was too siloed." You'll hear, you'll hear words like this. And then, you know, 40% of the time, actually, you know we had users say "We don't keep up with the business. Our business keeps evolving." If you just think about technology in the last ten years, we've gone through maybe four, four rounds of technology evolution, right? And the business keeps evolving. It's really hard to make it flexible and keep learning and growing.

4:51 And over the last six years at Atlan we have we have tried to, you know, systematically make progress on each of these failure modes of data catalogs, as we call them internally. I originally talked about this as the idea of data catalog 3.0 or what does the modern metadata for the modern data stack look like. We were the first back then to say, "Okay, how do you go from manual to automated?" Back in 2023, we launched Atlan AI. You know, we we launched playbooks and automations for, you know, automating metadata supply. We brought this back into the, you know, webhook kind of automation workflows, embedded documentation and data producer workflows.

5:34 There was all these things that we tried to do to say, "Okay, how do we just make it much easier to drive metadata supply?" Then you know, we said, "Okay," you know, we brought active metadata to market. We said, you know "How do we make this super collaborative? This should just be embedded in the daily workflow of end users and teams." If I'm in a BI tool and I say, "Can I trust this dashboard?" I should never have to go to a data catalog. I should just get that context where I am and when I need it. And you know, extensibility.

6:04 Like, you know, you should be able to keep up with how your business is growing. That became a core part of our thesis. We opened up Atlan with an... In a completely open, open ecosystem. And in that journey we made a ton of progress, right? Like, you know, when we started, we went from sort of like manual to a more automated metadata supply. We went from siloed to more collaborative metadata consumption. We went from closed systems to fairly extensible systems. And as a journey, our team was, was super proud of being able to take and, and drive innovation in, in every part of this in, in the ecosystem. But yet it wasn't enough.

6:47 Like I remember our team back in twenty twenty-four, despite all of this, it wasn't the easiest thing in the world to deploy. It wasn't a one-click deploy. Many people on this call would agree, you know many of these programs are about the people and the process and the program, not just about the technology. We took that to heart. And so in fact, we ran a project that we called the Data Governance Genome Project. And we studied what makes successful programs versus not successful programs. This was a slide from my keynote last last year where we launched a book called The Modern Data and AI Governance Blueprint.

7:26 It was a a parable of what it takes from a people process perspective to go implement this effectively inside organizations. And that trip down memory lane today was so nostalgic for me because going into twenty twenty-six I finally have begun to believe that we're on the precipice of fully solving the problem. Over the last couple of years, we've been trying to answer these questions. You know, for metadata supply, can you truly, fully get to a point where you can autonomously build metadata and context?

8:15 Metadata consumption, can users just fully, naturally adopt and get what they need? And then can you build almost a self-learning kind of data catalog? And over the next roughly ten minutes, I wanna talk to you about the learnings we've had in each of these questions that we've been exploring and, and some of what feel like breakthroughs that, that's, that we seem to have had in, in the ecosystem and industry.

Prukalpa Sankar

8:45 So question one, can AI really create and document data context that you can trust? We've been on this problem since twenty twenty-three. We launched Atlan AI in twenty twenty-three, right off GPT back then. And I have to say, like it was, it was a labor of love. Like we back then, no matter how much we pushed, you know, we couldn't get beyond a seventy percent acceptance rate. It was an ability for teams to basically go say, you know, auto-generate a description for a table or auto-generate a README. And it was much better than it not existing. But we were never able to push the boundary beyond seventy percent.

9:23 And so you know, our team started kind of questioning and asking these questions, saying, "Okay, well, as the models are going to start getting better, what, what's missing?" Like, if we bet on the fact that the models are gonna get better, what do we need to be able to actually move in a more autonomous direction? And this is when we realized that metadata infrastructure was never designed for agents. For agents to be able to process context a-about all of this, it needed to be able to read end-to-end lineage and query history and usage patterns and relationships between assets.

9:58 You know, and, and it needed to be able to do this at speed and low cost, right? Like, it needed to be able to operate almost like a data warehouse, not a metadata store. And that led to sort of this foundational reimagining. So last year, we launched what we called a Metadata Lakehouse. And we brought the same paradigms of... You know, we said metadata was gonna become big data. We brought those same paradigms into what we call the Metadata Lakehouse, right? So it's Iceberg native. It allows you to bring your own compute. It allows us to run compute, like scale level operations on, from a metadata perspective.

10:35 That was a foundation to this next evolution of what we call context agents in, in the product. The way we, we like to think of context agents are these AI teammates that make your data AI ready. And earlier this year we launched context agents to our first cohort of, of customers. And then the results were unbelievable. It was the fastest product launch and adoption ever in Atlan history. Eighty-seven percent of users said that AI-generated content was as good as or better than a human. We had just a phenomenal amount of, of user love that started coming in, in terms of what AI was, was able to do.

11:25 The scale over the next, you know, roughly now five months context supply in Atlan grew seven X this year from an autonomous agents going and go... Creating context about our ecosystem. And what was really cool, and we we started being able to analyze against that value realization. Customers that were using context agents and that where their data was enriched through context agents, they also had seven X more distinct assets viewed. Which means more end users were getting more value seven X more value realization than a than an average customer. This was, I have to say, a labor of love. It's been...

12:13 We've been on this for two years and you know, today you and I can go in and throw a table into Claude, and we can say, you know, "Create a, create a table description." And it'll... It's gonna give us something. It just doesn't mean it's gonna be accurate. And a lot of this, the journey that we've had, is that context quality compounds. So, for example, if AI is able to read, for column descriptions, lineage, SQL how someone uses it in BI, the source system that it comes from, it can generate a higher quality description. It can generate a higher quality popular filter.

12:50 Then if you're able to take that and add it to semantic logic and BI systems, you can start reverse constructing things like metrics, which is something new that we recently launched. And then, you know, if you're able to take all of these, you know, you can actually start reverse constructing relationships, which is kind of the, the holy grail of, of what we're trying to do from an AI perspective and you know, the idea of the ontology. And so our biggest learnings actually is that all of this goes back to base principles of data.

13:17 The same way you think about data quality pipeline, you have to think about a context quality pipeline, and that's been the journey that, that we've been on. Alongside this, what has been most amazing, and these are such... Like, I think seeing these messages go out on LinkedIn has just been some of the personally most gratifying moments this year for me. It's also starting to shift the role of the teams. We're starting to see teams go from, you know being necessarily the people who have to go and create context to the people that are validating information. And we're starting to see this new role of a... Or, or roles evolve into that of a context engineer.

13:59 Some of the best stories I've had have been teams that say, you know "Our teams now just review. Our teams now actually can go with 90%, not with a blank slate. We can go to our business teams, and our business teams just have to, like, review, help us review the, the next 10%." And it's just changed the, the leverage that we're able to drive and the speed and momentum that we're able to drive for the business. We're super excited about where context agents is going you know, where we've gotten a ton of feedback, ton of traction across the community. We have a few things that are coming up soon, right from agent templates.

14:34 So you're gonna start seeing this ability to start building, you know, more custom, purpose-built kind of agents inside of Atlan. MCP integrations bringing things from knowledge sources and unstructured sources has been a huge, a huge ask. We're already seeing great traction from people being able to bring in stuff on things that are documented in Confluence and being able to add that into your knowledge bases. That's coming soon. And then scheduling context agents. I have customers now saying "I want to be able to run this on a daily basis," and so on. And so we're starting to bring, bring those those in and, you know, that's going to be in, in pretty soon.

Prukalpa Sankar

15:13 So that was problem one, metadata supply. Let's go to problem two, metadata consumption. What does the catalog's interface for AI look like? You know, about two and a half years ago, all of us had the ChatGPT moment, right? Like, we. And it started changing something about what, what happens and how humans interact with technology. And the reality is humans don't really search anymore. We, we chat. And that's the fundamental human experience of technology that's starting to change. But getting them the right answer about their data estate is really complicated. I just want, like... We, we launched this thing called Conversational Search Atlan.

16:01 We've been on this for now well over a year. And you know, these are examples of questions that people ask us. For example, sometimes people ask search by association, so, "Where's the table someone built for Q3 conference spend analysis? I don't remember the name." Sometimes a question needs the agent to reason. It's not a search right? "EMEA revenue on the dashboard dropped. Is that real, or is it a data problem?" And then sometimes, you know, questions need you to just, you know, load a very specific part of the estate, right? Like, "If I change this table, what downstream dashboards would break?"

16:38 You don't want me to, like, go run a search on your entire data estate to be able to go answer that. And so our biggest realization was that being able to take metadata and truly make it ready for AI-ready context needs a foundational infrastructure shift. And that meant that in certain times, the AI needs to be able to go run a vector search. In some cases, an AI needs to be able to go run a semantic search. In some cases, it runs a hybrid search. It runs a a keyword search and a vector search. In some cases, it needs to run a graph traversal. All of these are different and unique.

17:20 We brought these as foundational primitives into the Context Lakehouse. And so now any foundation search query that happens on Atlan or question that happens on Atlan goes through this, all these four foundations, and we're able to choose and pick what's the right way and the fastest path to, to get to an answer. Then you go into, you know, availability of protocols and MCPs. One of our biggest realizations here has been no two MCPs are equal and no two MCPs are built equally. You know, how do you build. You know, over the last couple of years, we've had, you know, success, six, six more efficient search you know, faster search tokens per call.

18:09 It's been a lot of iteration and flows that it's taken to be able to go and, and drive and build The iterative loops that it takes to make something like an MCP work at scale.

18:23 And then I'm gonna bring it to the last part. This to me is the biggest gold that exists in catalog world. In the old world, we had to guess from an end user perspective what the end user wanted and now to a certain extent you don't, right? Let's say AI, you ask a question, what was our churn last quarter? You get an answer. It looks confident, it isn't really. User loses confidence, but nobody knows what went wrong, right? Is it that we don't have context? Did the AI search or the harness need improvement? Or, you know, are we using this on a model that isn't effective, right?

19:02 Like, is it the model, is it the context, is it the harness? That's the core problem. But in the software world, we had to guess what people were looking for, and now in the AI world, we don't. Users actually tell you. This is an example of all the kinds of traces that we have now where users are directly asking questions of... And we know what questions users are using and asking for. We decided to take this and translate this into a learning engine. And this to me is the last bucket of what it takes to close out a true self-improving catalog.

19:39 You know, you're able to take a question, you're able to take it and translate it into, why did the question fail? This is... We call, we call it Rex. It's a simulation engine. You can find, oh, hey, this is a completeness problem. We actually don't have this context. It can auto-trigger a context agent. This is a possibility. And then you can go back and say, "Hey, now you can answer the question the next time."

20:06 This to me is the final last bucket of the self-improving data catalog where you have like agents that are autonomously starting to create context, be able to go consume it with end users that are asking questions, and then you build this learning engine which can be a self-improving, self-updating kind of ecosystem. I wanna just end with that today. And just say that it feels like that we're on the precipice of something. And I wanna now hand it off to Zhenni to tell us a little bit about what, what the real journey... I'm gonna stop talking and let, let practitioners take over on what their experiences have been as they've gone through this.

21:04 Nandini, you might be on mute.

Nandini Tyagi

21:09 Wow. Making sure all the tech gods are after me today. Thanks for sharing that, Prukalpa, and I'm so excited to pass the mic to Zhenni. It's such an honor to introduce her. Zhenni is manager of AI and data governance at Mastercard, and it has been personally incredible to witness the partnership and what Zhenni and the team have gone through the entire context data journey and the impact they created. With that, over to you.

Zhenni Hu

21:35 Thank you.

Nandini Tyagi

21:38 Awesome.

Zhenni Hu

21:39 And would you be able to pull the slides for me Yes, there you go. Okay, perfect. Awesome. All right. Hi, everyone. I'm Zhenni Hu, and from the AI and data governance team at Mastercard. A quick word on the scale. Mastercard operates in more than two hundred and twenty markets with billions of credentials and around hundreds of billions of the card transactions. So all of that generates data across various domains and spread across dozens of the platforms built over the decades. So those data only helps if people can find it, understand it, and trust it, and that's what we would want to talk about today, how we use AI to catalog our data.

22:18 Before I get into it, I'm curious where this group sits. So two quick questions. First, have you used the AI at work this week? And give me a thumbs up. I will put in the quick questions to the chat. Anything counts. It doesn't have to be anything fancy. Okay. Cool. I'm seeing the number has been increasing. Plenty of you, about what I expected. And the second one will be a little bit harder. Would you be comfortable letting AI to making the decisions that you're getting held accountable for? Yep.

23:07 Not surprisingly, I didn't see any thumbs up comments yet. Awesome. So, right. Noticeably, no surprise that the second number will be much smaller, and most people are comfortable using AI now, but far fewer are comfortable trusting it, and that's the gap we are trying to close. So here's the dynamic we are dealing with, and I think most of you will recognize it. Most of the enterprise governance team's capacity stays roughly flat. The data estate from the other side keeps growing and expanding. Every migrations, every acquisitions, every new platform onboarding or even just a refresh you do adds onto it.

23:50 And as a data steward, the legacy way of manually cataloging the data assets is always a tedious work. It is slow, and it gets stalled the moment you finish. And I suspect most of people here have run an initiative that started strong but quietly lost the momentum. And whoever builds those tables understood them perfectly well, while once they changing the roles or moving to another teams, that understanding went with them And today, the stakes have risen. Cataloging is not only just critical to the compliance and regulations. Forbes says context is the oil of the AI economy. Documentation and metadata used to help people find the data. Now, it also helps AI understand the data.

24:37 When a person cannot find an answer, they reach out to a coworker, but when an agent cannot find an answer, it will often produce one anyway, which would cause the hallucination. So today, the quality of your metadata sets a ceiling on the quality of your AI. So next slide. What did we do? Three decisions, and honestly, the thinking behind it matters more than the specifics of our setup. The first one is the scope. Once you have the AI-generating descriptions, the temptation is to run it across everything, because why not? It's just one single click of a button.

25:14 We held back deliberately because generating hundreds and thousands of descriptions overnight, you build a review queue that your team will never be able to clear. And the first time a steward opens a weak one, that doubt spread across the whole batch. The goal was not just generating about volume, it was more generating something people could realistically review and trust. So we intentionally picked a bonded set of the critical assets small enough to review properly, visible enough that success would register with the business. And second, the evidence. This was probably the one of the most important learning for us like Prukalpa say, because it determines whether any of those is usable.

26:02 If the AI only see the column names, you just get a polished restatement of the schema. That's usually not what the steward really need. What made ours workable is that it reasons over the real signals how the data actually gets curated, either by the human or the systems, the real usage signals, and what it connects to the upstream and downstreams, and our existing enrichment information like the existing business glossary, classifications, data products, and et cetera. That's what made a human review realistic. Correcting a draft that's already closed takes minutes, but rewriting something too generic takes as long as writing it yourself, which is the job that everyone was already too busy to do.

26:46 And third, the certifications. A subject matter expert signs off before anything goes live for the consumption. That was part of the design from day one, and I will be direct about the first question we put here, what does that mean for my job? If you roll something like this out and nobody asks you that, it doesn't mean they are not wondering. At Mastercard, we believe AI is here to augment people, not replace them, and I will come back to what that looks like in practice soon. So here's what we learned. Control scopes makes it realistic, evidence makes the draft looks good, and people make it trustworthy.

Zhenni Hu

27:28 Now let's talk about the results, because eventually, every AI conversation gets to the same questions. What's the value? In a two-week pilot, we enriched approximately thirty thousand assets across roughly seven hundred tables and views and estimated about six thousand two hundred hours of the steward effort saved. From the speed perspective, two weeks means we can repeat... Oh, can we go back to the previous slide? Thank you. So yeah, it means we can repeat this. Traditionally, the manual cataloging takes months. So think about that. A twelve months effort gives you the results that you can only celebrate once. While a two weeks means we could scale the same approach more broadly now.

28:17 We can run it again next quarter on a different domain without the need for a whole steering committee or a task force dedicating all of their time. And when the leadership asks about return of investment question about AI, and they will, these are the numbers that can answer it. A fixed teams now covers a growing estate, and the capacity that came back went to the work that generally needs a person. So next slide, please. Here is the one point that worth emphasizing. The data stewardship role didn't disappear. It's got upgraded. Think about what writing a description actually involves.

28:59 Much of it is mechanical, restating the structure, describing what a field holds, and keeping phrasing consistent. It's real work, but it's less valuable thing a steward can do. Their value shows up in knowing when something is subtly off, recognizing that a term means one thing to finance and something slightly different to risk, knowing which of the three near identical tables is the authoritative one, and that judgment takes years to build, and it stays with the person. So the work moved up, less drafting, more judging, and the coverage the same team maintains went up with it because the reviews moves much faster than the authorship. And there's also a second effect worth naming.

29:46 When the steward certifies something, they are standing behind it. That gives whomever consume it downstream, either a person or an agent, a much stronger signals than an unverified model output. That's where the steward creates more value. And next slide, please. One last thought before I wrap up, and briefly on the direction. The order here is the message. So if the catalog is empty underneath, search won't be able to give you a reliable answer. AI increases the value and importance for having a good foundation. Some of the organizations start with the agents or the search, the last two boxes, because that's the exciting part, and it's what the leadership is asking about.

30:32 That's usually where it starts. Handling organizational context to an autonomous agent where you haven't verified that context yourself is a fast way to lose the trust in the whole program. So we started with the foundations. What we did here built on the years of the inventory, cataloging, and lineage work, which was generally hard and slow at the time. Now, we have the magic few to accelerate and scale it. How well we build a base sets the ceiling for the rest. With that, it concludes my sharing today. Thank you.

Nandini Tyagi

31:10 Awesome. Thank you so much for sharing your learnings, Zhenni. It was great to have you here as well. And I can already see a lot of questions and thoughts being sparked in the chat. So I'm sure community will have more things to come in. Please, please keep sharing them in the chat.

Nandini Tyagi

31:23 With that, I'm very excited to jump to our roundtable section. Please join me along with Zhenni to welcome our other two panelists. We are joined here by Rupal.

Rupal Sumaria

31:33 Hi.

Nandini Tyagi

31:33 I think Rupal is here. Hi, Rupal. Head of Data Management here at ASOS. And Keith, Principal Solutions Architect at Sophos. Welcome, everyone.

Keith Guyett

31:42 Hello.

Nandini Tyagi

31:42 I'm gonna stop there so that everyone can actually see you instead of the screen. There you go. If you all are not updating your laptops, I recommend you do it before the session. Mine decided to do it during, so there you go. Okay. Awesome. Well, hey, welcome. Welcome, everyone. It's so cool to have you all in the same room. Maybe we'll start with the first question here for us. Scale only means something if the output holds up, right? That's something we have heard a lot. I'm curious from each of you, like what did you accomplish?

32:15 How long did it take, and, you know, what did it actually take to get to the context and scale of the standard that you care about? Maybe Rupal, we'll start with you. Maybe you can introduce yourselves and give us a take.

Rupal Sumaria

32:28 Yeah, thank you. Yeah, thanks for having me. I'm really excited to be here, and, and thank you, Zhenni, for, for such a, a brilliant kind of illustration of exactly how context can really scale. We have a very similar story as well in terms of how we've scaled our assets.

32:43 I think what becomes really important to scale is sequencing and how you build that context and the order in which you build that context, because it does have a compounding effect, and the value that, that we now get out of it is, is really key on having actually some really high quality metadata and descriptions because the baselines that we started with were really strong as well, and I think that compounding effect is really impactful. I think one of the things now is data governance has, for, for a very long time, been talking about actually, you know, you really need these good semantic layers, and you really need this great metadata layer.

33:20 But it's always been very difficult to get to the execution of how are we gonna then use it. Actually, that problem shifts very dramatically and very quickly now in this era, and it becomes how, how best to package up that context, how best to make it portable because you have that solid semantic layer, and you have that solid context about the actual data that's built on the lineage and the code that we've actually developed. And that, that IP is really, really valuable to gaining the edge when it comes to using and creating more agentic solutions. So I think that, for me, has kind of been the biggest— Yeah...

33:54 Win overall is like we've won that, that battle. I think the human loop is still very important because it helps guardrail and make sure that it's making sensible decisions. But the human loop— Yeah... Doesn't have to go through masses of tedious work. I, I actually calculated on your 6,200 hours, that would be 258 days, which I think we can all say I don't think any business would ever do that, would ever commit to that amount of time. So that scale is, is really impressive.

Nandini Tyagi

34:22 Amazing. And I know, Keith, you feel very passionate about context and scale as well. I know you built your own methodology as well here. Would love to hear your take too.

Keith Guyett

34:31 Yeah. Really just mirroring Zhenni and, and Rupal and then, and even the stuff that Prukalpa said earlier we started out with a, a base question of can we do better than a blank page? And the answer to that was always yes. But over time, it evolved into we have to do better than what Claude can reason out on its own to provide value. And, and the answer to that is also yes. But from our point of view, we onboarded more than 600,000 assets to begin with back between January and, and before we started the accelerator in, what was it? End of March, early April.

Nandini Tyagi

35:14 Yeah...

Keith Guyett

35:14 And then during that accelerator, we, we enriched more than 9,000 assets before as we got started. But then we found out that doing it fast was not the correct answer because doing it fast just gave us, like Zhenni said just static restatement of the underlying model, and it didn't provide any business purpose to it.

35:37 So we then take, took a step back, improved the prompt with prompt engineering like you would with any other AI agent, and told it give it business purpose, give it grain and scope, look at metric relevance, give it examples when available, and just as importantly, look at the lineage and the source context, so the metadata around the metadata, to give us better answers. And then we went back, we re-enriched some of those initial ones. We added more enrichment. We ended up with like almost 43,000 assets that were enriched during the process between April and early May.

36:13 And just as importantly, that's a lot of assets, and we can't ask the business to evaluate every single one of those. So we built a test harness in Cortex Code that used the AI complete function and basically took the answers to school and told it to grade them A to E, basically a pass or fail. And we found that About 80% of them would give a A to B scale, and that gave us the confidence to say, "Business, these are what the descriptions are gonna be. Please overlook at them and give us a, a good go ahead."

36:51 And then we pushed those descriptions back left in our model, and we used those descriptions to enrich additional upstream data, and I think we're at almost 100,000 assets that are enriched and providing business value through cust— people using the AI, as well as the interface in the UI.

Nandini Tyagi

37:15 That makes sense. I remember seeing that grade sheet when you, when you shared that with us, Keith. It was cool to see how you went about it. Zhenni, do you wanna add anything to it before we take the next one?

Zhenni Hu

37:25 Sure. Just really quick, since I covered the numbers earlier, so I will talk a little bit about how we know it was good, right? So we went back to the subject matter expert, doing some reviewing and collecting the feedback and just asking two questions, like two simple questions. How much percentage that they can directly accept with no changes? And for the rest, were they, like, doing the really the editing or they have to rewriting from the scratch? So editing means that the draft worth having, rewriting would have been, like, the time is not being saved, right?

37:54 So on some of the example, on the one of the critical asset we sample with one team, roughly half of them came back with no changes needed at all, and most of the rest needed just small tweaks. So that's why we know something is working. But again, like report say, it wasn't uniformly good. It did well on the tables that people curate more oftens, have more signals on the curate insights and lineage, where, like, those are the real usage, right? And they can reasoning from. But it will be much weaker on the rarely touched assets and on the terms that meaning different things to different teams.

38:25 So which is where, like, you want a human anyway, so...

Nandini Tyagi

38:29 That, that makes sense. And I loved how we heard three different approaches on how you all went about and building your own trust and, like, does it work for us? We looked at, like, a number approach. Heard about, like, an emotional approach as well. So that was great to see. Awesome. I know it gets very hard to stay focused on what matters, and it's very easy to, like, boil the ocean. Would love to hear... Like, talk us through how you scoped the work. Like, what you chose to prioritize and why, and what does that process look like for you?

Rupal Sumaria

39:01 I think for me, scoping it really focused on those popular assets, and I think the, the context agents do a good job of pointing out which are your most popular assets.

Nandini Tyagi

39:09 Yes.

Rupal Sumaria

39:09 It makes sense to start there. I think particularly when you're starting out and you don't quite know what the context agents are capable of, you can test it on something actually you know that there's a lot of actually already inbuilt understanding that actually sits within your organization, and that's a really good way to start. I think the next thing is, as I say, sequencing is, is... I, I actually really preferred following the chain of my pipeline because I think you build knowledge and the transformation on top of each other, and I think that is then a really useful way of doing it.

39:38 We also have, like, great engineers that are very committed to making sure that whatever descriptions they can put together, they do, and they're really high quality. And again, that just means that the context that you're starting with really does have some really good leverage to start from. And then you just build on to your semantic models, and then from there you can build out your glossary terms and your metric layer, and that... Your metric layer at that point is actually grounded in the formulas that your business is using. I think that becomes really, really important.

40:06 So for us, it's start where you can in terms of just testing out, making sure you've refined it, and make sure that you have the context that you want to give. Tell it what your business is, tell it what you care about, tell it your strategy in terms of that prompt engineering. And then me, for me, it's left to right. I think that is the best way to go. I also think, same as, as Keith, it's making sure that it's good enough, you know, that, that actually it's better than a blank piece of paper. I think you can boil the ocean and worry about perfection, but perfection is not how your business runs.

40:40 Your business runs on a, can we get 80% there? So you operate the same way.

Nandini Tyagi

40:45 Makes sense. I think a related question to this, I would love opinion here, and it came in chat too. What happened to the documentation process that you had before? Like, did this change certain way of your processes? Did it retire certain ways you're working with or update them? We would love perspective on that too.

Rupal Sumaria

41:01 I think we definitely updated our templates. Like, you know, we, we made it a little bit more aligned to how actually the, the agent is providing. But I think the benefit there is we also know that's how other agents are gonna read this as well, so we're not building now just for other humans to, to consume. It is about how agents will consume as well. So that does change a little bit about your documentation approach. It also makes tagging much more crucial.

Nandini Tyagi

41:28 Makes sense. Keith, Zhenni, or do you... If you wanna add your perspective.

Keith Guyett

41:33 I, I can add perspective. We started out with a, a pretty tight focus, we thought in that we were targeting one or two domains in our business and, and working closely with them, but we quickly found out that even that was too large of a focus and, and we were losing sight of what we needed. So we quickly shifted over during the accelerator to the, the context agents, and they helped us with a, with a large portion of it to give us a good foundation. And then we ended up building some custom context agents that we were able to target a little better what we were doing.

42:11 And finally, we have actually even changed the scope further to build into our product life cycle. We are building out by data products, and we're defining those data products in Atlan, and we have built into the development life cycle all the way dev, QA, all the way to production that this metadata, this description, all of this terms need to be defined as part of the rollout and deployment, or it doesn't move forward. So the, the focus is now on we're providing value where the business is looking for that value in those data products Yeah, a lot of Keith's and Rupal's things resonate with our ap-approach as well, and I will probably do a high-level summary.

Zhenni Hu

42:57 The three things we use in doing the prioritizations, it coming from the business cred-criticality and where the stewardship already, already existing, and also the usage metric signals. So I think with all those, it makes us to find out the domain where that engages stewards, where will be the better first fit for the candidates because someone was ready to do the validation work, right? And also from a usage, we're starting with the most curated assets. So if a lot of people are already using something, that's where, like, we would like most of them to using them correctly. And also that's the things that we want AI to understanding first.

43:29 So that's kind of high level how we do the prioritization and scoping.

Nandini Tyagi

43:33 Makes sense. And that's super helpful too. Okay, moving from writing context to validating it and scoping, there is for sure one shift. If we zoom out, there is a bigger one that's happening about roles themselves changing. You all are part of different functions. Would love to hear your perspective on how has your team's role changed, and has this opened, like, door to new teams or, like, new conversations from partnership point of view internally?

Rupal Sumaria

44:01 I think maybe Prukalpa mentioned it the best, as context engineering becomes more prevalent, and it becomes more of a discussion that we have internally about, you know, is this the direction of travel? I think... I don't think we've got a, an answer to it yet, and I don't think we've solved it, but I think it definitely highlights where the governance team now takes on a much broader role than it previously did. And, and the traditional data governance model shifts as well with that. That, that's how I would view it.

Nandini Tyagi

44:31 Makes sense.

Zhenni Hu

44:35 Yeah.

Nandini Tyagi

44:35 Yeah.

Keith Guyett

44:35 Yeah, I think doing stewardship was our original thought was that we were going to have to help the stewards with definitions. But what ended up happening is we had to start defining through governance who owns these definitions, who approves the metrics can AI use this safely and effectively do they have the right entitlements how do we validate this stuff, things like that. So we ended up having to talk to engineering teams, business owners, identity and security to make sure that it was acceptable to do things this way. So it went from documenting to enabling trust.

Zhenni Hu

45:20 Yeah. And for us, we, as a governance teams, we used to be serving majorly for the privacy compliance and regulation asks. Application teams are being told this is mandatory, and please go in and fill in those descriptions for those fields and enrichments. So what changes today is people coming to us, they want to know what context we are building and how they can consuming it. And we are getting to pull into the strategy discussion earlier these days while things are being designed instead of reacting after, like, things have been built. And we're also being... Asking, like, literally the agentic downstream how they can consume, what's available today, what's on the roadmap to make it happen.

45:58 So I think people's mind has been shift on... It's by themselves, like we don't have to pushing them too hard.

Rupal Sumaria

46:05 Would agree with that. That, that's exactly what we saw internally. And, and I think what was then an added bonus is then being able to, like... Our, our business is asking us, "Okay, we really need to get this metric ownership and definition right because these, you know, we... These agents are sometimes veering a little bit off course. So how do we ma-make sure that it's, it is grounded in how ASOS uses this, this data?" I think what really then added that extra kind of I guess kudos to the whole thing was really that we were saying, "Actually, it's... The context is there, you just need to validate it.

46:36 You don't need to go and kind of draft it from scratch." And I think that has helped win over, I think, a lot of how governance has been perceived and really kind of shifted that, that role.

Nandini Tyagi

46:49 Yeah. Makes... I think, I think we found that— Please go on.

Keith Guyett

46:53 I, I was gonna say, I, I think we found that a good enough description for a human is not necessarily a good enough description for an AI agent because the human has that business knowledge ingrained, but the AI agent has to, to reason through it.

Nandini Tyagi

47:12 That makes sense. And I think I wanted to unpack a little bit what I think all three of you touched on this opening doors to teams who are building their own agents or, like, like internally, like there's some consumption of it happening. So we would love to unpack that a bit. Like, what's the shift that you're already seeing on context being consumed, and does... How does that change what you're building for? I know, like, everyone is at a different journey. Like, some folks are already building agents. Agents are the primary consumers. Some still have humans. Would love to see, like, how you are starting to see that shift internally.

Rupal Sumaria

47:44 I think we're being very strategic— It's just about building—... About the agents we build. I think that's the starting point is being strategic on those agents because even with context, there's context... Like, at the moment, Atlan is only holding a certain amount of context, and we need to obviously keep adding more. So being strategic about what you build, being very careful so that you, you don't lose that goodwill, and you don't lose that kind of trust in, in the data and trust in the context because you've added something or cr-created something that isn't quite fit for kind of business consumption.

48:14 So I think being strategic, being very clear on the scope of what you want, and focus your context engineering around that is, is really the focus that we have, particularly moving into the next year. I don't think we've solved it all yet. I don't think anybody could say that they have. But I think that's, that's really as, as Keith and Zhenni have said, like just being very strategic, scope it very clearly.

Keith Guyett

48:39 And, and I don't think it's all just agent building either. That, that's kind of our, our end goal, but we're having to support now people using Claude Desktop and just asking the data directly without Extra agentic guardrails in place. And we're building out tools that will allow them to have their Claude Desktop ask Atlan first before it tries to reason out an answer so that it can get a good business context.

Zhenni Hu

49:11 Yeah. And to add on that another big realizations we've been thinking about is enriching the individual asset might not be enough 'cause more and more we are seeing for specific business questions or use cases, the real context also come in with the relationships, whether you say it's lineage or ontology how the assets are meant to be using together, which one to join and how, and what's agreed metrics and what's the guidance or the prompts or instruction applies to them, right? So a lot of them lives not only in those relational, traditional kind of a cataloging. It might in some instruc— unstructured documentations and all these different places, right? So I think we are not very...

49:53 We are not there yet, we are but this where, where we think that we want to building maybe for the next years or so to be there. Yeah.

Nandini Tyagi

50:01 That makes sense. I know we got a few questions in the chat on your pre, your journey pre-Atlan. I know we didn't cover that in intros for anyone. I think few people are curious. Were you transitioning from something? Was this, like, your first attempt in your orgs setting out, like, the DG function? And, like, how did that look like for you?

Rupal Sumaria

50:23 This is ASOS's first data catalog, but it's not my first data catalog. And so the focus for us has always been around usability and accessibility. So I think making governance look modern and feel modern and actually be modern and automated is, is always been a key driver for us. You know, we, we really try and push our boundaries in terms of technical capability, and I think this has really helped in terms of this partnership. And we see it with... It's paid off in terms of how we've been able to scale context. So it, first for ASOS, not for me.

Nandini Tyagi

51:00 Awesome.

Keith Guyett

51:00 Yeah, I'll agree. It, it's not my first data catalog attempt. All the past ones have always been failures because of stale data, people losing interest, stuff like that. What's helping Atlan gain traction is, is the social aspect of it, the ability to talk to it, the ability to, to favorite things and go back and find it, and not having to go search a Confluence page that may not have been updated for two years on a asset that got changed last week.

Zhenni Hu

51:32 Yeah. It's also not our first cataloging tooling, but traditionally we have been more focusing on the privacy, so we're making all the data available for privacy search purpose, but it's just the whole ocean that it's too much information that people feeling it's hard to consume it and being valuable. So right now with Atlan, I think we have the opportunity to use the persona for different set of the group, showing their interest, and have the way to building a control scope for the data products that can better guiding people to the stuff they want, and also with conversational search.

52:02 Those so those are also give us all different opportunities to see increasing user experience and to making something really valuable to them instead of just saying we have the same story there.

Nandini Tyagi

52:15 Makes sense. Awesome. I know we are inching towards the end of our time together. I'd love to wrap the panel session maybe with, from the three of you. For everyone listening here today, I know everyone is going to be at different points in their journey. What's, like, one lesson or one learning you'd want to leave them with? One takeaway maybe.

Rupal Sumaria

52:35 AI isn't gonna wait— Yeah. AI isn't gonna wait for governments to catch up, so get moving.

Zhenni Hu

52:41 I was about to say the same. Just start. Don't wait to, waiting Yeah... Perfect strategy solution.

Keith Guyett

52:48 Yeah, and I think my, my takeaway is that metadata and these descriptions can't just be a bolt-on to the end of your process. You have to integrate it into your data flow, into your entire enterprise process.

Nandini Tyagi

53:03 Amazing. And one, one more learning I'd love, and I saw some questions here as well. Maybe one takeaway on building partnerships internally. Like, what has been, like, one learning you'd love everyone to leave with? Like, as you've done this, I know, I think few people mentioned, like for the data stewards, did you... Like, what did the review process look like? Different teams look at it differently. Like, what, what was one learning? You said get moving, move fast, Rupal. So what was one learning or takeaway for that?

Rupal Sumaria

53:33 I think we are... We, we don't shy away from AI, so we use AI in our training. So we, we actually enabled training for our whole business, not just on Atlan, but actually our wider data ecosystem for all of our ASOS's. And we did that in record time, and I think that push that push environment that we were in really does help. So I think timing is right. Making sure your culture is right is also important. I don't think just what works for me is gonna work for someone else, but I think, you know, if you've got a good sense of how your business operates, you'll find ways around it.

Keith Guyett

54:11 I think finding an evangelist, someone that trusts and believes in the process and the tool, and enabling them and putting them in front of everybody you talk to helps you with succeed.

Zhenni Hu

54:27 Yeah, and same for us, that finding a good partners that you can build in a quick POC and show the success, and it's only we see it's getting, getting more buy-in from the broader groups if you can show something real and have somebody be an advocate for you, seeing it's really works for their business and it should work for you as well.

Nandini Tyagi

54:45 Amazing. Thank you so much for sharing everything. I still see there are a bunch of questions coming in the chat, so maybe we'll do a follow-up session on this at some point. But I think we have covered most of the themes. I see Avinash, my colleague here on the team, is also answering some of them. I see one last that's open. Maybe everyone talked about measuring the time that you all saved and, like, the impact. Any learning on that? And we'll call that as the last one for audience.

Rupal Sumaria

55:16 I think we saved about six months' worth of work so the impact is real. You've got numbers. Everyone always asks what's the ROI on data governance, and I think that that's your clearest numbers that you can start to give than ever before. So we can actually quantify what we've been asking for for many years.

Keith Guyett

55:36 For us, I think it's really hard to quantify only because we don't have a baseline. Something like this would have never gotten done without this tool.

Zhenni Hu

55:48 And it's a little bit of both for us, the time we are saving as well as the empty field versus now at least we have something. It might not be perfect waiting people to review, but better than nothing.

Nandini Tyagi

56:00 Okay. Amazing. This was packed with insights and lots of takeaways. I hope everyone who joined got something for the journey they are in to get started. Thank you Zhenni, Rupal, Keith for your time. It was amazing to see you all in the room again. I know we all hang out on different community sessions but love doing these things together where you get to share with broader community too. I'm going to quickly screen share as we wrap up. One sec. All right. So with that for folks who are still here want to give a quick shout-out to Context Conference.

56:44 I know we get a lot of questions on what's the next thing we can go join to learn more. This is a conference for teams helping understand their business. You'll hear from leaders from AstraZeneca, BNY, Verizon, UKG about their architectures. Saw a bunch of architecture source questions as well come through in the chat today on questions. And you'll also hear like from builders of Apache Ossie, open knowledge formats, and on what open context generally looks like. And you'll hear from thought leaders like Jaya Gupta on how the space is evolving.

57:13 So as a follow-up it'll be a great opportunity for you to learn at broader industry level what's happening, and we'd love to see you all there as well. We do have two minutes remaining. We're giving you all a hydration break. Thank you so much for joining us on Thursday. I know it's morning, evening, afternoon, depending on your time zone. It was great to have you all here, and hope you have a lovely rest of the week. Awesome. And thank you again, our panelists, for being here.