Building Production-Ready AI Analysts

Anirudh Agarwal

0:00 Let's get started. So the agenda today in front of us is a really exciting one with everything that's happening with AI. We're gonna talk about how context plays a role in building you know, AI analysts and what the path that looks like to building production-ready analysts and shipping it to end customers look like. With me today, I have two amazing people joining me. First we have Josh, who's joining us from the Snowflake product team. He also leads the OSI initiative which talks about interoperability across semantic sources.

0:38 We also have Deb, a colleague of mine who works in the product team at Atlan, and he's gonna be doing an amazing demo of how we are looking at solving the context in AI analyst problems. And then myself I work in the product engineering team here at Atlan. And funnily, I used to be one of Atlan's earliest customers. I've been a data practitioner all my life, so now it's just, you know, super exciting to be solving the same problems I once experienced as a data practitioner, but now you know, leveraging AI and how the world's changing very, very quickly because of that. Cool. So let's get started.

1:18 So first, you know, even before we jump into the session, just wanted to share some, like, quick you know, housekeeping. Firstly, you know, this webinar is being recorded. We'll share it with you folks afterwards, so feel free to take notes if you want to, but know that you're gonna get this at the end of the session today. We love for you folks to engage with us. Please use the chat, ask us all your hard questions and, you know, we, we are all gonna respond and make sure we are able to help you with all of them.

1:47 We'll still have a Q&A at the end, so, you know, we're gonna take up some questions that we find really interesting right at the end of this discussion, and we're gonna be answering it live and we'd love to engage with you folks there. And then we're also gonna be doing some polls along the way. 'Cause polls helps us, you know, understand if the session we are doing it was, you know, helpful it helps us gather signals in where you folks are in your journey in building with AI and building analysts, and would be amazing if as many people can engage with them and give us their thoughts. So a quick agenda for today.

2:25 Firstly, it's gonna be myself, and I'm gonna be talking about the AI context gap and why it's very, very relevant for the AI analyst use cases. Josh is gonna be talking about how semantic views work, and he's gonna be talking about the OSI initiative. And then Deb and Josh are gonna do a quick demo of Atlan's Context Studio. It's a product we're working on at Atlan to help with you taking your AI analyst to production. And then Josh is gonna be talking about how all of this mixes well with OSI and Snowflake Cortex.

2:58 And then right at the end, all three of us are gonna take your hardest questions, and we're gonna do our best to try to answer them for you. All right. So even before we jump into the slide we would love to just, you know have a poll. F-for everyone here is: Where do you think your current state of context is in your enterprise today? We have some options in the you know, chat. We'd just love to hear from everyone what's the current context strategy for everyone, and this will help us understand where everyone is in their journey. All right.

3:48 I already see some answers coming in. Keep it coming, and we'll start with our session first just talking about the AI context gap. So you must have seen the MIT report that over ninety-five percent of AI experiments are not moving into production today.

Anirudh Agarwal

4:07 As we've been working very closely with our customers, we realized that it's not a problem with the models. These models are kept keeping on getting better. You're seeing what... The amazing things that are happening with Claude, with OpenAI. And what we are seeing, one of the biggest challenges is not that the not the models aren't good, it's actually the context doesn't exist, or it's scattered for these experiments to actually make it into production. And how it's materializing for our users is in, is in three ways. So firstly, what we are seeing is that there's actually a connectivity problem.

4:42 There is scattered context all across their data landscape, which, you know, their AI is not able to read from and be able to bring it into one single plane. The second problem that we are seeing is a knowledge problem. A lot of the existing knowledge that exists within an enterprise, it's in people's heads, it's in disparate, you know, sources of knowledge, such as your Confluence documents, Notion, all of your different sources of documents.

5:09 And then finally, we are seeing the meaning problem emerge, which is, hey, how do you take all of these different sources of data, all of these different sources of knowledge, and bring it together into a meaningful way where AI can work on top of to be able to answer, you know, all your questions that you're trying to build an AI use case for? So let, let's take an example, right? And, and this, this example is a really simple one, right? So let's say you build an AI analyst, and you ask this question, which is: What are top customers this quarter?

5:42 Now, this question is deceptively simple, but what it really requires is layers of different kinds of context to be able to answer this specific question. Now, the moment you ask this question, the AI now needs to know how do you calculate revenue, right? Is revenue net of discounts and refunds, or is this the total value of your transaction itself? It needs to know which tables to look up. It needs to know what each column means. And that's the first layer that your AI analyst should be aware of, which is just the data context itself. Now, it comes back to the meaning context, right? We asked, "Hey, what are our top customers this quarter?"

6:20 What does top mean for your business? Does this mean by, you know, your average order value? Does it mean by total number of orders that your customers are placing? How will your agent know which is the right definition of top ten for your business? Then comes what is the definition of the customer itself, right? Is, is this a customer who is, you know, a an individual customer? Is this an entire organization? Like, how do you define customer? How do you feed this context into your analyst will determine what the accuracy of the end result is.

6:52 And then finally, it's the user context, and we see this all the time, that finance teams will look at sales differently to you know, a sales team on how they are gonna look at total revenue. So all of that are the different layers of context that you need to bake into for your, any of your AI use cases to get the right outcome. And as you can imagine, this, this is quite a complicated and hard process to actually do. Now, as we've been working very, very closely with our customers who are trying to ship their AI analyst into production, we realize that teams are facing, you know, three core challenges.

7:29 So the first core challenge is just the context bootstrapping problem. Now, what does context bootstrapping here mean? It means that their meaning is scattered in not just their tables, but it, it lives across their systems of knowledge, which is your confluence. It, it lives in all of your different you know, documentation tools. It lives in their BI dashboards. It lives in, you know their data warehouses. So context is scattered, and teams need to do a lot of work to be able to take all of this context and bring it together into one place before they can actually start building out their AI use cases in the first place.

Anirudh Agarwal

8:09 The second problem that we're seeing in our custom— with our customers is something that we're calling testing hell, where customers are able to rapidly or quickly build some version of their AI analyst, but they don't know how good or bad it is, and it is stopping them from being able to roll it out to their end business users. We had a customer who told us that they've been stuck in this testing hell for six months for an AI use case, and they were writing thousands of test questions to be able to test it, but they weren't confident enough, confident enough to roll it out to their end customers.

8:41 And then the final challenge is context portability. Now, even when you've built out an analyst and rolled it out, one of the key challenges that we're seeing across enterprises is that enterprises are not just using one agent tool, they're actually using multiple agent tools. They're using Claude, they're using Snowflake Cortex, they're using Agentspace, and they want this context that they've created in a single plane to be able to, to be reusable across all of the different you know, agentic frameworks and agentic tools that they are using across their enterprise. And Josh will be talking a little bit more about how OSI helps with this particular part of context portability as well.

9:22 So how, how do we actually solve this problem, and how does this come together? Now, how Atlan helps you solve this context problem is by first connecting to all of your different sources. So Atlan has connectors that connects to your systems of record, like Salesforce. It connects to your systems of data, like Snowflake, Databricks. It connects to your system of semantics, which are nothing but your BI tools. It connects to, you know, systems of knowledge and systems of work and brings it all together into a singular enterprise data graph.

9:52 Now, what we do in the enterprise data graph is we process all of the SQL transformations and relationships between all of these different systems and build, bring it all together into a single view. And Deb is gonna show you how that looks like within Atlan today. And with the enterprise data graph, you have things like the semantics, tags, data profiling, data quality, business glossary. All of that lives together in, on the enterprise data graph. When you have the enterprise data graph, what we believe in Atlan and what we've seen work for our customers is this idea of an AI flywheel.

10:26 Now, what we do is we know your business context and semantics exist, but it's scattered across your enterprise data graph. We leverage that to be able to enrich and bootstrap more context for you. Now, imagine that we're able to add better descriptions for each of your tables, for each of your columns. We, you know, suggest popular filters. We build you know, semantic views out of your enterprise data graph. Build— And even bootstrap a complete ontology for you on top of your enterprise data graph. Once we do this is that we realize that once you do this, you actually need a lot of human colla— human and agent collaboration.

11:05 Context is a multiplayer game where agents and humans need to come together to collaborate and build together. So we build this workspace within Context Studio that allows you to collaborate with context, test and iterate, and keep building these feedback loops that can help you continue to improve your underlying context. And then finally, you have different interfaces by which you can access this context. You know, you have SQL, you have APIs, you have MCP servers, and also native integrations with various agentic interfaces like Snowflake Cortex as well as Atlan's, you know, Iceberg native context layer where you can directly access them just through SQL and integrate them in all, all of your AI use cases.

11:51 And we'll see more of it come to life as we go through the demo. Cool. I'll pass it over to Josh to, for OSI.

Josh Klahr

12:03 Oh, thank you. Thanks and good morning, good evening, good afternoon, folks. I thought I would just share a little bit about I, I think one of the critical elements that we're seeing in terms of the ability to take— The types of context that Anirudh was describing, and how do you persist it and apply it so that you can kind of use that context to answer analytic questions. And at Snowflake the way we've been tackl-tackling that, or one of the ways we've been tackling that is through a feature we launched several years ago or several last year called semantic views.

12:43 And you can view a semantic view as an object that allows you to store the context that is needed for analytic applications. These might be customer-facing applications, these might be your business intelligence tools. Increasingly these are agents and experiences of kind of AI-powered talk to your data or agentic experiences.

Josh Klahr

13:06 And what we were seeing in the market is that customers were struggling because the semantics actually sat in the consumption layer instead of the platform layer. And what they wanted was, how do I create a language that enables shared semantics across all of these consumers enables consistency enables kind of a, a shared view of the world regardless of how you're asking those questions. And so what a semantic view is, if you are a traditional business intelligence person this is a model that may look very similar.

13:40 One half of what we store in a semantic view is essentially a projection of key business concepts, metrics, dimensions the way that you might wanna slice and dice your, your data using business terms that make sense to your business users. These are projected on top so, on top of base tables and views that, that sit in your data. And if you think about maybe the signals Anirudh was talking about, harvesting context, if you think about the SQL queries that are running on your Snowflake platform or your data platform every day, there's context hidden in there. What are the joins? What are the aggregations people are, are using? How are they naming or aliasing those columns?

14:20 So semantic views allow you to store kind of that relational semantics and project metrics and dimensions to both AI as well as declarative clients that may wanna ask questions about those kind of those domains. The other half of what is in a Snowflake semantic view are instructions that are helpful for AI to reason about that data. That includes things like verified queries. What are the types of queries that have been asked in the past, and what does the SQL generation look like? This really helps an LLM reason about a complex domain. There may be custom instructions.

14:57 What are the, the extra rules of the business that maybe are stored in a business glossary or in a Confluence page somewhere? How do we take those instructions and actually attach them to the analytic domain, so that when you let an LLM loose and ask a question, it it respects your guardrails, it respects the, the meaning of your business? You can also use search services to do RAG search on kind of terms that people might look up. I, I might say I want to show, you know, all the credits for customer named Foo. I might type Foo wrong, or I might know that might not know that it's Foo comma Incorporated period.

15:34 So we can do fuzzy matching when you're asking questions. All of these things, if you think about this package represent kind of really important context that helps LLMs reason about data. In Snowflake, semantic views are stored like a database object. Just the way you would create table or create view, you can create semantic view. This means that you get things like RBAC. You can de-determine who can access those semantic views. Snowflake has cross-region, cross-account replication. Semantic views benefit from that just by being a native database object. And Snowflake semantic views, I'll demo this later have both a declarative interface as well as an AI interface.

16:20 So you can write a SQL query against a semantic view, and you can als-also pass that context to an LLM and ask the LLM to write the SQL query, but you're sharing the same semantics. And so this is... And, and I think the stuff, what, what, what I'm really excited about with the Context Studio work that Atlan is doing and Snowflake semantic views is the semantic view is a, is a, is a database object. It's, it's something that you have to build and create and curate, and that's actually the hard part. Like, how do you curate the semantics? How do you make sure it's high quality? How do you evolve it over time?

16:55 And so having this loop of context curation, context refinement, and then a place to store and persist those semantics I think both of those things are really critical. I'll mention one last thing. One of the things that we are hearing from our customers, from our partners, and from the market is yeah, semantics are really important but we also need semantics to be interoperable. We don't want to be in a position where semantics get stored in a single platform and can't be easily transported from system A to system B.

17:34 We want to be able to support easy interoperability and compatibility between kind of various players that may be participating in the semantics harvesting or semantics execution game. If you think about a typical BI stack, you may have kind of a data workflow engine like DBT. You might have a catalog and a business glossary that you're using. You might have an execution engine. You may have BI tools. And we wanna make sure that all of those tools can share the same definitions of semantics. That's why we're partnering with folks like Atlan, Atlan with something called OSI, the Open Semantic Interchange, which is essentially developing a shared standard language for defining semantic models.

18:22 And it allows those kind of documents that define semantics to be shipped back and forth between partners to enable synchronization and consistency. And so we think this is a, a really big unlock for some of the things that you'll see in, in the demo in just a moment.

Deb RoyChowdhury

18:43 Thank you, Josh. I couldn't have asked for a better segue. I would say I'm really tempted to try and answer that question, which is really loaded, how do we make LLM write good SQL queries? I mean, LLMs can already write good SQL, but I think the answer is a combination of things that have already been mentioned, and hopefully I can show you some aspects of that.

Deb RoyChowdhury

19:06 I have started sharing my screen. I'll remove the question away, and assuming that all of you can see, I will point your attention back to the slide that Anirudh was sharing a couple of slides back, where it was the flow from left to right of where your data assets are, what is your data landscape, and the enterprise data graph. My name is Deb. I have been in data for a long time, and I evaluated Atlan I've not been a user like Anirudh. I evaluated Atlan at Jungle Scout a few years back, but I've worked at GM, which is an Atlan customer, and a bunch of places in data in healthcare and automotive. And I...

19:49 These problems are really, really near and dear to all of us on the call, and I hope that all of you will also benefit from this conversation, so please join in. Now, with regards to Atlan's journey, the, the context layer story essentially began a long time ago, and it, it actually began with this aspect of connecting to all of these disparate systems, which has all of your data and your meaning. The challenge is not that an enterprise doesn't have enough context, but the fact that the context is fragmented.

20:22 So obviously, any player in the data space knows that we need to invest, and Atlan obviously has invested over the years heavily in connectors data warehouses like Snowflake, consumption layers like Tableau, Power BI transformation layers like Airflow, dbt, Fivetran, and, and more. And, you know, system of record goes, you know, you look at CRM systems and others. So this is the starting point where you start getting the data in. Now, what do we get when we get the data? We believe that a, a critical atomic part of your data graph is your tables. Is your tables, and your tables has your columns and descriptions. And not only that, there are transformations in lineage.

21:07 So this is just laying the foundation of wanting to build context, engineer context based on top of this. Now, as we all know, this lineage is not. I mean, it could be as simple and as complex as we make out to be. Now I apologize for this graph that is not so easy on the eye, but hopefully you can see and experience how deep and wide these things get. So knowing all of these things are extremely important for the first problem of bootstrapping context, which then needs to be made usable for collaboration with humans and AI. Now, the first problem in collaboration essentially is discovery.

21:50 How do we bootstrap con-co-context and in, in the context of answering a particular question like the one that Anirudh shared, which depending on a sales scenario, would rely on specific assets. So we think of this, I mean, obviously AI helps us with many of these things. We think of this problem as where do we start, and what do we use to get started quickly and fast? Because we are obviously all on the hook to deliver outcomes as soon as we possibly can with AI accelerating everything, right? So here, sales cri-critical assets is a bucket which gives us a view of critical assets with a bunch of agents.

22:32 Now, these agents can look at gaps in these assets and give suggestions, perform some automated enrichment, gives us a shortlist of the assets that matter, and helps us bring in this collaboration and human in the loop where we can do things like resolving ambiguity. If you have multiple dashboards consuming, and there are differences in the actual definition, how do we... What metric do we want to certify? Or how do we want to deal with this ambiguity? So we can, we can look at that, and this actually enables us to now enrich at scale quickly using these agents. We look at the reality of the consumption layer of SQLs, the joins, the filters, the calculated fields.

23:15 We look at the BI consumption. We look at the pipeline of transformations, and our goal essentially is to leverage all of the existing business logic, which is already encoded in the enterprise data graph, to build this out fast. And, and what we get is we get from the existing semantics, consumption-ready context baseline, right? So that, that is what spins the AI flywheel, right? And we still keep humans in the loop, and, and then we get to engineering context for humans and agents to answer natural language questions. So this is, this is Context Studio. And Context Studio as I answered in the question, is in preview with our early cohort of customers.

23:59 And in the engineering flow, we, we take it as a natural language flow and come in and ask to build the semantic view, or we call it context repository, and a manifestation of that is, as Josh showed us, semantic view. So how it works is we start with a prompt, a simple prompt of wanting to build, and now the the agent goes in and search for the assets and gives us the suggestions of assets. You can also browse and select assets Manually, if we know that there are things that we want to pull in, for example, if you want to pull in things like returns we can pull that in.

24:45 And we can basically layer in with these items selected the assets that we want to bring in to this particular context repository. Now, I'll click this, but this takes a moment which is why I, I have other screens. So what this does now is goes into this loop of building the context repository using all of what we have already shown.

25:14 So it's the tables, the columns, the metadata, the downstream consumption layers, the transformation logic, the SQL queries, the reports that we know, and it identifies the definitions, filter patterns in order to create a semantic a semantic YAML, which will then be useful for us to iterate. Now, the, the problem that this particular product needs to solve is a problem of confidence and repetitive testing.

25:46 So we talked about first, bootstrapping context is difficult, but once we have bootstrapped context, we know that LLMs are non-deterministic, and we know of customers who have been spending large number of months, even to the extent of years, to try and figure out the fine-tuning that is required to engineer the context to, to get it to a point where they are happy with the accuracy. So it's a problem of what does good look like for an AI use case, and how do we know that we are there, right? And we want to escape the testing hell, right?

26:19 But because we already have the enterprise data graph, when we generate this semantics, we can, we can create this semantic model. And I'll just go from here so that we don't wait for two, three minutes. But basically we'll get the recommendation of the metrics, the filters, the relationships, the instructions which form the models. I can select all and apply, and what I get then is what we call this context repository. And you will notice that this is similar to what Josh just showed earlier, right? So we, we are now in this place where we have a context repository, and we can ask questions of it.

26:54 Now, just to keep things consi-consistent, we will show this view. This is the context repository that is created from our Snowflake demo dataset. We have pulled in these tables, and we can, we can do a few things here. We can ask questions of the data, right? So let's say something like, "Show me total profit and revenue by store state." Now, when we ask this question, what it is doing in this demo is it will... You'll notice it will use this, this context repo, which has all the instructions, synonyms and everything, and it will use Snowflake Cortex as the underlying engine to create a SQL query and answer the question.

27:43 Now, while it does that, because obviously AI, it will, it will hit an API, get the query and come back with a response. While it does that, I want to show you a couple of things, right? Those things are simulation and deployment. So simulation is when we, when we are looking at building at scale and testing so that we get something consistent, what we need are the equivalent of a golden dataset of questions and SQL queries. We need to gain confidence that whatever changes we are making to the semantic YAML on the left is still reflected in actually improving the model to answer the questions correctly.

28:26 So there was a question earlier on how do we evaluate, right? The evaluation essentially is a natural language question which then has an expected SQL, and the question as a response generates a SQL and produces a data frame. And, and then we run that evaluation to see how many of them pass and how many of them fail. And obviously we would expect failures. When we get failures, we will get suggestions, right? So the AI would an agent would run on this and say, "What are the improvements?"

28:58 So there could be a, a relationship that can be intro-introduced, a synonym that can be introduced, filters that can be introduced, and we can choose based on our domain knowledge, this is the human part coming in, and apply those changes which would then in turn change the YAML, right? Finally, we talk about deployment. So this is the part where once we have created this, we talked about OSI and interoperability. We want this to be portable, absolutely, right? We want this to be deployed across different execution engines because different user groups in enterprise context would use different different products, interfaces. So Snowflake is obviously a deployment engine as Snowflake as a native engine. Databricks is another.

29:47 Also things like MCP. If you think about the consumption pattern of how the actual business users want to ask questions, they would want to ask questions via MCP and I can possibly show you an example of that here, where I have a question that I can ask on my cloud desktop, which is essentially got the skill to access this Context Studio and this context product built in, and it runs the command, and it will use this analyst to query and give me the response. So we want each to be interoperable and therefore, again, as the OSI part, there has to be some standardization, right?

30:32 Without standardization, interoperability is a struggle because we are in wide and deep realm of connectors and data models and query patterns and things like that, right? So we want, we want the interoperability and, and so a standard like OSI is certainly very helpful, and you will see something that Josh will show. Now, while this query is running, I'll come back and talk For a brief moment about observability. Whatever you saw in simulate or whatever you saw in chat and build, right? This chat and build is essentially you, you do it step by step, one by one. Whatever you saw in simulate is running it as a sequence.

31:11 But when you actually deploy it, in order to scale, you will need to know what's going on, right? So that's where we have the observability. And, and the idea with observability is like all of the live queries that are being run, and you can see all of the queries just recently run by me got synced. All of these queries that are being run are captured as traces, right? So there's telemetry to capture all of this information. This is to close the loop, right? We instrument how agents perform over time, what works, what doesn't work.

31:43 We can mark responses as correct, in-incorrect, and we also use this to modify, right, what we need to in our semantic model. So this is how we make context, which is really your org's IP, portable, right? The same context repo can be accessed via MCP, any other agents, and obviously, as you saw by humans. We can so-swap models, we can swap agent frameworks, we can swap warehouses, wherever your execution environment is, and give you the flexibility to run it and in a way that it is well, explainable.

32:22 I wouldn't say deterministic ever, but in a way that you have control and flexibility and guardrails while you are operating at the speed with which you want to operate, with some degree of certainty that, hey, you know what is the data graph that is used, how did we arrive at certain values, and why are you getting certain responses the way they are. With that, I will turn it over to Josh for the Snowflake part of the demo.

Josh Klahr

32:50 All right. Thanks, Deb, and if you've been following along, which it looks like people have essentially we just went through, we looked at a bunch of context and we asked to create a kind of a, a context grouping of how do I analyze. If you were paying attention, it was the TPC-DS dataset, everybody's favorite benchmarking dataset. And part of that was the creation of a semantic view, which we just just deployed to Snowflake. So that YAML file that represented kind of the, the language of defining semantics has been pushed over to Snowflake. And let's actually go to Snowflake, and we can look at where those, where that landed.

33:36 So I'm in Snowflake's catalog view right here. As I mentioned before semantic views are, are catalog objects, so I'm actually looking at my sample data. You can see here my TPC-DS kind of catalog, which has all of the tables, is, is available. So, you know, this is how a table shows up in, in the catalog. And I also have now some catalog objects, semantic views. There's a really long name in the title of this particular semantic view, but this is the semantic view that came over from from Context Studio. If I open that up and I look at the semantic information, you can see here that it contains the logical tables that were included.

34:25 We had the store table, the store returns. We had store sales. There are some relationships that were inferred. I didn't define these. These are, these are coming over from that metadata de-definition. But I now have this physical object that is sitting in in Snowflake that I can use to do my AI and my declarative analytics. I've also created an agent in Snowflake. The agent is what sits behind Snowflake Intelligence. You can see here I've registered those two semantic views. I've registered the customer the sales semantic view and the customer semantic view, and I can ask a question.

Josh Klahr

35:06 For example, show me which, which store manager has the most sales in Texas. And you can actually see here the context being used. This is a perfect question for the sales metrics tool, which is the tool I registered. It describes what information is in there by reading the semantics. And now it says, "I want to figure out how do I get this." I'm gonna query the sales metrics. I'm gonna look at if the results are ready for visualization. And so now the agent is using those, those details to go and answer my question. While we're doing that, I wanna show one other thing.

35:43 I mentioned before that Snowflake semantic views are built for both AI and BI, so we're providing context to an LLM or to an agent. But there's also a, a declarative query interface. You can say, "Select metrics and dimensions from semantic view," and get a result back. If you are a Tableau user, we will actually pre-configure a TDS file, a Tableau data source file. So I can download a TDS file that maps to the metrics and dimensions in the semantic view. And then in Tableau I've now opened up a pointer to that same semantic view. And you can see I've got my store tables, my store returns, my store sales.

36:26 The metrics that I defined on on my store sales table are in here, so the metrics that were defined, like the total discount, total net profit, that are part of that semantic model are there. And I've asked a question, I've asked the same question, "Show me the top store managers in the state of Texas." I've got Paul Vela here. Let's go and see what Snowflake Intelligence has said. Okay, it's given me the store managers with the most sales in Texas. You can see I've got the same answer. So now I'm in this world where both, Kind of Snowflake the A-AI powered experience and the BI powered experience are giving me the same answers.

37:07 And if you think back to the chart the picture that I showed in that slide deck, which is this one right here, we now have that shared semantic object, that shared context in a position to power my applications that I write, that write SQL queries, my BI tools that write SQL queries, and my AI tools that use an LLM to ask the same questions. So we now have this very consistent interface curated and, and developed by, by Context Studio. Okay.

37:44 I so that, I mean I I love, I love doing these demos 'cause it's pretty impressive to me how you can go from like zero to pretty rich context and, and something that performs well in AI pretty quickly. We did it in around forty minutes today.

Anirudh Agarwal

38:02 Yeah. Thank you, Josh. Thank you, Deb. What we'll do now is we'll start opening up the floor for some questions. Before we do questions we would like to run a second poll, which is that, you know, for all the builders out there and everyone today who's dabbling in AI analysts, we would love to know what are the biggest challenges you folks are facing today in building AI analysts. You'll see the options up. Would love to just hear thoughts. Would help us understand where all of you folks are stuck in any future sessions and where we should be helping focus on as well. All right.

38:43 Before we jump into the questions, you know up here is a QR code. This will help you folks, you know if you wanna chat with anyone from the Atlan team, if you liked what you just saw, if you want to try it, we'd love to give you folks a, a way to do it. So please scan the QR code. You can, you know, book a call with the team and, you know, we'd love to chat with everyone. And yeah, looking forward to, you know, as many people as we can talk to. Cool. I saw some, you know, really, really interesting questions on the chat, and I think we can start taking them one by one.

39:23 So I, I see one question from Shenat: "Are, are the context graphs time aware?" So Shenat, the way we look at it is that you have your enterprise data graph, which is a really, really large piece of context and data. It's gonna be hundreds of thousands, if not millions of assets. And versioning that is really hard, and that's why Atlan came up with the idea of context repositories, because they are a narrow, scoped down, well-refined version of context, and those can be versioned through Git controls and all of the o-other ways for you to be able to do versioning.

39:59 So those are essentially the touch points that you have of, you know, probably AB testing context be able to testing it before rolling it out to production. So we use context repositories as the Git control building block for any of your versioning or you know versioning use cases.

Deb RoyChowdhury

40:23 Maybe I can take the question that Carlos put out. Why should the context layer live within Atlan or in the catalog? What's the unique advantage? Why not in the database? Why not in the BI layer? That's a, that's a great question. And I would plug the blog that A16Z put out yesterday with the title, Your Data Agents Need Context. And the thing is that as we are building on top, and this is where Anirudh started, like all of the data that you have exists in disparate systems, and they have slightly different patterns.

41:01 So obviously databases, data warehouses and as Josh just showed, like has its benefit of storing the data as an object and helping you query and managing all of that, right? But there's also information that is outside of the database in the system of record. You can think of additional metadata, glossaries, tags, and things like that, which requires this aspect of human collaboration. So I, I wouldn't even assert and say that, "Oh, why should it live it here or there?" Where we started is that the context already exists. It's just that it exists in multiple places.

41:39 And in the representation of the enterprise data graph, which is the source of truth of how your organization's data landscape and assets and inventory looks like, we have an amazing view of that in Atlan because we have been making all those inv-investments over the years. And we are just finding out that because we alvios-obviously have all of this, it is that much simpler now to make all of this context AI-ready while keeping the human-in-the-loop collaboration and observability. So it's, it's, it's something like that. What is the best space for it to make it accessible for AI and then interoperable, right?

42:23 And in the data landscape, there is no like one answer because there are hundreds of products, right? So I think Atlan is like one viable and very solid solution just because of the evolution of how the product has evolved. Ani, you can add anything if you wanna add to that.

Anirudh Agarwal

42:38 No, I think Deb answered it perfectly. We have another question from Vishakha which says, "How the context that Atlan creates, where does that get stored?" Essentially what we believe in is openness. So any context, any metadata that gets created in Atlan is accessible via an Apache Iceberg Open Table formats. So now you can query all of this context and metadata through SQL directly through any of your execution engines, whether it's Snowflake, whether it's Databricks. So all of that context is open and accessible for you through the you know, what we call the metadata lake house here at Atlan, which is nothing but an Iceberg native solution to be able to access all of this context.

43:30 Yeah. I think Srinath had a question that, hey how essentially, how do these— what agents learn during execution doesn't easily flow back into a shared context layer that other agents benefit from. Are we doing something about this? So what we're building and what Dave showed in the observability workflow, now every time we instrument and capture any question that a user has asked your agent, what we have here, there are multiple options for you to do. The first thing you can do is that for any question that looks correct, you can add that as a simulation scenario, so that in the future when you're making any changes, it can regression test against that.

44:08 If it's a question that was unanswered or was answered incorrectly due to missing or lack of context, you can use that to update your context repository and/or even update the underlying assets in your enterprise data graph. So we believe in those closed loops where every time your user interacts with any of your agents, you can actually use that to help keep improving the context layer, not just for yourself, but for that use case, but for any other use case you're building in your enterprise. Yeah. I think Zitong had a question about ontology.

44:47 So what we're doing with ontology there at Atlan is that we're essentially using that AI flywheel to bootstrap starting from, you know, column descriptions, which we believe is the baseline enrichment you need to do. Now, think about like the biggest part of your you know, enterprise data graph is your column descriptions. We first enrich your column descriptions, use that to add metrics, terms, and then— or make its way all the way up to an ontology. The way the ontology helps is that it essentially connects multiple context repositories with each other.

45:22 So once you have multiple context repositories, the agent needs to reason about which context repositories to access and how do they connect to each other. And that's where we see the ontology helping from a reasoning perspective for an agentic use case.

Deb RoyChowdhury

45:42 There is another simple one, anonymous. Once the context repository is built, does the user get to change the YAML for his own use case? So as you saw in that, the context repository in a build session is obviously editable. And the idea is that if you're building on top of an existing context repository with additional data, you can do that as well. So the idea of portability includes the iterations.

Anirudh Agarwal

46:14 So we... If any other folks have more questions, please, please drop them in the chat or in the Q&A as well. Yes, I see I Go ahead.

Josh Klahr

46:28 I was gonna say, I saw, I saw kinda two related quest— I think related questions. One was around this idea of active metadata and, like, how do you update metadata kinda as you're interacting with it. And then there was another question about if you, if you are learning things through your analysis session or if the agent is learning something, how do you go and kind of share that context. And, and one of the things that we're seeing a lot is, we've talked about it like being a team sport or something that's got there's a loop.

47:03 Increasingly, we're seeing the implementation of basically memory systems, so that as the agent is learning how to answer questions, there needs to be a mechanism to go back and update the semantics or the context with what the agent is learning. This could be something the agent learns on its own, or it could be a piece of feedback that's coming from a user who says, "Hey, I didn't like this answer." And I think there was you saw a little bit of this in the Context Studio demo, this ability to provide signals back from consumers on the quality of the results and use that to kind of evolve the semantics.

47:39 And I think that's a, it's a long way of saying, yes, m-metadata is going to evolve, and there needs to be kind of mechanisms to store what is being learned as people are interacting with the data whether an agent or a person is learning that.

Anirudh Agarwal

48:00 Yeah. Well, th-thanks for that, Josh. So I, I see one more question, and this comes up a lot with our customers, is about unstructured data. How do you leverage context about unstructured data? I think there, there are essentially, like, two patterns that we are seeing emerge. One of the patterns is that customers are using LLMs to convert unstructured data into structured data. For example, they are adding fields into their CRMs like Salesforce by processing things like gong calls or, you know, meeting transcripts. So that's one way you can think about unstructured data, to add it into your add it, add it as a part of your structured data.

48:37 The second place where we are investing a lot in Atlan is something that we're calling knowledge tables. So what knowledge tables essentially are is that for any of your unstructured data sources, like Confluence, you can connect them through Atlan. It helps you chunk and embed those documents into an Iceberg native object storage that it builds a vector database on top of that can be leveraged by your agents as context. So that's how we are seeing you know, the unstructured data problem being solved, and those unstructured data sources connect back to your context repository so that your agents have the right context as well.

49:21 Yeah, I see one question by Keith, which is: Can Context Studio consume existing semantic views and enhance and republish them? So yes, Keith, we already see this use case coming up with our customers where they are leveraging things like dbt semantic views in order to use as a source for bootstrapping context, and then use those dbt semantic models to essentially combine it with all, with all of their other context sources like glossary terms, like business concepts, and create a new semantic model out of it, and then sync it back to their destination.

49:55 So that's additionally where we are already investing in, is to also you know, crawl and add all of these different semantic sources within Atlan.

Deb RoyChowdhury

50:11 Other one, which is, are there other patterns to consume the context other than OSI? I mean, OSI is obviously an interoperability layer, but in the, in the deployment patterns, you can consume the context created in in multiple levels in multiple ways. So one level is you use an engine like Snowflake Cortex or Genie Rooms. The other level is you can consume that in your agent orchestration framework that you're using to build agents, which has access to your data sources via MCP or API.

50:44 So you can consuming it consume it in many ways, and I don't know, you might have I, I might have not dwelled at the screen, but if you're asking, say, at the consumption layer questions on Slack or Teams or something, which then hits your service that, that uses this context to query a database. So there are many ways of creating consumption patterns of this context both for humans and for agents.

Anirudh Agarwal

51:12 Yeah. I see a question by Doug, which is asking here: Do you prioritize an enterprise data governance program or a functional area context parts? The core belief there is that you will have to do both because governance is important. It's making sure that, you know, your your, your data landscape is well-trusted, well-governed. What we've seen working really well with our customers is to just start with one specific use case that has an end business user impact and use that to kickstart and get going on any of their context use cases.

51:49 And then once they have their first success story, is that's when the compounding starts kicking in, where they can keep building and adding more use cases to it. So what, what we've seen work well with our customers that, you know, are fairly advanced is j-just starting with one use case to begin with. Yeah. I, I see one more question, which is: How how does Context Studio manage to find the proper repos to do the reasoning? So the way So there are two essentially problems, search problems here that we solve. First one is finding the right assets on top of which to build your context repository.

52:33 The way we are actually able to do this is because one, we understand your entire enterprise data graph, for which we captured some major signals such as, like popularity which are your most used assets. We captured through lineage what are not directly consumed but still important parts of your data graph. And we also captured you know, things like actual usage, which is what's the use case someone is using a particular asset for. Is it for finance? Is it for sales? All of this lands together within the enterprise data graph as enrichment.

53:04 And then you know, we reason on top of that to help you find the right assets to build your context repository on top of. In terms of finding the proper repos to do the reasoning, I think there are multiple patterns here. So there is you know, a pattern where you can just provide the relationships and the various context repositories and let the agent reason about which one to choose.

53:30 As well as, you know, other patterns that we are seeing around you know, contextual retrieval, which is more GREP-based, which is put all of your you know, context repositories into into your AI agent, let it you know reason and and iterate over them until it finds the right one to give you the answer. So I think it's multiple patterns that we are seeing evolve there.

Deb RoyChowdhury

54:01 I'm guessing that question regarding the decision trees being captured in simulation to see if the context is not proper, if that is the one by Srinath. That's, that's what the observability section is for. So when you are asking questions or running simulations, which are essentially putting a series of questions to a context repo we are obviously tracing that. And we are tracing the query, the SQL generated, and that's what we'll use that, that's what we use to generate suggestions also and find out where the context needs to be iterated on and tweaked, the context repo needs to be updated.

Anirudh Agarwal

54:48 I know we just have two minutes left. I think it was a wonderful session just interacting with everyone, hearing all your questions. I'm sure we'll do one more in the future as well. Any, any parting thoughts, Josh, that you would like to give to the team, and then...

Josh Klahr

55:05 No, other than I think this has been a great discussion. Clearly, this is a, a very hot topic that a lot of people are interested in, so appreciate everyone who stayed through the end and all the great questions.

Anirudh Agarwal

55:19 Yeah. Thank you for everyone. Thank you to everyone. I'll just drop in the last thing. If you were excited by what you just saw, you would want to play with it, you wanna get your team to use it, get hands hands-on with it, use this QR code. You can book a demo with the Atlan team. They can show you exactly how context works in real life and help you get your AI agents ready in production. So we're super excited to chat with as many of you as possible and thank you all for joining.

Deb RoyChowdhury

55:47 Yeah. Thank you. Thank you.