Watch the context layer get built

Daniel Bergquist

0:00 We've got a jam-packed agenda today. We're gonna be focused on all things context. So really thinking about how can you build out your enterprise context layer for AI and, and thinking about the agentic use cases that we can capture, whether that's talk to data, operational use cases, and really the, the power that context provides when it comes to building out these systems. So the first thing that we wanna talk about is really giving you some insights into what we hear from our customers on a day-to-day basis.

0:30 You know several folks come to us saying, "Hey, you know, I, I know I'm gonna have dozens of platforms, hundreds of agents, whether that's Sierra, Writer, Google Agent Space, Snowflake Cortex, Genie with Databricks," but what's happening is that they're all speaking slightly different versions of the truth. Now, coming from the data world, this was always the case. If you ask finance and sales the revenue number, you get two different numbers. Now, this is happening the same with these agentic systems but at a, at a much higher and compounding scale with AI.

Daniel Bergquist

1:03 Now, trust is easily lost but hard to earn, and what we're finding is that when we think about performance of agents, it's all about a function of intelligence and context. Now, intelligence, well, this is the model. This is GPT, this is Claude, this is an open source model. They're really interchangeable at the end of the day and becoming highly commoditized. You know, every week there's a new frontier model that's giving you more and more capabilities. But what we're finding is that context, on the other hand, is where we're getting the much more increase in control in our performance and our productivity.

1:41 You can think about context, you know, in the scope of knowledge, skills, tools, you know, everything someone learns while on the job. And because of that, it's really becoming a massive influencer when it comes to the performance of your AI agents. Now, I think it's, it's worth spending a few moments just kind of understanding how we break down context. Context is a loaded term. I just came from Snowflake Summit last week, and every person that came to the booth had a different version of what they thought context was. Now, it's important to note that we view context not just as semantics. It's much more broad than that.

2:21 So we think about it from this from this three buckets. You know, first, we've got knowledge. You can think about this in terms of your entities, your data assets, your definitions, metrics, relationships. Then we think about expertise. Think procedure, playbooks, workflows. How do you actually action upon insights to get the right outcome from a business standpoint? And then lastly, norms. What's actually allowed? These are gonna be your policies, your permissions, your guardrails. You know, if I'm in finance, there should be an expectation on what a metric means versus if I'm somebody in marketing. So all three of these together, knowledge, expertise, and norms, are how we view the actual underlying context layer that we're producing.

3:10 Now, this is very abstract. What we wanna do is then break it down into how we're viewing the architecture. So on the left-hand side, the first thing that you wanna do is think about your context mining. So this is very much aligned to getting information out of your systems of record your systems of data, systems of knowledge, systems of work, and even runtime or operational systems. Now, in practice, this could be, say, Snowflake and Power BI or Tableau could be documents from SharePoint or or Confluence, or runtime signals like DBT, Airflow, Spark, that give me insights into the, the data pipeline, as well as even observability tools.

3:51 All of these cohesively start to build out this context foundation that you're seeing in the middle of the screen. Now, we see this breaks down into a couple different areas.

Daniel Bergquist

4:00 First, being AI-ready through a data knowledge graph. So we've got accelerators to help to get you up to speed on building out your context pipelines and your, your on your data elements. And then we're also going to build out semantics and ontology, so making sure we have that shared meaning, business logic, and relationships fully understood. Again, if I ask revenue, there should be an expectation on how we're calculating that. And then lastly, breaking these down into skills. We're seeing that the agents architecture is kinda coalescing around skills for codifying workflows or business processes.

4:36 We're gonna talk a little bit about how we can enable that, but these three areas build out what we're calling is the context foundation. And from there, you wanna think about context almost as you would software. So in the same way that software development life cycle is built to deploy production grade software applications, our context development life cycle is built to produce production grade context for our AI agents. So in the same way we wanna think about build, test, review, approve, deploy, and learn, all of these are built into our platform, and essentially is for establishing the discipline around building out context for agents.

5:18 Once we're happy with this notion of a context layer across the enterprise, it's all about activation, so here on the top right-hand side. We've got several methods of activation because we don't want you don't want to get pigeonholed into a single interface because that might become unusable in, in the future. So we wanna be open and interoperable by nature. So a few of the interfaces that we're really focused on investing into are going to be MCP, support natural language search through semantics, push and pull of context. We also have APIs for more deterministic interactivity with your, your context, and even SQL can be run.

6:00 So all of our context is replicated into a, a context lake, so you can bring in iceberg format, you can bring your own compute, and you can actually run SQL on top of your context at high performance.

6:13 All of this then helps to build out what we call as a compounding learning loop. So once you deploy your, your agents into production, we want to establish a sense of traces and understanding of the model's reasoning. This serves a few purposes. Are we actually providing the right context at runtime? And secondly, how do we increase or improve the context over time? We want to avoid drift in underused or underutilized context a-across the enterprise. And then lastly, context governance and observability. We've got our, our background in, in governance and, and in working on metadata. It's really a natural evolution as we're seeing this play out the same as you would within context.

6:56 You can almost think about these as building out your own data marks. You want to make sure they're governed, trusted, and usable. It's the same with your context layer. You want to make sure that the context you're providing to AI agent is trusted, usable at a high quality with the right versioning and approval workflows. The last piece before I jump into the actual demo is going to be thinking about the path to contextual intelligence, the path to building and deploying your AI agents. The first wall that we're seeing folks run into is what we're calling is context bootstrapping. It's really easy to build an agent on Cursor or Claude in just a few minutes.

7:35 It'll spin up, build a YAML file, build out what it thinks is the right context, and deploy. Well, that's great. But when we want to build out a production-grade application, it takes much longer from what we're hearing from our customers. I've heard folks say that it's taking me several months to be able to build the right context for the right application, and that's really hard. That means that our time to value is slow, and our ability to to deploy in a, in a high velocity is, is challenged. Now we're seeing this is really the impact of maybe building out one to three agents.

8:11 And then once you get past this initial hurdle, context life cycle management and governance becomes a challenge. How do we present or prevent the the, the overuse of of certain context or govern what a metric or term actually is for the right repository? This becomes a challenge when we start thinking about middle scale, two to thirty. And then lastly, context sprawl and drift. Once you've gone live in production, how do you ensure that you're— the context you're providing your agents is not giving you diminishing returns through sprawl and building out too much context for your agents? And then drift, are we not answering the right questions or, or having challenges in that sense?

8:54 We're gonna showcase how we actually handle traces for developing that feedback loop once we're actually in production.

Daniel Bergquist

9:02 So let's go ahead and get into it, folks. We're gonna spend the next, say, twenty or so minutes in the actual platform showcasing the context development life cycle. If there's questions, please use our Q&A. We're happy to help facilitate that at the end. So jumping in, the first thing that I want to do is actually give you a side-by-side comparison of two agents and why enterprise context is important. So on the left-hand side we've got an agent that was given just raw SQL access. So the use case that we're taking is a fictitious burger chain where you might have a franchise operator who owns multiple stores, and they're asking a very basic question.

9:43 You know, "Why is my drive-through time up this week?" Now, why and how questions typically require additional nuance. It's not as easy as what questions, where it's inherent within a, a metric definition or calculation. Why questions require, well, what time of year is it? Are there any marketing campaigns going on? Are there any issues with the stores? You know, maybe there's a sporting event in town, and we're, we're over over over capacity when it comes to serving out food. These are all the considerations to have when providing an answer.

10:21 So when I actually fire off the question to, to two separate analysts, what we see on the left-hand side is almost that of an entry-level data analyst, somebody who's doesn't really know too much about the business. But what the the first thing that they start doing is simply querying tables, and they go directly to the raw tables and, and query, query, query to try to understand what's going on. Well, the good thing is it's at least telling me when, when it's making assumptions. And then finally, it's not hallucinating and telling me, "Well, actually, I can't even confidently tell you why."

10:54 So at least we're not losing trust, but we're not gaining value, and that's an important distinction. On the right-hand side is drastically different. We've given access to context at the persona, glossary, skill, and semantic model le-level. So before ever going to directly to the data, the analyst first understands, well, who's asking the question? How many stores do they own? How do we calculate drive-through time? What do I do if I need to diagnose an, an issue or check seasonality or perform what-if analysis? Everything that it takes to do the job is loaded up front using our context.

11:39 And finally, the agent then goes and queries the data mart, which contains the right answer. So a few things that we're seeing is that by providing this structured context up front, we're gaining a much more accurate answer. We're also reducing tokens, token use and, and having higher token efficiency because we're not having to go query, query, query. Models are very good at reasoning and very happy to run tool calls. But when we provide it the right context up front, those additional tool calls do not have to happen because we're giving it the right guardrails or the right map to get to the outcome or the answer.

12:18 So this is a bit of the how and how we enable agents and AI systems through our context. What we then wanna talk through is about how do we get there? Well, the first thing that we wanna think about is really around context mining. When we think about that architecture diagram I showed earlier, the first thing that we do is start thinking about gaining context from your system of records, systems of data, systems of knowledge, and so forth. So we've got over a hundred connectors already built in to platforms like Snowflake, Databricks, BigQuery, Tableau, Power BI, Airflow, Spark, data quality tools, you name it.

12:57 All of these provide really rich detail on how to, how to interpret the business. Secondly, we've also introduced the notion of uploading skills. We're seeing skills becoming a widely adopted paradigm for codifying business process or workflows. So if you've got skills already defined, perfect, we can go ahead and upload those into our platform to be centrally cataloged with all the other data elements. Otherwise, I'll showcase in just a few how we can actually help to generate skills. And then secondly, for folks that haven't seen Atlan in a while, we're also introducing this notion of knowledge folders and unstructured data.

13:36 I'll show what that looks like in a bit and how we translate that to ontologies and metrics. But what, what I will highlight is that we don't wanna be a catalog for all of your unstructured data. What we do want to do is become an ingestion point for your usable business processes, standard operating procedures, and so forth. In the same way that you've got bronze, silver, and gold data, you probably also have bronze, silver, and gold unstructured data sets. SharePoint and G Drive, well, that's just your data lake for unstructured data sets. But you probably have business certified policies and SOPs.

14:13 Those are the ones that you wanna upload into our platform to gain insights out of. Now, what does this build? This effectively is going to build out your data graph. So because we're gonna have insights into your data transformations, your data pipelines, we're gonna be able to showcase how data is being moved throughout the business and also being leveraged. So here is a perspective of data pipeline within Snowflake. The way in which we do this is through query mining. So I can actually expand this on the right-hand sidebar.

14:45 What we're gonna do is look at all the transformations that are being executed, whether that's through DBT, push down, store procedures, you name it, we'll have access to it. We're gonna parse these for your table to table as well as column level lineage. Now, this is massively important to understand again, how is the business truly architected when it comes to your data? This isn't just siloed to say your data platforms within your analytics warehouse or data lake, but also downstream to the BI level. So here I can see we've got a data source within Tableau, but this could be Power BI, Sigma, or so on or so forth. Any other platform is gonna be supported.

15:26 We connect into those platforms using their metadata APIs, and we've got custom connectors to extract the right context. I'll give you a little bit of hint of why this is important. What we're seeing is that your metrics are inherently defined within the BI. Think about your semantics. So what we wanna do is use this as a starting point to reverse engineer some definitions, and we've got some accelerators on how to do that. So directly connecting into th-these platforms only gets you so far. Now, what might be missing? That could be, say, descriptions on the table, descriptions on the, on the actual columns, readmes, how to use the datasets. These are inherently lo-lost or lacking documentation.

16:16 Now, in the old world, data catalogs were used and set up to, to solve this problem, but the human in the loop element prevented value from being reached. So our focus over the last year has been all about how do we automate the context generation across our data pipelines. A lot of the documentation is already operationally there. It's about reverse engineering and res-reciting what actually that documentation is. So we've introduced the notion of context agents across our customer base. And what we're seeing is by running a few cohorts, we're getting into the scale of millions of documentation updates in just a few short weeks.

16:59 Whereas in the prior year, we were just a few hundred thousand updates to our, our metadata. So massive acceleration when it comes to bootstrapping your context. Now, a way in which that lar works is that you define a collection of data assets. You don't necessarily need to boil the ocean here. Typically speaking, if you've got a gold layer that's already curated and ready for analytics and insights, it's probably a good starting point. But if you wanted to deploy this holistically, that's fully up to you and available. Now, to show you an example, let's say that I wanna focus on my worldwide assets within my plat— data platform, Snowflake and Tableau.

17:44 Now, once you're in a collection, you can then deploy the agent on a few different areas: descriptions, readmes, and SQL intelligence. Descriptions allow you to define the scope. And then the way it works is that we want to look at existing context. Think schema and column names, upstream lineage, looking at the actual pipeline. Perhaps you're aliasing, you're already adding in descriptions. Query history. How is the business actually asking questions of the data? And glossary terms. Have we actually established any underlying metrics, definitions, or terms that have been associated? We're taking all of these areas And reverse engineering into usable context.

18:31 So to showcase what this actually looks like in practice, if I was to jump back into that same table, I'm now going to see that we've got descriptions that have been updated via AI, as well as what we're calling is intelligence. Now intelligence is actually really important. Simplistically, I call this an automated FAQ, frequently asked questions. So what we're doing is looking at the actual query that a user is running, whether that's a select statement or so on, and we're translating that into a natural language question. Because all a structured query is doing is trying to understand a business term, a metric, or gain an insight.

19:14 So for the human coming to this table, I now know the questions that my co-colleagues are asking. For an agent coming this— to this table, I now have a sense of the metrics that are being derived from this data set, but also these become your simulations or your evaluation sets when it comes to building out your context in the future. So again, thinking about acceleration on all fronts. We also want to gain insights such as joins. You know, how can I connect data sets, filters, how do I actually produ-produce an insight in relationships? You know, maybe there's foreign keys that are associated with these tables.

19:54 All of this becomes usable information for both human and agents alike. Now, this is the, I'd say the business lens, right? Or I'm sorry, the data lens to these data, these data pipelines. We also want to augment this with your systems of knowledge. So I hinted at this a little bit earlier, but we're going to have these items called knowledge folders, where you can simply upload your usable, unstructured documents, you know, whether those are PDFs, PPT, CSV, and so forth. We want to gain a usable context from your underlying unstructured assets. Now, you can govern these as you would with anything else.

20:39 You can establish owners, purposes or descrip-descriptions of these folders, and then the actual PDF itself is going to have access. So we'll be able to see things like the underlying document and even lineage out to skills. We'll talk a little bit about skills generation in the future. But the important piece is that we don't want to just load an entire unstructured document into the context window for an agent. That's not usable, right? You're gonna have far too many tokens used, and the agent might r-run into additional drift or hallucinations because it's just too much information that's not readily usable.

21:20 So but what we want to do is take that, say PDF, and translate that into usable business terms, ontologies, and maps on the right-hand side. So an example of this is that embedded within this particular PDF is this notion of subscription and refund. Now, if I simply click on the subscription term, it's going to bring me to our ontology, our business graph. So subscription is a standalone asset where you can have an AI-generated summary, any usable custom metadata properties that we want to add in, as well as the map, the association, the graph related.

22:10 So for example, the subscription entity, if I was to zoom in a little bit, has four properties, last charge status, double charge flag, and so forth. It also has three metrics, failed charge count, lifetime charge count, and subscription monthly recurring revenue. So it's important that we establish this knowledge graph, this business graph to understand concepts, entities, and relationships. Once we get into agents, we're gonna condense this down into something that's usable, like a skill or bounded context. But having this holistic view of the relationships within the business gives us a sense into how workflows and business processes are accomplished and their relationships to metrics.

22:58 A final piece related to context bootstrapping that I'll cover off is going back to our context agents. You're going to be able to deploy these from a high level. So you'll, you'll get a sense of how are we from an AI-ready context layer by seeing the number of descriptions by agents, by humans, and how we want to translate this, or we want to load the by agent side of the house. You're going to be able to deploy these across descriptions, readmes, SQL intelligence, and get a full holistic view on the reporting sense.

23:41 So what— the, the idea or the mental model to keep here is that it's not the human in the loop for all context generation. That's just not sustainable, not scalable. It's the human on the loop. So you want to deploy the agent, trust that the AI is producing the right context, and allow you to scale out and focus on your objectives and outcomes. We're gonna talk a little bit about those outcomes in just a few moments. Now, taking a step back, going back to my architecture, we've talked a lot about context mining and building out the context foundations.

24:20 Now, the next important piece is going to be the context development lifecycle for building out your agents. The way in which we've done that is if I jump back into the Atlan platform We've got Context Engineering Studio, where you can think about this as almost a context engineering workbench.

24:42 How do we think about the use case or the outcome that we're after and gain the right context that's fit for purpose? So a use case could be maybe customer support, customer three sixty, financial analysis, marketing analysis, revenue, so forth.

Daniel Bergquist

24:59 Each of those requires its own bounded notion of what context is and how the enterprise interprets or produces functions on top. Now, to give you a quick glimpse into what that looks like, I'm gonna kick off a context repository build and showcase our, our workflow in action. So you simply give the point of view of what you want. So in this case, I said, "Build me a context repository for our McContext burger chain. I want the agent to explain why operational metrics like drive through time, order throughput, store performance shift week to week." So you give it an area to focus on.

25:40 Now, because we have a context layer in the back end, inclusive of your data, business graph, metrics definitions, and so forth, we're going to search across those da-data elements, Tableau, Snowflake, Databricks, Power BI, you name it. We're also gonna load in what we consider gold layer assets. We want to exclude the noise. Perhaps you've got a raw landing zone, bronze, silver, gold. We don't want to just take everything that could meet the criteria. We wanna take the most usable data sets for this outcome. We're also looking to other data assets. Maybe you've got some information in SAP, upstream in Postgres or Oracle or Salesforce. Each of these are rich repositories for context.

26:25 Now, because we're using an LM on the back, we're consistently reasoning o-over the results that are being returned and allowing you to look at the assets, view, refine, and continue. So this refinement is the human on the loop. Again, I'm not having to comb hand by hand and try to find the relevant assets. I'm being pro-provided what we believe are the right data elements in context for this particular purpose and allowing me to have the, the last call or the last certification. It's gonna continue to look across even data products as well as e-existing context repositories because the last thing we need is to have additional sprawl.

27:10 Now, what we're seeing is that time to value for AI agents is being massively compressed. So if you're somebody who's building out Snowflake Cortex, Databricks Genie, or your own talk to data architecture, you'll probably find that that it's taking a long time to actually provide the right context for the right outcome. Think about it in terms of providing a map to a non-deterministic system. We should have an expected outcome, but the map to get there can vary in a variety of different manners, but it's important for us to still get to that trusted outcome.

27:48 Now, once it's happy with the, the notion of the context required for this, it's going to build out on the right-hand repository of usable context. Now, this takes a few moments and would require some follow-up, so I'm actually gonna jump into an existing repository to showcase what that looks like. So this builds out your file structure, your contents embedded into a repository. Repository, think about in, in this— in the lens of a data product, if you're familiar with that methodology, or a use case, a purpose of context. At a high level, it's gonna build out your semantic models.

28:28 So it's going to take that information that we've mined from your data platform, such as name of the table, database schema, synonyms, descriptions, dimensions, filters, and so forth. Everything that we've been able to mine from, from that platform is translated into a semantic model, and we'll extract several of those that are gonna be relevant to the outcome. Again, thinking back to the notion of knowledge expertise and norms, this would be your knowledge. Expertise then becomes your skills. How do you handle situations such as daily sales trends, drive through timing, and so forth? You wanna embed interpretation in how a user would gain insights.

29:14 So for those that aren't aware of what skills are we're seeing that Anthropic pioneered this and that, that it's being largely used in the AI agent architecture and ecosystem as a way to precompile your context to tell the agent exactly how to do something, like give your workflows, your business processes, your standard operating procedures. So those unstructured documents that I showed you a little bit earlier, we're gonna translate those into a usable skill that's condensing not everything, but just what the agent needs to do. This could be inclusive of SQL, seasonality dimensions, and so forth.

29:52 Now, multiple skills can be embedded because an agent might have multiple purposes in terms of workflows that it needs to execute upon. So this allows the model, the harness, to reason over what are the most usable skills, and then the skill actually dictates or provides the map for how the agent needs to act. Now, because we want to embed this in terms of a life cycle, context CDLC, you can have your repository summary. So again, think about your purpose, linked assets, semantic definitions, model, and skills at a high level, as well as who owns this, and is it verified for use?

30:34 We're also going to have versioning That's applied, so as we're making iterations, the repository will be able to push back updates and then dictate whether something is active in a draft status or even archived.

30:47 So the life cycle is important here because it's not just a one-time build, but an ongoing iteration. And in some cases, we'll need to decommission our context because there's a better way to define that, and that's okay. But what we want to do is give a sense of how that life cycle can be handed with— handled within our platform. Once you've got your context built out at the repository level, simulations and evals are your next step. So think about questions. You know, what is a question that we ask of an agent, and what's the the SQL that it might generate?

31:23 So, so for something like a talk to data analyst, where we're trying to query Snowflake or Databricks, you might already have an expectation on what that question in SQL might be. If I was to look back, again, this is that usage and intelligence that we're mining. This is the acceleration that we can provide for building out your evals. Now, what we want to do is showcase what is the expectation versus the generated. I mean, this is a very simple unit test or a reconciliation. It's no different than if you were developing software. We don't care necessarily if the expected versus the generated SQL is exactly the same syntas— from a syntactic standpoint.

Daniel Bergquist

32:05 What we do care is that the query result is the same. So this allows you a sense of expectations and actuals in giving those guardrails to the agent. Now, this isn't in the world of manually annotating queries, but providing the right context. So let's say that the expectation versus the actual is providing differing results seventy percent of the time. This likely this likely means that we're not providing the right context. Maybe there's something inherently wrong in the semantics or the metrics definition that we're providing, or the description of a particular field or column.

32:49 So what we want to establish is this feedback loop to take an inaccurate response and feed that back in to the actual contents of the repository. So we're always taking a context-level mindset. It's never about hard coding or fixing queries in place, but how do we provide the right map to the agent? Now, once you're happy with your simulations, let's go ahead and deploy. Now, we view context as your IP. What this means is that it's important that you decouple your context production from the data platform or from the compute or from where you're actually running your model. The reason that's important is because you need your context to be open and interoperable.

33:35 You might have workloads that are more performant on Snowflake, whereas you might be having some operational workloads within cloud-managed agents, and that's okay. But what you don't want is that each of these agents are pulling from differing context. You want to pull from a centralized context layer. So our belief is that you centralize the context production, and then you deploy or you port this context into your runtime. So we enable this through Antlin MCP, so any model provider, OpenAI, Claude, Gemini, you can run this within Cursor VS Code, Copilot, and so forth. Anything that has a hook into MCP is compatible with our platform.

34:21 Then we also have one button deploy into Claude, Databricks and Snowflake to make your lives easy. Now, let's say that we're building a Snowflake a Cortex analyst or Cortex Agent. What this will do is actually translate that repository into a semantic view that then Cortex can call upon. So again, this is allowing us to be agnostic of the platform, but push or port out to whichever platform or end user interface that you all want to define. That's the port piece, portability. The last piece if we go back to our architecture, is this compounding learning loop. In the same way that software deployment or applications, you're never one and done.

35:05 There's always ongoing requirements, updates, and deployment changes. It's the same for context. So we want to bake in observability into this platform as well. So observability in the scope of traces. So we're seeing that the ecosystem is coalescing around open telemetry is one way to export traces. Many other platforms, Cortex, Managed Agents and others afal— also enable the ability to export an endpoint for traces. This effectively gives us a sense of how is the model reasoning over the, the request in the context, and what's a proposed edit that we can make.

35:47 So this reduces the scope of sprawl and drift because now we're, we're consistently taking back actual production runtime use and relaying this back into context refinement. So when we think about that life cycle, it's post the deploy is going to be learn, and this is how we can enable that. And we're actually seeing that observability and traces is probably becoming the most important data to gain access to when it comes to agentic use cases, because this will give us a sense of how do we continuously refine, improve, and maintain that level of accuracy.

36:26 So in summary, we, we walked across our entire platform, how we're thinking about AI agents, but most importantly, the context piece of the puzzle. We started off with context mining. How do we gain insights from your underlying systems of record, systems of data, and knowledge? Using that to build out our foundations of context, which is inclusive of data graphs, business graphs, semantics, ontologies, as well as skills, and then taking that centralized context and building a development life cycle on top of that. How do we build the right context for the right outcome and deploy that to any platform of our choice?

37:14 Once we're in production, then the compounding learning loop comes into play. It's not about static deployment, but continuous improvement and iteration, and using something like traces and observability to build out recommendations is how we want to go ahead and accomplish that. So that was the demo for today. What we wanted to do is just give you a sense of how we're accomplishing some of these use cases that we're seeing as agents and AI analysts are coming front and center for how folks are leveraging context within their environments. I'll go ahead and take a pause here. I'm gonna quickly check to see if there's any questions in the chat.

Daniel Bergquist

38:05 Okay. I see a question: How does the feedback loop work? Does it get the input from, say, Databricks? Yeah. So the idea is that we want to build in almost connections or integrations into whichever platform you, you execute your, your your agent on top of. If you're building out sort of platform-agnostic architecture, think LangGraph other, other agent builders typically speaking, you can enable open tele-telemetry, or you could provide an observ— a dedicated observability tool. That's one way in which we're seeing customers do this. For all-in data platforms like Databricks and Snowflake, we're seeing them starting to expose traces and observability.

38:51 And what we want to do is write an integration into those platforms to ingest that, that metadata, that context into ours. And the idea is that we have in-house harness for agents that actually parse out all of these traces in a structured fashion and proposed edits. So a long way of saying, yes, our goal is to provide or extract traces from all platforms dependent on, on how they're, they're exposing those. I see another question. How long does it take to build the context once you give it access to all these systems, and how much human in the loop we need initially to build the context layer? Yeah.

39:29 So we're seeing that folks are getting up and running in the span of days, if not weeks. So it's really the, the limiting factor is almost how quickly can you provide access to your platforms, provision a service account, and allow us to run our agents. So we've got customers that are able to say, give us access to their Snowflake, Tableau, Power BI environments very quickly, and the first thing that we wanna do is extract all of the information around your data assets, so again, that business, that data graph, data pipelines, usage and popularity, and so forth. That That's all done on initial run.

40:08 Once you have that in, as I was mentioning with our context agents, you then reverse engineer business usable insights, business context from those. Those are already built out as our own in-house harness for, for our context agent. So what you have to do is just deploy this selectively. So the actual generation of the context is very low lift from you all. The reason being is because once you get to the context generation piece within the within the repository, you're gonna allow for simulations to handle any fallout. So this is where you can actually ensure that the context you're providing is fit for purpose when it comes to building out your agents.

40:54 I see a question: Where does the context layer sit, in customer cloud or Atlan? Yeah. Yeah, great question. So everything that you're seeing here on screen is basically a service through our SaaS product. So we're fully managed SaaS. We deploy a platform into GCP, Azure AWS in the region of your choice. So everything is available as a platform. So you'll have full access, if I was to look at back at our architecture real quick, you'll have full access into retrieval via API, SQL, MCP et cetera. Now, if you wanted to then export those files from the repository into your runtime or into a GitHub, you absolutely can.

41:43 So you can still house the, the individual context within your, your sort of your runtime environment, your harness, but the Atlan platform itself is gonna be fully managed SaaS. How do you measure the accuracy of the context built by the agent? Yeah, that's a great question. So this goes back into our simulations. So simulations and observability. Simulations allow you to build those guardrails into what the expectation is. So for example this was the franchise analyst, but let's say we're building out a sales analyst or a financial analyst into a repository. You'd want to have questions and simulations that are relevant to that use case.

42:30 Now, a way in which we're seeing folks do this is by reverse engineering what they might already have built, say, in Power BI as a semantic layer or even Tableau or DBT or others. So you probably already have these gold layer SQL statements predefined. If not, we will be mining those directly from the SQL you execute in your data platform. So we can accelerate the creation of these simulations, but at the end of the day, simulations, evals, and traces are how, are how you certify that you're providing the right bounded context for the agent. And if these are failing, we'll have recommendations for context improvement.

43:11 You can actually have natural language conversation as well to produce the right level of context. Okay. I think that was all the questions that I can see here. The last thing that I'll, I'll have on screen is just going to be our QR code. So for folks that do want an additional deep dive demo, we're more than happy to showcase how our context layer can help with your specific use case by a follow-up demo.

43:42 We can talk again about how we're viewing the world when it comes to context, again, thinking about knowledge, expertise, and norms, and how we can execute on a single use case for you all, get you up and running in production, and showcase that we can build a repeatable discipline to build out not just a single use case, but many for a trusted, verified context. Well, I hope you all had learned a little bit about how we're viewing the world of context and how Atlan can help enable better, more trusted agents within your enterprise. Thanks for joining today.