Bringing data mesh to life with Atlan Mesh

Austin Kronz

0:00 All right. Welcome everyone. I am super excited for today's session. I think everyone has been waiting for this. This is the Atlan Activate, where we will be introducing Atlan Mesh. Thank you everyone for joining. Really excited things we'll be showing you and going over the Atlan philosophy, and a lot of great speakers from our product team, including our co-founder Varun. It is an Atlan event, and you know I like to start every Atlan event with getting the audience involved, and that means we have a question for you. We want to know about data mesh, but let's get some wrong answers only here. What do you want to accomplish with data mesh? And get creative, right?

0:40 Wrong answers only. Feel free to scan the QR code or follow the Slido link, and we'll be able to see these answers come in live and we'll be able to talk through some of these to, to kick off the event. So feel free to submit those answers and they should be coming in as we speak. I'm expecting a lot of, "We want to centralize," or, "We want to get to value slower," or, "Our dashboard should be wrong." Anything like that come to mind? Wrong answers only. Anyone love having multiple numbers, multiple KPIs in a single meeting? That's always fun, I think. Five brave participants starting this. More Slack messages. There we go.

1:22 I wasn't even thinking that route. There we go. Moving away from data contracts. Naturally, that's what you should do, of course. Yeah. One mesh to roll them all. Yeah. Let's put everything monolithic is the way to go. More confusion. I like this. I like this. I think now is probably a good time to set the record straight and get back on some serious answers. Varun, what do you think about this? At least we have some creativity going on in the audience. A lot of funny answers here. You know, losing weight in there. Slowing down, that makes sense too. Anything stand out to you, Varun?

Varun Banka

1:56 Losing weight is very interesting. Data teams do need to lose weight. I think there are a lot of challenges. So I, I love that.

Austin Kronz

2:03 Efficiency.

Varun Banka

2:03 It's more right or wrong. I don't know. It's maybe a right answer as well.

Austin Kronz

2:07 It's true. It could play as both. Well, I think we'll leave this up. Continue to submit some answers if you have them, but let's get into the meat of today's session. Let's talk about what you're here for data mesh. So Varun, I will pass it over to you to kick things off.

Varun Banka

2:24 Thanks, Austin. And once again, I'll just take this opportunity to welcome one and all to Atlan Activate. This is our flagship event where we share our vision future updates, preview new updates that are coming up, and share stories with... From our data teams and partners in the ecosystem. And most importantly, we open the gates for all kinds of feedback, suggestions from community and partners in the ecosystem. And in fact, this is very well validated the last Activate, and I'm sure some...

2:58 In case some of you missed in the last Activate, we we had a preview of Atlan's GenAI capabilities, where we opened up, made it easy for data owners to generate context, write descriptions, and I'm really thankful to twenty-five data teams who've been working with us in close preview and helping us shape these capabilities. And, and thank you for sharing all the feedback. I hope we'll be able to fine-tune the AI overloads and make this better and better to make it easier for data teams. Now, just like GenAI, today we're going to talk about one more change, shift in mindset, from centralized data governance to federated data governance.

3:44 And but today, don't worry, I'm not going to talk about what is data mesh. I'm sure there are hundreds of articles written around what is data mesh. We've been into multiple webinars. Today, we talk about our vision around what should the data experience look like in this federated world for data producers, data consumers, data domain owners. And we have a demo plan in very similar for all the three personas for all of you today. But before that, let's talk about... Little about our early days' journey as data team. So some of you know that Atlan started that we were a data team ourselves.

4:22 And we used to help data decision-makers drive decision-making around resource allocation, budget allocation, and we used to leverage a lot of public data source and to help drive these decision-making. Here, what you're seeing on your screen is a snapshot of a very rudimentary data supply chain that we had in the earlier days when there was no lake houses, modern data warehouses. So if you can follow along from the top, there used to be a analyst or a data engineer who used to write some API configuration. We used to pull these data source from public data sources, dump it into some common shared drives.

4:58 Then the analyst used to pick up these files, run some pipelines, run some R scripts, Python scripts to join clean data. I think cleaning are very interesting. I remember there was one day, because we had multiple analysts who was working on different finance different kind of domains, they one day got together and created some common guidelines of how every analyst should publish their CSV. What should be the data types? How should they write column names? Camel case, underscore, what should be written on that? How should they should publish? And I'm sure as we think about maybe in future as well around federation, this will come in handy.

5:34 But while we decentralize, how do we make sure there are some guidelines that all data producers apply? So it was an early experience for us when we started building our data supply chain earlier. Obviously, after that, you need to test the data because this data could actually impact lives of billions of people. So we ran some quality checks, make sure if the data consumer uses it, they have the right data, right reconciled data that is available, and we dropped it back again to the same shared drive. And this was a very rudimentary early version of our UI portal internally.

6:10 They used to take this, bring that in in a catalog and help people search and discover. But there was always this back and forth, which is, "Hey, I've got this data. Can you also give me data dictionary? Or can you write a README doc for this? Or what raw files did you actually use? Are you using any cleaning script for this? Can I reuse it for my data products?" We started seeing this a lot back and forth conversation around this.

6:36 And in fact, we moved, instead of a file, we moved into almost like a zip file or a folder where the producer used to bring this together, and then the catalog used to pull, pull that in. And in fact, a lot of what you see in Atlan, the Asset360 experience today in Atlan was inspired by this. How can you bring data, code, documentation, the sources, the lineage, the checks, all of them together?

Varun Banka

7:06 So in the last six months, we've started talking to some large enterprises, complex organizations, and started to talk to them about their journey. We thought it's important to learn from them before we think about our principles of how we'll think about federated data governance. Number one, all customers are at very different stage of their federated journey, right? Some are thinking about it. Some have started the journey. Some have already created their specs and managing these data products as part of the data workflow, whether in Git or whether in their pipelines. We've realized all customers are in different journey.

7:47 Second, and I'm sure a lot of you who've joined us will agree, it's still an evolving ecosystem. There's lack of still standard patterns in how customers are building data products, right? There are different kind of specs. People don't know how, what should be a data product, what should be built as a data product. Still a lot, lot of, lot of things to learn for all of us in the community. Third, need for decentralized ownership and accountability, especially for large organizations. I remember two conversations. One was a data producer never interacted with the data producer. They, like, they're in the same organization, but they didn't know each other. Like in a small organization, that's easy, right?

8:29 You can have— you build human relationships. But in large, complex organization, the producer had no clue about the data consumer, so there needed to be an ownership beyond human connection. A second— I, I remember the second story was one of the data teams, the data producer group was switching or changing every six months. Either they were transitioning out of the company or they were switching teams. Now, that created, again, massive lack of accountability if they're dependent on humans. And last but not the least, around data quality. While a lot of customers were thinking about data quality, some had already implemented it, but the biggest challenge was enforcement.

9:10 How do you make sure when a data consumer is using the data product, using the data asset, like how do they see that badge? That badge which says, "Hey, feel free to use it. This has been enforced. The contract that you see, we've decided the SLA, we've decided the schema checks, and you can be assured that this is monitored and enforced." Like, they didn't want to just see a lot of tests that are running, a lot of incidents. They wanted to see one check which said that, "Here was what agreed on. Can we make sure there's a big check saying this was enforced?"

9:44 Now, we took these learnings and started thinking about how at Atlan we should think about this.

Varun Banka

9:50 So presenting our approach here. One, we believe there needs to be a native experience for data domains and data products. For data discovery, data governance, lineage. It needs to be first-class citizens, no workarounds. Second, persona first. If, if, if you've talked to anybody at Atlan, you will hear this every time. Today, in the world for data teams, you can't build a product just for one persona. You can't just think about just the data store or just can't think about just a data engineer or data scientist. You have to think about all their needs. Or in this world, right, a data producer, data consumer, a data domain owner.

10:33 These are all these different personas, and they have their own needs. And in fact, today, we'll talk about all these per-personas as part of FIM. Scalable. We just touched upon that different data teams are in different stages of their maturity journey. So how do you build something that solves for all kinds of customers? Customer just that are starting their journey into federated governance, and customers who evolved, are more programmatic, have built things into their data pipeline. So just to give you a sense of the vision, what, what this will feel like. On the left, this could be customers who've started building their products, data contracts as part of spec, dbt projects.

11:11 We've seen multiple variations right now. Or custom YAMLs or JSONs sitting in there. Atlan will build deep integrations with these Git repositories to help you register and sync with Atlan, to help you drive product discovery, lineage, and governance. On the other hand, if you see on the right side, if you have— if you are in a journey where you're just getting started, you would just want to expose data products to your business users. You can use Atlan's UI, the workspace UI, to curate some data products. Or maybe if you have custom pipelines, you want to just embed and register these data products via your pipelines.

11:47 You can leverage our SDKs to drive and register these data products. What does it mean from a contract perspective? Again, this again, it's still an evolving ecosystem. Folks are still figuring out what's the best way to implement this. But some of the patterns we've seen is folks running contracts and tests in dbt or YAMLs that are set up in Git Where as part of the CI/CD pipelines, they're running some assertions, monitoring it. So the approach that we are saying is... And remember I talked about badges? So as part of the CI/CD pipeline, you can enforce as you run these checks. You can send a badge and enforce this on Atlan.

12:24 So when somebo— a data consumer is accepting the data product, they can see the badge that, hey, this is an enforced contract. This is not just for consumption, but this is also enforced. And I'm sure all of this will come together as we make progress in, in the journey. Now, before we jump to demo, I just want to put out our approach. Just like GenAI, the approach we are taking with federated governance mesh is we want to build with the community. There is a lot to learn together. I'm sure we'll all learn a lot as we try to solve some of these challenges, so that's our approach here. And second, partner with the ecosystem.

13:05 Atlan, it's, it's not just a tool software problem. We want to partner with the ecosystem to help drive that change and work with those data champions who've been pushing really hard to drive this change to bring more accountability and scale.

Varun Banka

13:20 So what will, will you experience today? So I'm, I'm sure you've heard me enough. You're like, "Let's jump to the demo." So what we'll see today? We'll see from three personas perspective. We'll look at chart for data consumer, we'll go to data producer, and then we'll end with a domain owner experience. So on that note, I'm gonna pass it to my colleague, Teresa, to take it over from here. Teresa.

Teresa Rojas

13:43 Thank you, Varun. So... Oh, no, still slides. Well hopefully it's a good... It's a short one. So indeed, we're going to dive into that data consumer experience. We are we are all users of data, so let's see what Atlan Mesh can do for us. So without further ado. The first principle that Varun just talked to you about when we started going for this investment, was that it could not be a workaround. So we had to think about this differently.

14:32 We kept on talking with customers, and they told us that they really had most of the things that are needed in Atlan already. That said, just some of the things, it's not necessarily what's going to really make a solution. So at some point in all these conversations, it hit us. It needs a home. It needs a home.

15:04 So it is with great pleasure that I introduce you to that brand-new home, where your consumers can discover domains, see the activity that's happening in them, know who are the owners, what's going on in Slack. And more importantly, they can discover data products. You can search for them the usual way, but personally, I prefer to have the things that I have worked on recently at my fingertips. Or maybe I'm interested on the popular ones in my domain or with my colleagues, or I want to see what's newly published. We also have the Star data products section.

15:51 So for the ones that you work often, you know how to search for them, just by clicking on the little star over there. You can search as well by domain, the different data products that there are. So let's look at this one in particular. So I showed you the bigger container, then the data product is the smaller container. And when we started thinking about this, the problem that we wanted to solve here was really around how do we give to that business user, to my data consumer, all the information they need to decide whether this is a data product that's worth exploring for them right now, or if they can move on.

16:33 We know they're working under a lot of pressure, time pressure for deadlines and so on. So it needed to be a good overview. So this is what can we come up with with. Latest information, announcements. We have a summary of what this is really about. We have very clear ownership. Criticality. Is this. If this would be low criticality, meaning your IT team maybe doesn't monitor it as much or it has a different SLA, would that still be the right product for the business question you're trying to to solve today? Sensitivity. Can I share this maybe with the partners outside the organization, or do I need to be cautious about it?

17:23 And freshness, of course, the eternal question. It's going to be very different, the the questions that you can be answering depending on how fresh the data is. We also have a bit of a summary of how the product is constituted. So we see the number of assets, the relations that they have who's been using it, whether it's on track with the SLA. We also have the output ports. Output ports are actually how I, as a business user, can consume this data product. We also have here what's happening either on Slack or Jira or Teams about specifically this data product.

18:09 And some more information over here, like tags, certifications, and so on Let me tell you about this product scorecard. So we've been working with several customers over the last few months, and this wasn't part of our, our original idea, but something that came really consistently from every customer we talked to, regardless of where they were in their data mesh journey, is that they needed to have a way to qualify their data products. And while as they mature, maybe the scoring is going to evolve over time, and it's not going to start with the perfect data product, they did need a way to track it. So this is what we came up with.

18:57 All of you data mesh enthusiasts, I'm sure you recognize these principles, and we have defined them in a way here in Atlan to compute them. But of course, we know organizations are going to have different needs and opinions on how this needs to be calculated, so it will be possible to customize it. But for now, for me as a business user, maybe just this part this general score is good enough. Let's move on to assets.

19:29 We know that different people like to understand the same product in different ways, and something we have heard a lot from our customers is how consumers maybe don't want to be overwhelmed by the hundreds of thousands of assets that can be in Atlan already. But they are going to care about the ones that constitute their data products. So that's what they're going to find here. They can see also the related products from this view. I'm now going to talk to you about lineage. But for that, first I feel we need to jump into how technical lineage looks like.

20:15 Those of you Atlan users, you are used to these great diagrams, right? Like we have common level lineage. They are great for impact analysis. But then when I look at them from my business user perspective, like I, I'm not sure I exactly know what I'm looking at. Maybe it's too much information, slightly overwhelming, and it's not necessarily answering the question that I have. So we had to think about this in a different way. How do we give that business user enough information without overwhelming them?

20:58 I feel like I need a drum roll here 'cause I'm going to introduce you to data products lineage, business lineage. So yes, it looks like a very simple de-depiction. That was completely intentional. It is simple, but it already gives me information like who are the owners and how are these products connected. I can look at the output ports as well and how they are related. This is a depiction probably of a organization that have just started with data mesh and doesn't have a lot of products right now. But let's time travel a bit.

21:44 In X amount of time, as that initiative matures, they're going to have more domains with more data products, hopefully making more boxes here and connections, leveraging the power of the mesh to really get to a state where we can say the more, the messier. So let's think about this one more time. Over time, this will become the map of your entire data estate. That is so powerful. The map of your entire data estate in a format that it's value-adding for all of your users, not just the technical ones. All right.

22:34 So now we have to talk about the journey between assets and data products. We kept on talking with customers, and something that really came up a lot was, "But what is the difference?" Like how do you go from one to the other? A data product can contain only one asset, or it can be a collection of many. But what really makes the difference is that data as a product thinking, thinking of the consumer, thinking of the use case. Yes, all of that works. But then ultimately, it came down to expectations. So yes, you guessed it, data contract.

23:19 In this view, you see, again, I'm a business user, so I am looking at the information that I need from the data contract. I can see the usage, the limitations, communications channels. I can see the versions. So it really has the information that I need. I can even check on the output ports, the terms, and look at the column preview or the sample data, which by the way, we see here there's a lot of descriptions missing, so let's take a mental note out of that. So indeed, this data contract gives me all the information that, that I need.

24:06 I know there's a lot of complicated stuff in there, but for my business person, it has everything that I need. Now that I think of it, I am really interested on this data product. I'll be using it. So I can subscribe to the to have alerts to changes. So I think it would be important for me to know when data contract is updated. And then I'll get notifications on the channel of my choice. Let's now move on to activity. So this is a good old history of this data product, right?

24:46 Like, so I can browse through to see or I can filter also to understand what have been the changes to this data product, which is important information for me. I can also see the views on it, who's working on it. Maybe some of my colleagues are there, maybe some of my customers are there, right? Like, because I produce data too as a business analyst. We also had to consider the aspect of data products being used downstream, not by people, but by other products. So this downstream usage widget over here gives you that answer. You're gonna have data products that are like that. Think of a customer table, for example.

25:37 We can also see the collaboration tab over here. This is all in good Atlan spirit, right? We love collaboration. So you can see who subscribed in their latest Slack. You can create your, your articles right there. And you remember we had seen that sample data that we didn't have a lot of descriptions with there. Yeah, like, so let me go right ahead and I'll just create a ticket for the product owner. So that's it. We did the tour.

Varun Banka

26:26 Awesome.

Teresa Rojas

26:26 Yeah. Thank you, Varun.

Varun Banka

26:29 Thanks, Teresa. Could we, could we go to the wrap-up? Let's, let's do... And thank you for sharing this journey. Let's do a quick recap. What did we look at? We looked at, we talked about how do we make it first-class native experience in Atlan. How do we discover domains? We made product a first-class citizen, so you can now explore product just like me and Teresa. We don't like to search. If you're doing a reset, less clicks, just find it right there. I, I could see a lot of feedback coming from lineage. So how do you bring business lineage, data product lineage? We love collaboration. There's no...

27:06 How do you bridge the gap between the producer and the consumer to collaborate, right? The producer could be driving Jira workflows, managing tickets. How do you bridge this gap together? And how do you get Slack notification so that if the contract gets changed, right, or if there is some update in the data product, you receive it right there so that you can take actions, reach out to the right owner and take the next step. So thanks, Teresa, for doing this, bringing the con-consumer experience. Now, let's, let's go to the next experience. Pete, do you wanna take this?

Peter Swigert

27:40 Thanks, Varun.

Peter Swigert

27:44 So I'll jump in. Let's talk a little bit about how Teresa covered our consumer experience and how Atlan Mesh is a seamless way to use data as a product and domain-oriented ownership to solve our problems. But where did that data product come from in the first place? So let's jump into our data producer experience. We're all users of data, but many of us are producers of data as well. What does this look like? Let's get back into our demo. So this is our product homepage, and before we dive too deeply into editing or creating any new data products, I wanna revisit again the classic Atlan experience.

28:22 I'll jump over here to our Assets page, and if you've used Atlan before, you know they're the home for your data estate, a key tool for data producers such as data engineers, scientists or analysts. You can see our instance here has connections to all sorts of partners and products across the modern data estate, and this could be from your upstream CRMs through your data warehouses and lakes into the BI layer. But particularly over the past six months, what we've heard from the industry and our customers is that with millions of tables, dashboards, DAGs and other objects, we need to pursue programmatic data mesh principles to better manage these complex estates.

29:04 Scalability across all these tools is fundamental. So if I'm a data producer, say the engineer who Teresa pinged with the Jira request, let's dig into how I can use Atlan Mesh to better service the data consumers in my organization. I'll jump back into our product homepage that Teresa demoed, say, "Hey, maybe I'd like to go about creating a brand-new product," the first step on our data mesh journey. Now, if I wanna start from scratch, could do simple things. Maybe I work in our finance domain, so I'm gonna work on a data product for accounts receivable analytics, and I'll tag this in our finance domain. I can come back, add some other information later.

29:46 But what is this data product gonna be made of? Well, we hear from customers all kinds of approaches. Some are building their products in DBT, some are building custom specs in GitHub, and some folks are just starting their journey and looking for a toolkit. What we've approached with is here is a programmatic way to bring in any kind of data assets across your entire estate. So these are custom filters you could connect based off of your source systems, your asset types. Maybe you wanna just bring in your first data product as just a table. Maybe you're looking for more governance-oriented things like ownership or certifications.

30:28 So with this approach, we have an easy wizard to get started to edit and create new data products with anything from scaling from a single data asset inside to thousands. So let's jump back into the data product that Teresa was looking at and see how we can edit this to meet the consumer's needs. The user had requested that we add column descriptions on one of the output ports, a table, so that it was more clear what those fields meant. We can easily jump into that, and we have edit access. So we can go into our product that she was looking at, the actual versus forecast financial KPIs.

31:05 We might wanna go ahead and make these changes, but before we do so, we've heard from so many of our customers that there's a huge gap between data producers and consumers. Oftentimes, the producers have absolutely no idea who's consuming their data and how. So if I'm a data engineer, maybe I'm just gonna go check our lineage and see who's gonna be impacted downstream by any changes I make here. Here again, we can see an easy quick view of our data products and get a sense for downstream what's gonna happen for any updates. Looks good downstream. In this case, we really feel safe. The request was just adding descriptions to a table.

31:44 If we jump back into that output port, we can see here's where the request for descriptions was, and here we can use some of our latest and greatest functionality at Atlan, Atlan AI. So Atlan AI will automatically read the metadata of your estate, take that context into account, and write a description for you, and this is all without any security concerns as it's not reading any of the underlying data at all, just the metadata that you give access to. Go ahead and apply that. And while we're here in the data contract, we can give a fresh look at it from the perspective of our data producer.

32:23 Teresa looked at this from a consumer trying to read this contract and understand how they could use it to solve a business problem. But I'm a data engineer. I think about this contract as code. To us, this is working in the tools that we feel most comfortable in. This could be a YAML file or a JSON. We could have Atlan integrated here with GitHub or Bitbucket. Perhaps we've started enforcing this at the pull request and, and review level, or maybe we're just using it for informational purposes. Again, whether you wanna crawl the source system to use or curate and enrich the metadata in Atlan, we're designing for the scale that you wanna work at.

33:05 So we finished reviewing our contract here, and we've made our update on those columns. So let's jump back in the collaboration section, and we can go ahead and close out our Jira ticket. We'll give ourselves a nice little pat on the back. And as you can see, we've never actually left many of the tools that we feel comfortable in. We've worked in integrations across Jira into Atlan, synced with our contract, and we've moved from producing individual data assets to more complex consumable data products. So I'll let Varun give us all a summary of just what we explored.

Varun Banka

33:52 Thanks, Pete. Just like Teresa took us through the consumer experience, Pete took us through the producer experience. So what did we go through? We looked, we looked through a data contract experience, both from a UI and a code perspective. Our goal is, as the data contract specs evolve, you can put, put this in something like a Git repository and push this to Atlan as part of the CICD pipelines, which can give you a first-class native experience on Atlan. Domain three sixty personalization and product creation. So for data product creation, again, the goal is same.

34:26 How do we balance both the worlds, people who are just starting up, and also customers who are building these product specs, YAML, dbt project in their GitHub, how to balance both the worlds going forward. Pete, it looks like we have one more persona pending, and you might have to play dual identity today. Are you ready to pick up the next one?

Peter Swigert

34:46 Absolutely. I mean, we work hard here at Atlan, always have to wear multiple hats. So I'm not just your data engineer today, but we have more in the bag. We're gonna talk today about our domain owner experience.

Peter Swigert

34:58 So I'm gonna jump right into our demo here. So I'll start here from Teams. After all, who among us doesn't have Slack, Teams, or similar open constantly? And right now I'm thinking more as a domain owner, so I'm not an individual data engineer trying to do one-off requests. I'm thinking holistically about how to manage all the data across my part of our complex organization. Now, what I can do here is jump directly in to, say, our finance domain. And again, that's sitting directly in Teams. No need to jump into a new interface. Atlan always exists in the tools that you wanna work in.

35:36 So this is our finance domain, and what we're bringing here is programmatic governance of all useful data in this domain. When I say useful, I mean curated data that drives business value. If I run finance analytics at our company, this means I don't want any more manually updated Confluence pages, SharePoint sites, or Notion docs, which all inevitably fall out of date. This microsite scales from my very first data product to hundreds of complex products seamlessly. So why do we need this? Well, we all know our data is getting more complex. Brilliant products across the modern data stack are making things easier and easier, but that just lets us do more and more with data.

36:19 This introduces new risks in security and compliance, new roles like analytics engineers, and higher expectations, like the move from batch to streaming. We hear constantly from customers and industry leaders about the need for managing these complex data states at scale without yet another tool that adds to the complexity A microsite like this gives you a one-stop shop for your data landscape as programmatic and scalable as you would like it, no matter where in your data maturity journey you are.

36:50 As you've no doubt seen, on this page, we capture all kinds of key information, the relevant products in our domain, use-usage, users and key stakeholders, downstream assets and related contracts, letting us know where in our data state this works and the context to effectively collaborate across our company. We've seen a lot of lineage so far today, but I'll jump into one more lineage view. This is the absolutely super simplistic, again, intentionally, domain-level lineage. So if I run finance data, I can see how my part of the company sits within all the other parts of the company.

37:26 I can jump into more details and see how the data products might flow from marketing into finance into IT. And again, this will evolve depending on your part of the step in the journey across data mesh in the modern data federated governance. So this is a scale and complexity challenge. And speaking of scale and complexity, who among us sometimes just doesn't wanna go to the old comfort, the old standby, Microsoft Excel? So you may have heard a few months ago that Microsoft launched Python in Excel. At Atlan, as a data team, we all thought that was hilarious. Things are coming full circle, data science moving back into Excel.

38:03 But if Microsoft can do that, we can do Atlan and Excel. I'll jump here into our Microsoft SharePoint Excel experience, Office three sixty-five, and we've got the Atlan add-in for Excel here. So I'm really excited to demo this and all the different features so we can take all kinds of actions directly from Excel. And again, I was speaking as a finance domain data owner. I'm in Excel all the time. I'm pivoting, I'm doing VLOOKUPs. I work out of spreadsheets, and that's where I wanna be. That's where I wanna operate my data products as well. So I can do things like run queries, import lineage, document columns. Today, what I'm gonna do is enrich product assets.

38:47 So what is that gonna do? That's gonna pull down all the data products in my domain and give me an option to add any information to them that I might wanna do directly from Excel. Maybe I don't wanna jump into the Atlan UI and have to do this. I'm a spreadsheet person. So what I've done here is I've chosen the data product that we were looking at recently. It's imported. These are row by row, all the different individual assets that we are looking at. Tables, Looker files, individual BigQuery columns. Maybe at my company, we've launched a new product and I'm worried about customer PII flowing downstream into my data products and reports.

39:29 I'd like to do some federated governance. So I'm gonna say, "Hey, this fact sales table, I'd like to tag this as PII." Now, this tag is gonna push back into Atlan when I sync it, and that's gonna apply access control policies. I'm using tags here as a nice federated way to shift left in my data governance approach. And if Excel isn't your cup of tea, it's an Atlan guiding principle that we operate where the humans of data are. So you could just as easily have been in Snowflake for this. Added your PII tag in Snowflake, sync that into Atlan. And you know what? We've taken this one step further.

40:13 You can be in Atlan itself, tag PII into Atlan, push that down into Snowflake. Things really come full circle. So as seamless as it can be, Atlan operates wherever you wanna be. Let's jump back into our domain. So we'll go over here to our finance domain. Now, when we manage our entire data domain, we can use our domain reporting center for all kinds of useful information to help us ensure that the data products we build are driving value, from usage to certification to trends over time. What I can look at over here actually is I can go down and see, hey, what tag's been pushed? All right, my PII tags have made it in.

40:58 Now, we've covered a lot in terms of creating and updating data products, monitoring them in a reporting center like this. One of the key things here is customers tell us all the time about legacy data debt, unused tables, pipelines, and reports. Those are getting lost in the millions of data assets. And especially with today's cloud computing requirements, this data isn't free. We can programmatically track that in a reporting center like this and ensure that the data that's being used is driving value, the data that's not is being deprecated. Well, Pete, you might say, "Cool, a dashboard. I have dashboarding tools. I don't want any more of those. Give me something automated. Give me something programmatic.

41:40 I thought this was computational federated governance." Well, how else can we automatically deprecate old content? Atlan comes with all kinds of automation. In-app, there's a seamless concept for bulk rules application called Playbooks. Outside the app, there's incredible extensibility via SDKs and APIs to to connect in a heartbeat to whatever you wanna use. So wherever you are, you can push the edge of things with Atlan. So I'll pause here and hand back to our co-founder, Varun, for a review of what we've covered and next steps.

Varun Banka

42:14 Thanks, Pete. So Pete just took over and took us through the domain owner experience. Just to do a quick wrap-up, I know we have gone through a lot. We looked at Excel, bringing the power of data products into Excel. I saw a lot of love on the comments as well. Yes we love Excel users as well. Like I said, different data teams are in different journey. We wanna make sure we solve for all personas. Domain-level reporting, which is how do you get aggregate reports for especially domain owners who are gonna be more responsible for their domains, all their data products, assets.

42:50 I'm sure we're really excited to bring Atlan into Microsoft Teams as an embedded experience. And also from a domain profile perspective, if you look at that microsite, our goal is if you're onboarding someone new in your domain, you're managing the domain for your data, how could this be that one landing page that has everything, your data products, your resources, your documentation, the people? This becomes the first page. If somebody has to learn about the finance domain, this is where they look at. And we talked about technical lineage, we went through data product lineage, and now opening up as well the domain lineage.

43:26 How do we make it easy, not just from a technical point of view, but how the domains can be connected across a landscape? So what's coming up next? So we've seen the consumer experience, we've seen the producer experience, we've seen the domain owner experience. Where are we heading next from a data mesh point of view? One, like we said, we all are just kickstarting this in the community right now around how do you register data products programmatically. We're seeing ton of different approaches right now from folks citing custom specs on their GitHub on their Git repositories or using DBT projects.

44:04 We'll be working with the community to find the best approach of how do we build, help create those programmatic integrations to drive those, register those data products which are sitting in programmatic worlds. Second, contract enforcement. This is where we'll work closely with our data quality partners to help drive enforcement, but bring that trust signal back to Atlan so that when somebody's seeing that data product, the quality score gets updated, the enforcement says big batch that this is enforced. So we'll be working with our partners in the ecosystem, our friends, to drive this. Bring your own contract. We're seeing...

44:43 I'm sure that peop— folks would have seen ton of open source specs now, thanks to the community that is opening those up. But the new kind of JSONs, YAMLs will evolve. So how do we allow folks to bring their own contracts? That's why if you see Atlan, we've thought about both the UI experience as well as the code experience, making it easy for you to manage irrespective of persona. And definitely more, right?

45:05 Like, like we said, we are just starting in this mesh journey, and I'm sure like a lot of you, we love— And this is a callout to everybody, please reach out even if you want just a coffee chat or want to talk half an hour to us. Talk, talk to us through your journey right now. Talk through us what challenges you're facing. We'll be happy to set up a call 'cause we have an amazing team that can help you and learn as well. We can do this. So that's on the mesh journey, the road ahead. Austin, do you wanna take next?

Austin Kronz

45:37 Yeah, absolutely. That was awesome, Peter, Teresa, Varun. Really exciting stuff. I think finally going from the conceptual data mesh frameworks to something tangible that data teams can really wrap their head around and start to, you know, implement data products in this federated approach and continue down this path of programmatic governance and shifting to the left. I think we're really fortunate at Atlan because we have a lot of amazing customers who are on this path already. And you've probably seen some of our master classes, you're attending an Activate right now. We really wanted to take it to the next level.

46:11 You know, all this talk of AI, data products, data mesh versus data fabric, should we centralize or decentralize? Or how do we go from reactive to proactive? Well, giving data teams those answers I think is something that we wanna give back to the community, and we have some awesome customers that are willing to help us do that. So on October fifth, we're really excited to announce first of its kind community customer conference where we will reimagine data governance. We'll be joined by teams like Autodesk, Nasdaq, HelloFresh, Fox, and many more, just to learn everything it takes to go down a data governance path, whether that's getting buy-in, building a strategy identifying the right technology.

46:53 How do you even evaluate the tools? What if you have something already and you're moving to a modern data governance platform? We're gonna hear it right from the customers themselves, and I think that's the best way to learn. The most common question I get when working with customers is, "What are other customers doing?" So really excited about this Regovern event. It'll be our largest event, bringing in the most customers we've ever had. And do check that out. Do register for that. It'll be a mixture of panels and presentations, and even some downloadable takeaways to help you on your journey. So join us on October fifth. I think it's gonna be the best yet.

47:26 And thank you again, Varun, Peter, Teresa. This is incredible. I, I'm really excited to see how the data mesh concept continues to emerge and permeate through the Atlan product. So really exciting. Thanks. And I'll just leave you with if you wanna stay in touch with us, if you wanna have a chat with anyone on the team, you know, we're constantly learning and getting feedback. So do reach out, whether it's on social media, on LinkedIn, following blogs, et cetera. I think anyone on the team will be happy to, to have a chat about data, data governance, data mesh.

47:59 So I really appreciate the time, and I'll give one last thank you to everyone who took time out to join today, and can't wait to see what's next. Thanks, everyone.

Varun Banka

48:08 Thank you, everybody. Thank you, everybody, for joining. Thank you.