Atlan Video Demo: A Home for Data Teams

Andrew Ermogenous

0:00 Hey there. I'm Andrew. I'm here to tell you about Atlan a home for data teams. In the same way that Salesforce built a home for sales teams or GitHub built a home for developers we've built a single collaborative environment for the humans of data to do their best work. And we believe we're uniquely positioned to do that because we started as a data team ourselves essentially doing data science for social good. You'll see the scale of projects we're working on, right? Billions of pixels of satellite imagery, sensitive data for hundreds of millions of citizens.

0:25 But internally, we face things like this every day, whether it's like a quick Slack request, a, a quick question, trying to give a, a new hire context. And ultimately the the reason for this is what we call the potato chaos problem. It's this whole kind of co-mingled, complicated process where multiple people are involved and, and, you know, constant context switching. But it's not just the ad hoc Slacks or Zoom calls it's the perpetual fire drills. It's the, the tribal knowledge that's lost when someone leaves. And ultimately, what we recognize is that data teams can't do their best work because they're always reactive.

1:01 And so Atlan was built, you know, fast-forward several years, to solve for, for some of that. We've built a collaborative workspace for modern data-driven teams. Great investors, recognized by Gartner, but most importantly, incredible customers. High growth startups like Plaid and Postman, as well as large publicly traded enterprises like Ralph Lauren and, and Unilever. And so I know you all came here to learn about data catalog, but we believe that data catalogs are fundamentally broken.

Andrew Ermogenous

1:26 They're yet another siloed tool that require implementation treat every consumer the same, and ultimately, like it takes a lot of work to get up and running. It, it lacks automation. And so we thought, why not take the best-of-breed kind of principles from folks like Netflix or Zapier, whether it's like a personalized experience or automation to build an active metadata platform. And so that's where Atlan comes into play. So we make all your data assets discoverable, whether it's a table, a dashboard, a model, et cetera.

1:56 We create that single source of truth, kind of like a GitHub profile so you can understand who owns it, where does it come from, what are the tags, classifications, the lineage. And then we create a personalized and, and curated experience on top of it. Because what we recognize is that, you know, a business user has different needs than a data scientist and a data engineer. And so we've enabled that, that personalized experience, and then in turn making the metadata available where you need it when you need it. Whether that's creating a Slack notification and, and pushing something to Slack as an example. Or, you know, taking a Slack conversation and pushing it back to Atlan.

2:31 And then down the line, doing things like logging Jira tickets or, you know, making metadata available where you need it, when you need it, via a Chrome extension. And the reason we're able to do this is because we're built by what we call the metadata orchestration layer. And so behind the, the scenes, we are built on, on this kind of open and extensible platform that can enable things like building a custom connector. So maybe you have MongoDB as an example, and you want to pull that in. Down the line, we'll start to enable things like creating like a data health dashboard. How do you use metadata as big data, essentially?

3:04 And then ultimately do what we call metadata activation. So this could be anything as simple as, you know, if you're gonna drop a column, how you notify the, the downstream consumers of impact. Or, you know, maybe you use metadata to tell you that a, a certain table isn't used frequently. And so now you want to take... Use that metadata to essentially like purge that, that data asset. So that's a little bit about who we are as a company.

Andrew Ermogenous

3:26 Now I wanna jump into the, the product itself. And so, you know, I talked about this concept as, of a personalized and curated experience. So every homepage will be different depending on what type of persona you are. So you'll see like my drafts, popular queries, announcements on, on assets that I have access to. But at the core of everything in Atlan is this concept of search. So like Plaid calls us Google Search for their enterprise. So when I search for something like customer, I could find terms, tables, dashboards, calculated fields, et cetera.

3:53 But oftentimes, you know, I don't necessarily know what I'm looking for in which case I'll jump over to the, the assets view here. And so what you'll notice here is... And maybe I'll just jump out to all access. You'll see that we have thousands of potential assets, right? Tens of thousands of tables, views, columns. What we've created and what's hyper unique to Atlan is this concept of a persona, which means that you could create a curated and personalized experience for every persona of data. Right access to the right person at the right time.

4:22 So for the purposes of the conversation today, let's pretend I'm like a BI specialist and, you know, if you're one of our customers like we were, they have 25 different personas, anyone from treasury ops to product that are accessing data. And so we can now create personalized experiences for each and every one of those folks. So today I'm I'm looking at consumer behavior for beverages. I'm an analyst. I just came to this, this group. I'm trying to learn quickly. So I'll search for consumer. You'll see something really neat, which is when I search for a consumer all the... Everything related to customer comes up.

4:51 You'll see consumer isn't mentioned anywhere here, but because it's a synonym to customer, it'll pop up. So kind of subtle nuances to enable you to browse and, and find the right assets you need. We do have, you know, some of those same type of traditional experiences that you may be used to, like verified assets, as an example. So if I search for verified, I can now get a quick view into what the verified assets are. And you'll see here at the right the sidebar, which enables us to get like really quick context into any individual asset. So if I'm looking at consumer behavior, you know, customer detail sounds interesting.

5:22 I can maybe quickly jump over to columns as an example. But I'll see there's only three columns here which likely means that this is a reference table, as an example. So I might filter down further. Maybe I'll look at, you know, when the ETL last ran, as an example. And we're enabled to do this because we have this powerful custom metadata capability which I'll get into. But now it's kind of filtered down to, to two assets. And so if I'm trying to see what the most popular table is, I probably wanna see if it's used downstream. So we can use that sidebar again to get a sense of the lineage of the asset.

5:54 And we'll see here it's not used anywhere. What about this beverages table? Interesting. Okay. We'll see it's used both upstream and downstream in this Looker model. So probably something we wanna drill into so this is where we we come into the data asset 360 view, kind of single source of truth for all things data.

Andrew Ermogenous

6:11 Owners, experts, verified status, data dictionaries front and center, README. You can also embed like Google Sheets or Confluence pages here. And I want to to highlight a really important nuance here with respect to Atlan which is governance. And so you'll see that if I click on this column to the right, I'll see that this customer mobile column has a PII tag. And so that's something that's happening at the point of integration. We're automatically categorizing PII. And what you can now do is create a, a policy such that when you, you jump over to the sample data preview it'll mask the PII.

6:48 So you'll see here customer name and customer mobile are masked, but I could still get value out of the data asset here. So okay, great. I found the asset. I've got more context. This seems like something I want to do or, or use. I'll jump over to the lineage view. And so the goal here is I can get quick context on how it's used upstream and downstream. And for context on how we're using lineage or, or creating lineage we'll parse your SQL query history or your, you know, dbt manifest.json files and get some visibility into what these transformations are here. We can also go down to the column level.

7:21 So if I click on, you know, customer name as an example, I can see how that's used upstream and, and downstream. Coming back to my original story I can see that this is used in a Looker model. And now I can go downstream and start to get a sense of how else this, this asset is used, right? And so this gives me quick context, and if I click on this individual table, I'll see that Himan— or explorer rather, see that Himanshu owns that, and he seems to own all these assets. So now I know that Himanshu's probably the right person to talk to. So I can use some of our embedded collaboration.

7:55 I could ask a question. I could say Himanshu, can you confirm these are the right explorers?" And he'll get notified where he needs it, when he needs it. So now I'll shift gears a little bit and I'll jump over to Slack.

Andrew Ermogenous

8:12 And so you'll see that where that message came up I tagged Himanshu. He gets tagged. He can just give a quick check mark, and I know the answer to the question. Maybe just to kind of switch it up a little bit, let's pretend for the purposes of the conversation today I got a message that like, "Hey, you know, CAC is off." Great. I can actually search Atlan via Slack, and so I can kind of quickly get a picture of what customer acquisition may, may look like. So I'll see here customer acquisition cost, that's a term in Atlan. I could send it to a channel. Or, you know, I could jump directly into Atlan.

8:50 And so what this will do now this will pull up customer acquisition costs in Atlan. And I'll quickly get the answer to my question via this announcement feature here. So I'll see that the calculation method moved from the complex method or moved to the complex method from the simple method. And so now I know that that's exactly what, what happened. I can even dive a little bit deeper, and I'll see there's two different calculations here. So there's the complex method and the simple method, which is deprecated.

9:20 I could jump over to our insights tab, and now I could get quick, quick context on, A, how it's calculated as well as, B, if I want to run a results I could potentially do that.

Andrew Ermogenous

9:31 And so we've actually created this SQL ID type experience. You'll see I just ran the query and got an answer to CAC. I could do the same thing if I wanted to jump over to the, the simple method as an example. And you'll see I just ran that and we'll see it's five as compared to fourteen. So that makes sense as to why CAC exploded. One other kind of quick thing I wanted to highlight, you know, a big value add for Atlan is the concept of our visual query builder. That enables us to create queries in a kind of no-code environment.

10:03 So anyone who's familiar with, with with a pivot table as an example could use this. We can also create this concept of a parameterized query here. So for the data team, you can enable better self-serve. And so now the business users can kind of change the, the parameters and go back and forth. So that's kind of like the discovery exploration kind of use cases. I did want to jump over to some of the admin capabilities here. So we'll quickly jump over to the workflow center.

Andrew Ermogenous

10:31 And keep in mind, this is not an exhaustive list of our integrations but just a few examples. So you could create workflows which are both integrations, and down the line will be some of those like metadata orchestration use cases that I highlighted. We support all the cloud-native BI, data warehouse, data lake solutions, anything from Snowflake to Glue to Tableau, you know, you name it. And we could jump into some of that, as well as some of the relational databases like MySQL and Postgres. Finally, I wanted to jump over to the concept of personas in Atlan. This is a r-really unique capability to Atlan.

11:07 Coming back to some of our core principles, we've recognized that the humans of data are diverse. It's not just a data scientist or a data engineer, an analyst. It could be a marketing data analyst versus a finance data analyst. And then you bring in, you know, folks like finance or marketing, like they each have different, different needs, right? The depth and breadth of data, the level of access, the level of security. And so what we've enabled here is we've separated... So first and foremost, we've created this concept of persona, which is different from a user or group. A persona is a very specific user of data within the organization.

11:38 And then we've also separated the concept of a metadata access policy from a data access policy which is really, really powerful. Which means that you could provide metadata for all and you can, by the way, give them kind of edit access or or not, depending on what you want to enable. So metadata for all you could see all the metadata for all the assets. But you can now separate the, the concept of data access, whether it's on that preview or in the, the Query tab as well. And so actually I'll just open up this asset specifically. I can actually turn on or off the asset.

12:14 And then we also have this concept of masking, which I talked about earlier. So some really deep and, and granular policies that we can enable. We also have this, this concept of a purpose. A purpose you could think of a, as a data domain or a data use case. So PII data protection as an example. You know, global policy to turn off data access to PII. These are things that you could start to enable within Atlan. You know, endless possibilities for, for purposes. You can even start to think of like churn analytics or very specific like data projects. Few other things I, I wanted to, to highlight here.

12:49 With respect to classifications I talked a little bit about the, the PII classification. You can also create like GDPR or public. This could also be like silver, gold, bronze. We're also opening up our bot ecosystem so that customers can create their own bots or alter the logic of the bots to, to meet their unique data needs. Along with the concept of you know, access policies, we have the repla-request workflow. Right now what you'll see these are really bot-driven request workflows, but this is where human-driven request workflows would come into play as well. And then the, the final piece is this concept of custom metadata. It's a really unique component to Atlan.

13:32 What you could do here is start to create different use cases for individual end users. This could be anything from bringing metadata from your, your ATL creating kind of unique data stewards or business stewards, or, you know, bringing metadata from tools like DBT from tools like a data quality solution, et cetera. So variety of different use cases, really extensible governance center, all backed by an OpenAPI. Thanks for watching, and looking forward to sharing more in the future.