Key Takeaways From Gartner Summit 2024

Austin Kronz

0:00 Welcome everybody. I'm really excited about today's session, where we will go through some of the key takeaways and kind of nuggets of information that we got from the Gartner Data and Analytics Summit in Orlando. We have 30 minutes, so it's gonna be a really jam-packed session. My name's Austin Kronz. I'm the director of data strategy here at Atlan. I'm particularly excited about recapping a Gartner event because I used to be a Gartner analyst on this team that puts together the event. So I'm really excited to see what old teammates were presenting and, and learn from them, as always.

0:29 Well, I'll give a, I give a quick review of kind of what the event is, and then I'll talk about the goals of this session and what I'll, what kind of information I'll be able to provide for you. So, seems like we're evening out. Some people were able to attend, so maybe this would be a good refresher as well. Quick overview of what I plan to go through. I'm really excited to go over the results of a survey that the Atlan team ran at the Data and Analytics Summit.

0:52 So we had over 600 respondents to a survey about what is the state of data and analytics in terms of what are data leaders' priorities for 2024. And then I'm gonna go through the key themes and sessions of the actual event. So this is gonna be a mixture of both what Gartner has sent as follow-up in terms of the overview of the event, as well as just sessions that we were able to attend. And I, I've chosen three to give the, I'll call it, like, a five-minute turbo review of what happened in these sessions, 'cause I think they really represent a lot of the key themes really well.

1:22 And then I want to save at least five minutes or so at the end to get just open question and answers, and turn it over to the audience. So with that, I'll jump right in. For those of you who said you haven't attended, I see the comments already. "I want to know if data mesh is still dead in Gartner's view." We'll see. For those of you who weren't able to attend this is a, a massive event, right? There's over 150 sessions from Gartner analysts. There's 37 interactive roundtables that have limited availability.

1:49 There were 136 vendors, exhibitors that were there in attendance, which meant there were 61 sessions that you could possibly attend, even just from an exhibitor standpoint. So if you only went on the show floor, you could probably pack your entire week full of sessions, which adds up to one really long week. Which is why I think this recap is gonna be really helpful, 'cause hopefully you can get in 30 minutes what a lot of people spent the week getting. All of these sessions add up to a lot of information overload, right?

2:16 I can't be the only one that built a Gartner agenda, and I found out that I was supposedly trying to attend seven different 11:00 AM sessions. So with that, I think we'll dive in a little bit to key themes of all the sessions. So let's start with what we were able to uncover.

Austin Kronz

2:33 Get a baseline on what are data and analytics leaders experience in terms of pain. So hopefully resonating with things that you are experiencing as well. The first question on the survey that we asked, and again, this is over 600 respondents for these types of questions across industries and across sizes of companies. Ultimately, just what are your priorities? Now, this isn't, we didn't include gen AI in this for partially on purpose because we wanted to see what are the foundational elements that you actually are prioritizing for 2024. I think what we found is 65%, over half of the respondents were saying that data governance is a top priority.

3:06 I think this is really reassuring because ultimately, to achieve any type of analytics, AI, data science type success, I think data governance is resurfacing as a crucial element in making these things happen. Next, we got into trends, right? Now, the next follow-up is, okay, so what are you following closely? What are you most excited about? And surprise, I don't think any spoilers here. Generative AI was over 54% of respondents saying that this is what they're most excited about. There was a question about data mesh. I think still a third of respondents were saying that is next priority.

3:40 So probably more of how are we organizing for success, how are we approaching more of a decentralized organization structure, how are we achieving governance at scale through automation, things like this. Surprising, only 5% of data leaders talking about data contracts, which is a little counterintuitive to what I had experienced just in calls with customers. Data contracts, almost everyone talks about data observability, quality, and the ability to, you know, agree on what things are. And so I think data contracts maybe will start to gain some traction throughout 2024 in terms of terminology, but I think in concept, almost every customer I talk to is trying to achieve what data contracts are aiming to promise.

4:19 So a really exciting opportunity, I should say. Next phase of this is, okay, you're interested in gen AI. Makes perfect sense. How would you describe your current position and your current ability to achieve gen AI success? And I think we only saw about 40... Sorry. Yeah. We only saw about half of the respondents saying they're just in a research phase. I think a little less than a third talking about, "We're doing proof of concepts." And as you can see here, 14% are actively using gen AI use cases in production. So I think there's still room to grow.

4:59 The vast majority are still in an exploration and prototyping phase, which means we have a lot of opportunity in 2024 to, to move past that. The next obvious question, again, continue with this train of thought, is, well, if only 14% are able to or are prepared or are currently delivering use cases in production, what is the main contributor to, you know an absent gen AI strategy, or why are you stalled? What are the gaps? And I think what we found is most of it is just trying to figure out what use cases we should approach. Like, we don't know the business impact of the technology just yet.

5:35 We haven't really applied it to our business or done that type of analysis. Followed by, we just don't know if we have the right people. And what I thought would be potentially higher, and maybe it's just because organizations haven't gotten to this point yet. If they haven't established value, they probably haven't gotten to this 17% who are saying, "We don't even know if we have the data, and we also don't know if it's of acceptable quality."

5:57 So data, I think, following value is honestly a good signal, but obviously we want all of these numbers to be closer to, all of these respondents to be closer to the 86% that are saying that they haven't approached an AI yet. So hopefully shifting those numbers throughout the year. I think I think a positive signal is only 13% are saying technology is not really the problem.

6:16 Technology is pretty readily available. It's more of how do we apply this to our organization and those are typically the more challenging obstacles. So last but not least, if we're not quite sure how to apply this to an organization, I was curious about, okay, well, if you don't know what the value is, are you tracking ROI of these different initiatives today, right? How are you tracking your ROI of your data, your analytics, your and I think we kind of are getting to the root of why it's challenging to figure out the value of this. Only 11% of data leaders were saying that they track both qualitatively and quantitatively the ROI of their investments.

6:55 Now, I could easily add the 11% to the 20%, 27% that you see here to say 38% track some type of value. But I think it's important to call out that 27% is more operational KPIs. I think of this as you have a data quality initiative and you're tracking data quality metrics, and you're tracking it over time to say, "Yes, generally, we're seeing a return on our data stack investment because data quality is improving," or they're using things like monthly active users of analytics tools or data governance tools, right? So it's more of an operational metric. It's not necessarily we're actually driving increased revenue, decreased cost, reduced risk. It's not innately tied to those things.

7:36 So I think in terms of what we should hope for the rest of 2024 is a push more towards that business value connection, and hopefully this 11% increases year over year. So with that, I'll get into, to key themes.

Austin Kronz

7:50 I'll start with, I think, just what Gartner was able to communicate after the event as here's overall everything we learned this week, what does Gartner say are the key themes? Obviously, value. Focus on value, and the way we generate value is increased collaboration, so they were saying generating value together is extremely important. So getting more cross-functional, connecting to the business, blurring those lines between IT and business teams I think is definitely critical. Achieving AI readiness, I think no surprise there. AI readiness meaning do we have the people, process, technology in place, the data in place to achieve AI success? Getting back to governance fundamentals was, I think, a great topic.

8:32 Governance is not a new topic. You can find governance best practice textbooks 30 years ago. But I think AI has introduced, I think, I don't want to say popularity of governance, but a renewed sense of governance has its place as almost a first order task within data and analytics and within organizations. It means more than just policy and regulation. I think it means more about just generally how do we optimize how we do data and analytics. We want to invest in metadata to future-proof strategy.

9:01 I'll talk a little bit more about this later and why seems like Gartner thinks and also Atlan thinks metadata is really the backbone to achieving success, especially with things like gen AI, but as well as just traditional data analytics, your descriptive diagnostic type reporting. Ultimately, all of these changes, there's going to be a shift in operating models. I think we see this as relevant in data mesh, where there's more of a push towards decentralization. I think a common theme that I see with customers is this, I hear a lot of, "We're actually centralizing to prepare to decentralize," and so they're almost coming in so they can more effectively distribute and federate.

9:37 And all of that comes down to new ecosystems and new operating models. And lastly, data and analytics for good. I think just ultimately how can we use data to become not just better companies, but make a better impact on the, the world as a whole. With that, I, I'll get into the three sessions that I picked to diagnose for today's webinar. And I picked these both on data that we got from Gartner in terms of the most attended sessions, the most highly rated sessions, but also just ones that I thought summarized these themes very well. And I'm not gonna redo the whole presentation. These are 30 to 45-minute sessions.

10:12 What I've done is taken maybe one or two slides from them, and then I'll give you the, the Austin's versions of the key takeaways, and hopefully that'll be beneficial and, like I said, give you a week's worth of value in 30 minutes. I'd be remiss if I didn't mention Atlan had two sessions there. It was a really great experience with two, two... Three of our customers, excuse me. We had a session with Fox, where they were going over their essentially approach to modern data governance and how they were able to build out a really great technical architecture following the medallion structure, right? There's bronze data, silver data, and then a gold kind of semantic layer.

10:52 But then what they found is it was great from a technical standpoint, but Atlan came in and was the bridge between that technical architecture, or the physical architecture, and the business users who actually come in and leverage that data, business users, analysts, et cetera. And then we had a panel with TaxSlayer and Ingersoll Rand, which is more of a, I call it, like, getting real. Let's actually talk about what it means to start a data governance program, and they touched on a a lot of really great things.

11:18 If you look at data governance framework, there's probably eight things that data governance could mean, and Angela and Laura both gave great examples of their own organizations and how they started with particular subject areas. They started with one domain. They started with some proof of concepts and really just focused on getting value as soon as possible, rather than building out this eight-point perfect strategy. Focus on execution over, over excessive planning.

Austin Kronz

11:46 But with that, let me get into the first session. There's a lot of talk about governance, and so there was a session by Guido De Simone, again, a former team member of mine, on the foundations and future of data governance. And I thought a really great slide that he shared was this great level-setting slide, where traditional data governance was more of a reactive, "We have some problems with data."

12:14 We are going to buy a bunch of tools and create processes to fix that data, and maybe if we're lucky down the road, think about how it impacts the business. It was very submit tickets. I can... Sorry, reading the, the chat as well. It's very reactive, very tactical, and I think business value was an afterthought. It was more about execution in executing in the moment. Whereas modern data governance needs to almost be the exact opposite. We're going to start from our key strategic objectives. We're going to start from things that drive the business forward and work our way backwards to prioritize use cases, and ultimately repeat this cycle.

12:50 So again, coming back to what I said earlier, data governance is becoming more about just how we do data and analytics than it is explicitly policy and regulation. I think before it had like a red tape connotation. That's certainly a part of it, policy and regulation, but data governance is also about just optimizing for value and having a team that can actually be problem-solvers in the organization. And, and I think that's the modern role of governance teams. In order to do that, I think another paradigm shift that Guido talked about was we can't stop looking... We can't keep looking at governance as one size fits all. Every use case will be completely different.

13:35 We almost have to be prepared and create operating models that support varied tiers of, I'll say, control, right? Fundamentally, there's... You're not going to get creative with externally facing compliance reporting. There is no self-serving on that, right? That is, "Here is the template. You must follow this, otherwise you'll be fined." Sure, that follows more of a control-oriented operating model for governance. However, there's also room for things like sandboxing and prototyping, new data science use cases, data that is very well understood, like customer data, that we want to build predictive models on top of.

14:11 So having these structures that support not only control, but more of this outcomes and agile, and as we get into the AI world, we're going to need to work on how do we build an autonomous governance model. Ultimately, we need to be flexible, take more of this adaptive approach to governance rather than one size fits all. So to wrap that session of what did I get out of the foundations and future presentation, like I said, governance needs to go from reactive and this tactical kind of mindset to more of this proactive and business value-driven approach. Not every data use case will be the same. It shouldn't be treated the same.

14:49 We have to be adaptive when it comes to data and analytics governance. And another point within the presentation that I thought was great, I think we all know, is technology has improved tremendously. That's fantastic. Technology is not a strategy, though. Tools will only support and amplify the approach that you choose, which means there's an exercise and a planning and thought that has to go into this, and I think a lot of collaboration that has to go into the overall approach to governance. And then you can have tools that support the workflows that you decide to go with. So technology first, not the way to approach it.

15:22 It's more of technology as a amplifier or support system for your data governance strategy.

Austin Kronz

15:31 The next presentation I'm really fond of because it's by Kurt Schlegel, again, a teammate of mine. He and I worked on a lot of self-service research whenever I was at Gartner, and I would also describe him as the master of analogies. So Establishing Analytics Franchises is the name of the presentation. I, I think I was there when Kurt was tossing this idea around. The analogy I think is just a cooler way to describe data mesh to a non-technical person. And the way that... I'll just give the analogy first. Data and analytics org models probably should follow the same type of structure that a corporate franchising model follows. What do I mean by that?

16:10 Well, think, I'll just use McDonald's, for example, right? Like McDonald— What is McDonald's but a core entity that creates a set of best practices and operating procedures, but then has a bunch of individually operating franchisees and franchise owners that operate in the day-to-day, but within the guardrails of what the, the overall corporate strategy is, as dictated. And that's the analogy when it comes to data and analytics, right? We want this core group of, that reaches consensus and best practices, but we don't want... We want autonomy for the federated teams or the franchisees.

16:45 And so the presentation follows this analogy and essentially sets out to answer this question, which is, like, how do we balance control and freedom? We want self-service. We want people to do what they need to do with data to find the insights that they need. But at the same time, there is probably a best practice, best practice in how we develop and create data products and how we build dashboards, et cetera. One of the things I built when I was still at Gartner to take this a level deeper was think of this, think of your domains in terms of different types of personas, right? You're going to have a consumer.

17:17 Self-service to a consumer is really just, like, sort, drill, filter. They'll never ask you for access to the database. They're executives, they're frontline workers, they're consuming. Explorer, I think, was brought about by these low-code, no-code BI tools, which was, "Hey, if the data is perfect and you have these nice, neat models ready for me, I can drag and drop things in Power BI. I can build things in Tableau. I can search in ThoughtSpot." That's great. And then there's more of the innovator persona, who's essentially saying, "Nope, I want credentials to Snowflakes and Snowflake and Databricks. Make sure I have access to that at any given time."

17:49 And what you'll see in organizations if you go through all of your domains and assess them at this level is every domain is going to be slightly different. If you think of domain understanding as consumers, quant skills as more of the explorers, tech skills more of the innovators from the previous slide, being aware of where each domain sits will help you understand the level of support. Meaning I might have to do ninety percent of the data and analytics pipeline for the HR domain in this model, where a supply chain is entirely self-sufficient. They actually hired their own data engineers. They have their own analysts. All they need from us is a operating model, right?

18:25 They're a self-sustaining franchise, and, and we only need to support them in terms of best practices and guidance.

18:32 So with that, identify your domains, determine their maturity level and their data needs to figure out where they fall on that spectrum. Create feedback loops. It's in a franchise. The franchises also provide feedback to the central entity, so it's not a one-way street. It's not just we determine things centrally and push it out. It's very much a bidirectional process. And lastly, in this model, you have to adopt a prototype first approach, which is get to value as fast as possible.

19:00 Then evaluate, does this need to be an enterprise-grade, IT data supported data product, or is this more of a, as an ad hoc one-off analysis, we're gonna use it this month, but we don't need it again. There's different tiers of outputs. And with that, we, we'll come to a, a GenAI presentation, right? I, I couldn't not address GenAI. Arun, longtime Gartner analyst, VP, distinguished analyst gave a great session on ten best practices. He also has a research note on this. If you have access to Gartner, I, I highly suggest reading it.

Austin Kronz

19:37 Setting the stage, there's a lot of benefits we can get from GenAI. Sorry, reading the chat here. There's a lot of benefits that you can have from GenAI. I think we're aware of the productivity benefits. There's clearly, according to our survey data, a lot of room for exploring how else it could add business value. There's also a lot of risks, right? We've all... I think if you follow anything on LinkedIn, you see this concern of hallucinations or what I call is LLMs being confidently wrong. There's issues with copyright. Where did this text that this is trained on actually come from? Is this producing net new text, or is it actually plagiarizing someone's work?

20:13 So all these things that we need to be aware of. It's a ten best practices session, so I don't really have the slides that go through all ten of them, so I'll just get right to my takeaways of how do we address those challenges and, and maximize those benefits. Ultimately, much like governance, this is taking a right-size approach, meaning there's a lot of opportunity for AI use cases, LLM use cases. Balance value with feasibility, meaning, and obviously considering regulatory requirements with everything from the AI Act and EU, just data requirements across the world. Balance these three things. I say explore minimum investment for maximum value.

20:53 What I mean by that is not just the development process of yes, prototype and pilot very quickly. I think that's always helpful, but more so the second point, which is you have a lot of options now when it comes to utilizing LLMs, which is say, I can build them, I can buy them, I can customize them, and more than likely, you're gonna do all three, right? This is using a productized LLM as a managed service from Azure and OpenAI, but I'm gonna build a RAG architecture around it that actually helps customize it to my organization's data. Or you could get as complex as fine-tuning the model or even building your own model yourself, right?

21:31 All of these, I think, incrementally cost more and more, and so you really need to understand how accurate and contextualized does my model need to be for my use case, and where can I balance not only the effort, but if you're fine-tuning models and building them from scratch, your cost to produce that is gonna go up significantly. There was a study I just saw on LinkedIn. I and forgive me for not knowing where it was from, but I think it was something like the difference between using ChatGPT's 3.5 and ChatGPT-4 Turbo for two hundred and fifty users was, like, tens of thousands of dollars to support two hundred and fifty users with 3.5 versus...

22:12 I'm sorry, you could support thousands of users querying on 3.5 versus 4 was only a hundred users or something like that, is the same cost. So really need to balance cost with model fit and the amount of contextualization that you need. Third, second bullet here. AI's only as good as the data that underpins it. We've said this for a long time just with analytics as well. I think it's more important in, in the AI world.

22:39 Another adage I, I love, I think in traditional reporting, we have garbage in, garbage out, whereas in AI, it can easily become garbage in, gospel out, which means we need to educate our people on what they're using, where the AI comes from, the data that underpins it. There's a lot of transparency involved there. And last, but certainly not least, in order to contextualize this, you need to invest in capabilities like metadata. Metadata is gonna become the holy grail of building highly relevant and contextualized LLMs that support your business. And I think this is for a couple reasons.

Austin Kronz

23:14 I'll transition out of this presentation into just how we've been seeing it. If I look at metadata, it's really that trust and context layer that you need for both building the LLMs, supporting transparency once they're in production, as well as contextualizing them. So from a just data standpoint, I need to know where the data is. Can I trust it? Is it of high quality? Who owns it? Do I have lineage, so I can figure out where it came from and where it goes?

23:44 On the LLM side of things, we have a lot of customers, and I talk to a lot of organizations that are building LLMs on top of not only their unstructured data, but their structured data as well. And the way they're optimizing their RAG architectures to contextualize the responses for end users is they're tapping into that business metadata associated, and technical metadata, associated with their structured data sources. So they know that this column for country and price is actually the one that the user's asking about.

24:13 So metadata that we may have historically looked at just for searchability of data assets will now be, I think, more front and center in the LLM world because it's gonna become automated, which means we now need to rationalize where we got it from and if we have enough metadata to actually create accurate models in production. And lastly, like transparency, like the ethical use of data, all of this is gonna be Hey, how did that embedding model break down our documents that we used in our RAG architecture?

24:44 What was the structure and the chunks that it's retrieving? All of this is just more metadata, and so mastering this metadata world is gonna become... Or navigating the metadata world will become extremely important. I included this because one of the questions I hear the most is, "Hey, we're being told to invest in gen AI," and, or, "We're being told to explain to leadership what we're doing when it comes to gen AI."

25:09 So I created this impact radius that I think you could use as a tool for breaking down what your organization is doing, and essentially what it is saying is even if you're in that, like, 60-some percent from our survey data that says we're still exploring what AI could even be, well, more than likely, all the tools and technology that you're using are already incorporating some form of LLM or AI. And so I think you can almost produce this for leadership and say, "Hey, there's individual tools that we have and all the capabilities," like Atlan has Atlan AI.

25:39 There's also all of the AI for just generally our modern data stack and copilots, so we are utilizing and we are taking advantage of AI. And then when you get to the outer tier, this is much more of, like, what are we building to benefit our business specifically outside of what's already in the tools and technology? So I think it's a really good framework to help answer that question because almost no one is starting from scratch. You're gonna have these first two tiers. It's more of that third tier that you really need to figure out and it probably has the broadest scope for impact. Hey, though, we're right at time.

26:13 I think we're gonna have to end. If you have any questions, you can find me on LinkedIn. Happy to answer them and follow up on there, but I, I really appreciate the time, and thank you for joining. If you want to hear more about Atlan and what our customers are doing and just what we're doing in this metadata data governance space, definitely join us in May for our Re:Govern event. It'll have a lot of, I think, not only examples of what Atlan is releasing throughout 2024, but real examples from customers. And if you've seen our great data debates, we're gonna roll that into Re:Govern, so we can actually get the thought leadership angle.

26:46 Really exciting event. I, I'm looking forward to it and hope to see you there. So thanks for joining. I know that was a lot for 30 minutes. I'm gonna take a drink of water, and hopefully talk to you all soon. Thanks, everybody.