Gartner Active Metadata Management: Concept, Market Guide, Peer Insights, Magic Quadrant, and Hype Cycle
Share this article
Active metadata management by Gartner refers to capabilities that enable continuous access and processing of metadata to support ongoing analysis of data and analytics use cases.
By 2024, these capabilities will also include monitoring, evaluating, recommending design changes, and orchestrating processes in third-party data management solutions.
In this article, we’ll look at Gartner’s take on active metadata and its management. We’ll also interpret the recommendations and tooling capabilities suggested by Gartner’s market guide for active metadata management.
Table of contents
- How does Gartner help you pick the right active metadata management tool for your enterprise?
- What is active metadata according to Gartner?
- What is active metadata management according to Gartner?
- 4 recommendations for evaluating solutions
- 8 essential tooling capabilities
- What is Gartner Peer Insights?
- What is the Gartner Magic Quadrant report?
- What is the Gartner Hype Cycle?
- Atlan: A pioneer in the active metadata management space
- Related reads
How does Gartner help you pick the right active metadata management tool for your enterprise?
Gartner is a $5.5 billion market research company with 40+ years of experience in providing research insights and guidance on technology and offering tool recommendations across industries and functions.
Gartner also enables peer connections with conferences and its peer communities to help your team use peer-driven insights for faster and better technology decisions.
Gartner leverages its research primarily through two mediums: Research publication and tools
- Gartner Market Guide
- Gartner Magic Quadrant
- Gartner Peer Insights
- Gartner Cool Vendors
- Gartner Vendor Ratings
- Gartner Hype Cycle
What is active metadata according to Gartner?
That’s because the purpose of metadata is to help organizations understand their data assets. However, passive metadata doesn’t provide the complete picture as it doesn’t drive action and relies on human effort to curate, document, and update metadata.
“Active metadata is the continuous analysis of all available users, data management, systems/infrastructure and data governance experience reports to determine the alignment and exception cases between data as designed versus actual experience.”
Gartner further elaborates on the characteristics of active metadata as metadata that:
- Is the continuous analysis of various types of metadata to determine alignment and deviations between “data as designed” and “operational experience”
- Creates a continuously evolving operational knowledge graph of which data is used, how often, by whom, for what purpose, and on which platform
- Enables automation in the data pipeline, delivers insights into characteristics of data, and optimizes user engagement
- Is a key enabler of data fabric, data mesh, microservices, data as a service (DaaS), and their continued evolution
- Supports self-service analytics and application development by automating data content, structures, availability, and discovery
Also, read → Active Metadata 101
Here’s an analogy to establish the difference between passive and active metadata clearly.
What is an example of active metadata? Comparing active and passive metadata with an analogy
Passive metadata is a static map that shows the road networks, prominent landmarks, pitstops, etc. Meanwhile, active metadata is dynamic like your car’s GPS which takes things a step further by dynamically adjusting your route based on real-time traffic conditions, road closures, etc.
So, active metadata monitors the traffic situation continuously and responds to changing circumstances to guide you through the best possible route.
Also, read → 14 active metadata use cases for enterprises
Next, let’s look at what’s involved in managing active metadata.
What is active metadata management according to Gartner?
According to Gartner, active metadata management operationalizes analytic outputs in the form of operational alerts and generated recommendations. It identifies the nature and extent of patterns in data operations, ultimately resulting in AI-assisted reconfiguration of data itself and operations that use that data in active metadata.
Gartner predicts that, by 2026, organizations adopting active metadata practices will increase to 30% as it will help in accelerating automation, insight discovery, and recommendations.
As a result, Gartner also predicts that a “stand-alone metadata management platform will be refocused from augmented data catalogs to a metadata “anywhere” orchestration platform.”
This means active metadata platforms will rise to prominence as always-on systems that create intelligence from metadata and drive action across your data estate.
Gartner also predicts that organizations adopting aggressive metadata analysis across their complete data estate will decrease the time to delivery of new data assets to users by as much as 70%.
How does this translate into tooling? Let’s look at Gartner’s recommendations while evaluating available solutions for active metadata management.
Gartner market guide on active metadata management: 4 recommendations for evaluating solutions
According to Gartner, you should consider the following factors while looking for active metadata management solutions:
- Adopt metadata management solutions that accept parameterized instructions to alter design inputs such as job flow, and resource allocation, and manage precedent/dependent processing calls to third-party tools.
- Assess current data management tools and platforms by evaluating their capability to share internal metadata to support broader platform-to-platform orchestration.
- Replace tools that share their design-based metadata with those demonstrating active metadata capabilities, i.e., tools/platforms that can export their own metadata and import “foreign” metadata relative to existing metadata, processing instructions, and optimization strategies.
- Enable support for automated system changes in adjacent data management systems by leveraging tools that support metadata analytics workflow management capabilities, including collaborative design capabilities.
What do these factors look like as tool capabilities? Gartner highlights 8 core capabilities of active metadata management platforms. This isn’t an exhaustive list, but a starting point when comparing active metadata management solutions.
Gartner market guide on active metadata management: 8 essential tooling capabilities
Gartner highlights 8 essential capabilities that will define an active metadata management tool. These include:
- ML over profiling: Active metadata management tools will leverage ML models to profile data and offer predictions on how this data can be used to get insights.
- Content analysis: Active metadata management tools will evaluate text (documents, oral communication, and graphics) to draw inferences from data and highlight how it could drive action.
- User/use-case clustering: Active metadata management tools can create clusters by grouping together similar users and use cases. These tools can also recommend similar content to help data practitioners get richer context. They can also analyze these clusters to automatically detect anomalies and outliers, thereby improving data quality and completeness.
- Resource allocation metrics: Active metadata management tools can automatically distribute resources — storage allocation, task scheduling, app scaling, etc.
- Alerts and recommendations: Active metadata management tools generate alerts and offer recommendations by continuously monitoring metadata information.
- ML by case example with the trend, with usage: Active metadata management tools analyze usage metadata to offer suggestions on better using your data assets.
- Orchestrate recommendation and response: Active metadata management tools constantly track technical metadata (design and runtime metadata) from connected data products to send alerts and offer recommendations.
- Use case to new asset inference: Active metadata management tools also compare newly generated metadata with existing passive metadata to identify trends, analyze usage patterns, and offer suggestions.
These tools are an evolution from passive data catalogs that merely populated data asset descriptions and offered lineage mapping to provide a picture of your data estate.
Active metadata management tools are dynamic, constantly updating data assets as and when things change. They enable a bidirectional flow of metadata from the tool to all the connected data products and vice versa.
Gartner’s market guide for active metadata management also identifies some vendors who provide active metadata management and data cataloging capabilities. These include:
- AWS Glue
- Google Cloud
- Microsoft Azure Data Catalog and its evolution Azure Purview
- Oracle Enterprise Metadata Management (OEMM)
Besides the market guide, Gartner generally offers guidance with its Peer Insights, Magic Quadrant, and Hype Cycle.
Let’s understand this further.
What is Gartner Peer Insights?
Gartner Peer Insights is an open platform to source software product reviews from verified users. The platform covers more than 400 categories, 18000 products, and 380,000 appraised reviews.
You can filter through the reviews by using criteria, such as geography, business size, total revenue, industry, and ratings. You can also compare products to understand the key differences.
Presently, Gartner does not have peer insights specifically for active metadata management tools, but you will find reviews of some of the above-mentioned tools under Enterprise Metadata Management tools.
What is the Gartner Magic Quadrant report?
Gartner Magic Quadrant is a proprietary market research methodology to compare and contrast tools and technology providers across various categories.
According to Gartner, the Magic Quadrants “offer visual snapshots, in-depth analyses, and actionable advice that provide insight into a market’s direction, maturity and participants.”
The magic quadrant evaluates and places the tools in 4 quadrants — Niche Players, Challengers, Visionaries, and Leaders.
As of now, Gartner does not have a magic quadrant for active metadata management tools.
You can go through the (now scrapped) magic quadrant for metadata management solutions and evaluate these tools based on the recommendations and key capabilities mentioned earlier in this article.
Gartner resources to understand the trends in active metadata management:
- Quick answer: What is active metadata?
- Market guide for active metadata management
- Metadata management (EMM) solutions reviews and ratings 2023
- How to activate metadata to enable a composable data fabric
- Case study: An active metadata-augmented data classification system to boost analytics efficiency
- Deploying effective metadata management solutions
What is the Gartner Hype Cycle?
Gartner’s Hype Cycle is a graph that shows you how a technology or application will evolve over time. It is a great way of understanding emerging technologies and whether it’s worth taking a risk and investing in them to achieve your business goals.
According to Gartner, the Hype Cycle is “a graphical depiction of a common pattern that arises with each new technology or other innovation.”
The five stages of the Gartner Hype Cycle are — Innovation Trigger, Peak of Inflated Expectations, Trough of Disillusionment, Slope of Enlightenment, and Plateau of Productivity.
Here’s an example of the Hype Cycle for emerging technologies in 2022.
Currently, Gartner does not have a Hype Cycle graph for the active metadata management space.
Hype Cycles related to active metadata management
- Hype Cycle for Data and Analytics Governance, 2022
- Hype Cycle for Analytics and Business Intelligence, 2022
- Hype Cycle for Data Science and Machine Learning, 2022
- Hype Cycle for Emerging Technologies, 2022
- Hype Cycle for Data Management, 2021
Atlan: A pioneer in the active metadata management space
Atlan is the only modern data catalog that activates metadata. With Atlan, you can set up a data ecosystem that’s personalized, collaboration-first, and automated.
As a result, you can allocate compute resources dynamically, proactively alert downstream consumers of potential changes/issues, define access control policies at scale, and more.
Humble brag alert! Atlan has been featured in Gartner reports on active metadata management, data cataloging, DataOps, and more:
- Market Guide for Active Metadata Management
- Augmented Data Catalogs: Now an Enterprise Must-Have for Data and Analytics Leaders
- Cool Vendors in DataOps
- Tool: Vendor identification for data and analytics governance platforms
- Hype Cycle for Data Management
- Hype Cycle for Emerging Technologies
- Hype Cycle for Enterprise Information Management
The latest Forrester report on Enterprise Data Catalog for DataOps also named Atlan as a leader, receiving the highest possible score in 17 evaluation criteria including Product Vision, Market Approach, Innovation Roadmap, Performance, Connectivity, Interoperability, and Portability.
If you’re looking into active metadata management platforms, then get hands-on with Atlan and explore how it can help you with data enablement, discovery, quality, security, and more.
Gartner active metadata management: Related reads
- Gartner Data Catalog Research Guide — How To Read Market Guide, Magic Quadrant, and Peer Reviews
- A Guide to Gartner Data Governance Research — Market Guides, Hype Cycles, and Peer Reviews
- Active Metadata: Your 101 Guide From People Pioneering the Concept & It’s Understanding
- The G2 Grid® Report for Data Governance: How Can You Use It to Choose the Right Data Governance Platform for Your Organization?
- The G2 Grid® Report for Machine Learning Data Catalog: How Can You Use It to Choose the Right Data Catalog for Your Organization?
Share this article