Modern Data Catalogs: 5 essential features and evaluation guide

July 27th, 2021

What is a data catalog? Image of catalogued books in a library

What is modern data catalog?

What is a modern data catalog? As defined by Gartner, modern data catalogs “automate various aspects of data cataloging, including metadata discovery, ingestion, translation, enrichment, and creation of semantic relationships between metadata.”

With that being said, a modern data catalog isn't just an inventory of your data. It is built around the premise of "embedded collaboration" in today's modern workspace and is also just as fast, flexible, and scalable as the rest of the modern data stack.

Modern data catalogs are inclusive & welcome more data practitioners into the fold of metadata management, by empowering non-technical users to participate in understanding, enriching, and using metadata to maximize the value of the data they produce and consume.


Why do we need modern data catalogs?

The modern data stack is no longer an aspiration, it’s a requirement for companies looking to build a data moat for themselves.

Data leaders recognize that they need to bring their metadata management up to speed, ensuring to wade through the struggles of bringing easy discovery, trust, and more context to data.

Naturally, basic data catalogs are out of the running. Enter the modern data catalog or data catalog 3.0 as we like to call it.


What are the features of a modern data catalog?

In general, there are 5 must-have features of a modern data catalog:

  1. Activates metadata
  2. Organizes and consolidates metadata in a single repository
  3. Enables embedded collaboration
  4. Provides granular governance and access control
  5. Built on open API architecture

Activates metadata

We live in a world where metadata itself is becoming big data. As metadata increases, the intelligence we can derive from it increases, so does the number of use cases that metadata can power.

It’s no longer sufficient to just organize and store technical metadata. We need data catalogs that activate metadata i.e. ones that can pervasively find, inventory, enrich and use all kinds of metadata - enabling manual or automated use of the same across the stack.


Organizes and consolidates metadata in a single repository

Data assets are usually spread across different places — data lineage tools, data quality tools, data prep tools, and more. Modern data catalogs generally act as the unified access layer from all data sources & stores - and are powered by visual querying capabilities to ensure democratic access to all - technical and business users.

Modern data catalogs must ensure to be the single source of truth for all data assets in the organization.

Google Like Search

Google like search. Image by Atlan


Enables embedded collaboration

Data teams include analysts, engineers, business users, product specialists, scientists, and more. They each come with tooling preferences, skill-sets, and workflows. Modern data catalogs recognize the strength in this diversity and are inclusively designed seamlessly to serve them all in their natural habitats.

Enables Embedded Collaboration

In-line chats. Image by Atlan

In-line chats, annotations, data assets & queries as shareable links, easy sharing amongst workspaces like Slack, & project management & issue tracking tools like JIRA, etc. characterize modern data catalogs that enable embedded collaboration.


Provides granular governance and access control

Data increases in value as more people use it. Usage and adoption must reach a critical mass before organizations start seeing the true value of their own data. But then again, comes the question of governance & access to sensitive data. Modern data catalogs nullify this challenge and ensure granular control not only for data stewards but for every data producer and consumer in the organization.

Access Control

Access Request Management. Image by Atlan


Built on open API architecture

The modern data stack is burgeoning and evolving. Thus one benefits from catalog tools that are open by default and easily extensible with the tools you use. Today your stack might look like something, but tomorrow you may look to add fresher capabilities, even re-engineer some - the modern data catalog - should allow data teams to build their data infrastructure on top of it, with ease.


How to evaluate a modern data catalog?

The 5 key steps to evaluating a modern data catalog are as follows:

  1. Define organizational needs for a data catalog
  2. Create a customized evaluation criteria
  3. Understand the tools and vendors available in the market
  4. Take demos from select data catalog vendors
  5. Execute hands-on Proofs-Of-Concept

Define organizational needs for a data catalog

Start by identifying the top 3 challenges, map organizational challenges, and also remember to evaluate non-functional conditions.

Create a customized evaluation criteria

Finalize a clear set of objective criteria that will guide the evaluation process.

  • Core capabilities mapped to organizational needs
  • Other high priority considerations and how to evaluate them

Understand the tools and vendors available in the market

There are broadly three types of Data Catalog offerings in the market:

  • Traditional Data Catalogs
  • Open Source Data Catalogs
  • Modern Data Catalogs

Take demos from select data catalog vendors

After conducting secondary research on market offerings, reach out to select data catalog vendors to set up deep dive demos.

Execute hands-on Proofs-Of-Concept

After taking demos, reach out to selected data catalog vendors to set up hands on PoC.


We understand investing in the right modern data catalog is easier said than done. We’ve put together a step-by-step guide along with easily usable templates that will make finding the right catalog a breeze! In this detailed guide, we take you through each of the following steps, and elaborate on what all to consider and tick through.

Download the ultimate guide to evaluate data catalogs.


Summary

It's safe to say, we at Atlan, know a thing or two about Modern Data Catalogs, and can surely help you through your evaluation process.

If you are evaluating modern data catalogs or data catalogs in general, we cannot emphasize enough on the importance of going through each of the 5 steps mentioned above and expanded in the guide. The evaluation process will help you define the problem better even before you start looking for the probable best options.

Post step 3 in the evaluation process, surely you’d like to evaluate Atlan - a modern data catalog. Here are some quick links to the demo, or best, speak to Team Atlan and discuss all your evaluation queries directly.

Ebook cover - metadata catalog primer

Everything you need to know about modern data catalogs

Adopting a modern data catalog is the first step towards data discovery. In this guide, we explore the evolution of the data management ecosystem, the challenges created by traditional data catalog solutions, and what an ideal, modern-day data catalog should look like. Download now!