Gartner on Data Mesh: The Debate, Challenges, and the Future of Data Architecture

Updated June 30th, 2023
Gartner Data Mesh

Share this article

The data mesh paradigm has stirred discourse across the industry. Gartner’s endorsement of the approach is both challenging traditional views and inviting speculation about potential successors.

The Ultimate Guide to Data Mesh - Learn all about scoping, planning, and building a data mesh 👉 Download now

In this article, we’ll summarize Gartner’s take on data mesh, see why some people disagree with their assessment, and discuss why the future of data architecture remains open-ended.

Table of contents

  1. What is data mesh?
  2. Gartner’s stance on data mesh
  3. Challenges of data mesh architecture
  4. Is data mesh DOA?
  5. What the creator of data mesh thinks
  6. Conclusion
  7. Gartner data mesh: Related reads

What is data mesh?

Data mesh is a paradigm shift in how we approach data architecture.

Instead of traditional centralized architectures, the data mesh model promotes a distributed system where individual teams manage and build their own data as discrete products. These data products are then federated across different teams with mutually agreed-upon standards.

As Zhamak Dehghani, the creator of the concept of the data mesh:

“Data mesh, at the core, is founded in decentralization and distribution of responsibility to people who are closest to the data in order to support continuous change and scalability.”

Read more → Data mesh 101

Gartner’s stance on data mesh

Gartner’s perspective on the data mesh paradigm from their 2022 Hype Cycle for Data Management has stoked vigorous discussion in the industry. The Hype Cycle is a visual representation that outlines the potential risks and rewards of adopting specific technologies or applications.

Gartner situates data mesh in the “innovation trigger” phase in 2022, which is often seen as the conception of a technology’s lifecycle. Interestingly, Gartner also predicts that the data mesh concept will become “obsolete before plateau.”

The Gartner Hype Cycle for Data Management, 2022

The Gartner Hype Cycle for Data Management, 2022 - Source: Gartner.

This forecast does not necessarily imply that the paradigm is currently obsolete. Rather, Gartner thinks it may be supplanted by a different approach before reaching full maturity.

In terms of market penetration and benefits realization, Gartner rates both as “low”. The company projects that as businesses mature in their data governance capabilities, a more centralized approach might once again gain favor.

This shift might seem counter-intuitive, given the current trend towards decentralization. However, it underlines the complex, evolving dynamics of data management.

Challenges of data mesh architecture

The allure of data mesh comes with considerable challenges that might have influenced Gartner’s predictions.

Among these hurdles, the demand for consistency in data governance stands as a significant consideration. Before an organization can contemplate adopting a data mesh, it must already have a decentralized approach to governance in place.

This necessitates a high level of maturity within the organization, both in terms of culture and processes.

However, many organizations aren’t there yet. Gartner’s 2021 Data and Analytics Governance Survey indicates that only 18% of organizations have reached the maturity level necessary to adopt the data mesh approach successfully.

Many of the advantages of a data mesh approach involve inherently associated challenges:

  • Domain boundary identification: It can be challenging to define clear boundaries for data domains, especially in complex organizations with overlapping business functions.
  • Technical complexity: Implementing a data mesh requires significant changes to the existing data infrastructure and could necessitate complex technologies and techniques, which could be difficult for some organizations to manage.
  • Team skills and training: A successful data mesh implementation requires each team to possess data management skills. Training teams and ensuring they have the required skillset can be a significant task.
  • Interoperability: Ensuring seamless interoperability between different data domains can be challenging and requires robust standards and protocols.
  • Cultural change: A data mesh approach requires a significant cultural shift from a centralized to a decentralized approach, which may be met with resistance within the organization.
  • Data quality management: Ensuring data quality across different data domains managed by different teams can be a significant challenge.
  • Regulatory compliance: Complying with data protection regulations can be complex in a decentralized environment. This is especially challenging when data is distributed across geographical regions with different regulatory landscapes.

Is data mesh DOA?

As we’ve noted, Gartner’s prediction about data mesh becoming “obsolete before plateau” prompts the question: What will replace data mesh if it truly becomes obsolete?

Ex-Gartner analyst and current Head of Data Strategy at Profisee Malcolm Hawker believes that Gartner is positioning “data fabric” as the alternative to data mesh.

What Malcolm Hawker, ex-Gartner analyst and the current head of data strategy at Profisee, has to say about data mesh.

What Malcolm Hawker, ex-Gartner analyst and the current head of data strategy at Profisee, has to say about data mesh. - Source: LinkedIn.

Data fabric is another emerging paradigm in data architecture that aims to seamlessly integrate data from various sources to support a range of business use cases.

However, this view is not universally shared.

Scott Hirleman disagrees with Gartner’s stance, calling it a “vendor-first, technology-first” approach. He suggests that such a perspective may overlook the practical and business realities that are driving the adoption of decentralized data management practices like data mesh.

Scott Hirleman on data mesh and Gartner’s hype cycle analysis.

Scott Hirleman on data mesh and Gartner’s hype cycle analysis. - Source: LinkedIn.

Additional data mesh alternatives include:

  • Flexible data lakes
  • Traditional data warehouses
  • Convergent data hubs
  • Abstractive data virtualization layers
  • Federated database systems

What the creator of data mesh thinks

Zhamak Dehghani originally created the concept of a data mesh while working for Thoughtworks. She now leads her own company, Nextdata, which is creating its own set of “data mesh-native” tools to enable the next generation of data sharing.

Dehghani isn’t keen on what she sees as Gartner’s premature declaration of data mesh’s demise.

In a comment on Scott Hirleman’s LinkedIn thread, Dehghani says, “If Gartner’s opinion is the result of quantified analysis, I would love to see the raw data behind it.”

Dehghani goes on to note that she works with companies on a daily basis that are implementing data mesh architectures. “…the gap between a single analyst’s point of view and the realities on the ground seems quite vast.”

Conclusion

As we contemplate the future of data mesh architecture, it’s crucial to understand that different approaches may suit different organizations, depending on where they are in their data journey. Both data mesh and data fabric present viable paths, each with its unique strengths and considerations.

The choice between these paradigms should align with the organization’s key challenges and objectives. For companies where data accessibility is the primary concern, the data mesh model could be a fitting choice. On the other hand, for organizations that prioritize data quality and trust, the data fabric approach might be more suitable as it emphasizes seamless data integration and interoperability.

Regardless of the approach chosen, one element remains critical: the data catalog. In either a data mesh or data fabric model, a data catalog is vital for data discoverability and data governance.

Share this article

[Website env: production]