The Current State of MDM

MDM has always necessitated integration. By definition, MDM is supposed to be the central location for entity data. It should be fed and accessed by dozens of systems. So how well integrated is MDM? 

There are mixed results in the market. On average, it’s fair to say that is only partially integrated into an overall data architecture, and it is not always a central component. To figure out why, let’s look into the integration functionality in more detail.

Batch Ingestion:

There has always been a significant emphasis on ingesting data in batch. Most MDM solutions integrate with ETL/data integration tools, or have OEMed or built ETL ingestion capabilities. However, there are still shortcomings. There is a lack of pre-built ETL integration to common source applications. MDM also lacks metadata integration with ETL tools, so much so that the metadata profiled in ETL isn’t always shared and stored within MDM. Third, most MDM tools don’t utilize data profiling, quality and cataloging to understand data sources before they are ingested into MDM. This is one of the biggest reasons for long MDM implementations; the source data wasn’t well enough understood and took longer to master than anticipated. What’s more, MDM tools don’t often integrate with data governance tools to understand how to master new data sources. MDM capabilities are designed to “get the data in there” and the integration is very much at the tool level (i.e, not at a pre-built functionality and integration).

Batch Export:

There are far fewer pre-built capabilities to export data from MDM. The same integration tools for ingestion can be used to export data, however, much of that work needs to be done manually. 

Real-time Create, Update and Read:

Much like batch capabilities, most MDM tools have generic, tool-level real-time capabilities. Services exist to ‘add entity’, ‘update entity’, ‘search entity’ and ‘view entity’. These services will contain the data you define within that master data ‘entity’. But they lack pre-built definitions of what that entity is. They also lack business logic. Sure, that can all be added, but that’s a lot of work. Most MDM tools only have these generic service capabilities, with one notable exception. 

The functionality explains why MDM isn’t as well integrated as it should be. Most MDM solutions are more tools than applications. They have technical capabilities, but data users have to define “entities”, configure data models, define services, add business logic and rules to those services, or create business-entity level services as a layer above those generic services. This “tool mentality” has left MDM more isolated and not as central in modern architectures as it intended to be. 

This has become worse in the last few years due to budget cuts across data and IT teams. There are fewer resources and smaller budgets to support building out the integration around MDM to make it an integral part of a modern data architecture. And that’s a shame. Because it should be an integral part of a modern data architecture. As data architectures evolve towards concepts such as data mesh, which emphasizes a logical business model to integrate various data sources, MDM would be the perfect application to power a data mesh with centralized “core” business entity data, and links to other sources of business entity data sources. With growing adoption of AI, a modern data architecture will evolve more towards real-time data management and consumption, and trusted master data is essential for effective AI applications.. Real-time MDM services will be essential, but batch-oriented MDM systems might be left behind. The need to consume business entity data, for example customer data, will only increase. The key question is whether MDM will modernize fast enough to be the system to meet that business demand.