5 tips for deploying a data mesh approach in your organization

Over the last few years, the data mesh concept has grown in popularity, driven by its ability to optimize the value of data by promoting sharing and reuse.

However, many organizations are structured in ways that it is difficult to implement data mesh, with still siloed in different departments. What are the best practices around deploying the data mesh approach in your organization?

Before implementing data mesh, it is vital to understand and define it. To begin with, it is neither a product nor a specific technology. It is an approach, linked to data governance and management, that allows you to create greater value from your data through increased sharing and reuse.

This concept was first defined by Zhamak Dehghani in 2019. Since then the data mesh approach has become seen as one of the best methods of ensuring you can securely create data flows that are then provided to the right people.

When it is simply stored within data lakes or data fabrics within organizations, it is difficult to fully leverage data to create uses and value. The data mesh approach adapts to any type of structure and empowers those who are closest to the data, while keeping a common governance model that guarantees interoperability, consistency and security.

In order to promote data sharing and reuse, the data mesh approach is based on four main principles.

Data mesh organizes data by domain, based on which team or department has created it. Even if this data is then stored in a data lake it is available through dedicated sub-domains for each department. It is managed by the business with the support of data experts.

This allows each team to define the way data is stored and analyzed, according to their specific needs. The benefit is twofold:

• Stakeholders are empowered to use the data;
• Information is more accessible and understandable to all.

Through the data mesh approach, data is no longer just isolated information, or components of a more global digital product. It becomes a service in its own right. This is the “Data as a product” (DaaP) model.

Data products are different from simple data because :

• they are available on dedicated sub-domains, and are gathered in an internal self-service data portal for quick access
• common rules of governance are associated with them,
• they can be accessed and distributed through specific ways, such as via APIs, connectors to business applications or data analysis tools.

With the data mesh approach, the objective is to make data available through self-service. This means that employees no longer need to go through intermediaries (data experts) to obtain the information they want. Through an internal self-service portal, they can access and analyze data in real-time in order to become more agile and efficient.

With a single point of access to centralized data, it’s easier to manage different access permissions and to ensure a smooth and secure flow of data.

Implementing a data mesh approach, based on decentralization, presents a risk: the creation of incompatible data silos that are unable to work together.

To avoid this situation, it is essential to establish shared governance rules. Knowledge and alignment between the different departments is necessary to ensure that data is always up to date, processed in a consistent way and that it can always be reused.

This ensures data interoperability, consistency and security, while providing flexibility to the different departments.

Implementing a data mesh approach improves efficiency on multiple levels:

• Speed: as data access times are shortened, decisions are made more quickly.
• Resource allocation: the skills of different teams are fully utilized, bringing maximum value to the company.
• Risk reduction: Overlong implementations can cause a project to fail. This risk is reduced with the data mesh approach.
• Collaboration: This is facilitated by the implementation of a shared language. As a result, projects move forward much more quickly.

The objective of the data mesh approach is to democratize data for the entire organization. To achieve this, priority is given to the data, not the technology behind it. As data is considered as a product in its own right (and not just as a tool), departments are more committed to improving its quality and reliability. It is this Data as a Product approach, with the notion of responsibility, that enables true data governance to be put in place.

The data mesh approach focuses on creating value from data. Thus, only high value-added data is used to create data products that have an impact on employees and help them in their roles.

Through the deployment of a self-service platform, employees can conduct their own analysis, without going through data experts. Every sub-domain is designed to closely meet the needs of business experts. Thus, data processing and analysis are adapted to the requirements of each department.

All of this helps to strengthen the commitment and involvement of teams in sharing and using data, regardless of the department (HR, marketing, finance, R&D, etc.) they are in.

While federated governance implies defining common rules for the entire organization, each team is free to define its own standards according to its internal needs, such as reinforced security, or data sharing through APIs, for example. This flexibility allows everyone to work without too many constraints.

Unlike some data architectures, the data mesh approach adapts to all types of companies, regardless of their budget, size or industry.

Moreover, as it does not mandate a specific technology, organizations that already use centralized tools (such as data lakes or data warehouses) can implement a data mesh approach. In the same way each department can use tools adapted to its own problems, without preventing the wider deployment of a data mesh approach.

To develop the right structure, it is essential to have data managers in each department, as well as data/IT specialists to support them.

Most mature organizations opt for a hybrid organization, with data managers integrated in the departments, along with a central team. This team defines the global data strategy and supports the business units on the technical aspects of their projects.

Data mesh is not a one-off project. It is a long-term commitment that determines the way your organization manages data. It implies changing employee behavior, implementing governance rules, developing new skills and sometimes using new tools.

But not all organizations are ready to deploy a data mesh approach. Before doing so, they must have a specific vision of data processing and analysis, but also commit to investing and developing new value-creating uses. If this level of data maturity is not yet in place, it is advisable to gradually prepare for this transformation of company culture.

The data mesh approach relies above all on a strong, well-understood data governance policy. Successfully implementing this approach relies on the creation of shared standards. It requires:

• Metadata: with centralized standards to simplify understanding throughout the organization.
• An internal data marketplace: to facilitate access to data by all.
• A monitoring and management plan: including around data security, quality and privacy.

While the data mesh approach can be adapted to any environment or architecture (such as a data lake, lakehouse, or data warehouse), some tools make it easier to implement. For example the Opendatasoft platform allows you to create your internal data marketplace thanks to features that include:

• More than 80 connectors to integrate all your tools and automate connections to your data sources
• The creation of sub-domains, linked to a parent domain, to create data products and share them easily within the organization
• Built-in governance capabilities to strengthen data quality and security across the organization
• Rights management for sharing with other teams as well as outside your organization

Our vision is based on the same federated data management concept as the data mesh, enabling non-specialist teams to leverage their data. With our platform you can very quickly become independent and empower teams to create new data services.

Deploying the data mesh approach requires an in-depth transformation of your organization. That means it is important to start with small projects, and then expand them as you move forward. For example, you can start with a single department, where the data mesh approach is most relevant. Through this pilot project you can explore what works and what doesn’t. And once the data mesh approach is well established there, you can extend it to other departments. Often this is driven by demand from employees who can see the benefits that their colleagues are already receiving.