How and Why to Automate Data Mesh Governance with Ontology Integration

Orion’s Enterprise Information Intelligence Graph (EIIG) is a self-defined data fabric that is built on real and factual metadata. An integrated and centralized approach to metadata management creates a unified view of data across the organization. ML/AI-Powered semantic integration adds business meaning and governance capabilities to maintain a centralized view of various data sources and assets at scale.

Data Mesh

Data Mesh is a modern architectural approach to designing and managing data at a scale. It addresses the challenges of traditional centralized data architecture. In Data Mesh’s decentralized data ownership and access model, individual domain teams are responsible for the quality, security, governance, and semantics of the data they own. This approach allows organizations to take advantage of the expertise and context of different teams, while still ensuring that data is integrated and shared across the organization.

Domain Models

Domain ontologies are used in EIIG to support the development and implementation of an integrated Data Catalog and a Data Mesh architecture by providing a standardized and structured approach to data modeling, integration, sharing, and governance. Pre-packaged domain ontologies are designed to capture the concepts and relationships within a particular domain or industry. Organizations can share and integrate the data assets with governance support using domain ontologies that reflect the structure and semantics of the data owned by different teams.

How EIIG Works

Orion’s EIIG includes built-in automated solutions for data governance and business semantics using domain ontology learning, engineering, and mapping. The EIIG integrated Business Glossary provides a wide range of information management and governance functions for ontological knowledge integration with various types of data products and assets. This includes the ability to capture and maintain domain knowledge and business semantics as plain sets of business terms and definitions, conceptual business models, or full-scale domain ontologies with logical relations, rules, and restrictions between business concepts.

Domain knowledge can be captured and integrated automatically in multiple ways using the EIIG platform:

• Derived automatically from existing technical assets, such as database schema or business intelligence and reporting models, using ontology learning methods (bottom-up approach). This method allows for the use of relational database structures and constraints, such as primary and foreign keys, to derive business concepts, types, definitions, and their structures and relations.

• Imported from external files or other tools’ native formats, such as rdf/owl, xml, xsi, csv, etc. This method allows for the rapid reuse of existing domain knowledge within an organization, with automated asset matching capability based on predefined dictionaries, similarity matching, and machine learning techniques (top-down approach).

• Predefined and preloaded industry-standard ontologies and domain models that are ready to use as the fastest option to automate asset classification and governance with a minimum amount of manual work.

EIIG provides various tools and file format support that can be used in different combinations to achieve the best result for each domain terminology capture and data governance role automation. For example, one can start with business terminology generation from a relational database schema and then switch to a manually enriched and handcrafted approach, or start from manual entry and continue with a mixed approach with automated concept discovery and mapping.

Why This Method is Best

By using domain ontologies and terminology, organizations can achieve various benefits within a Data Mesh architecture, including:

• Automation – Keep the Mesh in sync whenever the IT landscape changes. Leveraging a supervised Machine Learning model ensures that changes are captured and classified correctly. This significantly reduces the human capital needed to keep the mesh current.
• Management of data governance, ownership, and responsibility relations (such as ownership, data stewards, domain experts, and contact persons) at the domain level and automatic propagation of these relations to connected business and technical assets.

• Maintenance of business terminology and semantics in a single location, which can be reused across different domains, with the meaning automatically propagated downward to technical assets.

• Propagation of governance roles, business meaning, and semantics downstream to data structures or upstream to business intelligence structures using the underlying data lineage graph in the Data Fabric, thereby providing better coverage and interconnection of Data Mesh assets.

• Standardized and structured data representation and modeling, which enables easier data integration and sharing between different domain teams.

• Improved interoperability between systems, datasets, and products based on shared semantics using connected ontologies or documented and automated interfaces for sharing data.

• Improved quality and consistency using standardized data models and semantics.

• Reduced data duplication and fragmentation by providing a common framework for data integration, sharing, and meaning.

In summary, Orion’s Enterprise Information Intelligence Graph is a self-defined data fabric that provides built-in support for preloaded or imported domain models and automated mapping of business and technical assets using machine learning algorithms and models. These essential functions enable integrated and efficient data mesh governance and business semantics capabilities in centralized or distributed environments.

About the Author: Kalle Tomingas, PhD is the Chief Data Scientist at Orion Governance, Inc. Connect with Kalle on LinkedIn.