Enhance Data Governance in A Data Vault 2.0 Implementation with A Self-defined Data Fabric

Data Vault 2.0 is a well-established data management architecture with a proven methodology and unique modeling design pattern. It is gaining popularity due to its agility, accuracy, scalability, cost-effectiveness, and trustworthiness.

Data governance and DataOps are integral to a Data Vault 2.0 implementation. Since Data Vault is not a product, enterprises who are adopting Data Vault architecture and methodology need to incorporate a data governance solution. Orion Governance’s Enterprise Information Intelligence Graph (EIIG) is such a solution best suited for Data Vault 2.0 deployment.

An All-in-one Self-Defined Data Fabric Platform

Orion’s Enterprise Information Intelligence Graph (EIIG) ingests metadata from a wide variety of technology sources and intelligently weaves it into a comprehensive knowledge graph that connects all systems/applications within the enterprise together, thereby creating a self-defined data fabric. A self-defined data fabric is one that is built based on factual metadata.

It is one platform with natively integrated capabilities such as data lineage, data catalog, impact analysis, metadata analytics like similarity analysis, augmented data quality, and trust propagation. Enterprises can meet all their data governance needs in one product and thus avoid the pain of dealing with a patchwork of poorly integrated products/modules.

Comprehensive Technology Support

In the last twenty plus years, Data Vault has evolved into Data Vault 2.0 to support the Lakehouse type of environment. This development demands a more robust data governance regime with the capabilities of supporting both legacy and modern technologies in hybrid cloud. With EIIG, Data Vault 2.0 customers are able to automatically ingest metadata from a wide range of technology sources from the legacy to modern, on-prem to cloud. As a result, they can get end-to-end lineage no matter what technologies they deploy– cloud warehouses from Snowflake, AWS, or Google, programs developed in Python and/or Java, or BI tools such as PowerBI, Tableau, or Qlik.

What is more, thanks to EIIG’s capability of ingesting metadata directly from CI/CD pipelines, enterprise customers can get the DNA of their data with Zero Impact to their Production Systems (Orion’s ZIP technology). This granular visibility into their data enables more accurate discovery of data and more thorough auditability for regulatory compliance.

As a metadata management platform, Orion’s EIIG integrates seamlessly with the CI/CD pipelines to provide near real-time impact analysis, cataloging, and lineage.

Enhancement of Data Vault 2.0’s Value Proposition

Enterprises adopt Data Vault 2.0 to shorten the time to value, achieve accuracy and trustworthiness of their BI reports, as well as agility. EIIG enhances all these benefits as an integral data governance/DataOps component.

Corresponding to Data Vault’s quick time to value, EIIG can be deployed in weeks rather than months, thanks to AI/ML-powered automation. Customers can also choose to implement it on-prem or in the cloud to meet their specific needs.

As mentioned above, EIIG supports a wide range of technologies and covers all types of metadata, be it business metadata, technical metadata, or operational (process execution) metadata. EIIG intelligently and automatically stitches all this metadata in a knowledge graph and shows lineage with granular details from the source through all the systems to the BI report. With hierarchical options, different users can view the lineage from their chosen perspectives. EIIG also supports the masking of complex logic in the Data Vault and exposing it on an “as-is-needed” basis.

What is more, EIIG activates all this metadata throughout the data pipeline. One example is the augmented data quality. EIIG calculates a data quality score based on quality metrics and visualizes the changes when data travels through various systems and before and after a transformation until it reaches the BI report. So, users understand not only where the data comes from but also how good its quality is. By the same token, this capability can help propagate trust, ensure accuracy, and build the users’ confidence in the reports in front of them.

Data Vault 2.0 customers can also leverage EIIG’s powerful term-to-asset mapping capability to capture metadata for mapping from source tables in the staging area to Raw Data Vault tables in an automated and flexible manner. Whether it is the mapping from a staging table to a Data Vault hub, to a Data Vault satellite, or to a Data Vault link, EIIG can facilitate and automate the process. EIIG also captures the business policies and rules that govern the data elements in the data vault through supervised machine learning.

Last but not least, EIIG enriches Data Vault 2.0’s agility particularly through near real-time impact analysis. With EIIG, enterprise customers can use automated and accurate impact analysis to gain a central point of control and remove fear of changes. Customers can leverage impact analysis as part of every change analysis within their Data Vault 2.0 implementation. They can identify and visualize the dependencies between datasets and executables that are impacted by a proposed change, and thus enable small, reliable, and frequent changes. As a result, they are able to adapt to new business requirements and accelerate the value of their information assets in a more agile fashion.

In short, with EIIG, a Data Vault 2.0 customer has all their data governance and metadata management needs covered all the time.

About the Author: Niu Bai, Ph.D. is the Head of Global Business Development at Orion Governance, Inc. Connect with Niu on LinkedIn.