The Synergy between Databricks Lakehouse and Orion Governance’s Self-defined Data Fabric
Published On: July 11, 2023
Sign up for emails you actually want to read
Join the Orion Governance email list
Thoughts from Databricks Data & AI Summit
Despite the mostly dreary weather in San Francisco, Data and AI were hot inside the Moscone Center, where Databricks’ annual user conference was being held. The theme of this year’s Data & AI Summitwas “Generation AI.” Under this thematic umbrella, besides LLMs and lakehouse architectures, an important topic is data governance around Databricks’ offering Unity Catalog. As a sponsor of this event, Orion Governance, had a chance to interact with Databricks customers and partners and found great interest in our Enterprise Information Intelligence Graph (EIIG), a self-defined data fabric which offers tremendous synergy with Databricks Lakehouse and Unity Catalog.
This synergy is multifaceted with two main use cases.
Accelerate Lakehouse Migration
Along with end-to-end granular data lineage and dynamic data catalog, EIIG provides near real-time impact analysis to visualize what effect one change may have on downstream systems and identify dependencies. In addition, EIIG’s data quality augmentation and trust propagation capabilities enable organizations to have confidence in the data they are migrating.
Enterprises can also leverage EIIG to manage costs during migration and lakehouse implementation. One such example is the use of metadata analytics such as similarity analysis. EIIG can help automatically and visually identify duplicate reports, ETL (Extract Transform and Load) jobs, and tables so that organizations can delete them and prioritize the right workloads to migrate. As a result, they can reduce cost and minimize risks.
In short, as the next-generation metadata management and data governance platform, EIIG can help speed up the migration process and enable enterprises to implement their Databricks Lakehouse projects on time and within budget.
Complement Unity Catalog and Enable Enterprise-wide Data Governance
Unity Catalog is Databricks’ unified governance solution for data and AI on the lakehouse. EIIG complements Unity Catalog to create significant synergy.
The reality of enterprise IT landscape is a hybrid cloud of which lakehouse is a part. EIIG connects disparate systems in this hybrid cloud and creates one federated view of enterprise information flow. With bidirectional API, EIIG integrates with Unity Catalog. This integration enables organizations to send metadata feeds from other systems to Unity Catalog to enrich its coverage. And at the same time, enterprises can use EIIG to get information from Unity Catalog to be included in an enterprise-wide data catalog.
Since EIIG is one product that natively integrates all the data governance capabilities including lineage, catalog, impact analysis, active metadata, and metadata analytics, organizations can implement a global data governance regime to cover the entire enterprise with the lowest total cost of ownership.
To sum up, Unity Catalog and EIIG can work together to help enterprises achieve unified governance and comply with regulations so as to harness the power of data and AI more effectively, quickly, and economically.
The Control Objectives for Information and Related Technology (COBIT) framework, developed by ISACA, stands as a pivotal framework for managing and governing enterprise information and technology. Releasing its latest iteration as COBIT 2019 or COBIT [...]
In today's technology landscape, mainframe systems continue to play a pivotal role in large organizations across various sectors, including finance, healthcare, manufacturing, retail, and government. These robust systems are chosen for their unwavering reliability, [...]
Orion Governance’s Enterprise Information Intelligence Graph (EIIG) is an award-winning active metadata management solution. Since EIIG only works on the metadata level, there is ZERO impact on production data. This means EIIG can [...]