Notes from Databricks Data + AI Summit 2024
The annual Databricks Data + AI Summit is always a valuable opportunity for learning, engagement, and inspiration for Data and AI enthusiasts and practitioners. One of the major announcements at this year’s event was the open-sourcing of Unity Catalog. Databricks’ emphasis on data and AI governance as a cornerstone of data intelligence highlights the increasing importance of this discipline.
I had the privilege of speaking with numerous customers and partners. The consensus is that while Unity Catalog will be crucial for data governance within the Databricks Lakehouse environment, organizations still require a cross-system, enterprise-wide data governance regime integrated with Unity Catalog.
In evaluating current data governance offerings, these customers and partners have identified three major gaps:
- Technology Coverage Gap: Enterprises have deployed various technologies over time. Traditional tools offer limited support for this diverse technological landscape, both in breadth and depth. For complex technologies like mainframe systems (COBOL, JCL, PL/1) or programming languages such as Python and Java, these tools are ineffective. Additionally, they provide only high-level views, hindering organizations from achieving their data governance goals due to broken data lineages and inadequate data asset inventories.
- Capabilities Gap: Many existing solutions are a patchwork of poorly integrated products. For instance, some data catalog vendors offer subpar or nonexistent data lineage capabilities, relying on third parties to fill the gap. Others claim to have these capabilities but provide disparate products that are difficult to implement due to incoherent design or incompatible modules from acquisitions.
- Automation Gap: Traditional data governance tools require extensive manual work due to inefficient architecture and insufficient use of AI and machine learning. This lack of automation complicates the establishment of a data governance regime, preventing enterprises from quickly harnessing the value of their data.
Prospective customers and partners were excited to learn that Orion Governance’s Enterprise Information Intelligence Graph (EIIG), a self-defined data fabric, can address these gaps. EIIG automatically harvests metadata from over 70 technologies, enabling enterprises to establish both intra- and inter-system connectivity. This creates a dynamic, detailed map of end-to-end information flow.
EIIG is an all-in-one active metadata and data governance platform, integrating data lineage, catalog, impact analysis, and metadata analytics. Designed and integrated natively as a single product, EIIG simplifies data governance processes and helps organizations achieve their business outcomes faster.
Moreover, EIIG is highly automated, thanks to its innovative architecture and efficient use of AI and ML. It is easy to implement and can be deployed in weeks rather than months or years, as is common with other vendors. By minimizing human intervention, EIIG helps lower the total cost of ownership.
In summary, with EIIG, enterprises can quickly build an enterprise-wide data governance regime, enabling data-driven decision-making and accelerating key business initiatives such as data modernization and regulatory compliance.
Ready to see EIIG in action? Schedule a free demo today.
About the Author: Niu Bai, Ph.D. is the Head of Global Business Development at Orion Governance, Inc. Connect with Niu on LinkedIn.
Sign up for emails you actually want to read
Join the Orion Governance email list
Recent Articles
A Solution to Solving the Stubborn Data Silo Problem | Evanta NYC 2024 Review
Thank you to Evanta for organizing this CDAO Executive Summit in New York, where leaders in digital transformation shared insights and best practices. Several key takeaways emerged, including the leverage of Generative AI and [...]
EIIG Identifies and Enables Efficient Usage of Golden Data Sources
Unlock the full potential of your data assets with Orion Governance's EIIG (Enterprise Information Intelligence Graph). This in-depth explainer video showcases how EIIG identifies and enables more efficient usage of “golden” data sources, [...]
Video Explainer: How does EIIG identify data assets that have the most value and risk to ensure governance and compliance?
In this video, we explore how Orion Governance's Enterprise Intelligence Information Graph (EIIG) helps organizations identify data assets that hold the most value and risk, ensuring effective governance and compliance. Through EIIG’s powerful [...]