Unlock Seamless Efficiency: SAS Viya’s No-Code/Low-Code Experience to Democratize Databricks

2 Likes

Democratizing in this context means making Databricks accessible to a wider range of users within an organization, not just code focused data scientists or engineers. By combining SAS Viya's user-friendly interfaces and Databricks Lakehouse, more people within an organization can participate in data analysis, decision-making, and innovation, regardless of their technical background. This approach enables organizations to maximize the value of their data assets and drive better business outcomes.

Democratizing tribal knowledge in a digital catalog

Democratizing Databricks with SAS Viya involves converting informal and tacit tribal knowledge about data, analytics, BI dashboards, pipelines, business rules, and decisions into easily accessible digitalized knowledge. This is facilitated by the SAS Information Catalog, which provides effortless access to such information.

Irrespective of the data's location, which in this instance is a Databricks Lakehouse, the SAS Information Catalog offers users a comprehensive understanding of their data from both business and technical perspectives. On the technical front, it provides details like data size, column count, and storage format (e.g., Spark). Meanwhile, from a business standpoint, it covers aspects such as status (approved, under review, flagged etc.), associated business terms and tags, data ownership, automatic explanation, information privacy, and semantic type detection (for PII compliance).

Lastly, from an analytics standpoint, SAS Information Catalog automatically analyzes your columns to aid in understanding quality issues and determining if the data is suitable for analytics purposes.

Figure 1: SAS Information Catalog shows technical and business metadata for a table in Databricks.

Democratizing data literacy through appropriate tools

The SAS Information Catalog is a valuable resource for all users within an organization and facilitates the pursuit of a highly data-literate organization. Improving data literacy is also about providing the proper tools to users. For those eager to delve deep into their Databricks sources, SAS Visual Analytics offers a comprehensive suite of capabilities. Suitable for a wide array of users, including statisticians, data scientists, data engineers, and business analysts, SAS Visual Analytics facilitates ad-hoc analysis for a thorough understanding of Databricks sources. Users can leverage both basic and advanced analytics features extensively. Furthermore, for individuals aiming to create highly informative and visually compelling dashboards that accelerate time to market, SAS Visual Analytics seamlessly integrates robust statistical analysis with appealing business intelligence dashboards.

Figure 2: Dashboard providing insight to readmissions for hospitals in an area of Sweden.

Navigating from SAS Information Catalog to SAS Visual Analytics is as simple as clicking on the Actions menu and choosing "Explore and Visualize." This action automatically transfers the Databricks Spark source into high-speed SAS memory, ensuring immediate accessibility for a wide range of analysis capabilities within SAS Visual Analytics.

Figure 3: Analyzing a Databricks data source in SAS Visual Analytics.

Democratizing code development with the right tools

Truly skilled coders, along with exceptional data scientists and engineers, are a rare and highly sought-after group. This scarcity can lead to a potential lag in important innovations and hinder the achievement of business growth and objectives. However, trying to convert other groups of business users within an organization into proficient coders is a misguided approach. So, how can you empower the rest of your organization—individuals who possess deep business knowledge and a profound understanding of the data—to develop data and analytics pipelines without investing extensive time in enhancing their technical skills?

Patric Hamilton elucidates this development process in his latest blog post, Data Brilliance Unleashed: SAS Data Quality against Databricks - Precision, Performance, Perfection. The key lies in offering user-friendly tools capable of handling both straightforward tasks and intricate, technical coding without requiring users to write a single line of code. The crucial aspect is to ensure these tools facilitate collaboration, enabling both coders and non-coders to assist each other in transforming innovations into production.

Figure 4: SAS Studio data pipeline for entity resolution to create golden or master records.

Conclusion

This comprehensive approach to making Databricks accessible with SAS Viya involves three main strategies: using SAS Viya's user-friendly interfaces and Databricks Lakehouse to increase access, making tribal knowledge easily available through the SAS Information Catalog, and equipping users with the necessary tools for data literacy and code development. By making Databricks available to a wider range of users and providing intuitive tools for analysis and collaboration, organizations can maximize the value of their information assets and achieve better business outcomes.

Learn more about SAS and Databricks

Please note: Data used in screenshots are either fictive demo data or open public data.