SAS Grid Manager and SAS Viya: Use Cases
This is the third post of a series about SAS Grid Manager and SAS Viya, in which we discuss how the two compare, complement, or even differ in providing advanced capabilities to SAS customers.
The initial post covers the integration points that SAS Grid Manager has with SAS Viya, building on the capabilities offered by SAS Foundation since SAS 9.4 M5.
The next one compares how they address specific topics such as workload management, scalability, ease of maintenance, availability, and parallelization.
Finally, this post shows different use cases in which customers have successfully adopted both SAS Grid Manager and SAS Viya to exploit their capabilities in a seamless environment.
Customers adopting SAS solutions can be at different levels of maturity in their journey to implement SAS technology. Here you can find some use cases seen along the way. In each case, SAS Grid Manager and SAS Viya can both provide complementary benefits.
The evolution pattern
Many customers have a large user base consisting of SAS Enterprise Guide users, who enjoy SAS Enterprise Guide mostly for the “visual programming” aspects, on top of its capabilities as a code editor. How can they evolve their environment?
Many of these customers have already embraced, in the past, a shift from multiple, individual desktop SAS installations, to a centralized environment that uses Enterprise Guide as client. Some costumers have kept evolving following a pattern that moves along these phases:
(1) SAS Enterprise Guide connected to a departmental server running SAS 9 --> (2) SAS Enterprise Guide connected to a SAS 9 grid --> (3) SAS Enterprise Guide connected to a SAS 9 grid that submits code to CAS.
The next figure shows the initial phase, where some desktop clients running SAS Enterprise Guide are connected to backend SAS workspace servers hosted on a central server.
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
SAS Enterprise Guide Using a Single Server
As the number of clients grows, it might become difficult to avoid resource contention and prioritize users of the shared environment. The next figure shows the second phase, where the single backend server is migrated to a SAS grid environment, which provides the required workload management capabilities.
SAS Enterprise Guide Leveraging a SAS Grid
Keeping the pace with the ever-increasing amount of data to be consumed and analyzed requires another shift to add distributed in-memory capabilities. The next figure shows the third phase, which adds SAS Viya to the previous environment, bringing new advanced algorithms and CAS in-memory processing capability.
SAS Enterprise Guide Leveraging a SAS Grid Integrated with SAS Viya
In this new environment, SAS Enterprise Guide can even leverage SAS Studio tasks; they provide native SAS Viya tasks, that can be part of any project, on top of traditional SAS Enterprise Guide tasks. The next figure shows some of these tasks, which can generate code that can be spawned from a grid session to execute directly in CAS.
SAS Studio SAS Viya Tasks Leveraged by SAS Enterprise Guide
Looking further into the future, SAS Viya is moving away from desktop clients and towards web-based interfaces. The natural evolution for SAS Enterprise Guide users is to migrate to SAS Studio. Although the current release of SAS Studio on SAS Viya is focused on providing interfaces for programmers, work is in progress to develop visual perspectives where users design the flow between nodes, in a similar way as they are familiar with SAS Enterprise Guide projects.
This “evolutionary” approach can be also implemented for other SAS solutions. For example, customers currently using SAS Enterprise Miner in SAS 9, can leverage a SAS 9 grid environment to parallelize models execution and balance the workload between modelers; the addition of SAS Viya can bring new models, new algorithms, and in-memory, distributed execution. SAS Viya models and procedures can be called directly from SAS Enterprise Miner, starting with release 14.2, by using the SAS Viya Code node.
A variation of this “phased evolution” approach can be obtained by implementing a SAS Grid Manager environment integrated with SAS Viya since the beginning. For example, a customer with an extremely overworked SAS 9 environment, whose users are mainly Base SAS programmers using DATA step, SQL, and analytical procedures, might wonder whether to move into a grid environment or directly switch to SAS Viya. Are the growing number of technical and analytical capabilities in SAS Viya ousting the need to contemplate a SAS grid?
As we have already explained in the previous posts of the series, implementing both SAS Grid Manager and SAS Viya can provide an efficient and highly available environment, in which they complement one another, and each uses its own strengths. SAS Grid Manager can provide the workload management capabilities required to avoid resource contention and to efficiently prioritize the diverse user base. SAS Viya can augment the analytical capabilities by providing advanced algorithms and accelerate the time to results, thanks to CAS distributed and in-memory processing.
The pipeline model
Multiple-use cases can be found in environments where SAS Grid Manager and SAS Viya provide complementary services in different steps of a project.
SAS Viya Feeds into SAS Grid Manager
SAS Data preparation and SAS Visual Data Mining and Machine Learning, both of which are based on SAS Viya, can be used by analysts to prepare data and build models in development. They can benefit from large-scale data manipulation, interactive and scalable visualization, and advanced model training. When models are ready to be pushed as batch jobs running on the corporate data warehouse, SAS Grid Manager can be used to orchestrate their execution in production, providing operational batch workload management.
SAS Grid Manager Feeds into SAS Viya
Analysts working in SAS Viya want to leverage advanced analytical capabilities on large amounts of data. Traditional ETL systems can lag in feeding the required data into their environment. In these cases, data quality and data preparation steps can be run in a SAS grid environment. The SAS Grid environment can leverage grid scaling and parallelization capabilities to perform parallel data manipulation for a lot of users, for a lot of applications, and on a lot of data. Grid jobs can include both traditional SAS 9 and CAS steps, integrated in a seamless flow.
A recent implementation can be found in the project of the Copenhagen Regional Psychiatric Centers dashboard to monitor COVID-19 pandemic’s impact on their care centers. By using functionality in their SAS Grid environment for data manipulation and automation, they automated the whole process from source data, trough validation and transformation to the final daily updated COVID-19 dashboard, implemented with SAS Visual Analytics on SAS Viya.
Managed analytics
SAS Visual Analytics customers that have already embraced SAS Viya can benefit from SAS Grid Manager, too. They may face issues such as batch jobs or interactive, ad hoc sessions that connect to CAS and that exhaust resources. When that happens, online reporting might become sluggish, and reports might sometimes display quickly and sometimes seem strangely slow.
In this case, SAS Grid Manager can provide workload management capabilities. As we have briefly discussed in the previous posts, it is possible to create specific grid queues for SAS jobs that interact with CAS. Queue policies can set a fixed limit for the maximum number of concurrent jobs, which leaves enough resources on CAS workers for interactive SAS Visual Analytics users.
A more advanced integration can be obtained by creating custom metrics for SAS Grid Manager. For example, a custom script could monitor the CPU consumption of CAS workers or the number of existing CAS user sessions. SAS Grid Manager could then be configured to use the custom script output to dynamically adjust the limits on grid queues based on these metrics.
Conclusion
If you are a SAS customer who has business critical applications or batch jobs that are managed by SAS Grid Manager, then you can leverage the new functionalities that SAS Viya provides. You can use the two side by side without having to convert over your processes all at once.
If you are already using SAS Viya, you can benefit from SAS Grid Manager workload management capabilities.
In the end, you are free to choose the path that best meets your needs, using the best from both worlds.
To further understand how to get the most from both an architecture and an administration perspective, you can get additional information from my SGF 2020 paper SAS® Grid Manager and SAS® Viya®: A Strong Relationship, or its accompanying YouTube video.
