SAS Grid Manager and SAS Viya: Integration Points

[#1]

As SAS Viya adoption increases among customers, many discover that it fits perfectly alongside their existing SAS implementations, which can be integrated and kept running until major projects have been migrated over. Conversely, SAS Grid Manager has been deployed during the past years to countless production sites. Because SAS Viya provides distributed computing capabilities, customers wonder how it compares to SAS Grid Manager.

SAS^® Grid Manager and SAS^® Viya^® implement distributed computing according to different computational patterns. They can complement each other in providing a highly available and scalable environment to process large volumes of data and produce rapid results. At a high level, the questions we get the most from SAS customers can be summarized in four categories:

• I have SAS Viya and SAS Grid Manager. How can I get the most value from using them together?
• I have SAS Viya. Can I get any additional benefits by also implementing SAS Grid Manager?
• I have SAS Grid Manager. Should I move to SAS Viya?
• I am starting a new project. Which platform should I use - SAS Viya or SAS Grid Manager?

This post is part of a series that will try to answer these questions.

SAS 9 and SAS Viya - Integration Points

Beginning with SAS 9.4M5, SAS Viya and SAS 9 solutions can play nicely together:

• Interfaces and integration points are built into SAS 9 to enable easy access to CAS.
• Integration capabilities with SAS Viya are available both in SAS^® Foundation and in many clients (such as SAS Studio, SAS Enterprise Miner, SAS Enterprise Guide, and others).
• SAS/CONNECT^® allows SAS 9 and SAS Viya to work together by exchanging data and by permitting remote code execution.

From a SAS session you can connect to a CAS server. After you connect to the server, you can load data to CAS server memory, submit DATA step code, and run SAS Viya analytic procedures, CAS server utility procedures, and Base SAS^® procedures. Finally, you can save tables that are in CAS server memory or send back any result to the original SAS session.

Although it is also possible to use SAS/CONNECT, when licensed, to connect SAS grid sessions with the SAS programming and runtime environment in SAS Viya, it is more efficient to directly submit data and code to CAS from the SAS sessions.

In short, by using both SAS 9 and SAS Viya in parallel, you can continue to benefit from any investment in SAS 9 while making use of SAS Viya functionality and features. From within familiar SAS 9 interfaces, projects, and code, you can access the advanced analytics and performance enhancements that SAS Viya provides. As @MarkTomas states, “The integration of SAS 9 and Viya has steadily evolved over the past couple of years”.

SAS Studio 3.x (on SAS 9): Tasks and Snippets Include Portions of Code to Interact with SAS Viya.

Select any image to see a larger version.
Mobile users: If you do not see this image, scroll to the bottom of the page and select the "Full" version of this post.

 

SAS Grid Manager and SAS Viya - Integration Points

Now that the tight integration between SAS 9 and SAS Viya is clear, it is easy to understand how SAS Grid Manager can provide benefits to SAS Viya like those that a centrally managed environment brings to solutions traditionally running on SAS 9:

• SAS Grid Manager can run multiple SAS sessions in parallel that offload computing tasks to CAS, which increases overall utilization and keeps CAS CPUs full. Code can be submitted to both SMP or MPP CAS servers, and both can analyze data using CAS multi-threaded capabilities. In an MPP CAS environment, large analytic problems can be broken down into chunks that CAS simultaneously processes across many machines.
• SAS Grid Manager workload management capabilities can avoid resource contention and efficiently prioritize a diverse user base. SAS processes that spawn CAS sessions can be managed and prioritized.

To summarize, SAS Grid Manager can help to fully utilize CAS hardware resources while at the same time prevent over-saturating CAS nodes with too many sessions.

In the next figure, the left side represents a job started on a grid node. The job code can connect to a CAS server and establish a new session. Execution is offloaded to CAS, which can accelerate the processing of steps within the job code by distributing the processing across multiple compute nodes on an MPP CAS server. On the CAS server, the CAS controller is responsible for managing a cluster of CAS workers. The controller distributes the data to the workers to load into their memory, submits processing tasks to CAS workers, and aggregates the results from the CAS workers when processing is complete.

A SAS Grid Manager Job Spawning CAS Parallel Processing

It is worth remembering that SAS Grid Manager cannot manage SAS Viya, or more specifically, workloads running inside of CAS, as if they were native grid jobs. However, you can use SAS Grid Manager - with any of the available providers - to throttle the workload that gets submitted to CAS.

For example, you could submit all jobs that access CAS to a common grid queue and limit that queue to five simultaneous jobs at any given time.

 

Conclusion

The common answer to all the initial questions is that SAS Grid Manager and SAS Viya complement each other in providing a highly available and scalable environment, as will also be better explained in the next posts of the series.

To better understand how to get the most from both an architecture and an administration perspective, you can get additional information from my SGF 2020 paper SAS^® Grid Manager and SAS^® Viya^®: A Strong Relationship, or its accompanying YouTube video: