The "openness" of SAS Viya has been advertised and touted for the last couple of years. Many, if not most SAS customers have skills and investments in open source resources that they want to exploit and leverage to their business advantage. To a certain extent, SAS delivered some of this capability prior to SAS Viya. Three years ago I wrote an article on deploying SWAT for Python and showed how it opened the door to CAS. But the capabilities of SAS Viya have grown significantly since that blog was written.
Along the way you may have heard about an integration point or some slick functionality with regard to interfacing with SAS software, but you may not be aware of the many ways SAS Viya can interact with various open source technologies. In this article we'll try to capture many of the touchpoints and catalog them to provide a quick reference (sorry no pictures). In a subsequent article we'll look at some of the more common scenarios from an architecture and topology perspective.
Open Source > SAS and SAS > Open Source
Let's state the obvious. The integration with open source software is a two-way street.
For example, a user working with R may want to transfer some local data from their environment to CAS and subsequently join with data already loaded. She then executes models in CAS followed by returning the results to their R environment. Therefore she is reaching out to SAS resources from her open-source programming environment.
Alternatively, a modeler may be working with SAS Model Studio (VDMML) and have a block of Python code that he wants to invoke as part of their modeling scenario. In this scenario he is exploiting the ability to integrate Python code into their VDMML project.
These are just two examples of interactions, but from an architect’s perspective, it may be useful to be aware of the many ways SAS Viya interacts with open-source software.
We'll use the following categories to group the open capabilities.
- Open-source applications leveraging SAS execution engines
- SAS applications leveraging open-source execution engines
- Application development/integration
- Data integration
Open-source applications leveraging SAS execution engines
Python
Let's start with the most popular language used to access SAS Viya and CAS. There are multiple touchpoints when interacting with SAS Viya and CAS.
Getting Started with SAS Viya for Python (developer.sas.com)
- SWAT (Scripting Wrapper for Analytics Transfer) - The most obvious package, and probably the most popular, is SWAT. Once installed this plugin opens the Python doors to access the powerful capabilities of CAS.
- saspy - This package allows a programmer to connect to and submit code to a SAS session. There are numerous connection methods, but only one (HTTP) that is specific to launching a Compute Server on SAS Viya. The remaining methods are primarily used to connect to SAS 9.4 sessions.
- DLPy - SAS Deep Learning Python package provides high-level Python APIs to deep learning methods in SAS VDMML.
- ESPPy - The ESPPy package provides the capability to create SAS ESP models via programming in Python. It can connect to a running ESP server and interact with running projects, continuous queries, windows and various other components of the ESP environment.
- sasoptpy - The sasoptpy package introduces a modeling interface to SAS Viya Optimization and SAS/OR solvers. It provides a quick way for users to deploy optimization models and solve via CAS actions.
- sas-kernel - The sas-kernel package for Jupyter Notebooks opens the data manipulation and analytics capabilities of SAS within the notebook interface. This interface can be used to execute SAS code and view results inline.
R
- SWAT - Similar to Python when the SWAT module is installed a user can be developing R programs and call CAS actions directly from their programming environment.
SAS Viya and CAS REST APIs
- The REST APIs provide a mechanism to access both Viya and CAS services from many different programming languages that provide REST call functionality.
Java
- The SAS Java Client interface is the equivalent of the SWAT package for Python and R. It enables Java programmers to access CAS resources. This includes loading data and running CAS actions against that data loaded to CAS.
Lua
- SWAT - As we saw with Python and R, SWAT for Lua enables the programmer to unleash the power of CAS from their Lua programs.
SAS applications leveraging open-source execution engines
SAS VDMML - Python and R
- SAS Model Studio is the interface used to create, execute and evaluate models in SAS VDMML. There are numerous nodes available for predictive analytics when working with a project. One of those nodes is the Open Source Code This node enables the user to seamlessly insert R or Python code into the project to provide complementary analytics processing
SAS Visual Forecasting - External Languages Package
- The External Languages Package was released with SAS Visual Forecasting on SAS Viya 3.5 to provide SAS > (R, Python) > SAS roundtrip communication while running in distributed mode across CAS nodes. Running in distributed mode provides scalability to Python and R. It handles all data communication from CAS tables to Python or R and back to CAS
SAS Model Manager - PMML Support
- PMML is an XML markup language developed to support exchanging models between modeling systems and scoring platforms. SAS Model Manager supports importing a majority of standard-compliant PMML models and scoring them within a SAS environment using the SAS PSCORE procedure.
SAS Model Manager - PyMAS
- SAS Model Manager enables a customer to store models in a common repository, as well as organize them within projects and folders. The PyMAS package supports models that contain a custom Python code file. This package is delivered during the installation, but must be enabled via configuration.
SAS Intelligent Decisioning - PyMAS
- SAS Intelligent Decisioning provides decision management capabilities to automate the process. If a customer has existing Python code files or writes new code for decisions, these can be executed as part of a decision flow by loading the code to the SAS Micro Analytic Service. The PyMAS package provides this ability to load and execute Python code.
SAS Event Stream Processing - PyMAS
- SAS ESP can use the SAS Micro Analytic Service to execute a block of Python code as part of a running ESP model. That block of code, defined at the project level, can contain functions written Python.
Application development and integration
There are numerous statements, PROCs, functions and SAS programming features that interact with open-source resources. In this section, we will touch on a few of them.
SAS Event Stream Processing
- SAS ESP Connectors and Adapters - SAS ESP ships with many adapters and connectors, the data source/target interfaces, that retrieve data and load data to many common open-source data sources. A few examples are Kafka, Cassandra, HDFS, and RabbitMQ.
- WebSocket API - The WebSocket API enables two-way communication between a client written by a developer and a remote host that accepts that communication. In the ESP context, the ESP server would be the remote host server and the languages invoking web sockets could be written in Java, JavaScript, C++, Python and IOS.
- JavaScript API - The ESP JS API can be used to create and manage ESP projects in an ESP server from within web pages on platforms independent of the ESP server.
- Publish/Subscribe API - SAS ESP provides publish/subscribe APIs for C, Java, and Python. These APIs can be used to publish event streams into a Source window of a running project as well as subscribe to windows within a project to retrieve events.
- Apache Camel - Apache Camel is a framework that enables a user to integrate applications into a single, cohesive architecture to consume and produce data.
- Apache Nifi - Apache Nifi is a system used to automate data flows between multiple systems.
Mobile SDKs (IOS and Android)
- Available Software Developer Kits used to create mobile apps accessing SAS Visual Analytics.
Visual Analytics SDKs
- The Visual Analytics SDKs enables web developers to use a collection of JavaScript libraries to embed SAS Visual Analytics content with custom web pages and web apps
PROC FCMP
- The SAS Function Compiler (FCMP) procedure enables a programmer to create, test and store SAS functions, CALL routines, and subroutines that can then be invoked in a DATA step or SAS procedure. These user-created functions can subsequently call Python functions, allowing a customer to leverage Python functions they may have already created and stored. The Micro Analytic Service provides the execution environment for Python functions
Git functions
- There are many Git functions available in SAS programming that enable a developer to access and manage Git projects from SAS code. They include many of the common actions taken with a Git project.
Hadoop
Although most customer Hadoop deployments are commercial releases, the basis of these deployments is built on open-source software.
- PROC HADOOP can be used to submit HDFS and MapReduce commands to Hadoop instances from a SAS program.
- PROC GROOVY enables programmers to push Groovy statements as uncompiled code to a JVM.
Others
- The FILENAME statement provides access to many data sources, of which there are numerous open-source sources.
- PROC HTTP issues HTTP requests to a specified server and is available in SAS 9 and SAS Viya.
- PROC LUA allows a programmer to run LUA statements with SAS code.
- The X statement provides the capability to execute an operating-environment command from within a SAS session.
Data integration
Like SAS application integration points, there are numerous open-source data sources that can be accessed by SAS Viya.
SAS/ACCESS Interfaces and SAS Data Connectors
- The ACCESS interfaces and data connectors provide access to open data sources from SAS compute and CAS sessions. Example interface and connectors are MySQL, Spark, and PostgreSQL.
SAS Cloud Data Exchange
- SAS Cloud Data Exchange provides data connection capability for SAS Data Preparation. The connection provides secure data transfer from an on-premises data store to cloud-based instances of SAS Viya, enabling SAS Viya applications to access to those data sources. The SAS Data Agent is deployed on-premises and configured to transfer data to and from SAS Viya. It is capable of using many of the interfaces and connectors noted above.
A quick look at SAS 9
While digging into this topic I came across this presentation that highlights many of the basics when looking at SAS 9 and open source. To become familiar with some of those integration points navigate here to check it out: Open Source and SAS 9 (support.sas.com).
Wrapping up
As you can there are many touchpoints regarding SAS Viya and open-source software. This list is not a comprehensive list but has been compiled to raise awareness. Now that we are aware of many of the integration components it begs the question for architects of what software is needed and where should it be placed. We'll cover some of those scenarios in a future article.
My thanks to Simon Williams for his contributions to this article.
Thanks for reading.
