ModelOps and DecisionOps for the Modern World
ModelOps and DecisionOps are popular buzz words today. At the simplest level, these terms are all about deploying analytical models and decision flows into critical business systems for making the best data-driven outcomes. The latest version of SAS Viya includes several deployment capabilities and has plans to further expand its range / support. This article explores the many options you now have to operationalize your models and decisions using SAS Model Manager and SAS Intelligent Decisioning.
Model and Decision Publishing
First, let's talk about what is model publishing? Once developed, the model or decision can be exported as executable run time code that takes input data and actually makes a decision. Your goal is to deploy this code to a runtime system to support some business application. In the context of dev, test, and prod systems, publishing helps you make the journey from dev to prod.
You have two options. In the first case, you can create your own pipeline. The number of steps in the process, and their complexity, are often frustrating for users that want to quickly react to business changes with new score code. In the second case, you can use tools like SAS Model Manager (MM) and SAS Intelligent Decisioning (ID) that make use of the SAS model publishing service to consolidate many of those steps within the act of model publishing, thus reducing the cost and critical time to market. Consider the two flows below.
1. Do it yourself
2. Use the product
How do these options differ you might ask? Well, let's start with how are they the same: you start by building a model and end with a published model and score code. Also, both methods use the model publish service provided by SAS Model Manager and SAS Intelligent Decisioning, in some fashion. However, the journeys differ significantly. In the DIY option you have choices and freedoms, but these can be cumbersome and time consuming. Why not let the SAS model publishing service do what it was built to do and handle all these steps programmatically? In the end, you choose which better services your needs.
Publish Options
Several products at SAS can be used to publish runtime code to operational business systems. SAS Viya 3.5 provided batch and real time scoring options. With the new SAS Viya version, we added several new publishing options including multiple container-based scoring destinations and adding Git as a publish option.
The full list of runtime publish destinations now includes:
Let's take a look at each of these publishing destinations in more detail.
Scoring in CAS and MAS
The SAS Cloud Analytic Services (CAS) and SAS Micro Analytic Service (MAS) are scoring workhorses. They serve as companions for running models in offline batch processes (CAS) or online transactional events (MAS). Both services underwent improvements in the latest release of SAS Viya. Let's take a closer look at the two items.
CAS is installed as in the base install of your SAS Viya environment. It supports SAS Data Step and DS2 code, Astore objects, and open source Python and R code functions. The model publishing action set stores the runtime score code files and metadata as rows in dedicated CAS tables. Your batch scoring process will run this action from SAS code or directly from CASL code. Both ID and MM use CAS for routine score testing, and Model Manager uses CAS scoring for all model monitoring execution.
MAS is also part of the base SAS Viya environment. MAS allows you to score SAS models, Python and R models, and SAS models mixed with SQL or Python or R. When you publish to MAS, a new microanalyticScore REST API endpoint is created for the scoring function. Your business application will call the API to make critical business decisions. Both MM and ID can publish to MAS and run a validation test to confirm successful use of the REST API. MAS is in routine use by some of our biggest customers for mission critical application services.
Bottom line:
- Use CAS for scoring large data already loaded in CAS, integration into other CAS applications, interactive and scheduled batch processing, strategic decisions, and data analysis of model output and root cause analysis.
- Use MAS for scoring of online business events like customer recommendations, credit decisions, billing and collections, office automation, and any other online operational decisions that needs to be delivered right now.
- Together, MAS and CAS form a complete scoring environment in SAS Viya for many applications and users.
Example SAS Viya scoring environment using CAS and/or MAS
Teradata, Hadoop, and Synapse
Database vendors are traditionally stronger in offline data management and OLAP applications, but vendors are pushing capabilities in machine learning systems through scoring functions. SAS applications can publish some Data Step and DS2 modules to the SAS Scoring Accelerator and for in-database scoring in the SAS Embedded Process (EP).
The score execution can run in the CAS server which will push down the scoring job to the configured database, or it can be executed from SQL statements on the database system. The result is no data is transferred out of the database system for scoring. See the SAS Scoring Accelerator for more information.
Teradata and Hadoop support is available in all SAS Viya versions. SAS Scoring Accelerator for Microsoft’s Synapse analytical database will be available in newer versions of SAS Viya later this year.
Bottom line:
- Use the Scoring Accelerator when the data is already stored in the target database system and you want to minimize data movement.
- Use the Scoring Accelerator when you are using a database vendor application that can execute SQL for scoring the data.
- This pattern is common in general data management, data warehouse, and ERP systems.
- Open source models and complex decision flows with SQLSTMT, HTTP, or subject-contact storage may not be published to these sources.
Example SAS Viya 4 in-database scoring environment using SAS Embedded Process (EP)
Event Stream Processing
While SAS Event Stream Processing (ESP) is not a publish destination from Model Manager, it is a valid streaming analytics runtime system. ESP is an embeddable high-speed service for processing streams of time-ordered data to make analytical decisions. The SAS Event Stream Processing Studio user will select the analytical model from the model repository service and integrate the model into the event stream process flow. After testing, the user can push the model out through the Event Stream Manager application to ESP servers. ESP can run SAS DS2 models, Astore models, ONNX format, and Python models.
Bottom Line:
- Use ESP when you have a function that operates on multiple high-speed events in sequence before it’s persisted.
- Use ESP when you need to integrate with IOT devices and join static data with streams.
- Use ESP when you need to filter, detect patterns, calculate across retention periods, or transform high-speed streams of data events.
- Use ESP when you have a system already built around ESP.
Example SAS Viya streaming data scoring environment using SAS ESP
Git
This feature was recently added and is continually enhanced for both MM and ID with SAS Viya 2020.1.1. The runtime model code files are written to the user’s selected Git repository. The files written include all the code needed for execution and all selected descriptive files. Use this set of files to deploy into any existing SAS or MAS environments, or to seed a custom SAS container process.
Bottom Line:
- Use this when you want to archive the MM or ID runtime files outside the SAS Viya environment
- Use this when you want to develop your own CI/CD pipeline to deploy your models and decisions to runtime environments using tools like Jenkins and GitOps.
- A CLI tool has been developed to deploy decision flows to any set of CAS and MAS services.
Example SAS Viya scoring environment leveraging GIT
SAS Container Runtime
Customers consistently ask for a light weight runtime environment. They want to reduce their hardware and administration costs, focus their staff on general purpose IT skills, and minimize data movement. Run time containers fill all these needs.
In a nutshell, a container is a complete package of software that can be stored, moved, and executed in any container run time environment. The SAS container runtime combines a base layer of essential system files provided by SAS with the model or decision run time code files developed by the MM or ID user. A separate article will detail the SAS Container Runtime.
Ultimately, runtime containers produced by SAS run in private Docker or Kubernetes systems or cloud service providers such as Microsoft Azure, Google Cloud Platform, or Amazon Web Services. The publish and deploy process is depicted below. SAS software will publish the complete container image to a user-defined Docker registry. This repository will serve as a staging area for containers ready to be deployed into a runtime environment.
MM and ID can run a publish validation test; however, users are responsible for deploying the container into the runtime. The following image shows one of many possible runtime systems. In this case, a common pool of SCR containers is serving both batch processes through Kafka integration and online event processes through the same REST API. The number of active containers is managed by a runtime system such as Kubernetes according to demand.
Example SAS Viya scoring environment using SCR containers
Bottom Line:
Use containers when you want to:
- score in Docker and Kubernetes environments
- use cloud-based systems for resource management and availability
- implement modern CI/CD based workflows
- integrate with cloud-based resources such as Kafka or CamelK
- integrate with cloud-based data services such as S3 or BigTable
- move into Software as a Service implementations
- support batch and real time with one service type
More on containers
The latest version of SAS Viya produces three types of containers and each are described in more detail below:
- Open source containers for Python and R models
This feature originally shipped in SAS Viya 3.5 and is still supported with the new SAS Viya. Use this container when you have a predominant SAS Viya system and need to add open source model management and deployment. Only MM supports this container destination.
- General containers for SAS models – SAS Container Runtime (SCR)
This feature is new with the 2021.1.3 release and supports many SAS code models. Also, decisions can now be published to a runtime container. Use this when you have a model produced by VA, VML, or SAS Studio. Use this when you have a decision with no SQLSTMT, HTTP, Python, or subject-contact nodes.
- Microsoft Azure Machine Learning (AML) containers for Python and SAS models
This feature is new for release 2021.1.3 and allows users to publish SAS models and Python models from MM directly into Azure Machine Learning. This approach is recommended when users you want to leverage a Microsoft-managed environment and score data available in your Azure subscription. Use this when you want to score SAS runtime code or Python runtime code, but not both in the same process. Model Manager is the only product currently supporting this feature.
Conclusion
SAS has made a strong commitment to supporting modern ModelOps and DecisionOps systems. Our goal is to remove any barrier to deploying SAS models. The recent push into container deployment and Git archives will enable more customers with the ability to deploy runtime code according to their own corporate standards and skills. In particular, the SAS Container Runtime is a unique offering that enables true model and decision portability with maximum scalability, availability, and reproducibility while maintaining true minimum resource requirements.
