Have you ever wanted to see all your published models in one view? We did!   With the SAS Viya September 2021.1.5 stable release, SAS Model Manager now offers a new Deployments view.   SAS Model Manager's new Deployments view   This view displays the full list of published models by Published Destination. Users select, or multi-select, a Publishing Destination of choice to filter the list. This makes it easier to track which items have been published and allow users a different view of the model repository more focused on deployment.     This video shows a quick example of the new Deployments view.   This is the first step in a series of planned ModelOps enhancements. Our goal is to make it a lot easier to audit where, when and how performant your deployed models are. Please stay tuned for increased functionality from this new Deployments view.   For more information about using SAS Model Manager integration with SAS Model Risk Management, refer to the SAS Model Manager: User’s Guide documentation.   Discover new SAS Viya 2021.1.5 features:   Discover new SAS Model Manager features: Home page Publish Python models to Azure Machine Learning  Simplified workflow-based KPI alerting Generate standard and custom KPIs Updated performance report Publish SAS models to Azure Machine Learning SAS Runtime Container Enhanced Git Publishing Integration with SAS Model Risk Management   ... View more

“Financial Services technology is currently in the midst of a profound transformation, as CIOs and their teams prepare to embrace the next major phase of digital transformation.”-McKinsey, 2020    This is a primary driver for SAS to release its next generation risk solution ecosystem including SAS Model Risk Management on SAS Viya. This modular, API-first design will facilitate injection of model governance and oversight in a seamless and automated fashion. Closer integration between MRM, model ops, model execution and model performance monitoring will allow for opportunities in process automation.    SAS Model Risk Management (MRM) is an enterprise class model governance solution which allows companies to manage the information and risks associated with their entire model development lifecycle.    SAS Model Manager (MM) is a tool that helps streamline model deployment and management to operationalize AI everywhere.    Combining MM and MRM provides a more wholistic vision of a unified enterprise platform needed to capture the entire model lifecycle for the financial services industry. Essentially, the combination of these offerings is like 1 + 1 = 3.   This video shows the native integration of an MRM Model Risk Card directly within MM's Model View.   The MM and MRM teams are working closely to develop native integration features to notify others in the model risk discipline of potential issues. With the SAS Viya September 2021.1.5 stable release, MM provides an easily toggle switch to govern model risk within the MRM solution. This correlation pushes details of model risk to all relevant stakeholders and thereby turns the data captured by MRM into actionable information by relevant parties.    The following video walks through how to set a model linkage between MM and MRM.   For more information about using SAS Model Manager integration with SAS Model Risk Management, refer to the SAS Model Manager: User’s Guide documentation.   Discover new SAS Model Manager features: Home page Publish Python models to Azure Machine Learning  Simplified workflow-based KPI alerting Generate standard and custom KPIs Updated performance report Publish SAS models to Azure Machine Learning SAS Runtime Container Enhanced Git Publishing   ... View more

Making models available in Git from SAS Viya just got a lot easier! SAS Model Manager 2021.1.4 stable release includes enhanced Git publishing. Now you can select a Git directory path and folder, or create a new folder, directly from SAS Model Manager when you want to commit your model to a Git repo.   The following video demonstrates how easy it is to publish SAS and open-source (Python/R based models) to Git.   Publishing to Git was added to SAS Model Manager with the 2020.1.1 stable release. This feature allows runtime model code files to be written to a user’s selected Git repository. These include all code files needed for execution and all selected descriptive files. Use this set of files to deploy into any existing SAS or MAS environment, or to seed a custom SAS container process.   Publishing models to Git is an option when you want to access runtime files outside the SAS Viya environment. Users can leverage these files to develop CI/CD pipelines to deploy models registered into SAS Model Manager to runtime environments using tools like Jenkins and GitOps.   Example SAS Viya scoring environment leveraging GIT   This video walks through how easy it is to create a new Git Publishing Destination.   For more information about using SAS Model Manager publishing destinations, refer to the SAS Model Manager: User’s Guide documentation. For information on how to configure Git Publishing Destinations go to this link. For a helpful example on how to create the necessary domain ID and  a Git publishing destination using a Python notebook, go to this example on our Model Manager Resources GitHub.   Needing help to determine which publishing destination is best for you, check out this SAS Communities library article  "ModelOps and DecisionOps for the Modern World" written by David_Duling.   Discover new SAS Model Manager features: Simple filtering dashboards plus enhanced search options Customize SAS Model Manager performance monitoring reports with embedded web components pzmm (Python Zip Model Management) module updates New/updated Model Management Macros generating JSON files Enhanced support for R models New Project Wizard Home page Publish Python models to Azure Machine Learning  Simplified workflow-based KPI alerting Generate standard and custom KPIs Updated performance report Publish SAS models to Azure Machine Learning SAS Runtime Container ... View more

Big news...SAS Model Manger now publishes SAS models as containers. Now you can execute SAS models ANYWHERE!   With the SAS Model Manager 2021.1.3 stable release, you can now publish SAS models using a lightweight OCI (Open Container Initiative) compliant Docker container for executing SAS models. This will greatly simplify efforts to move code from your Dev system to your Prod system using immutable containers. Simply register your SAS containerized model to your favorite cloud or docker location. Create an app or use restAPIs to score new data against your published model leveraging SAS Container Runtime. This provides a modern, bullet proof, reliable, non-proprietary system for transferring and executing runtime code.   The following video walks through how easy it is to publish SAS models to the new SAS Runtime Container.   Now that your SAS model is published using SAS Container Runtime to any container registry, you can easily take advantage of the cloud infrastructure, or local installation, to deploy applications with small footprint and are highly scalable and highly available. This ensures resources can be fully utilized to quickly execute the largest number of models and reduce the effort that is required to manage the environment.   ​Why did SAS create SAS Container Runtime? Customers consistently ask for a light weight runtime environment. They want to reduce their hardware and administration costs, focus their staff on general purpose IT skills, and minimize data movement. Run time containers fill all these needs.    In a nutshell, a container is a complete package of software that can be stored, moved, and executed in any container run time environment. The SAS container runtime combines a base layer of essential system files provided by SAS with the model or decision run time code files developed by the SAS Model Manager (MM) and/or SAS Intelligent Decisioning (ID) user.    Ultimately, runtime containers produced by SAS run in private Docker or Kubernetes systems or cloud service providers such as Microsoft Azure, Google Cloud Platform, or Amazon Web Services. The publish and deploy process is depicted below. SAS software will publish the complete container image to a user-defined Docker registry. This repository will serve as a staging area for containers ready to be deployed into a runtime environment.    MM and ID can run a publish validation test; however, users are responsible for deploying the container into the runtime. The following image shows one of many possible runtime systems. In this case, a common pool of SCR containers is serving both batch processes through Kafka integration and online event processes through the same REST API. The number of active containers is managed by a runtime system such as Kubernetes according to demand.     Example SAS Viya scoring environment using SCR containers     Bottom Line: Use containers when you want to: score in Docker and Kubernetes environments use cloud-based systems for resource management and availability implement modern CI/CD based workflows integrate with cloud-based resources such as Kafka or CamelK integrate with cloud-based data services such as S3 or BigTable move into Software as a Service implementations support batch and real time with one service type   Summary SAS Container Runtime (SCR) hits all the marks: Modern – industry standard tools for cloud enterprises​ Bulletproof – completely self contained with minimal requirements​ Reliable – runs the same in every environment​ Non-proprietary – users only need common skills for transfer and execution ​ ​ For more information about using SAS Model Manager publishing destinations, the SAS Model Manager: User’s Guide documentation. For more information about SAS Container Runtime, see the SAS Container Runtime. Looking for helpful "how-to" use the SAS Container Runtime for publishing SAS models to Kubernetes on the Azure cloud, check out this blog.   Needing help to determine which publishing destination is best for you, check out this SAS Communities blog entitled "ModelOps and DecisionOps for the Modern World" written by David_Duling.   Discover new SAS Viya 2021.1.3 features:   Discover new SAS Model Manager features: Simple filtering dashboards plus enhanced search options Customize SAS Model Manager performance monitoring reports with embedded web components pzmm (Python Zip Model Management) module updates New/updated Model Management Macros generating JSON files Enhanced support for R models New Project Wizard Home page Publish Python models to Azure Machine Learning  Simplified workflow-based KPI alerting Generate standard and custom KPIs Updated performance report Publish SAS models to Azure Machine Learning ... View more

Have you ever wanted to expand utility of your SAS models? Have you wanted to score SAS models within a Power BI report or Microsoft Power Apps low-code app? Great news! This is now a lot easier with the latest SAS and Microsoft integration. Deploying SAS and open-source models to Azure Machine Learning has never been easier.   With SAS Viya 2021.1.3 stable release, SAS Model Manager now supports publishing SAS models directly to Azure Machine Learning. This is the second phase of SAS Model Manager's planned integration point, with more integrations planned. Check out this article for the initial integration effort supporting publishing Python models to Azure Machine Learning.     Users with an Azure Machine Leaning (AML) tenant and SAS Model Manager on SAS Viya (MM) license can use their Azure AD single-sign-on to quickly publish SAS models directly into AML. Users need to register SAS models, already registered into MM, using a configured Azure Machine Learning publishing destination allowing MM to dynamically discover available AML workspaces for model registration. MM takes care of passing all the necessary artifacts to AML. AML then creates an AML container, using a specially configured base container. This might sound complex, but the user experience is incredibly simple. Just choose your AML publishing destination, select your AML workspace and select "publish". SAS Model Manager and Azure Machine Learning developers took care of everything for you!   The following video walks through the simple and integrated SAS and Microsoft connection to publish SAS models into Azure Machine Learning.   SAS Models Which SAS models you might ask? Any SAS model with executable score.sas or .sasast (astore) registered in SAS Model Manager on SAS Viya  is supported. This includes models created in SAS Enterprise Miner, Base SAS, SAS Enterprise Guide, SAS Visual Data Mining Machine, SAS Visual Text Analytics, and SAS Studio.    Azure Machine Learning Publishing Destination Admin users can leverage this example notebook  to create an Azure Machine Learning (AML) publishing destination in SAS Environment Manager using their AML Subscription ID.   Score SAS models with confidence in Azure Machine Learning  SAS Model Manager offers a simple Publish option that dynamically discovers available AML workspaces for registering your SAS models. Once published, it's easy to validate whether SAS models will run successfully in AML using SAS Model Manager’s Publishing Validation option. This feature dynamically creates a temporary Azure compute resource. Select test data, and SAS Model Manager will pass the data directly into the Azure Machine Learning Inference Computer cluster. Once AML runs the data in the container, the compute cluster shuts down automatically. Results of the Publishing Validation process return to SAS Model Manager for review. SAS Model Manager also offers an option to Deploy published models to Azure Kubernetes Service (AKS) to immediately provide a useable endpoint to score your SAS models in Azure.   Check out how easy it is to deploy SAS and Python models from SAS Model Manager to Azure Kubernetes Service.   For more information about using AML as a SAS Model Manager publishing destination, see the SAS Model Manager: User’s Guide documentation.   With Microsoft Azure as the preferred cloud provider for SAS Cloud, we are working together to provide the best experience and value to our customers as they transition to the cloud. We will build deep integrations of our SAS® Viya® Platform with Azure services, develop joint market-ready solutions in different industry verticals, and share in go-to-market activities across the globe. For more information on the strategic partnership, visit www.sas.com/Microsoft.   Discover new SAS Viya 2021.1.3 features:   Discover new SAS Model Manager features: Simple filtering dashboards plus enhanced search options Customize SAS Model Manager performance monitoring reports with embedded web components pzmm (Python Zip Model Management) module updates New/updated Model Management Macros generating JSON files Enhanced support for R models New Project Wizard Home page Publish Python models to Azure Machine Learning  Simplified workflow-based KPI alerting Generate standard and custom KPIs Updated performance report   Now's your turn. We're thinking about other SAS Model Manager and Microsoft integrations. Perhaps ways to enhance Team collaboration. What do you suggest? Leave us your suggestion in a comment below. ... View more

ModelOps and DecisionOps are popular buzz words today. At the simplest level, these terms are all about deploying analytical models and decision flows into critical business systems for making the best data-driven outcomes. The latest version of SAS Viya includes several deployment capabilities and has plans to further expand its range / support. This article explores the many options you now have to operationalize your models and decisions using SAS Model Manager and SAS Intelligent Decisioning.   Model and Decision Publishing First, let's talk about what is model publishing? Once developed, the model or decision can be exported as executable run time code that takes input data and actually makes a decision. Your goal is to deploy this code to a runtime system to support some business application. In the context of dev, test, and prod systems, publishing helps you make the journey from dev to prod.    You have two options. In the first case, you can create your own pipeline. The number of steps in the process, and their complexity, are often frustrating for users that want to quickly react to business changes with new score code. In the second case, you can use tools like SAS Model Manager (MM) and SAS Intelligent Decisioning (ID) that make use of the SAS model publishing service to consolidate many of those steps within the act of model publishing, thus reducing the cost and critical time to market.  Consider the two flows below.   1. Do it yourself 2. Use the product How do these options differ you might ask? Well, let's start with how are they the same:  you start by building a model and end with a published model and score code. Also, both methods use the model publish service provided by SAS Model Manager and SAS Intelligent Decisioning, in some fashion. However, the journeys differ significantly. In the DIY option you have choices and freedoms, but these can be cumbersome and time consuming. Why not let the SAS model publishing service do what it was built to do and handle all these steps programmatically?  In the end, you choose which better services your needs.   Publish Options Several products at SAS can be used to publish runtime code to operational business systems. SAS Viya 3.5 provided batch and real time scoring options. With the new SAS Viya version, we added several new publishing options including multiple container-based scoring destinations and adding Git as a publish option.    The full list of runtime publish destinations now includes: CAS server MAS server Teradata, Hadoop, and Synapse Event Stream Processing Azure Machine Learning containers Azure containers AWS containers Private Docker containers Git repositories SAS Container Runtime   Let's take a look at each of these publishing destinations in more detail.   Scoring in CAS and MAS The SAS Cloud Analytic Services (CAS) and SAS Micro Analytic Service (MAS) are scoring workhorses. They serve as companions for running models in offline batch processes (CAS) or online transactional events (MAS). Both services underwent improvements in the latest release of SAS Viya. Let's take a closer look at the two items.   CAS is installed as in the base install of your SAS Viya environment. It supports SAS Data Step and DS2 code, Astore objects, and open source Python and R code functions. The model publishing action set stores the runtime score code files and metadata as rows in dedicated CAS tables. Your batch scoring process will run this action from SAS code or directly from CASL code. Both ID and MM use CAS for routine score testing, and Model Manager uses CAS scoring for all model monitoring execution.   MAS is also part of the base SAS Viya environment. MAS allows you to score SAS models, Python and R models, and SAS models mixed with SQL or Python or R. When you publish to MAS, a new microanalyticScore REST API endpoint is created for the scoring function.  Your business application will call the API to make critical business decisions. Both MM and ID can publish to MAS and run a validation test to confirm successful use of the REST API. MAS is in routine use by some of our biggest customers for mission critical application services.    Bottom line: Use CAS for scoring large data already loaded in CAS, integration into other CAS applications, interactive and scheduled batch processing, strategic decisions, and data analysis of model output and root cause analysis. Use MAS for scoring of online business events like customer recommendations, credit decisions, billing and collections, office automation, and any other online operational decisions that needs to be delivered right now. Together, MAS and CAS form a complete scoring environment in SAS Viya for many applications and users.   Example SAS Viya scoring environment using CAS and/or MAS   Teradata, Hadoop, and Synapse Database vendors are traditionally stronger in offline data management and OLAP applications, but vendors are pushing capabilities in machine learning systems through scoring functions. SAS applications can publish some Data Step and DS2 modules to the SAS Scoring Accelerator and for in-database scoring in the SAS Embedded Process (EP).    The score execution can run in the CAS server which will push down the scoring job to the configured database, or it can be executed from SQL statements on the database system. The result is no data is transferred out of the database system for scoring. See the SAS Scoring Accelerator for more information.   Teradata and Hadoop support is available in all SAS Viya versions. SAS Scoring Accelerator for Microsoft’s Synapse analytical database will be available in newer versions of SAS Viya later this year.   Bottom line: Use the Scoring Accelerator when the data is already stored in the target database system and you want to minimize data movement. Use the Scoring Accelerator when you are using a database vendor application that can execute SQL for scoring the data. This pattern is common in general data management, data warehouse, and ERP systems.  Open source models and complex decision flows with SQLSTMT, HTTP, or subject-contact storage may not be published to these sources.    Example SAS Viya 4 in-database scoring environment using SAS Embedded Process (EP)    Event Stream Processing While SAS Event Stream Processing (ESP) is not a publish destination from Model Manager, it is a valid streaming analytics runtime system. ESP is an embeddable high-speed service for processing streams of time-ordered data to make analytical decisions. The SAS Event Stream Processing Studio user will select the analytical model from the model repository service and integrate the model into the event stream process flow. After testing, the user can push the model out through the Event Stream Manager application to ESP servers. ESP can run SAS DS2 models, Astore models, ONNX format, and Python models.    Bottom Line: Use ESP when you have a function that operates on multiple high-speed events in sequence before it’s persisted. Use ESP when you need to integrate with IOT devices and join static data with streams. Use ESP when you need to filter, detect patterns, calculate across retention periods, or transform high-speed streams of data events. Use ESP when you have a system already built around ESP.   Example SAS Viya streaming data scoring environment using SAS ESP   Git This feature was recently added and is continually enhanced for both MM and ID with SAS Viya 2020.1.1. The runtime model code files are written to the user’s selected Git repository. The files written include all the code needed for execution and all selected descriptive files. Use this set of files to deploy into any existing SAS or MAS environments, or to seed a custom SAS container process.   Bottom Line: Use this when you want to archive the MM or ID runtime files outside the SAS Viya environment Use this when you want to develop your own CI/CD pipeline to deploy your models and decisions to runtime environments using tools like Jenkins and GitOps.  A CLI tool has been developed to deploy decision flows to any set of CAS and MAS services.    Example SAS Viya scoring environment leveraging GIT   SAS Container Runtime Customers consistently ask for a light weight runtime environment. They want to reduce their hardware and administration costs, focus their staff on general purpose IT skills, and minimize data movement. Run time containers fill all these needs.    In a nutshell, a container is a complete package of software that can be stored, moved, and executed in any container run time environment. The SAS container runtime combines a base layer of essential system files provided by SAS with the model or decision run time code files developed by the MM or ID user. A separate article will detail the SAS Container Runtime.    Ultimately, runtime containers produced by SAS run in private Docker or Kubernetes systems or cloud service providers such as Microsoft Azure, Google Cloud Platform, or Amazon Web Services. The publish and deploy process is depicted below. SAS software will publish the complete container image to a user-defined Docker registry. This repository will serve as a staging area for containers ready to be deployed into a runtime environment.    MM and ID can run a publish validation test; however, users are responsible for deploying the container into the runtime. The following image shows one of many possible runtime systems. In this case, a common pool of SCR containers is serving both batch processes through Kafka integration and online event processes through the same REST API. The number of active containers is managed by a runtime system such as Kubernetes according to demand.     Example SAS Viya scoring environment using SCR containers    Bottom Line: Use containers when you want to: score in Docker and Kubernetes environments use cloud-based systems for resource management and availability implement modern CI/CD based workflows integrate with cloud-based resources such as Kafka or CamelK integrate with cloud-based data services such as S3 or BigTable move into Software as a Service implementations support batch and real time with one service type   More on containers The latest version of SAS Viya produces three types of containers and each are described in more detail below:   Open source containers for Python and R models This feature originally shipped in SAS Viya 3.5 and is still supported with the new SAS Viya. Use this container when you have a predominant SAS Viya system and need to add open source model management and deployment. Only MM supports this container destination.    General containers for SAS models – SAS Container Runtime (SCR) This feature is new with the 2021.1.3 release and supports many SAS code models. Also, decisions can now be published to a runtime container.  Use this when you have a model produced by VA, VML, or SAS Studio.  Use this when you have a decision with no SQLSTMT, HTTP, Python, or subject-contact nodes.   Microsoft Azure Machine Learning (AML) containers for Python and SAS models This feature is new for release 2021.1.3 and allows users to publish SAS models and Python models from MM directly into Azure Machine Learning. This approach is recommended when users you want to leverage a Microsoft-managed environment and score data available in your Azure subscription. Use this when you want to score SAS runtime code or Python runtime code, but not both in the same process. Model Manager is the only product currently supporting this feature.    Conclusion SAS has made a strong commitment to supporting modern ModelOps and DecisionOps systems. Our goal is to remove any barrier to deploying SAS models. The recent push into container deployment and Git archives will enable more customers with the ability to deploy runtime code according to their own corporate standards and skills. In particular, the SAS Container Runtime is a unique offering that enables true model and decision portability with maximum scalability, availability, and reproducibility while maintaining true minimum resource requirements.   ... View more

Models can begin to degrade the moment that you put them in production. With SAS Model Manager, you can monitor the performance of your analytical models to see whether they continue to behave as expected after changes occur in market conditions or customer behavior, new data becomes available, or there is concept drift. By monitoring their performance, you can avoid model decay and re-evaluate the business value of your models in production to keep them performing at the highest levels. Put simply...if your organization focuses only on building models, and not assessing model performance over time...your organization will greatly minimize the value your Data Science and IT teams.   Updated SAS Model Manager performance monitoring report   The SAS Model Manager 2021.1.2 includes significant updates to it's out-of-the-box performance monitoring. Now you can compare performance for multiple models within the same report. A new Summary page highlights critical information about the models analyzed within the completed performance job. Quickly see if your champion model is still out-performing challenger models or other models within your project. Users can easily compare how all the different models performed over time. Simply click the different model tabs to compare performance charts like variable distribution, characteristic, stability, lift, ROC, KS and Gini per model. The performance charts now include natural language to help users better understand how to interpret their model's performance. Customization using SAS Visual Analytics embedded web components is also supported (see blog for more details). This is a great way to augment input and output variable tracking to look for changing conditions, concept drift and other issues affecting your model's performance.   Check out the differences between the original and updated SAS Model Manager performance monitoring report.   The new Summary page contains assessment information about the models that are included in the project’s performance results. It also includes scoring observations, performance definition high-level details, and the performance rank of models within the containing project. The performance rank is the model with the best latest performance relative to the project's defined model assessment criteria. If the model assessment criteria is not specified in the project’s properties, the misclassification KPI is used to determine the rank of models in a project with a classification model function. Likewise, the average squared error (ASE) KPI is used to determine the rank of models in a project with a prediction model. Please see this blog to learn more about standard and custom KPIs supported with SAS Model Manager.   New Summary page   SAS Model Manager enables you to monitor and evaluate model performance of your champion, challenger, and other models to ensure that your models are performing efficiently. Once you create a performance definition, with as many models within a project you're interested in seeing, run it. The updated performance monitoring report will allow you to explore performance results, such as Variable Distribution, Characteristic, Stability, Lift, Gini, ROC, Kolmogorov-Smirnov (KS), Average Squared Error (ASE), and Feature Contribution (FCI) charts for each model. Now you can easily compare performance results inline for each model with natural language helping you read the results.   New individual model pages   The SAS Model Manager team has several other features are in development to further augment the updated performance monitoring report. New features include a new Compare page to visualize performance result trends for multiple models. New performance monitoring charts for multinomial models. Plus performance monitoring charts for registered/deployed SAS Visual Text Analytics models including text categories, text concepts, text sentiment and text topics. These VTA text performance monitoring trend charts are being created by our SAS Summer intern @JoeCabral. Check out what Joe created in just a few weeks.   Design ideas for text category performance monitoring charts:      Remember, SAS Model Manager is not limited to supporting just SAS models. It supports open source models too. Check out this blog about Python model support using pzmm. Here's another blog about recent enhancements to support R models.   Also check out other examples in our SAS Model Manager documentation.   Discover new SAS Viya 2021.1.2 features:   Discover new SAS Model Manager features: Simple filtering dashboards plus enhanced search options Customize SAS Model Manager performance monitoring reports with embedded web components pzmm (Python Zip Model Management) module updates New/updated Model Management Macros generating JSON files Enhanced support for R models New Project Wizard Home page Publish Python models to Azure Machine Learning  Simplified workflow-based KPI alerting Generate standard and custom KPIs   What's New with SAS Model Manager Quiz Aside from performance monitoring report updates, what new feature is coming soon to SAS Model Manager that needs it’s own new ‘view’? (Hint: Look closely at the SAS Model Manager images in this blog.)     ... View more

Want to use common statistical measures to track model performance over time? SAS Model Manager includes a new MMKPI macro and action set to generate out-of-the-box standard KPIs and run user-provided KPIs. Now standard and custom key performance indicators (KPIs) can be defined, computed, and used to generate alerts. Check out this article to see how KPIs are integrated in a new out-of-the-box workflow-based email notification when model performance exceeds the project's model assessment criteria threshold value.   A KPI is a computed value derived from the SAS Model Manager performance job or from a user's own data and jobs. Common KPIs for classification projects and interval prediction projects are "Misclassification" and "Average squared error" respectively. The number of possible custom KPI metrics is limitless. Here are the current standard KPIs generated by the the new SAS Model Manager MMKPI macro and action set: ​Average Squared Error (_ASE_) Area Under the ROC Curve (_AUC_) Cumulative Lift at a Depth of 5 (_Cumlift5_) Cumulative Lift at a Depth of 10 (_Cumlift10_) Cumulative Lift at a Depth of 15 (_Cumlift15_) Cumulative Lift at a Depth of 20 (_Cumlift20_) Cumulative Lift at a Depth of 30 (_Cumlift30_) F1 Score (_F1_) False Positive Rate (_FPR_) Misclassification (_MISC_) R-Squared (_RSquared_) True Positive Rate (_TPR_)   Example output of the new MMKPI macro and action set:   With the 2021.1.1 (May 2021) stable release of SAS Viya, SAS Model Manager includes a new Files tab to run the standard KPIs for a project, once a performance monitoring report job has run successfully. Notice the project UUID (_projectID) is automatically populated for you. Simply click Run for the ProjectKPI.sas file and View results to see the output (shown above) code and log for generating standard out-of-the-box KPIs.   Additional enhancements to the new Files tab came with the 2021.1.3 (July 2021) stable release with support for custom KPIs. Need some help writing custom KPIs? Please see this link for some examples.   Generate standard out-of-the-box KPIs and custom KPIs directly within SAS Model Manager:    Results of these standard KPIs are already integrated into an out-of-the-box, simplified model monitoring alert notification workflow and the updated performance monitoring results. Soon, you'll see KPIs used in model health dashboards, model and project comparisons, alerts, and notifications.   New SAS Model Manager performance monitoring report including latest model assessment criteria KPIs:   ​See the SAS Model Manager documentation for more information about the new MMKPI macro and action set. Essentially, the %MM_KPI_ACTIONSET macro enables users to: create standard KPI data table from the performance monitoring results using the runKPI action set add custom KPI data and labels to the standard KPI table using the addCustomKPI action set, and/or run user-provided custom KPI code using the runCustomKPI action set   In addition to running standard KPIs, the upcoming stable release of SAS Model Manager will allow users to integrate and run custom KPIs directly within the new Files tab. In the interim, leverage the new MMKPI action set via APIs and via SAS code within SAS Studio.   In SAS Studio, here's the code to start a new CAS session and activate the new MM KPI action set. /* Start a CAS session and enable new MM KPI action set */ cas _mmcas_ uuidmac=session_uuid; %put session _mmcas_ uuid: &session_uuid; caslib _ALL_ assign; /* update string with your Project UUID */ %let projectUUID=%nrstr(23a309cc-7451-47b3-a667-7976ed9d726e); libname mmkpi cas caslib="ModelPerformanceData" tag="&projectUUID."; %mm_kpi_actionSet;   Here's the code to generate standard out-of-the-box MM KPIs using this new action set: /* Generate Model Manager Standard KPI Table (<projectUUID.>MM_STP_KPI) Need to have already run the performance job first */ proc cas; _projectID = "&projectUUID."; builtins.loadactionset / actionSet="mmkpi"; mmkpi.runKPI result=r status=s / projectUUID = _projectID casout={ caslib="ModelPerformanceData", name=_projectID || ".MM_STD_KPI", promote=True} ; if dim(r.Error) > 0 then do; describe r; print r.Error; end; run; quit;   Here's a very simple example of creating custom KPIs using the new KPI action set. You have lots of flexibility for creating custom KPIs. /* Generate custom KPIs to calculate the rate of change from one time period to next for Misclassification (MISC), Average Square Error (ASE), and Area under the curve (AUC) and append the custom KPIs with the standard MM KPI table */ data work.MM_USER_KPI; set MMKPI.MM_STD_KPI; run; proc sort data=work.MM_USER_KPI; by ModelUUID TimeSK; run; data work.MM_USER_KPI; set work.MM_USER_KPI; by ModelUUID; /* put your custom code here */ MISC_RateChange = (_MISC_ - lag(_MISC_))/lag(_MISC_); ASE_RateChange = (_ASE_ - lag(_ASE_))/lag(_ASE_); AUC_RateChange = (_AUC_ - lag(_AUC_))/lag(_AUC_); run; /* Load your new custom KPIs into an in-memory table. This code ensures MM variables are loaded with varchar format, not character. */ data casuser.MM_USER_KPI (promote=Yes); set work.MM_USER_KPI (rename=(ModelName=ModelNameOLD ModelUUID=ModelUUIDOLD ProjectUUID=ProjectUUIDOLD TimeLabel=TimeLabelOLD)); length ModelName varchar(18); length ModelUUID varchar(36); length ProjectUUID varchar(36); length TimeLabel varchar(2); ModelName=ModelNameOLD; ModelUUID=ModelUUIDOLD; ProjectUUID=ProjectUUIDOLD; TimeLabel=TimeLabelOLD; drop ModelNameOLD ModelUUIDOLD ProjectUUIDOLD TimeLabelOLD; run; /* Integrate the custom KPIs back into the MM project KPI table */ proc cas; _projectID = "&projectUUID."; mmkpi.addCustomKPI result=r / kpiTable = {caslib='casuser', name='MM_USER_KPI'} projectUUID = _projectID ; run; quit;   Additional custom KPI sample code is available on the Open Model Manager samples page on GitHub.   Also check out other examples in our SAS Model Manager documentation.   Discover new SAS Viya 2021.1.1 features   Discover new SAS Model Manager features Simple filtering dashboards plus enhanced search options Customize SAS Model Manager performance monitoring reports with embedded web components pzmm (Python Zip Model Management) module updates New/updated Model Management Macros generating JSON files Enhanced support for R models New Project Wizard Home page Publish Python models to Azure Machine Learning  Simplified workflow-based KPI alerting Updated performance reports ... View more

SAS Workflow Manager is a BPMN standard process management solution used by other SAS products for business process customization. For instance, SAS Model Manager includes access to SAS Workflow Manager allowing customers to create custom model life cycles and other custom business processes. Specifically, SAS Workflow Manager allows users to design, execute, audit, and refine their common business processes to provide: routing and management of tasks requiring user interaction automated triggering of basic actions and alerts Users customize and build business processes that include user tasks, manual gates, and/or approvals for integrating with third-party orchestration tools like Jenkins and other MLOps tools via REST APIs. SAS Workflow Manager automates and versions these tasks, allowing users to work on tasks, be prompted with available options, and receive notifications. Additionally, all tasks are tracked from time started to time completed with a full breakdown by task for governance purposes.   Customizable workflow definition example:   Review the status of a running workflow instance:   With the SAS Model Manager 2020.1.5 April stable release, we included a simplified workflow-based KPI email notification alerting when the project's model performance exceeds the assessment criteria threshold value. The workflow is called "SimpleModelMonitor" and is included out-of-the-box  and will appear in the SAS Workflow Manager definitions list.   This video shows the implemented workflow-based integration.   Sample workflow definition (BPMN) file To access the SAS Workflow Manager workflow, please download the SimpleModelMonitor_April2021.bpmn file from the Samples folder within the Model Management Resources for Open-Source Models GitHub repo.   Setting the preferences For each workflow user wanting to receive an email notification as part of a flow, add the following preferences: Notifications.Directed.Channels and OfflineLocale.DefaultLocale.   To add the preferences use the following REST API call:   POST /preferences/preferences/<user login id> Headers: Content-type: application/json Body: { "items":[{ "id": "Notifications.Directed.Channels", "description": "user notification directed channel default", "application": "notifications", "value": "mail", "version": 1 },{ "id": "OfflineLocale.DefaultLocale", "description": "user notification locale default", "application": "notifications", "value": "en-US", "version": 1 }] }   Configure Email Service In order to send email notifications, it is also necessary to configure the email service. This is done in the Environment Manager.   Set the configuration of the Mail Service with a host:     It is also common to set the fromAddress and the fromPersonalName properties. This applies to all email sent from by the Mail service. Full configuration is for the service is available in the Viya admin guide.   Why integrate SAS Workflow Manager with SAS Model Manager? Including simplified workflow-based KPI alerting is one example of how SAS Model Manager integrates with SAS Workflow Manager. The pairing of these technologies allows users to define, automate, audit and refine business processes. Integration of custom workflows orchestrated through SAS Model Manager allows users to functionally integrate business objectives and inputs into an automated workflow. In turn, adjust considerations based on model accuracy for which models should stay in operation, when users need to be notified, when organizational accountability risk is increased and many other business related impacts are at risk or experienced.    SAS Workflow Manager provides the tools to rapidly integrate fundamental workflow management into business operations and business offerings based on SAS solutions and products.   Want to learn more about SAS Workflow Manager? Interested to learn how to improve model management and workflow management by ensuring your tools, processes and environments are as efficient as possible? Check our the video on the  Automating Model Life Cycle Management with SAS Model Manager and SAS Workflow Manager article.   Also, refer to our SophiaRowland's article Building Performance Monitoring Processes in SAS Workflow Manager for more information about how to use SAS Workflow Manager.   See the SAS Workflow Manager: Help Center documentation for more information about SAS Workflow Manager.   Discover new SAS Viya 2020.1.5 features   Discover new SAS Model Manager features Simple filtering dashboards plus enhanced search options Customize SAS Model Manager performance monitoring reports with embedded web components pzmm (Python Zip Model Management) module updates New/updated Model Management Macros generating JSON files Enhanced support for R models New Project Wizard Home page Publish Python models to Azure Machine Learning  ... View more

How Do I Start My ModelOps Journey with SAS ® Model Manager. Q&A, Slides, and On-Demand Recording   Watch this Ask the Expert session to learn how to put your SAS and open source models into action using SAS Model Manager..    Watch the webinar     Join Diana Shaw as she teaches you how to use SAS Model Manager to store and manage all types of models within one common model repository and organize them within projects and folders. You can also evaluate models for champion model selection, monitor performance, and publish models. Users have a variety of options to interact with SAS Model Manager from the web application, a REST API, SAS code or Python packages available on GitHub. During this webinar you will learn: How SAS Model Manager can unite all types of users in their ModelOps journey to support analytical model deployment and management. How embedded AI and machine learning are used in SAS Model Manager. How continuous integration/continuous delivery (CI/CD) is used in SAS Model Manager.   The questions from the Q&A segment held at the end of the webinar are listed below and the slides from the webinar are attached.   What value is SAS Model Manager generating for customers? Rapidly deploying models to real-time, edge, and batch scoring applications. For instance, one customer is using SAS Model Manager to productionalize over 100+ models per day for use in the real-time alert notification for their manufacturing process. Another customer leverages SAS Model Manager output to reoptimize their production settings using nightly batch processing. SAS Model Manager is largely used by Banks, Financial Services, Telco, and Retailers for managing A/B testing and suggesting the best offer.   What types of businesses are using SAS Model Manager? Banking, Financial Services, Telecom, IT Hosting Services, Manufacturing, Electronics Manufacturing, IT Technology, Automotive, Retailers, Government Agencies, and Consulting Firms. Essentially SAS customers have more than 100 models in production are often SAS Model Manager customers.   What SAS and open-source products are natively integrated into SAS Model Manager? All SAS products (Visual Analytics, Model Studio, Enterprise Miner, Credit Scoring, SAS Studio, Intelligent Decisioning, Event Stream Processing, and others) are natively integrated. Upcoming SAS solutions like SAS Model Risk Management and SAS Risk Solutions are all planning to be natively integrated using SAS Model Manager as the ModelOps framework. SAS Model Manager is fully integrated with SAS 9.4 technology.   SAS Model Manager natively integrates Python and R models and can deploy SAS and open-source models to batch, real-time, and containers for storing and reporting. SAS Model Manager works with the users existing Python and R distributions. We have this all documented and it’s very customizable.   Within this framework using SAS Viya, how do people manage different runtime needs for different models? Ex. different versions of Python, different package configurations, etc. SAS Model Manager’s Model Publish service automatically formats model execution code based on the desired deployment destination. Users can provide and specify different open-source distributions and supporting assets. We have this all documented and it’s very customizable. Additionally, users can also create model execution code that supports databases and restful endpoints.   Recommended Resources Model Management  SAS Model Manager Decisioning SAS Decisioning Community PZMM (Python Zip Model Management SAS Global Forum 2021   Want more tips? Be sure to subscribe to the Ask the Expert board to receive follow up Q&A, slides and recordings from other SAS Ask the Expert webinars.   ... View more

Hi SAS Decisioning Community,   I’m presenting a live “Ask the Expert” webinar on April 13 th , 11 AM – Noon ET entitled How Do I Start My ModelOps Journey With SAS ® Model Manager? Learn to put your SAS and open source models into action using SAS Model Manager. SAS Model Manager is created to store and manage all types of models within one common model repository and organize them within projects and folders. You can also evaluate models for champion model selection, monitor performance, and publish models. Users have a variety of options to interact with SAS Model Manager from the web application, a REST API, SAS code or Python packages available on GitHub. Join this free webinar to learn more! I’ll host a live Q&A session at the end of my presentation You will learn: How SAS Model Manager can unite all types of users in their ModelOps journey to support analytical model deployment and management. How embedded AI and machine learning are used in SAS Model Manager. How continuous integration/continuous delivery (CI/CD) is used in SAS Model Manager. Click here to register for the webinar.   Want more tips? Be sure to subscribe to the Ask the Expert board to receive follow up Q&A, slides and recordings from this and other SAS Ask the Expert webinars.   Can't join the live event? You can view this and other Ask the Expert sessions on-demand here.    ... View more

SAS and Microsoft have formed a long-term strategic partnership around analytics, artificial intelligence, and the cloud. As part of the extensive technology and go-to-market strategic partnership, the SAS Model Manager team established a very simple-to-use and tightly integrated connection between SAS Viya 4 and Microsoft Azure Machine Learning.   Initial development efforts are available with the SAS Model Manager 2020.1.3 release. Users with an Azure Machine Leaning tenant and SAS Model Manager on SAS Viya license can use the Azure AD single-sign-on to simplify connectivity and provide a simple way to register Python models created in SAS Viya directly into Azure Machine Learning. This allows the Python model to leverage a cloud-native model execution environment. SAS Model Manager dynamically discovers available Azure Machine Learning connections. This allows easy access to SAS Viya registered Python models for all Microsoft Dynamics and Power Platforms applications.   The following video walks through a simple and highly integrated SAS and Microsoft connection registering Python models from SAS Model Manager into Azure Machine Learning.   Admin users are able to create an Azure Machine Learning (AML) publishing destination in SAS Environment Manager with simply knowing their AML Subscription ID.   SAS Model Manager offers a simple Publish option that dynamically discovers available AML workspaces to register the Python model. The Python model must contain correctly formatted score code. See the pzmm article for more details about using Python models directly with SAS Model Manager.   To validate the pickle file and score code, use SAS Model Manager’s Publishing Validation option. It will dynamically discover available compute resources. Select a test data. SAS Model Manager passes this data directly into the selected Azure Machine Learning Inference Computer cluster. Once AML runs the data in the container, the compute cluster shuts down automatically. Results of the Publishing Validation process return to SAS Model Manager for review.   Publishing SAS models to Azure Machine Learning arrive with the July 2021.1.3 release.   For more information about using AML as a SAS Model Manager publishing destination, see the SAS Model Manager: User’s Guide documentation.   With Microsoft Azure as the preferred cloud provider for SAS Cloud, we are working together to provide the best experience and value to our customers as they transition to the cloud. We will build deep integrations of our SAS® Viya® Platform with Azure services, develop joint market-ready solutions in different industry verticals, and share in go-to-market activities across the globe. For more information on the strategic partnership, visit www.sas.com/Microsoft.   Discover new SAS Viya 2020.1.3 features Be sure to check out the new Model Manager features in the SAS Viya 2020.1.3 release highlights video.   Discover new SAS Model Manager features Simple filtering dashboards plus enhanced search options Customize SAS Model Manager performance monitoring reports with embedded web components pzmm (Python Zip Model Management) module updates New/updated Model Management Macros generating JSON files Enhanced support for R models New Project Wizard Home page ... View more

With the 2020.1.3 stable release, SAS Model Manager now has a new Home page. This Home page appears the first time you sign in to the web application. It contains the following information: the most recent items that you opened links to the most recent how-to videos and What’s New articles on SAS Communities links to the What’s New, overview, and user documentation for SAS Model Manager Here's a video that shows a user’s experience with the new Home page:   We plan to continue updating and refining this Home page to include notifications and keyword search. In fact, below is a screenshot highlighting some UX enhancements we added since our 2020.1.3 initial release.   Let’s talk a bit about the Stay Connected section.   As you may know, the latest cloud-native release of SAS Viya employs a continuous integration, continuous delivery (CI/CD) process for its release practice. This means customers can choose their release intervals. In addition, new features and enhancements are released each month for all products. Fortunately supporting SAS documentation updates with the same cadence. For some users, seeing the new features/enhancements in standalone blog posts and videos helps users quickly understand how to make use of these new capabilities and stay current.    The Model Management team at SAS created a What's New with SAS Model Manager series, covering  ModelOps and other cutting-edge functionality now available. Each article is available directly from the new SAS Model Manager Home page, which appears the first time you sign into the web application or later on-demand as needed.   These “What’s New” articles include a short explanation of the new feature and how it can help users in their ModelOps journey. Each article aims to include an embedded video showcasing the feature in action with any necessary supporting code or how-to documentation links.   Customers running on the new SAS Viya choose their own release intervals. An RSS feed of all available “What’s New” articles are accessible, assuming an internet connection is available. Users can easily scroll through a dynamic list of available how-to videos and “What’s New” articles posted on the Decisioning board on SAS Communities.   The Stay Connected section of the Home page helps users stay current with quick access to learn about new SAS Model Manager capabilities and helpful how-to videos. This section also includes direct links to the latest SAS Model Manager marketing overview, What’s New, and SAS Model Manager: User’s Guide documentation.   Discover new SAS Viya 2020.1.3 features Be sure to check out the new Model Manager features in the SAS Viya 2020.1.3 release highlights video.   Discover new SAS Model Manager features Simple filtering dashboards plus enhanced search options Customize SAS Model Manager performance monitoring reports with embedded web components pzmm (Python Zip Model Management) module updates New/updated Model Management Macros generating JSON files Enhanced support for R models New Project Wizard Publish Python models to Azure Machine Learning  Simplified workflow-based KPI alerting Generate standard and custom KPIs Updated performance reports ... View more

Lucas De Paula, Customer Advisor at SAS, put this great video together providing an overview of SAS Model Manager in Spanish. Enjoy!   Lucas De Paula, Asesor de Clientes de SAS, armó este excelente video que brinda una descripción general de SAS Model Manager en español. ¡Disfrutar!   ... View more

Watch ScottLindauer walk through the use of pzmm to easily operationalize his Python model without recoding using SAS Model Manager.   Here is a link to the notebook ScottLindauer used in this demo.   For more information about pzmm, please check out What's new with SAS Model Manager? pzmm (Python Zip Model Management) module updates.     ... View more