Get Hands-on With ModelOps
Are you ready to get hands-on with ModelOps? We’re offering a free trial of SAS® Viya® so you can explore the capabilities. Discover how to apply it – deploy your own models and test scenarios. To get started, take a look at the guide:
- Getting Started - registering and accessing the free SAS Viya trial
- Exploring SAS Viya - reviewing the ModelOps interfaces on SAS Viya
- Building Models - training models using pipelines
- Deploying Models - putting models into production
- Testing Models - ensuring models work as expected within production
- Monitoring Model Performance - ensuring models work as expected over time
- Additional Challenges & Scenarios - applying and extending your knowledge
Getting Started
First, register for your SAS Viya Free Trial environment. To register for the free trial, you will need to use a commercial email account and you can sign up for the free trial once per year. This trial environment will be available for 2 weeks, giving you plenty of time to explore. You will receive an email once the environment is ready. Use the information provided within this email to login to SAS Viya and click the “i” information icon.
Next, watch the last 2 videos. These short videos show how a Data Scientist builds modeling pipelines and then deploys a champion model into a production system.
Exploring SAS Viya
Now, let's explore the ModelOps interfaces within SAS Viya. Start by opening the applications menu.
We will be using Build Models and Manage Models interfaces within this guide.
You can explore these interfaces at your leisure, as inspired by the 2 Exploration videos.
Building Models
Next, start exploring the Customer Churn project in Build Models. From this interface, you can train model pipelines to turn data into predictions. These pipelines all start with data, which can feed data processing nodes to clean and prepare the data for modeling. Next, supervised modeling nodes train machine learning models to predict the event of interest.
⭐ Hint: Double click a project tile to open it.
The name of the table containing the training data can be found in the right-hand pane within the Data tab. Now, explore the Type and Role columns. The Role specifies how the variable will be used within the pipeline. For example, the variable with the Target Role represents the one the model pipeline will try to predict.
Click the Pipelines tab and select a ‘node’ (box) in a pipeline to see the configured properties in the right-hand pane or right-click to open the options menu to access the results.
⭐Hint: Use these icons:
Click the Pipeline Comparison and Insights tabs. Notice the menu options in the Pipeline Comparison tab: Register Models and Manage Models. The Register Models option will register the selected model pipelines into the Manage Models interface.
Deploying Models
From Manage Models in the Applications Menu, open the Project pane (3rd item of the left menu) and then open the Customer Churn project. This project contains a treasure-trove of metadata and information about the project, such as the variables used, model evaluation properties, project history and much more. This interface can also be used to deploy, test, and monitor models.
Explore the information and actions available on each tab.
Click a model name to open it and view the scoring code and model properties.
Next, click the Publish button and publish a model to the CAS (cas) cloud destination. Publishing a model makes it available for production scoring.
The notification reports the status of the publish action:
Testing Models
Once the model has been published successfully, a scoring publishing validation test will be available to execute the score code in the production environment. This validates that our model is working as expected and ready for production workloads. Navigate back into the project by closing the notification dialogue and clicking the name of the project near the top of the screen. On the Scoring tab, click Publishing Validation and then the model name with today’s date.
Click the folder icon to set the Input table to CUSTOMERCHURNPR_1_Q1 and run the test.
When it completes, click the icon under Results. The Log contains details of the scoring test.
Monitor Model Performance
Models become less accurate over time, in a process known as model decay. To ensure that models are continuing to work as expected, monitoring the model's performance over time is critical. Click the Performance tab to view performance reports and their explanations.
⭐Hint: If a report appears blank, click refresh:
Click the Edit button at the top right of the screen for the settings to create the reports. Notice the Tables settings.
Go to Manage Data in the Applications Menu and search for tables starting with CustomerChurnPR to locate the performance data sets used to create the reports.
Close the window, return to Manage Models, click Home, and view the links under Stay Connected to find the latest updated news and postings on ModelOps.
Additional Challenges
Now that you are comfortable with the interface and common ModelOps tasks, try improving the existing Customer Churn project in Build Models.
Create a new model pipeline by clicking 
Use the gallery of nodes to prepare data and train models or start with a pre-built pipeline template if you prefer.
Open the Pipeline Comparison tab of Model Studio and register a model into Model Manager.
From Manage Models in the Applications Menu, open the Project pane (3rd item of the left menu) and then open the Customer Churn project. Your model should now be available within the project models.
Challenge Your Skills in a New Scenario
Next, let's apply what we've learned in a new scenario: using sensor readings on truck performance metrics to determine when to schedule maintenance for trucks in the fleet.
Open the Fleet Maintenance Maintenance Prediction project in Build Models.
Adjust the pipelines to your liking and run them.
Register a champion model into the model repository.
⭐ Hint: Open the Pipeline Comparison tab and click
From Manage Models, open the Project pane and then open the Fleet Maintenance Maintenance Prediction project. First, open the model and publish the model to the CAS or MAS location.
⭐ Hint: Click the 'Replace item' toggle if an error occurs.
Next, head back into the project and run a scoring publishing validation test using the ‘Fleet_train’ data source within the Scoring tab.
Now, create a new performance definition to create the performance reports. From the Performance tab, create a new definition or edit an existing performance definition. Walk through the New Performance Definition wizard and fill out the required information.
For input data, use the Public library with datasets starting with fleetp as shown above.
Finally, save and run the performance definition and view the results when it finishes.
Thank you for test driving the SAS Viya Free trial environment! Now you have trained, registered, reviewed, deployed, tested, and monitored your own machine learning models. Ready to learn more? Learn all the basics of SAS Model Manager in under a hour from our recent free webinar: How Do I Use SAS Model Manager? If you are ready to be an expert, try the Managing Models in SAS® Viya® course from our excellent education team!
