October was a BIG month for new Python functionality in Python-sasctl and SAS Model Manager. SAS Model Manager is SAS’s leading solution for helping organizations manage, test, audit, deploy, monitor, and search their models. And yes, this includes Python models. Data scientists using Python lean on the Python-sasctl package to interact with SAS Model Manager and ensure that their models are included within a wider ModelOps and MLOps strategy. And we’ve release three new capabilities in the v1.10.0 release of python-sasctl, including a new function to assess model bias, Key Performance Indicator (KPI) and Hyperparameter integration, and support for automatic score code and metadata generation for TensorFlow Keras models. Let’s dive into each.
Assess Bias for Python Models
Python-sasctl now provides a function to assess the bias and fairness of Python models! This function leverages SAS’s Assess Bias action to take pre-scored data and determine if there are differences in the model’s accuracy and predictions across values of the specified sensitive variable, such as race or gender. The new function within python-sasctl can return the following results:
Group metrics, which contain the model assessment statistics and average prediction for each level of the sensitive variable.
Max differences, which contain the maximum absolute difference between all pairs of levels from the sensitive variable.
Bias metrics, which contain the bias metrics for the predictive model.
The results of these tests can help support that a model is treating groups similarly or let the data scientist know that they need to take action to build a fair model.
The new assess model bias function can be called using the following parameters:
pzmm.JSONFiles.assess_model_bias(
score_table,# table scored by the model
actual_values,# variable with the actual target valables
sensitive_values=,# variable to assess for bias, can add more as a list
prob_values,# variables for predicted probabilities (classification)s
levels,# same order as prob_values
json_path,# directory for json files, optional
return_dataframes=True# returns group metrics, bias metrics, and max differences)
Key Performance Indicator (KPI) and Hyperparameter Integration
SAS Model Manager generates Key Performance Indicators (KPIs) automatically based on performance monitoring definitions. However, we can extend the KPIs tracked by SAS Model Manager through the addition of custom KPIs. From Python, we can perform local tests on models we’ve registered to SAS Model Manager, then pass up the resulting values as custom KPI values. For certain python models, sasctl will also generate a json file containing the hyperparameters of the model, making them easily accessible for future use.
Keras is the high-level API of the TensorFlow platform. It provides an approachable, highly productive interface for solving machine learning (ML) problems, with a focus on modern deep learning. And now, python-sasctl can generate the metadata files and score code to run models developed using TensorFlow Keras in SAS Model Manager! To learn more, see this example and the following demo:
TensorFlow Keras.mp4
Video Player is loading.
Current Time 0:00
/
Duration 1:32
Loaded: 0%
0:00
Stream Type LIVE
Remaining Time -1:32
1x
Chapters
descriptions off, selected
captions settings, opens captions settings dialog
captions off, selected
default, selected
This is a modal window.
Beginning of dialog window. Escape will cancel and close the window.
End of dialog window.
This is a modal window. This modal can be closed by pressing the Escape key or activating the close button.
Four days of inspiring keynotes, product reveals, hands-on learning opportunities, deep-dive demos, and peer-led breakouts. Don't miss out, May 6-9, in Orlando, Florida.
This site uses cookies and related technologies for site operation, analytics and third-party advertising purposes, as described in our SAS Privacy Statement. You may consent to our use of these technologies, reject non-essential technologies or further manage your preferences. To opt out of SAS making information relating to cookies and similar technologies available to third parties for advertising purposes, select "Required only." To exercise other rights you may have related to cookies, select "Manage cookies."
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
