Model Cards were added to SAS Model Manager in July 2024 and our team is continuing to add enhancements and broaden their scope. The first iteration of our Model Cards focused on prediction and classification models developed in SAS Model Studio and models developed using Python. With the release of SAS Viya 2024.11, we are now adding support to build a complete model card for SAS classification and prediction models trained in SAS Studio through updates to our Import Model macros.
The SAS Model Manager macros provide SAS programmers the ability to interact with SAS Model Manager from SAS code. SAS programmers just need to add values for the various parameters and then run the code. The Import Model Macro and the Import ASTORE Model Macro have been used to take models developed in SAS code and register them directly into SAS Model Manager, along with their various metadata files. And with our latest release, this macro now generates all the files to start building a complete model card.
Within these macros, two new parameters were added, and one existing parameter is now required. The TRAINTABLE parameter was previously optional and specifies the model’s training data. This parameter is now required and is used to calculate relative importance for the model’s input variables. The two new parameters, SENSITIVEVAR and VARIMPTABLE, are both optional. The new SENSITIVEVAR parameter is used to specify a variable to use for assessing model fairness. For each value of the sensitive variable, SAS calculates the model’s accuracy and average prediction to determine if the model is treating classes differently. This uses SAS’s assess bias action under the hood. You can read more about the assess bias action in the documentation. Since tree-based models have a built-in mechanism for determining variable importance, the VARIMPTABLE parameter is used to specify the variable importance table from a tree-base model.
To have the most robust model card, we recommend that you split your data into training, testing, and validation sets prior to model training. When leveraging the macro, specify the training percentage using the SAMPLEPCT parameter and the testing percentage via the SAMPLEPCT2 parameter. Splitting your data into training, testing, and validation sets allows more points of comparison within the model card.
Once the models are registered into SAS Model Manager, users can add additional information through their interactions with the model and project in SAS Model Manager. You can learn more about these steps in the last article on Model Cards.
To see the updated Import Model and Import ASTORE Model macros in action, check out this short demo:
Next for the model card, we are evaluating updating PROC REGISTERMODEL so SAS programmers on SAS Viya Workbench can register a model into SAS Model Manager and generate a complete model card. We are also evaluating exporting the model card as a PDF. But what would you like to see next for the SAS Model Card? Leave a comment below!
