The Model Summary section of the model card includes information about the model during training. This data is automatically collected and registered with machine learning models trained in SAS Model Studio. To support more models, methods for generating and registering these metrics from SAS Data and AI Studio (formerly SAS Studio) and the python-sasctl package were added. To support even more models, including external models, SAS Model Manager can now collect information about the model training process to create metrics for the Model Summary section.
For models where metrics can be generated and the complete model card populated, a button will appear on the model card. This will specifically be for tabular binary classification, nominal classification, and interval prediction models that are within a project and aren’t read-only. If your model is read-only, you can create a new version of the model from the Versions tab that will be editable. If your model is not in a project, you can move your model to a project.
Clicking the button to generate a full model card will open a dialog to collect information about the model at the time of training. Some important things to keep in mind are:
- The process will require the table used to train the model.
- Providing a training table that is already scored by the model is required for models that do not have a score code type of Data Step, a flavor of DS2, Python or R.
- A prescored training table can be supplied for models with a score code type of Data Step, a flavor of DS2, Python, or R, which will speed up generation.
- To calculate metrics across the training and testing or training, testing, and validation splits, the training table or scored training table should have a column for the partition variable. If a partition variable is not supplied, the entire training table is treated as the training partition level and measures of generalizability (which compare the training partition level to the validation and testing partition levels) will not be available.
- Bias assessment can be included for variables marked as “Assess for Bias”.
- Which output information you’ll need to supply will depend on the type of model.
Generating the metrics will take a few moments, but in no time at all you’ll have a model card with a full Model Summary! If you need to regenerate the metrics for any reason, head over the Properties for the model and then the new Model Card Settings section. Here, you can regenerate the metrics, but this will create a new version of the model. Regenerating is only recommended if a user specified a parameter to the generation process incorrectly. Regeneration is not available for models without a full model card (because how can you re-generate what was never generated?)
Check out the process for a Binary Classification Model, Interval Prediction Model, and Nominal Classification Model, including specifying partition, using prescored data, and adding bias assessment:
And alongside our new lighter model card and model card PDF export, that rounds out our batch of model card enhancements! What would YOU like to see next for the model card? Add a comment below to let us know!
