Model Interpretability for Models with Uninterpretable Features
The purpose of this blog is to provide a method in SAS Model Studio to make use of model interpretation tools for a model based on generated, non-interpretable features.
SAS Viya users, specifically those who build machine learning models using Data Mining and Machine Learning tools, likely have used or at least are aware of model interpretability features to gain insight into black-box models. Although model interpretability can be deployed through code, for example using SAS Studio, this blog will focus on the point-and-click interface for SAS Viya, Model Studio. The model interpretability features in Model Studio use tables and plots to provide the user with insights and interpretable information about how the model makes predictions which are otherwise typically not available for black-box, machine learning models.
Machine Learning models often trade interpretability for accuracy. Neural networks, forests, gradient boosting models, and support vector machines are perfect examples of machine learning models that can provide accurate predictions but are considered black-box models. Model Interpretability tools do not magically make the black-box model directly interpretable (in the same way a regression model or decision tree are interpretable), but they do provide insights such as the relative importance of inputs in the model and how the model is making predictions based on the inputs. The following tools are all available for model interpretability: a Variable Importance table, Partial Dependance (PD) plots, Individual Conditional Expectation (ICE) plots, a Local Interpretable Model-Agnostic Explanation (LIME) method, and a Shapley Explainer (HyperSHAP) method. All these tools, aside from the variable importance table, are visual plots to help explain model predictions. The tools can provide insight into the model globally (variable importance and PD plots) or locally for individual observations in the data (ICE, LIME and HyperSHAP plots). Resources on model interpretation are provided below.
Another common practice in predictive modeling is to generate new features from the available data and use these new features as inputs into the model. New features can be created by doing straight forward mathematical transformations on inputs, such as taking the log or squaring an input, but can also be based on more advanced mathematical transformations which may be constructed using all inputs. One node in Model Studio that generates features using advanced mathematical transformations is the Feature Extraction node. This node generates new features using methods such as principal components, singular value decomposition, and autoencoders. All these methods, however, generate new features based on combinations of original variables in the data set, and these newly generated feature are typically not interpretable on their own. The new features are often linear or nonlinear combinations of all original inputs. Resources on feature creation are provided below.
So, what happens when an analyst wants to gain insight into a model based on such uninterpretable features? Let’s walk through an analysis which is based on telecom data used to predict customer churn. Below is the initial Model Studio pipeline.
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
The imputation node solves the missing data issue. The feature extraction node generates new features. In this case, the feature extraction node is used to create principal components; reducing the dimensionality of the input space, by creating new features based on linear combinations of the original inputs. (By default, principal components are constructed only using interval inputs, where nominal and ordinal inputs would be ignored by the node. This will be relevant later.) The principal components and the original categorical inputs are used to construct a gradient boosting model. Gradient boosting models are powerful machine learning models, that are based on an ensemble of individual decision trees. They are not directly interpretable, and thus are considered by most data scientists to be black-box models. To gain insight into how the gradient boosting model makes predictions, an analyst may choose to use model interpretability tools. However, as can be seen below, these model interpretability results would be based on the principal components. Below are the PD, ICE, and LIME plots for the gradient boosting model used in the analysis:
Notice that the plots are based on the uninterpretable principal components (for example, PC10 and PC14) as well as categorical variables ignored by the feature extraction node. The other model interpretability tools, namely the variable importance table and the HyperSHAP plots, would similarly be based on a combination of the original categorical inputs and principal components. The trick to have these same model interpretability tools based only on original inputs, is very simple! Build an ensemble model using only the gradient boosting model and have the ensemble produce the desired model interpretation tools. This is done by connecting the gradient boosting model to an Ensemble node. This trick, of course, works for any machine learning model. Not just gradient boosting models. The updated pipeline will appear as follows:
When model interpretation tools are turned on for the ensemble model, they will only be based on original inputs. Keep in mind that in this case, the ensemble model is just the original gradient boosting model. Nothing in the model itself changes. It is still based on the same exact combination of categorical inputs and principal components. It is the same gradient boosting model, making the same exact predictions. The only difference is that the model interpretability tools will all be based on original inputs only. Below are the same plots as above but being generated from the Ensemble node.
Now the analyst can gain insight into the gradient boosting model by having model interpretation based only on the original, interpretable inputs.
For more on model interpretability: https://www.youtube.com/watch?v=6LcyVSLwVck
https://www.youtube.com/watch?v=ZjhvxsE9n4I&t=19s
For more on generating new features: https://www.youtube.com/watch?v=D9_jXt_sdX8
