topic LIME in SAS without Viya in SAS Data Science

LIME in SAS without Viya

Ullsokk — Fri, 01 Nov 2019 09:06:06 GMT

I have a gradient boosting model that I want to to do some LIME on. But we don't have Viya yet. I have tried using python, but it seems I have to run the model in python as well, so the model I get LIME plots from is not identical to my SAS model.

Is there a way to run LIME in sas without Viya?

Re: LIME in SAS without Viya

RalphAbbey — Fri, 01 Nov 2019 15:03:12 GMT

Unfortunately to my knowledge the only way would be to write a SAS macro that performs the steps of LIME yourself. This might be a little short on details, but cover these steps cover tabular LIME:

1) Run an analysis to obtain the variance for each input variable

2) Choose an observation which you wish to run LIME on

3) Create N samples, where the observations are sampled from a Gaussian normal distribution with mean equal to the variable value for observation from 2) and the variance equal to the variance from 1)

4) Calculate weights for each observation using the RBF kernel between the observation in 2) and the generated observations in 3)

5) Score this new data set using the model

6) Run a weighted regression on the scored data set using the output of the model as the dependent variable and the inputs of the model as the independent variables

Let me know if this helps.

Re: LIME in SAS without Viya

Ullsokk — Mon, 04 Nov 2019 12:07:19 GMT

Thank you! It helps knowing there is no easy sollution. Looks like I will be running the models in python until we are on Viya. Implementing LIME by hand does sound a bit excessive, can't risk spending that much work on something I am not even sure I can pull off.