topic Re: Is it possible to produce residuals of a regression model source dataset in SAS in SAS Data Science

Is it possible to produce residuals of a regression model source dataset in SAS

FeliciaFung — Thu, 01 Aug 2019 13:48:24 GMT

Hi everyone!

I am using SAS miner workstation.

I need to extract the residuals of a regression model and feed them into a new model. Is it possible?

Many thanks!

Re: Is it possible to produce residuals of a regression model source dataset in SAS

Reeza — Thu, 01 Aug 2019 14:51:00 GMT

Yes it's possible. There's usually an option to create an output data set and that should have the predicted values and residuals or you could score the data in another node again to get the predicted vs residuals and then pipe that into your next step.

Re: Is it possible to produce residuals of a regression model source dataset in SAS

FeliciaFung — Thu, 01 Aug 2019 15:08:08 GMT

Thanks for answering. But I could not find the option which can create the output data set of residuals?

Re: Is it possible to produce residuals of a regression model source dataset in SAS

Reeza — Thu, 01 Aug 2019 15:45:34 GMT

Try the scoring option then? I'm very surprised it wouldn't be in the default output data set. What variables are in that data set?

Re: Is it possible to produce residuals of a regression model source dataset in SAS

FeliciaFung — Thu, 01 Aug 2019 15:51:18 GMT

Those are financial data such as CAR and turnovers

Re: Is it possible to produce residuals of a regression model source dataset in SAS

FeliciaFung — Thu, 01 Aug 2019 15:52:54 GMT

And I would like to ask how to obtain output data set?

Re: Is it possible to produce residuals of a regression model source dataset in SAS

Reeza — Thu, 01 Aug 2019 16:01:20 GMT

Residuals are listed as included in the output data set from the documentation:

https://documentation.sas.com/?docsetId=emref&docsetTarget=n1jqzz8cssr9m2n1ktx2iyv87q56.htm&docsetVersion=14.3&locale=en#p1bjbum4ergaobn1o3r8ck6zof9o

The Regression node must run before you can view the Regression node output data sets. After a successful run, ensure that the Regression node is selected in the Diagram Workspace, then select the button to the right of the Exported Data property. This opens the Exported Data — Regression window, which lists the data sets that the Regression node outputs.

Two types of data sets are output from the Regression node:

Scored Data Sets — are the scored training, validation, and test data set that contains original inputs and scores (prediction, residuals, classification results, and so on).

Here is a list of the name prefixes of the scores in the scored data sets:
- BL_ or BP_ — best possible profit or loss for any of the decisions
- CL_ or CP_ — profit or loss that is computed from the target value
- D_ — level of the decision that is chosen by the model
- E_ — error function
- EL_ or EP_ — expected profit or loss that is chosen by the model
- F_ — normalized category that the case comes from
- I_ — normalized category that the case is classified into
- IC_ — investment cost
- P_ — posterior probabilities for categorical targets or predicted values for interval targets
- R_ — residuals
- ROI_ — return on investment
- U_ — un-normalized category that the case is classified into.
When the model selection methods of forward, backward, or stepwise are applied, the Regression node automatically changes the model roles from input to rejected for variables that are not included in the regression model.

Re: Is it possible to produce residuals of a regression model source dataset in SAS

FeliciaFung — Thu, 01 Aug 2019 16:29:35 GMT

Thanks, I opened the exported data property after running the basic regression but the scored data are not in the default output data set. I will try to add the scored node between the regression and next model as you mentioned then.

Re: Is it possible to produce residuals of a regression model source dataset in SAS

Reeza — Thu, 01 Aug 2019 16:31:35 GMT

Are you saying you don't see a data set that has a variable named R_ in it?

Re: Is it possible to produce residuals of a regression model source dataset in SAS

FeliciaFung — Thu, 01 Aug 2019 16:32:49 GMT

yes

Re: Is it possible to produce residuals of a regression model source dataset in SAS

FeliciaFung — Thu, 01 Aug 2019 16:37:08 GMT

Oh sorry I saw the variable residual but I am not able to extract it

Re: Is it possible to produce residuals of a regression model source dataset in SAS

Reeza — Thu, 01 Aug 2019 16:49:40 GMT

What do you mean by 'extract it'?

Re: Is it possible to produce residuals of a regression model source dataset in SAS

FeliciaFung — Thu, 01 Aug 2019 16:53:09 GMT

which means to produce those residuals into a new dataset or to have it in the original dataset

Re: Is it possible to produce residuals of a regression model source dataset in SAS

Reeza — Thu, 01 Aug 2019 16:54:43 GMT

You can use a task on the output data set to filter/extract only columns of interest? Use the output dataset name as your input or add it to the process flow or a new process?

Re: Is it possible to produce residuals of a regression model source dataset in SAS

FeliciaFung — Thu, 01 Aug 2019 17:04:48 GMT

But I am not sure why I cannot save the output. I can only 'plot' after opening the exported data property

Re: Is it possible to produce residuals of a regression model source dataset in SAS

Reeza — Thu, 01 Aug 2019 17:07:33 GMT

The dataset should exist in your work library, you now have the name. You should be able to use it as any other table.

Re: Is it possible to produce residuals of a regression model source dataset in SAS

FeliciaFung — Thu, 01 Aug 2019 17:12:05 GMT

Thanks so much

Re: Is it possible to produce residuals of a regression model source dataset in SAS

Urban_Science — Thu, 01 Aug 2019 21:41:06 GMT

I'm sorry to chime in after this thread has been marked as "Solved!", but the easiest way to use the residuals from a regression model as the target of another model is to:

1. Connect a Metadata node after the regression node.

2. Click on "..." In the Train Property value

3. Find the column that has Role = "Target" and change New Role to "Rejected"

4. Find the column that has Role = "Residual" and change New Role to "Target" (This column should also start with R_ like Reeza said)

5. Click "OK" (in the lower right corner of the window)

Any nodes after the Metadata node will now use that residual column as the target variable.

Attached is an example picture of the options.

Re: Is it possible to produce residuals of a regression model source dataset in SAS

Reeza — Thu, 01 Aug 2019 22:15:38 GMT

Thanks for chiming in, always best to know the most efficient way to do something. I don't have access to EM anymore so most answers are off memory or documentation.