topic Re: R-Square on test and training dataset in Statistical Procedures

R-Square on test and training dataset

csetzkorn — Sat, 11 Feb 2017 10:19:26 GMT

Using this code snippet:

PROC GLMSELECT DATA = WORK.Training TESTDATA = WORK.Test;
MODEL
.... / selection=none stb showpvalues;
ods output
"Fit Statistics" = WORK.Model_Fit
"Parameter Estimates" = WORK.ParameterEstimates
Nobs = WORK.Nobs;
RUN;

The dataset WORK.Model_Fit contains:

Root MSE
Dependent Mean
R-Square
Adj R-Sq
AIC
AICC
SBC
ASE (Train)
ASE (Test)

Is the R-Square measured on the training dataset? Is it possible to obatin the R-Square fro the training and test dataset?

Re: R-Square on test and training dataset

PGStats — Sun, 12 Feb 2017 05:02:03 GMT

One option is to calculate R square yourself from the residuals as 1 - USS(resid)/CSS(dependent) for each data subset.

Re: R-Square on test and training dataset

Ksharp — Sun, 12 Feb 2017 12:22:36 GMT

You can use SCORE or CODE statement to score new dataset and calculated R-square.
OR
proc glmselect;
model.........
run;

proc reg.......
model y=&_GLSIND ;
........

to get that R-square.

Re: R-Square on test and training dataset

csetzkorn — Sun, 12 Feb 2017 13:31:01 GMT

Thanks, sorry this does not make much sense. I am able to use proc plm. I could use this to potentially calculate the r-square against the test dataset myself. The final aim is to put it into a macro variable.

Re: R-Square on test and training dataset

PGStats — Sun, 12 Feb 2017 20:49:34 GMT

Here is an example:

/* Split a dataset into training and test subsets */
data splitClass;
set sashelp.class;
if mod(_n_, 3) > 0 then role = "training";
else role = "test";
run;

proc glmselect data=splitclass;
class sex;
model weight = sex height / selection=none;
partition rolevar=role(test="test" train="training");
output out=outClass residual=resWeight;
run;


proc sql noprint;
select 1 - uss(resWeight)/css(weight) as rsquare format=7.4
into :r2_training trimmed
from outClass where role="training";
select 1 - uss(resWeight)/css(weight) as rsquare format=7.4
into :r2_test trimmed
from outClass where role="test";
quit;

%put &=r2_training &=r2_test;