topic Linear regression vs nonlinear regression performance measurement in SAS Data Science

Linear regression vs nonlinear regression performance measurement

viva0521 — Tue, 22 Sep 2015 20:45:03 GMT

Fellows,

I am wondering that how do you evaluate your regression model. R square value may be the most popular one. However, it is not appropriate when it comes to nonlinear regression model, especially when one wants to compare linear regression VS nonlinear regression model.

My question is: is there any systematic performance measurement out there for linear or nonlinear regression models? Any comment is highly appreciated and thanks in advance!

Re: Linear regression vs nonlinear regression performance measurement

PaigeMiller — Wed, 23 Sep 2015 12:11:04 GMT

However, it is not appropriate when it comes to nonlinear regression model, especially when one wants to compare linear regression VS nonlinear regression model.

I'm not sure why you say this. I would certainly compare the R-squared on a nonlinear model to the R-squared on a linear model on the same data.

Re: Linear regression vs nonlinear regression performance measurement

BrettWujek — Wed, 23 Sep 2015 12:17:52 GMT

There is a difference between a non-linear fit using linear regression (ie, including higher order terms) and non-linear regression. This article explains it well:

http://blog.minitab.com/blog/adventures-in-statistics/what-is-the-difference-between-linear-and-nonlinear-equations-in-regression-analysis

This aticle does a good job of explaining why R-squared is not a valid error assessment for non-linear regression.

http://blog.minitab.com/blog/adventures-in-statistics/why-is-there-no-r-squared-for-nonlinear-regression

Re: Linear regression vs nonlinear regression performance measurement

M_Maldonado — Wed, 23 Sep 2015 16:20:53 GMT

In the context of data mining or predictive modeling you care about how accurate are your predictions.

I would look at misclassification and ROC index if I am predicting a binary or nominal target, and at average square error for any other target or response.

I hope it helps!

Re: Linear regression vs nonlinear regression performance measurement

viva0521 — Fri, 25 Sep 2015 18:45:00 GMT

Fellows,

Thanks for your replying. After do some search online, I happened to get some ideas from " Technical Descriptions and User’s Guide
for the BOOT Statistical Model Evaluation Software Package, Version 2.0". The authors gave some statistical performance measurement as basis for a quality model evaluation, including following:

Where Cp is the predicted value and Co is the observation value, and C bar is the mean value.

Any comment?

Re: Linear regression vs nonlinear regression performance measurement

viva0521 — Fri, 25 Sep 2015 19:04:12 GMT

Miguel,

I have a question for you. Suppose I am going to implement these mentioned equations below (FB, MG, etc) to evaluate my predictive model in SAS code node. My model is a linear model using stepwise selection like following:

data mydata;
set &EM_IMPORT_DATA(in=a) &EM_IMPORT_VALIDATE(in=b) &EM_IMPORT_TEST(in=c);
if a then _partition="_Train";
else if b then _partition="_Valid";
else if c then _partition="_Test";
run;
proc glmselect DATA=mydata namelen=100;
effect MyPoly = polynomial(A B... F/degree=5);
model Y = MyPoly / selection=stepwise(select=SL SLE=0.05 SLS=0.05);
partition rolevar=_partition(TEST='_Test' TRAIN='_Train' VALIDATE='_Valid');
run;

How can I perform these measurements in SAS? Is there any way I can calculate the residuals without knowing the prediction model? Thanks!