Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

Linear regression vs nonlinear regression performance measurement

Reply
Occasional Contributor
Posts: 19

Linear regression vs nonlinear regression performance measurement

Fellows,

 

I am wondering that how do you evaluate your regression model. R square value may be the most popular one. However, it is not appropriate when it comes to nonlinear regression model, especially when one wants to compare linear regression VS nonlinear regression model.

My question is: is there any systematic performance measurement out there for linear or nonlinear regression models? Any comment is highly appreciated and thanks in advance!

 

Trusted Advisor
Posts: 1,615

Re: Linear regression vs nonlinear regression performance measurement


However, it is not appropriate when it comes to nonlinear regression model, especially when one wants to compare linear regression VS nonlinear regression model.
I'm not sure why you say this. I would certainly compare the R-squared on a nonlinear model to the R-squared on a linear model on the same data.
SAS Employee
Posts: 23

Re: Linear regression vs nonlinear regression performance measurement

There is a difference between a non-linear fit using linear regression (ie, including higher order terms) and non-linear regression.  This article explains it well:

http://blog.minitab.com/blog/adventures-in-statistics/what-is-the-difference-between-linear-and-nonl...

 

This aticle does a good job of explaining why R-squared is not a valid error assessment for non-linear regression.

 

http://blog.minitab.com/blog/adventures-in-statistics/why-is-there-no-r-squared-for-nonlinear-regres...

 

Super Contributor
Posts: 336

Re: Linear regression vs nonlinear regression performance measurement

In the context of data mining or predictive modeling you care about how accurate are your predictions.

I would look at misclassification and ROC index if I am predicting a binary or nominal target, and at average square error for any other target or response.

I hope it helps!

 

Occasional Contributor
Posts: 19

Re: Linear regression vs nonlinear regression performance measurement

[ Edited ]

Miguel,

 

I have a question for you. Suppose I am going to implement these mentioned equations below (FB, MG, etc) to evaluate my predictive model in SAS code node. My model is a linear model using stepwise selection like following:

 

data mydata;
set &EM_IMPORT_DATA(in=a) &EM_IMPORT_VALIDATE(in=b) &EM_IMPORT_TEST(in=c);
if a then _partition="_Train";
else if b then _partition="_Valid";
else if c then _partition="_Test";
run;
proc glmselect DATA=mydata namelen=100;
effect MyPoly = polynomial(A B... F/degree=5);
model Y = MyPoly / selection=stepwise(select=SL SLE=0.05 SLS=0.05);
partition rolevar=_partition(TEST='_Test' TRAIN='_Train' VALIDATE='_Valid');
run; 

How can I perform these measurements in SAS? Is there any way I can calculate the residuals without knowing the prediction model? Thanks!

Occasional Contributor
Posts: 19

Re: Linear regression vs nonlinear regression performance measurement

Fellows,

 

Thanks for your replying. After do some search online, I happened to get some ideas from " Technical Descriptions and User’s Guide
for the BOOT Statistical Model Evaluation Software Package, Version 2.0"
. The authors gave some statistical performance measurement as basis for a quality model evaluation, including following:

QQ截图20150925143105.jpg

Where Cp is the predicted value and Co is the observation value, and C bar is the mean value. 

 

Any comment?

Ask a Question
Discussion stats
  • 5 replies
  • 463 views
  • 0 likes
  • 4 in conversation