topic Re: How to compare change in regression results from two models: one with N and one with N-1 in Statistical Procedures

How to compare change in regression results from two models: one with N and one with N-1

jlp2ba — Thu, 10 May 2018 20:34:49 GMT

Hello,

Release: 3.7 (Enterprise Edition)

Java Version: 1.7.0_151

I want to compare the change in F-value, R-squared value, t-value, etc of two regression models (one predictor, one criterion in each). Right now I'm running two separate regression models as my way of comparison, but I'd like to see output that details the change as a result of running the second model which removes one observation that had high Cook's D.

My code looks like this right now:

title2 'Model1 with DMUS_pre only';
proc reg data = wombat.popedataset2 plots(label)=(RStudentByLeverage CooksD);
  model dmus_post = dmus_pre / ss1 ss2 stb clb corrb influence r cli clm;
run;

*REMOVE HIGH COOK'S D STUDENT;
title2 'Model2 with DMUS_pre only MINUS ONE PARTICIPANT';
proc reg data = wombat.popedataset2 plots(label)=(RStudentByLeverage CooksD);
  model dmus_post = dmus_pre / ss1 ss2 stb clb corrb influence r cli clm;
  where participant_ ^= 804; 
run;

Thank you!

Re: How to compare change in regression results from two models: one with N and one with N-1

PaigeMiller — Thu, 10 May 2018 21:52:10 GMT

I'm not aware of any statistical test to compare a regression on n observations with a regression run on n-1 observations. In fact, the idea of doing a statistical test in this instance seems to me to be unnecessary and improper. In fact, except for some trivial situations, the R-squared and t-value are different, not in the statistical hypothesis testing framework (which doesn't make sense here), but just different because they are different numbers.

Re: How to compare change in regression results from two models: one with N and one with N-1

jlp2ba — Thu, 10 May 2018 22:35:22 GMT

PaigeMiller,

Thank you for your reply. I'm a novice at this so it may be a poor question. I've just noticed that removing the high-influence observation improved the fit and increased the r-squared value and F-value. I was wondering if there was a way to identify the change in r-squared, or if I can do model 2 r-sq minus model 1 r-sq.

Re: How to compare change in regression results from two models: one with N and one with N-1

PGStats — Fri, 11 May 2018 05:12:37 GMT

Try running

proc robustreg data = wombat.popedataset2;
  model dmus_post = dmus_pre / diagnostics;
run;

to see if your suspected outlier is idenfified by the procedure.

Re: How to compare change in regression results from two models: one with N and one with N-1

PaigeMiller — Fri, 11 May 2018 10:44:28 GMT

@jlp2ba wrote:

PaigeMiller,

I was wondering if there was a way to identify the change in r-squared, or if I can do model 2 r-sq minus model 1 r-sq.

I would recommend actually making a plot of the data with the two different regression lines shown, and see what the difference is.

Re: How to compare change in regression results from two models: one with N and one with N-1

jlp2ba — Fri, 11 May 2018 14:14:54 GMT

Gotcha - Thank you!

Re: How to compare change in regression results from two models: one with N and one with N-1

jlp2ba — Fri, 11 May 2018 14:15:09 GMT

Will do - thanks for this!