To solve some questions, I used PROC REC with FORWARD/BACKWARD/STEPWISE options. The questions is exactly asked by following:
"For the forward selection, she specifies significance level for entry equal to 0.3, for the backward selection, she specifies significance level for staying equal to 0.3, whereas for the stepwise selection method she uses default significance levels for entry and staying."
Is there way to specify the p value? or just run and read it?
my code-----
PROC REG data = work.Datafile ;
model rses = bfio bfic bfie bfia bfin / SELECTION = FORWARD ;
RUN ;
and
result summary is below.
Forward Summary
1 | bfin | 1 | 0.1770 | 0.1770 | 43.3071 | 40.23 | <.0001 |
---|---|---|---|---|---|---|---|
2 | bfic | 2 | 0.1058 | 0.2828 | 15.9682 | 27.43 | <.0001 |
3 | bfie | 3 | 0.0536 | 0.3364 | 3.1043 | 14.94 | 0.0002 |
4 | bfio | 4 | 0.0027 | 0.3391 | 4.3596 | 0.75 | 0.3885 |
- Backward Summary
1 | bfia | 4 | 0.0013 | 0.3391 | 4.3596 | 0.36 | 0.5495 |
---|---|---|---|---|---|---|---|
2 | bfio | 3 | 0.0027 | 0.3364 | 3.1043 | 0.75 | 0.3885 |
- Stepwise Summary
1 | bfin | 1 | 0.1770 | 0.1770 | 43.3071 | 40.23 | <.0001 | |
---|---|---|---|---|---|---|---|---|
2 | bfic | 2 | 0.1058 | 0.2828 | 15.9682 | 27.43 | <.0001 | |
3 | bfie | 3 | 0.0536 | 0.3364 | 3.1043 | 14.94 | 0.0002 |
Which model has the best fit? How to justify the best fit?
Thanks in advance.
So did you check the documentation firstly?
proc reg ALPHA=
Default alpha is 0.05, you can specify it by yourself.
The options on the model statement for entry and staying in are SLE= and SLS=. The documentation says the default SLE for FORWARD is 0.5, and for STEPWISE 0.15. For SLS, the default for BACKWARD is 0.10 and for STEPWISE 0.15.
Given all of that, these methods result in biased selection, with the standard errors biased small. There are are a variety of methods for selecting "best fit" including adjusted R-squared and various information criteria, but still the method is flawed. Google search for a paper by Peter Flom regarding the dangers of stepwise (and forward/backward) selection of variables.
Steve Denham
I use the options SLENTRY and SLSTAY.
/*FORWARD SELECTION*/
TITLE "FORWARD SELECTION" ;
PROC REG data = work.Datafile ;
model rses = bfio bfic bfie bfia bfin
/ SELECTION = FORWARD
SLENTRY = 0.3;
RUN ;
TITLE ;
1 | bfin | 1 | 0.1770 | 0.1770 | 43.3071 | 40.23 | <.0001 |
---|---|---|---|---|---|---|---|
2 | bfic | 2 | 0.1058 | 0.2828 | 15.9682 | 27.43 | <.0001 |
3 | bfie | 3 | 0.0536 | 0.3364 | 3.1043 | 14.94 | 0.0002 |
/*BACKWARD SELECTION*/
TITLE "BACKWARD SELECTION" ;
PROC REG data = work.Datafile ;
model rses = bfio bfic bfie bfia bfin
/ SELECTION = BACKWARD
SLSTAY = 0.3;
RUN ;
TITLE ;
Backward elimination
1 | bfia | 4 | 0.0013 | 0.3391 | 4.3596 | 0.36 | 0.5495 |
---|---|---|---|---|---|---|---|
2 | bfio | 3 | 0.0027 | 0.3364 | 3.1043 | 0.75 | 0.3885 |
/*STEPWISE SELECTION*/
TITLE "STEPWISE SELECTION" ;
PROC REG data = work.Datafile ;
model rses = bfio bfic bfie bfia bfin
/ SELECTION = STEPWISE ;
RUN ;
TITLE ;
1 | bfin | 1 | 0.1770 | 0.1770 | 43.3071 | 40.23 | <.0001 | |
---|---|---|---|---|---|---|---|---|
2 | bfic | 2 | 0.1058 | 0.2828 | 15.9682 | 27.43 | <.0001 | |
3 | bfie | 3 | 0.0536 | 0.3364 | 3.1043 | 14.94 | 0.0002 |
Can help me to fill out?
1. Which model has the best fit? Please justify your answer.
??? can help me? less step? r-square?
2. Do any models reach the same conclusions with regards to regression coefficients? If so, which ones?
my answer is FORWARD and STEPWISE is same conclusions
3. Provide an interpretation of an intercept for the model with the best fit.
4. Provide an interpretation of partial regression coefficients for the model with the best fit.
I agree with Steve's comments. You shouldn't waste your time with antiquated statistical methods.The only selection criteria available in PROC REG that account for the number of estimated parameters are ADJRSQ and CP. Model selection, especially when it is based on small datasets, requires a lot of expertise and ... humility.
PG
And I wish I could go yell at your stat instructor, who gave you this problem, and never considered the drawbacks of these methods. Yes, we need to know that they exist, but they are just an intensive and frustrating way of making bad decisions. , I don't mind helping with analytic methods and approaches, but answering questions 1 through 4 looks exactly like homework or a take-home final exam, and the answers should be left to the analyst to come up with.
Steve Denham
Build your skills. Make connections. Enjoy creative freedom. Maybe change the world. Registration is now open through August 30th. Visit the SAS Hackathon homepage.
Register today!Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.
Find more tutorials on the SAS Users YouTube channel.
Select SAS Training centers are offering in-person courses. View upcoming courses for: