topic Re: PROC REC can I specify "p-value" ? in SAS Procedures

PROC REC can I specify "p-value" ?

newmkka — Wed, 28 Nov 2012 07:59:50 GMT

To solve some questions, I used PROC REC with FORWARD/BACKWARD/STEPWISE options. The questions is exactly asked by following:

"For the forward selection, she specifies significance level for entry equal to 0.3, for the backward selection, she specifies significance level for staying equal to 0.3, whereas for the stepwise selection method she uses default significance levels for entry and staying."

Is there way to specify the p value? or just run and read it?

my code-----

PROC REG data = work.Datafile ;

model rses = bfio bfic bfie bfia bfin / SELECTION = FORWARD ;

RUN ;

and

result summary is below.

Forward Summary

1	bfin	1	0.1770	0.1770	43.3071	40.23	<.0001
2	bfic	2	0.1058	0.2828	15.9682	27.43	<.0001
3	bfie	3	0.0536	0.3364	3.1043	14.94	0.0002
4	bfio	4	0.0027	0.3391	4.3596	0.75	0.3885

- Backward Summary

1	bfia	4	0.0013	0.3391	4.3596	0.36	0.5495
2	bfio	3	0.0027	0.3364	3.1043	0.75	0.3885

- Stepwise Summary

1	bfin	1	0.1770	0.1770	43.3071	40.23	<.0001
2	bfic	2	0.1058	0.2828	15.9682	27.43	<.0001
3	bfie	3	0.0536	0.3364	3.1043	14.94	0.0002

Which model has the best fit? How to justify the best fit?

Thanks in advance.

Re: PROC REC can I specify "p-value" ?

Ksharp — Wed, 28 Nov 2012 08:52:27 GMT

So did you check the documentation firstly?

proc reg ALPHA=

Default alpha is 0.05, you can specify it by yourself.

Re: PROC REC can I specify "p-value" ?

SteveDenham — Wed, 28 Nov 2012 10:29:12 GMT

The options on the model statement for entry and staying in are SLE= and SLS=. The documentation says the default SLE for FORWARD is 0.5, and for STEPWISE 0.15. For SLS, the default for BACKWARD is 0.10 and for STEPWISE 0.15.

Given all of that, these methods result in biased selection, with the standard errors biased small. There are are a variety of methods for selecting "best fit" including adjusted R-squared and various information criteria, but still the method is flawed. Google search for a paper by Peter Flom regarding the dangers of stepwise (and forward/backward) selection of variables.

Steve Denham

Re: PROC REC can I specify "p-value" ?

newmkka — Wed, 28 Nov 2012 21:37:18 GMT

I use the options SLENTRY and SLSTAY.

/*FORWARD SELECTION*/

TITLE "FORWARD SELECTION" ;

PROC REG data = work.Datafile ;

model rses = bfio bfic bfie bfia bfin

/ SELECTION = FORWARD

SLENTRY = 0.3;

RUN ;

TITLE ;

1	bfin	1	0.1770	0.1770	43.3071	40.23	<.0001
2	bfic	2	0.1058	0.2828	15.9682	27.43	<.0001
3	bfie	3	0.0536	0.3364	3.1043	14.94	0.0002

/*BACKWARD SELECTION*/

TITLE "BACKWARD SELECTION" ;

PROC REG data = work.Datafile ;

model rses = bfio bfic bfie bfia bfin

/ SELECTION = BACKWARD

SLSTAY = 0.3;

RUN ;

TITLE ;

Backward elimination

1	bfia	4	0.0013	0.3391	4.3596	0.36	0.5495
2	bfio	3	0.0027	0.3364	3.1043	0.75	0.3885

/*STEPWISE SELECTION*/

TITLE "STEPWISE SELECTION" ;

PROC REG data = work.Datafile ;

model rses = bfio bfic bfie bfia bfin

/ SELECTION = STEPWISE ;

RUN ;

TITLE ;

1	bfin	1	0.1770	0.1770	43.3071	40.23	<.0001
2	bfic	2	0.1058	0.2828	15.9682	27.43	<.0001
3	bfie	3	0.0536	0.3364	3.1043	14.94	0.0002

Can help me to fill out?

1. Which model has the best fit? Please justify your answer.

??? can help me? less step? r-square?

2. Do any models reach the same conclusions with regards to regression coefficients? If so, which ones?

my answer is FORWARD and STEPWISE is same conclusions

3. Provide an interpretation of an intercept for the model with the best fit.

4. Provide an interpretation of partial regression coefficients for the model with the best fit.

Re: PROC REC can I specify "p-value" ?

PGStats — Wed, 28 Nov 2012 22:57:48 GMT

I agree with Steve's comments. You shouldn't waste your time with antiquated statistical methods.The only selection criteria available in PROC REG that account for the number of estimated parameters are ADJRSQ and CP. Model selection, especially when it is based on small datasets, requires a lot of expertise and ... humility.

Re: PROC REC can I specify "p-value" ?

SteveDenham — Thu, 29 Nov 2012 12:54:05 GMT

And I wish I could go yell at your stat instructor, who gave you this problem, and never considered the drawbacks of these methods. Yes, we need to know that they exist, but they are just an intensive and frustrating way of making bad decisions. , I don't mind helping with analytic methods and approaches, but answering questions 1 through 4 looks exactly like homework or a take-home final exam, and the answers should be left to the analyst to come up with.

Steve Denham