adjusted p-value

[knighsson]

Hello,

I am using proc surveyreg to compare the mean of different groups. But I do not quite sure what the adjusted p-value exactly mean in the output.

 

the code:

the output:

I wonder whether the adjusted p-value is due to  Bonferroni adjustment? or Tukey-test?  

Thank you!

[sbxkoenk]

Hello,

You ask for a multiple-comparison adjustment of LS-means differences by the ADJUST= option on the LSMEANS statement, so you get it. 🙄

As method, you specify TUKEY (ADJUST=TUKEY) so it's a Tukey adjustment for the p-value.

More info in the doc.

https://go.documentation.sas.com/?docsetId=helpcenterwlcm&docsetTarget=home.htm&docsetVersion=1.0&lo...

Cheers,

Koen

[knighsson]

Thank you, But I want to know if I still want to use bonferroni adjustment, should I consider adjust p-value <0.05/6 as the significant differences?

[sbxkoenk]

Hello,

There's no need to do a BON (Bonferroni) correction on top of the TUKEY correction. 

To me a TUKEY correction is often a good choice (less conservative than Bonferroni).

But feel free to select the best correction method for your situation:

Here's the doc of the ADJUST= option (for multiple comparison p-value adjustment).

https://go.documentation.sas.com/?cdcId=pgmsascdc&cdcVersion=9.4_3.5&docsetId=statug&docsetTarget=st...

LSMEANS statement (across procedures): 

https://go.documentation.sas.com/?cdcId=pgmsascdc&cdcVersion=9.4_3.5&docsetId=statug&docsetTarget=st...

If you do not know what to choose: choose ADJUST=BON.

Cheers,

Koen

[sbxkoenk]

You can also read this and have a look at the recorded webinar:
What Are Best Practices for Using SAS® Survey Procedures? Q&A, Slides, and On-Demand Recording
https://communities.sas.com/t5/Ask-the-Expert/What-Are-Best-Practices-for-Using-SAS-Survey-Procedure...

[knighsson]

This is helpful! Thank you so much!

[knighsson]

Hello, just one more question, in the table above, if the adjusted p-value <0.05, then that comparison is significantly different? Or do I need to calculate a significant level for p-value, if I used tukey test?

Thank you!

[StatDave]

For any adjustment method, the adjusted p-values can be compared directly to the level you choose to use - such as 0.05 if that is your choice.

[knighsson]

Thank you so much!

[sbxkoenk]

Hello @knighsson ,

A small add-on to the comment made by @StatDave .

The adjustment method you choose does not influence in any way the significance level that you set for the hypothesis test (a comparison is also a hypothesis test). The alpha-level (for your hypothesis test) is the probability of making a type I error and is subjectively chosen by the user UPFRONT.

If you choose 0.05 as alpha, it stays that way after adjustment (with BON or TUKEY or ...) because the p-values are inflated. SAS should inflate the p-values instead of reducing the alpha because SAS does not know what the significance level / alpha is (because it's in the head of the user).

Good luck,

Koen

[knighsson]

Thank you so much!