Programming the statistical procedures from SAS

Adjusting for multiple comparisons in Logistic Regression models

Accepted Solution Solved
Reply
N/A
Posts: 0
Accepted Solution

Adjusting for multiple comparisons in Logistic Regression models

Hello, I need to adjust my p-value for significant interaction term in the adjusted logistic regression model with quite a few covariates which i have to keep in the model. I have no clue how to do that. I tried PROC Multtest but then it is not logistic and does not allow to use for covariates, and then I have no possibility to input raw p-values so my raw and bonferroni appear the same! (?). Is there any option in PROC LOGISTIC that would allow me to adjust for comparisons?
What should I do if not?
Thanks very much for any input!

Accepted Solutions
Solution
‎07-03-2017 09:51 AM
SAS Employee
Posts: 241

Re: Adjusting for multiple comparisons in Logistic Regression models

You can adjust any set of p-values by providing them to PROC MULTTEST in a data set. For example, these statements save the ParameterEstimates table to a data set in which the p-values are in a variable named ProbChiSq. PROC MULTTEST is then used to adjust them using the step-down Bonferroni method (which is less conservative):

ods output parameterestimates=pe;
proc logistic data=remiss;
model remiss(event='1') = smear blast cell infil li temp;
run;
proc multtest inpvalues(probchisq)=pe holm; run;

View solution in original post


All Replies
Trusted Advisor
Posts: 2,114

Re: Adjusting for multiple comparisons in Logistic Regression models

Rather than use an adjusted p-value, you could easily use an adjusted alpha level using the Bonferroni method. If you want to protect for an overall alpha of 0.05, then just divide 0.05 by the number of tests.

An alternate that may be preferable is to use a "chunk test" to evaluate a number of interactions in one test (which would cut the number of tests in the Bonferroni method by a lot. See Frank Harrell's book; it's published by Springer.
N/A
Posts: 0

Re: Adjusting for multiple comparisons in Logistic Regression models

Thank you for the input. Bonferroni method is too conservative. I am testing multiple polymorphisms (hundreds), one per model though so that each model has one polymorphism, and one- two way, and one 3 way interaction term.
Is there a simple SAS code in LOGISTCIC regression to do permutation test or FDR but utilizing the same LOGISTIC MODEL? Thanks a lot!
Solution
‎07-03-2017 09:51 AM
SAS Employee
Posts: 241

Re: Adjusting for multiple comparisons in Logistic Regression models

You can adjust any set of p-values by providing them to PROC MULTTEST in a data set. For example, these statements save the ParameterEstimates table to a data set in which the p-values are in a variable named ProbChiSq. PROC MULTTEST is then used to adjust them using the step-down Bonferroni method (which is less conservative):

ods output parameterestimates=pe;
proc logistic data=remiss;
model remiss(event='1') = smear blast cell infil li temp;
run;
proc multtest inpvalues(probchisq)=pe holm; run;
N/A
Posts: 0

Re: Adjusting for multiple comparisons in Logistic Regression models

Wow, this actually worked! thanks so much! can i obtain fdr-adjusted p-values this way? also, can I do permutations this way? Thanks again!!!
N/A
Posts: 0

Re: Adjusting for multiple comparisons in Logistic Regression models

I also have a question, if I can write macro for all those SNPs I am testing and for their multiple comparisons adjustment? Thanks!
SAS Employee
Posts: 241

Re: Adjusting for multiple comparisons in Logistic Regression models

There are many p-value adjustment methods available, including FDR. See the MULTTEST documentation:

http://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/multtest_toc.htm

However, resampling-based adjustment (permutation and bootstrap) cannot be done when inputting a set of p-values. These can only be done for the tests that MULTTEST does (via its TEST statement) since it must have the actual data in order to do resampling.
N/A
Posts: 0

Re: Adjusting for multiple comparisons in Logistic Regression models

Thanks. Sorry for so many questions, this is the last: I realized that all the HOLM, FDR adjusted for is n covariates in the model. How would i perform adjustment, for let's say 200 other models like this that I am running yet with only one difference: different polymorphism (predictor) is included in each of those 100 models?
SAS Employee
Posts: 241

Re: Adjusting for multiple comparisons in Logistic Regression models

You can adjust any set of p-values to control the experiment-wise error rate. The collection of p-values to be adjusted can be from different analyses (regression parameter tests, t-tests, etc.). A method like Holm's is always applicable as long as the p-values themselves are valid. So, you can use MULTTEST if you are doing several regressions and want to control the experiment-wise error rate across all the parameter tests.
Contributor
Posts: 57

Re: Adjusting for multiple comparisons in Logistic Regression models

Hi everybody,

In this case, how can we generate adjusted odd ratios and confidence intervals?

Thanks a lot for your help.

Best,

🔒 This topic is solved and locked.

Need further help from the community? Please ask a new question.

Discussion stats
  • 9 replies
  • 5105 views
  • 0 likes
  • 4 in conversation