topic Re: Which p-values to report as part of multivariate analysis in proc logistic? in Statistical Procedures

Which p-values to report as part of multivariate analysis in proc logistic?

Dcicantab5 — Sat, 09 Jul 2016 17:26:53 GMT

Hi,

Need clarification:

Clinical papers often quote p-values obtained from multivariate analysis (most would be by logistic regression). Which p-values do we report as multivariate analysis?

Take the stepwise selection method for example, say for 9 variables, there's p-values in the

1. analysis of effects eligible for entry, even at step 0

2. type 3 analysis of effects

3. summary of stepwise selection

Most of the time, the number of significant variables would have reduced in number when we reach the summary table.

Thank u in advance.

Saiful.

Re: Which p-values to report as part of multivariate analysis in proc logistic?

Ksharp — Sun, 10 Jul 2016 02:12:21 GMT

I guess it is "2. type 3 analysis of effects " .

Re: Which p-values to report as part of multivariate analysis in proc logistic?

SteveDenham — Mon, 11 Jul 2016 18:10:47 GMT

And be sure to include the summary of stepwise selection, as this brings out all of the multiple comparisons/multiple testing that went on, and unless adjusted for (LASSO or LAAR) renders the type 3 p values inaccurate.

Steve Denham

Re: Which p-values to report as part of multivariate analysis in proc logistic?

Dcicantab5 — Tue, 12 Jul 2016 06:55:25 GMT

Hi Steve,
You mean run LASSO under PROC GLMSELECT and then run PROC LOGISTIC based on the selected variables?

To be honest I am not familiar with LASSO but it appears that depending on PROC LOGISTIC alone seems unwise when it comes to variable selection (one depends on intuition and study design and what is so far known about the topic studied)?

Re: Which p-values to report as part of multivariate analysis in proc logistic?

SteveDenham — Wed, 13 Jul 2016 17:25:46 GMT

The problem with almost all variable selection methods, other than "expert knowledge", is that the estimates are biased, and that the predictive ability of the model is poor. See http://www.lexjansen.com/pnwsug/2008/DavidCassell-StoppingStepwise.pdf.

Currently, the best automated methods seem to be LASSO based--unless you go the neural net/machine learning route, and that leads to a question of interpretability.

Steve Denham