How to compare predictive validity of two variables in multivariable model??

[iressa131]

Hello,

I am looking to assess the predictive validity of two measures (M1 and M2) on a binary outcome. After constructing my model, do I just calculate odds ratios and compare the odds ratios for the two measures? I am not looking to assess the predictive value of the whole model just 2 specific predictors. Thanks for any clarification!

[PGStats]

Compare the AIC for the two fits in the model fit statistics table. The model with the lower AIC is better.

[iressa131]

Thanks for the help! I didn't realize I needed two separate models. Now if I then wanted to determine if M1 and M2 are independently associated with the outcome, would I construct a new model containing both in order to test for independence?

[PaigeMiller]

---

@iressa131 wrote:
Thanks for the help! I didn't realize I needed two separate models. Now if I then wanted to determine if M1 and M2 are independently associated with the outcome, would I construct a new model containing both in order to test for independence?

---

The way you have worded the question, I don't think this is a question that can be answered by fitting a model that has both variables.

If M1 and M2 have a correlation of zero, they have independent effects on the outcome. If they have a correlation that is not zero, then the effect of M1 and M2 will be correlated and not independent.

[iressa131]

Okay I think I understand that thank you! If its not too much trouble, could you clarify why 2 separate models is better for my initial aim of comparing predictive validity of M1 and  M2 on the outcome? Would fitting a model with both M1 and M2 allow for potential bias?

[Reeza]

For logistic regression make sure to look at the confusion matrix and the AUC as well. 

---

@iressa131 wrote:

Hello,

I am looking to assess the predictive validity of two measures (M1 and M2) on a binary outcome. After constructing my model, do I just calculate odds ratios and compare the odds ratios for the two measures? I am not looking to assess the predictive value of the whole model just 2 specific predictors. Thanks for any clarification!

---

[pau13rown]

a sas macro is available for pencina's net reclassification index: https://analytics.ncsu.edu/sesug/2010/SDA07.Kennedy.pdf

pencina's method would be relevant for the scenario you describe ie 'not the whole model' just the added variables: "Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond" https://www.ncbi.nlm.nih.gov/pubmed/17569110.

I'm going to write a brief blog post about about it's implementation when i find time....