Calcite | Level 5

6 REPLIES 6
Opal | Level 21

## Re: How to compare predictive validity of two variables in multivariable model??

Compare the AIC for the two fits in the model fit statistics table. The model with the lower AIC is better.

PG
Calcite | Level 5

## Re: How to compare predictive validity of two variables in multivariable model??

Thanks for the help! I didn't realize I needed two separate models. Now if I then wanted to determine if M1 and M2 are independently associated with the outcome, would I construct a new model containing both in order to test for independence?
Diamond | Level 26

## Re: How to compare predictive validity of two variables in multivariable model??

@iressa131 wrote:
Thanks for the help! I didn't realize I needed two separate models. Now if I then wanted to determine if M1 and M2 are independently associated with the outcome, would I construct a new model containing both in order to test for independence?

The way you have worded the question, I don't think this is a question that can be answered by fitting a model that has both variables.

If M1 and M2 have a correlation of zero, they have independent effects on the outcome. If they have a correlation that is not zero, then the effect of M1 and M2 will be correlated and not independent.

--
Paige Miller
Calcite | Level 5

## Re: How to compare predictive validity of two variables in multivariable model??

Okay I think I understand that thank you! If its not too much trouble, could you clarify why 2 separate models is better for my initial aim of comparing predictive validity of M1 and  M2 on the outcome? Would fitting a model with both M1 and M2 allow for potential bias?

Super User

## Re: How to compare predictive validity of two variables in multivariable model??

For logistic regression make sure to look at the confusion matrix and the AUC as well.

@iressa131 wrote:

Lapis Lazuli | Level 10

## Re: How to compare predictive validity of two variables in multivariable model??

a sas macro is available for pencina's net reclassification index: https://analytics.ncsu.edu/sesug/2010/SDA07.Kennedy.pdf

pencina's method would be relevant for the scenario you describe ie 'not the whole model' just the added variables: "Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond" https://www.ncbi.nlm.nih.gov/pubmed/17569110.

I'm going to write a brief blog post about about it's implementation when i find time....

Discussion stats
• 6 replies
• 1779 views
• 0 likes
• 5 in conversation