BookmarkSubscribeRSS Feed
iressa131
Calcite | Level 5
Hello,
 
I am looking to assess the predictive validity of two measures (M1 and M2) on a binary outcome. After constructing my model, do I just calculate odds ratios and compare the odds ratios for the two measures? I am not looking to assess the predictive value of the whole model just 2 specific predictors. Thanks for any clarification!
 
6 REPLIES 6
PGStats
Opal | Level 21

Compare the AIC for the two fits in the model fit statistics table. The model with the lower AIC is better.

PG
iressa131
Calcite | Level 5
Thanks for the help! I didn't realize I needed two separate models. Now if I then wanted to determine if M1 and M2 are independently associated with the outcome, would I construct a new model containing both in order to test for independence?
PaigeMiller
Diamond | Level 26

@iressa131 wrote:
Thanks for the help! I didn't realize I needed two separate models. Now if I then wanted to determine if M1 and M2 are independently associated with the outcome, would I construct a new model containing both in order to test for independence?

The way you have worded the question, I don't think this is a question that can be answered by fitting a model that has both variables.

 

If M1 and M2 have a correlation of zero, they have independent effects on the outcome. If they have a correlation that is not zero, then the effect of M1 and M2 will be correlated and not independent.

--
Paige Miller
iressa131
Calcite | Level 5

Okay I think I understand that thank you! If its not too much trouble, could you clarify why 2 separate models is better for my initial aim of comparing predictive validity of M1 and  M2 on the outcome? Would fitting a model with both M1 and M2 allow for potential bias?

Reeza
Super User

For logistic regression make sure to look at the confusion matrix and the AUC as well. 

 


@iressa131 wrote:
Hello,
 
I am looking to assess the predictive validity of two measures (M1 and M2) on a binary outcome. After constructing my model, do I just calculate odds ratios and compare the odds ratios for the two measures? I am not looking to assess the predictive value of the whole model just 2 specific predictors. Thanks for any clarification!
 

 

pau13rown
Lapis Lazuli | Level 10

a sas macro is available for pencina's net reclassification index: https://analytics.ncsu.edu/sesug/2010/SDA07.Kennedy.pdf

 

pencina's method would be relevant for the scenario you describe ie 'not the whole model' just the added variables: "Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond" https://www.ncbi.nlm.nih.gov/pubmed/17569110.

 

I'm going to write a brief blog post about about it's implementation when i find time....

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 6 replies
  • 1600 views
  • 0 likes
  • 5 in conversation