topic Comparing stratified logistic regression models to an unstratified model in Statistical Procedures

Comparing stratified logistic regression models to an unstratified model

greylondyn — Thu, 19 Mar 2020 21:11:36 GMT

I am using SAS Studio and I'm trying to compare an unstratified logistic regression model to three models created after stratifying by one variable to see if there is a significant difference. I can visually look at the models and see that there is a difference, but is there a way to run a comparison to see if the difference is statistically significant?

Thanks in advance!

Re: Comparing stratified logistic regression models to an unstratified model

PaigeMiller — Thu, 19 Mar 2020 21:21:08 GMT

In any regression, a term in the model such as

stratum

will provide a test to see if the intercepts are different in the different strata, while a term such as

stratum * x

will provide a test to see if the slopes are different in the different strata.

Re: Comparing stratified logistic regression models to an unstratified model

Ksharp — Fri, 20 Mar 2020 11:12:50 GMT

After getting predicted Y, could try K-S test for each model and

see which model's k-s value is higher .

proc npar1way data=final_total_score plots=edfplot edf ;
class good_bad;
var total_score;
run;

Or you could compare AUC of these two model.

@Rick_SAS wrote a couple of blog about AUC .

data all
 set A B;
run;
proc logistic data=all ;
model pred_y=score_a score_b/ nofit;
roc 'Model A' pred=score_a;
roc 'Model B' pred=score_b;
run;

Re: Comparing stratified logistic regression models to an unstratified model

JacobSimonsen — Mon, 23 Mar 2020 13:56:04 GMT

It depends on what you mean by "stratified model". If you mean that you run your model with or without a effect modifier on some covariate of interest, then you can indeed test if the effect modifier is significant (in terms of p-value). It is just to test if there is an interaction effect.

But, If stratified model means that you run a conditional logistic regression model by using the strata-statement, then I don't see any way to test if the variable used in the strata-statement change the result significant (in terms of p-value). You will have to judge if you find the strata variable has changed the result so much that your find the change "significant" (but then done use the word significant as it indicate you use a p-value). Also, It will most often not make sense to remove the variable in the strata statement, as it is given by the design of the model that the strata-variable has to be there (as in forexample a 1:k sampled case control study) .

Re: Comparing stratified logistic regression models to an unstratified model

StatDave — Mon, 23 Mar 2020 14:41:26 GMT

I don't think these are models that you compare and i'm not aware of a valid way to compare them. If the data was collected in strata resulting in observations that are correlated, then the analysis should reflect that structure. A conditional model is one way for the analysis to deal with that sampling scheme. An unconditional model would ignore the correlation and provide incorrect standard errors.