Proc Glimmix: Type III Results Change With A Change In Specified Reference Cell

[susheel73]

Hello:

I've been struggling with a problem. Here is the scenario.
I have been using proc glimmix to fit a hierarchical logistic regression (dichotomous outcome) consisting of level I and level II fixed effects. The model contains fixed effects included in the class statement (categorical variables) and fixed effects that are continuous. Some categorical variables are dichotomous, some are polytomous. Some are level I variables (i.e. patient level characteristics) and some are level II variables (i.e. hospital level characteristics). It also contains an interaction term using two of the categorical variables cited in the class statement. The model includes a random intercept of hospital ID and no random slope.

I noticed that when I change the reference cell for some (i.e. 2) of the class/categorical variables (both happen to be dichotomous) and nothing else, the p-values for one of the associated Type III tests change as does the p-value for the interaction term. The difference for the main effect is negligible but the difference for the interaction term makes me think I am missing something. Does anyone know what might be driving this?

Please see below for the code and the output.

First set of code/results, note the “female (0/1)" and “pcfcat2 (“$0”,”> $0”) variables and the assigned reference cell for each:

 proc glimmix data = modelanly order=formatted method=laplace oddsratio;
 class agecat (ref='18-36')  female (ref = '1') risscat (ref='16-20') pay1 (ref='3') pcfcat2 (ref='> $0') race (Ref='1') ahaid bsc2 (Ref="< 100") mapp5 intgroup_v2 (ref='Appropriate Triage');
 model died (event='1') = agecat female race risscat elixsum pay1 bsc2 mapp5 intgroup_v2 pcfcat2 intgroup_v2*pcfcat2 los
                          /dist = binary link=logit ddfm=bw cl solution oddsratio;
 random intercept/subject=ahaid ;
 covtest /wald;
 title 'Reduced Model, Intgroup, Retriage and Field Triage Only';
 where intgroup_v2 not in ("High Level Interfacility Transfer", "Low Level Interfacility Transfer");

And the code/results with the reference cells for female and pcfcat2 changed:

proc glimmix data = modelanly order=formatted method=laplace oddsratio;
 class agecat (ref='18-36')  female (ref = '0') risscat (ref='16-20') pay1 (ref='3') pcfcat2 (ref='$0') race (Ref='1') ahaid bsc2 (Ref="< 100") mapp5 intgroup_v2 (ref='Appropriate Triage');
 model died (event='1') = agecat female race risscat elixsum pay1 bsc2 mapp5 intgroup_v2 pcfcat2 intgroup_v2*pcfcat2 los
                          /dist = binary link=logit ddfm=bw cl solution oddsratio;
 random intercept/subject=ahaid ;
 covtest /wald;
 title 'Reduced Model, Intgroup, Retriage and Field Triage Only, Flipped PFCCAT/Female Ref';
 where intgroup_v2 not in ("High Level Interfacility Transfer", "Low Level Interfacility Transfer");
 

Again, the differences here are not great enough to change conclusions though not enough to rule out coding/user error.  Plus there are other models, albeit more problematic and less stable, where the differences are more dramatic. 

The odds ratios for the two covariates in question do also change by a degree greater than the 1/OR that one would expect when running the two sets of code.  It is difficult to share this output in a tidy, legible manner though I would characterize the differences to be greater than one could attribute to some sort of discrepancy due to rounding.

Any help or hints you could provide would be helpful.  I'm afraid the datasets are much too large to share (n=243000 observations) though I could provide additional details or agnostic descriptive summaries if that would be helpful.

Kind regards,

Susheel Reddy

[susheel73]

Sorry here is the SAS code and associated results again:

proc glimmix data = modelanly order=formatted method=laplace oddsratio;
 class agecat (ref='18-36')  female (ref = '1') risscat (ref='16-20') pay1 (ref='3') pcfcat2 (ref='> $0') race (Ref='1') ahaid bsc2 (Ref="< 100") mapp5 mapp8 intgroup_v2 (ref='Appropriate Triage');
 model died (event='1') = agecat female race risscat elixsum pay1 bsc2 mapp5 mapp8 intgroup_v2 pcfcat2 intgroup_v2*pcfcat2 los
                          /dist = binary link=logit ddfm=bw cl solution oddsratio;
 random intercept/subject=ahaid ;
 covtest /wald;
 title 'Full Model, Intgroup, Retriage and Field Triage Only';
 where intgroup_v2 not in ("High Level Interfacility Transfer", "Low Level Interfacility Transfer");
 run;
quit;

And with the reference cells changed:

proc glimmix data = modelanly order=formatted method=laplace oddsratio;
 class agecat (ref='18-36')  female (ref = '0') risscat (ref='16-20') pay1 (ref='3') pcfcat2 (ref='$0') race (Ref='1') ahaid bsc2 (Ref="< 100") mapp5 intgroup_v2 (ref='Appropriate Triage');
 model died (event='1') = agecat female race risscat elixsum pay1 bsc2 mapp5 intgroup_v2 pcfcat2 intgroup_v2*pcfcat2 los
                          /dist = binary link=logit ddfm=bw cl solution oddsratio;
 random intercept/subject=ahaid ;
 covtest /wald;
 title 'Reduced Model, Intgroup, Retriage and Field Triage Only, Flipped PFCCAT/Female Ref';
 where intgroup_v2 not in ("High Level Interfacility Transfer", "Low Level Interfacility Transfer”);
 run;
quit;

Thank you,

Susheel Reddy

[jiltao]

There are several possible reasons for this. Can you send us the log and complete output from running the two models?

Thanks,

Jill

[jiltao]

I see that you opened a technical support track with us. I will communicate with you via the track.

Jill

[Rick_SAS]

FYI, there seems to be a problem with the second set of code, which does not seem to match the output that you posted.

The first model includes mapp8 as a classification effect, but the second does not. But the output for the first model does not include mapp8 in the parameter estimates table. 

[susheel73]

Thank you both so much for your kind replies.  I'm terribly sorry for the messiness of the post and the included code/output.  I've attached versions of the output and the log that are cleaned up and hopefully are conducive to review.

[susheel73]

The output for the Type III tests using the two model runs are on p. 29-30 and p.59-60 of the output respectively.

Thank you,

Susheel

[susheel73]

Here is an abridged version of the output with just the Type III test results (provided to facilitate easy review).

Thank you,

Susheel