PROC GLIMMIX non-zero estimate for class reference category

[Huggies_23]

Occasionally when fitting mixed effects models to categorical data using PROC GLIMMIX, the returned coefficient estimates for the specified fixed effects reference categories (e.g. cat3 in var1 from the statement class <var1>(ref = 'Cat3') <var2>(ref = 'Cat2') etc.) are non-zero. The coefficient estimates for all other categories of the variable are also shifted by the value of this non-zero estimate and this value is "counteracted" by the intercept. A completely made up toy example is set out below:

Expected behavior of fixed effects parameter estimate output:

Effect       Cat       Estimate    Standard Error    DF    t Value    Pr>|t|

Int.                        1.02           0.05                   100     22.30    <0.05

Var1         1          -0.02           0.08                   100   -0.31      >0.05

Var1         2          -1.04           0.04                   100   -24.32    <0.05

Var1         3           0                  .                         .         .              . 

Var2         1           0.27          0.08                   100      3.11      <0.05 

Var2         2           .                   .                        .         .               . 

Actual "unusual" output:

Effect       Cat       Estimate    Standard Error    DF    t Value    Pr>|t|

Int.                         16.02           0.05                  100      122.30   <0.05

Var1         1          -15.02           0.08                   100    -100.31   <0.05

Var1         2          -16.04           0.04                   100    -124.32   <0.05

Var1         3          -15.0                  0                   100      Infty      <0.05 

Var2         1            0.27            0.08                   100      3.11      <0.05 

Var2         2           .                   .                        .         .               . 

We can see from the above that an estimate for the reference category of Var1 has been produced by PROC GLIMMIX, the Intercept estimate has shifted in the opposite direction by the same value as this estimate and all other categories of Var1 have decreased by the value of the estimate. As a result of this the previously non-significant estimate for cat1 of Var one is now significant. There is no appreciable difference in any of the model fit stats 

This feels to me like the parameter optimization has found a local minimum and the intercept is being used to offset this. As the relative difference between the chosen reference category and the other categories of the variable has remained constant it seems like the relative estimates may be sound, and the corresponding t-values and p-values could be re calculated in this basis.

From this perspective this unusual solution seems "fine" (as the expected behavior result confirms) but in any case I would like to understand this feature further.

I hope this makes sense! If anyone has any suggestions then I'd love to hear them. If you require any further information please do ask.

Best,

Dave

[PaigeMiller]

This is not a "local minimum" as you say. It is an issue that in the case of CLASS variables, the parameter estimates are not unique, and any solution found by changing the REF level is equivalent in terms of predictions — in other words, they are all the same model.

For example, suppose you have a study of the heights (in inches) of males and females in a given population. If the males are ten inches taller than the females, then you could have an estimate of +5 for males and –5 for females, or +10 for males and zero for females, or –10 for females and 0 for males. These are all the same. Also note that in each of these cases, the intercept would have to change as well.

So, in your case, I strongly recommend looking at LSMEANS instead of using the parameter estimates. LSMEANS are unique, and avoid this problem of the parameter estimates being not unique; and LSMEANS are usually much more interpretable than the parameter estimates as well.

[Huggies_23]

Hi PaigeMiller,

Thank you for your clear and concise response. It is much appreciated and I am glad it echos my thinking.

I am still quite interested in why PROC GLIMMIX in certain cases estimates the reference category, when I would assume this would be fixed to be 0 by design. 

Best,

Dave 

[PaigeMiller]

I'm sure if I thought about it long enough, the answer would come to me. But since I never look at the parameter estimates for class variables any more, and I don't even think about parameter estimates for class variables, I just go straight to the LSMEANS, I don't have an answer right now. Remember, the different models are all equivalent.

[sld]

I've never seen this sort of output from GLIMMIX. You might need to post your code and an exemplary dataset to get any insightful answers.