How to interpret Glimmix model covest ZEROG/Wald

[xuyi]

I am running a proc glimmix model for a multi-site study, we have only three sites, each site has about 400-700 patients, the outcome is a binary variable. First I would like to see if the random site (siteID) effect is necessary in the model. Here are my codes. 

proc glimmix data=SubjectPQ_combine method=quad;
class SiteID;
model CSPrepQual2CAT (event='0')=/dist=binary link=logit solution;
random intercept/subject=siteID type=vc;
covtest zeroG/wald;
title1 'GLIMMIX model-site as random effect';
title2 'Test variance of random effect';
run;

I am not sure how to interpret the covtest zeroG/wald results, although I read the proc Glimmix procedure, covest.

The "covariance Parameter Estimates" indicates a p-value 0.148, however, the "Tests of Covariance Parameters
Based on the Likelihood" indicated a p-value <0.0001.

My understanding is that based on the first p-value, there isn't significant variance for random effect. However, the second p-value means there is significant variance for random effect. Where am I wrong?

Thank you in advance.

[Mike_N]

I can't say with 100% certainty what is going on without having access to your data, but it looks like you are getting the results of two different tests of the null hypothesis that the variance of the random effects is zero. The default for the covtest statement is a likelihood ratio test, and then you have also requested a Wald test. These two types of tests are not guaranteed to produce the same results. You can read more about them in the 'Statistical Inference for Covariance Parameters' section of the documentation (https://go.documentation.sas.com/doc/en/pgmsascdc/9.4_3.4/statug/statug_glimmix_details27.htm ). You'll note that there is subsection that specifically discusses likelihood ratio vs. Wald tests. In general, likelihood ratio tests are often preferred to Wald tests. 

[xuyi]

Hi Mike,

Thank you so much for your response. Based on your response and the SAS manual that you referred to, it seems that I should use the likelihood ratio test. 

However, these two results are so much different, and I see people used Wald test very often.

I am attaching the dataset in case you would like to look at it.

Again, thank you so much for your time.

[Mike_N · Accepted solution]

I can't tell you for sure which one to use, you will have to make the final decision. However, if you haven't already, I would suggest reading the 'One and two-sided testing, mixture distributions' subsection of the documentation (https://go.documentation.sas.com/doc/en/pgmsascdc/9.4_3.4/statug/statug_glimmix_details29.htm ). In the third paragraph, there is a discussion about hypothesis testing when the null value of the parameter is on the boundary of the parameter space. In your case, when you test the null hypothesis that the variance of the random intercepts is zero, you are testing a null hypothesis about a parameter on the boundary of the parameter space (i.e., variance must be greater than or equal to zero). As detailed in the linked subsection, there has been a lot of research into the behavior of the likelihood ratio test statistic under those conditions, such that the likelihood ratio test implemented in PROC GLIMMIX will often provide an asymptotically more powerful and reliable test than the Wald test.  For more details, read the papers cited in the documentation. 

[xuyi]

Thank you so much. This is very helpful.