I will have to study your detailed note later to see if I agree with everything. But I think you have captured many important details. I must point out that I am not an expert on the parameterizations other than the GLM. This is the main one I use because of its value in hypothesis testing of expected values (main effects and interactions) and in calculating estimable functions. There are many ways of parameterizing factor level effects, and they all have value for different applications. The individual parameters may all have direct interpretation on an individual basis, but tests of groups of them (say, for a collection of terms making up a main effect) may be hard to interpret. If you are going to be adding random effects into a model, you will have to get used to the GLM parameterization, the only choice available. ... View more

I was originally quite surprised by your results, so I generated a binary data set last night with two factors. I did not have your problem: I got identical chi-square results for GLIMMIX and GENMOD (I did not use LOGISTIC). Your problem is that you used LR based Type 3 testing in GENMOD and Wald (chi-squared or F) testing in GLIMMIX (at least that is what you showed). (Note that the F statistic is just a scaled Wald statistic). It is well known that LR and Wald tests can give different results. With very large sample sizes (depending on the complexity of the model, structure of the data, etc.), they can give the same results, but I would not assume this for most situations. Often they give reasonably close p values at moderate to large sample sizes, but there are exceptions. The Wald statistics work better at smaller n because LR is based on asymptotic properties of the test statistic. To get Wald type 3 statistics in GENMOD, add wald as an option on the model statement. You also made the following statement: "First, with dichotomous variables, I would expect the Type 3 F test statistic to simply be the parameter t statistic squared, as the F stat has 1 num df and the same denom df as the t stat.". This is not true, in general, but it is an easy misunderstanding to have. For a simple regression problem, where binary variable is not an indicator for the factor level, the square of the t statistic (est./se) is the F Wald statistic. But it is a different situation when the binary variables are being used to give the expected values or other estimable functions for levels of a factor (especially in a factorial). With a factorial, the test statistic for the main effect of A involves more than just the A parameters. In particular, the main effect mean for A1 is based on a linear combination of A1, the average of the B terms, and the average of the interaction terms that involve A1. So, the type 3 test statistics are based on more complicated contrasts than you are supposing. To see this, add the e option to the model statement in GLIMMIX to see the coefficients used for generating the tests of A, B, and A*B. You can see these involve more than just single terms for the main effects. One final note. Although this is not affecting your results at all (I don't think at this point), you are using different parametrizations for the class variables in GLIMMIX compared to the others. That is, GLIMMIX only uses the so-called GLM parameterization, where there is always a 0 for the last level. You have no choice for this procedure. The reference parametrization is slightly different, which you elected to use for the other two procedures. Here, there is one less level for each factor. The GENMOD documentation describes the many different ways of parameterizing factors with the CLASS statement. This won't affect your global type 3 results, but it is important to be aware of the differences.The developers of linear model procedures going back to the 1970s have made it clear why they favor the "GLM" parameterization. That is why you only find this in GLM, MIXED, and GLIMMIX. ... View more

There are several sources on-line. Check out: Lecture 17—Friday, February 10, 2006 for a detailed example of what you want. Or read: http://www.turkupetcentre.net/reports/tpcmod0016.pdf Good luck. ... View more

proc standard will not give you a standardized or studentized residual. Each residual has a unique variance, var(r_i), and you cannot recover this from proc standard (even though I saw a website that says that you can). ... View more

With the addition of one parameter (e.g., one variable in a linear model), -2LL has to decrease by at least 2 for there to be a decrease in AIC. ... View more

proc standard would not be giving you the standardized or studentized residual. You need to obtain the estimated standard error for EACH residual, which is not directly given by NLMIXED.The standard procedure is just scaling by the the variation in the raw residuals (not the same thing). I think getting standardized residuals from NLMIXED will require some coding (post-model-fitting) which would depend on the chosen conditional distribution. It would require the "hat" matrix. ... View more

I think the comment by stat@sas is a bit misleading. If you add variables to a model, the -2 log-likelihood (-2LL) will go down. But the AIC may or may not go down. The AIC adds a penalty for the number of parameters in a model. If the addition of a variable (i.e., parameter) does not reduce the -2LL enough, then the AIC will go up. That is why the AIC is a useful measure of goodness of fit penalized by the number of parameters (it prevents drastic overfitting of a model). It is not exactly a hypothesis test, however. In programs like MIXED, GLIMMIX, etc., a scaled Wald statistic (t or F) is used to test hypotheses. This does not use the -2LL directly. The Wald test can lead to similar conclusions to that obtained by looking at AIC, but not always. There are tests possible based directly on the -2LL (likelihood ratio tests). But even these do not not exactly correspond to the AIC. You have a case where there is a disagreement. It is well known that model selection based on the AIC may lead to an overly complex model. You will have to decide on whether the interaction matters. ... View more

I don't like the idea of automating this based on fixed p values. Model building should involve judgements and interpretation. All the stepwise procedures are VERY dangerous to start with, unless derived models are tested on independent data (at a minimum). p values can be very misleading in stepwise procedures. I highly recommend that you check out the GLMSELECT procedure for better ways of selecting predictor variables. The SAS Global Forum has some nice articles on use of this procedure. ... View more

I just wanted to praise Jill Tao for an outstanding explanation of sweeping and the order of random effects. Explanation of the role of Z and G in constructing V is also great. Many will benefit from this posting. ... View more