Proc genmod - discrepancy between p-values

[confooseddesi89]

I'm running PROC GENMOD (SAS 9.4) with a Poisson distribution with the factors condition, values_relatedness, condition*values_relatedness, order (of conditions), and condition*order.

The code is as follows:

proc genmod data=DS_WPAT.Night_morn_aftern plots=none;
where condition ~='First' and Valid_Overlap=1;
class Condition Order Subject_ID_byDate values_relatedness;
model Morn_vs_Night_Accuracy= Condition values_relatedness Condition*values_relatedness
Order Condition*Order/ dist=poisson;
repeated subject=Subject_ID_byDate / TYPE=AR(1);
lsmeans condition / pdiff;
title 'Effect of Condition on Morning vs Night Accuracy not including overlapping words';run;title;run;

I'm seeing a discrepancy between the p-values for the effect of Condition reported in the table "Analysis Of GEE Parameter Estimates: Empirical Standard Error Estimates" and the table "Differences of Condition Least Squares Means" (see screenshot 1). This discrepancy does not appear when I run a model with an effect of Condition only (see screenshot 2). For the model in screenshot 1 - which p-value should I "trust"? Any further explanation regarding why these values are different would be much appreciated.

[StatDave]

An LS-mean does not involve just a single parameter estimate in a model like yours involving interactions. For such a model, an LS-mean is a linear combination of several model parameters. To see how each linear combination is defined, add the E option in the LSMEANS statement. For more information on the construction of LS-means, see this section of the documentation.

[confooseddesi89]

Thanks for your reply.

Would you be able to indicate why the PROC GENMOD procedure specifically (compared to another procedure I've used, PROC MIXED, which has the same p-values for effects of condition under "Type 3 Tests of Fixed Effects" and "Differences of Least Squares Means" tables) has a mismatch between the p-values? (I understand these are different procedures/model distributions and may not be directly comparable). Also, most importantly - is my effect of Condition on my outcome significant if the p is < .05 for the "Analysis Of GEE Parameter Estimates" but not "Differences of Condition Least Squares Means"?

[SAS_Rob]

I think this is the difference between the tests themselves.  The parameter estimates/LsMeans are tested using a Wald test while the Type3 test uses a Score test.  If you add the WALD option to the MODEL statement, then it will give you Wald statistics for Type 3 contrasts as well.

[StatDave]

You are asking about two differences. Concerning the p-value difference between the Condition Type3 test and the Condition parameter estimate test, this is due, as Rob says, to the difference in test type. Concerning the p-value difference between the Condition LS-means comparison test and the Condition parameter estimate, the bottom line is that your model involves Condition in an interaction and by definition, there is no single difference between Condition levels. The presence of interaction in the model means that the effect of Condition depends on the levels of the variables it interacts with. If you feel that the results indicate that Condition does not interact with the other variables and you remove the interactions from the model, then the p-value difference should disappear. Alternatively, if you retain the interaction model, then the LS-means at the Condition levels are estimates averaged over the levels of the interacting variables. The way that this is done can be seen in the coefficients applied to the parameter estimates to compute the LS-means. As I mentioned to your earlier, you can see these coefficients by using the E option in the LSMEANS statement. Of course, there are many ways that coefficients could be chosen and the LSMEANS statement does it one way by default. You can alter this using the OM, OM=data-set, or BYLEVEL options.

[confooseddesi89]

bump