ANOVA test: Modify degrees of freedom using a complex survey

[alexgonzalez]

Hello,

 I am trying to compare multiple means using a sample from a survey with a complex design (500 bootstrap weights available) using SAS EG 5.1. In this type of surveys, the denominator degrees of freedom (df) = # of clusters – # of strata. When using PROC SURVEYREG to perform an ANOVA analysis, the default is # of observation - # of parameters. There are cases where I perform multiple comparison tests, and the results from multiple comparisons are in contradiction to the results from ANOVA. One possible solution would be to correct and fix degrees of freedom in those tests. Nevertheless, the denominator degrees of freedom for the ANOVA test cannot not be modified. Could you please advise?

Thanks,

AG.

[ballardw]

You should show your Proc Surveyreg code as a minimum. That would avoid getting suggestions to implement options you are already using and possibly give us an idea of where to make suggestions.

Also, how are doing "multiple comparison tests"?

[alexgonzalez]

Hi,
Thanks for your suggestion. Let's say in comparing mean age by quartiles (Q1-Q4) of a given variable C. The code I'm using is:
proc surveyreg data=data varmethod=brr;
class C
model age = C / noint solution clparm anova ;
weight bsw;
repweights bsw1-bsw500;

estimate 'Q1 vs Q2' C 1 -1 0 0;
estimate 'Q1 vs Q3' C 1 0 -1 0;
estimate 'Q1 vs Q4' C 1 0 0 -1;
run;

If the option DF is used, it modifies the DF of the multiple contrast (i.e. ESTIMATE option), but not for the ANOVA table. In this case, the DF of the comparisons by two is 500 while for the ANOVA test is number of observation - 1.

Thanks,

AG.

[ballardw]

From the documentation for Surveyreg:

PROC SURVEYREG produces tests for the significance of model effects, regression parameters, estimable functions specified in the ESTIMATE statement, and contrasts specified in the CONTRAST statement. It computes all these tests taking into account the sample design. The degrees of freedom for these tests differ from the degrees of freedom for the ANOVA table, which does not consider the sample design.

So it sounds like trying to force the same DF in both is going to be fruitless.

Note this bit in Degrees of Freedom:

For the BRR method (including Fay’s method ) without a REPWEIGHTS statement, the denominator df equals the number of strata.

Since you do not include a STRATA variable you one 1 stratum and it appears that without the repweights statement you might force the Estimates to 1DF matching Anova but I doubt that is what you really want.