topic Difference between proc multtest and proc mixed in Dunnett test in SAS Procedures

Difference between proc multtest and proc mixed in Dunnett test

sskt — Sun, 29 May 2022 16:15:32 GMT

Hello.

I have run the Dunnett test on both proc multtest and proc mixed, and the p-values before adjustment match perfectly, but the adjusted p-values do not match and a large difference occurs.
I think the reason why the multtest does not match is because of the seed value set, but the difference is too large for that.
What algorithm is working for these two?
I am using ddfm=satterth as an option.
If it is not possible for me to explain, I would appreciate it if you could provide me with some reference material.

Re: Difference between proc multtest and proc mixed in Dunnett test

Rick_SAS — Mon, 30 May 2022 10:49:24 GMT

Please post the SAS statements for each analysis. In particular, how are you using PROC MULTTEST? I do not see Dunnet's test in the documentation. You mention a seed value, so are you using some sort of resampling?

I assume for PROC MIXED you are using the LSMEANS statement such as

LSMEANS Treatment / diff adjust=dunnet;

but please provide the full statements for each procedure.

Re: Difference between proc multtest and proc mixed in Dunnett test

sskt — Mon, 30 May 2022 14:24:53 GMT

Thank you for your reply.

The following is the code used.

proc multtest data=raw permutation order=data out=OUT
                            nsample=100000 seed=1;
  class group;
  test mean(epoch / ddfm=sat);
  contrast '1-control' 1 -1  0 0 0;
  contrast '2-control' 1 0 -1 0 0;
  contrast '3-control' 1 0  0 -1 0;
  contrast '4-control' 1 0 0 0 -1;
run;

proc mixed data=raw order=data;
  class group;
  model epoch=group/ddfm=satterth;
  repeated /group=group;
  lsmeans group/adjust=dunnett pdiff=control('control') ;
run;

Re: Difference between proc multtest and proc mixed in Dunnett test

Rick_SAS — Sun, 05 Jun 2022 11:54:46 GMT

Sorry for the delay. It was a busy week.

I don't have your data, so it is difficult to assess the problem.

I suspect the reason that you are not getting similar answers is because your CONTRAST statements are not consistent with DUNNETT's comparison. The DUNNETT comparison is between each treatment group and a control group. Your CONTRAST statements seem to be comparing the first level of the GROUP variable with the 2nd, 3rd, etc. This is correct if the first level is the control group, so please verify that that is the case.

Here is some data and an analysis that we can all use. When I run this program, the permutation p-values from PROC MULTTEST agree with the adjusted p-values from PROC MIXED to the second decimal place.

data raw;
call streaminit(1);
array g[5] $ ('control', '1','2','3','4');
array mean[5] (1 1.4 0.9 2 1);
array sd[5] (1 1.4 0.9 2 1);
length group $7;
do i = 1 to 5;
   group = g[i];
   do obs = 1 to 25;
      epoch = rand('Normal', mean[i], sd[i]);
      output;
   end;
end;
keep group epoch i obs;
run;

proc multtest data=raw permutation order=data out=OUT
                            nsample=100000 seed=1;
  class group;
  test mean(epoch / ddfm=sat);
  contrast '1-control' 1 -1  0 0 0;
  contrast '2-control' 1 0 -1 0 0;
  contrast '3-control' 1 0  0 -1 0;
  contrast '4-control' 1 0 0 0 -1;
  ods select Continuous pValues;
run;

proc mixed data=raw order=data;
  class group;
  model epoch=group/ddfm=satterth;
  repeated /group=group;
  lsmeans group/adjust=dunnett pdiff=control('control') ;
  ods select lsmeans diffs;
run;