BookmarkSubscribeRSS Feed
sskt
Obsidian | Level 7

Hello.

 

I have run the Dunnett test on both proc multtest and proc mixed, and the p-values before adjustment match perfectly, but the adjusted p-values do not match and a large difference occurs.
I think the reason why the multtest does not match is because of the seed value set, but the difference is too large for that.
What algorithm is working for these two?
I am using ddfm=satterth as an option.
If it is not possible for me to explain, I would appreciate it if you could provide me with some reference material.

3 REPLIES 3
Rick_SAS
SAS Super FREQ

Please post the SAS statements for each analysis. In particular, how are you using PROC MULTTEST? I do not see Dunnet's test in the documentation. You mention a seed value, so are you using some sort of resampling?

 

I assume for PROC MIXED you are using the LSMEANS statement such as 

LSMEANS Treatment / diff adjust=dunnet;

but please provide the full statements for each procedure.

 

sskt
Obsidian | Level 7

Thank you for your reply.

 

The following is the code used.

 

proc multtest data=raw permutation order=data out=OUT
                            nsample=100000 seed=1;
  class group;
  test mean(epoch / ddfm=sat);
  contrast '1-control' 1 -1  0 0 0;
  contrast '2-control' 1 0 -1 0 0;
  contrast '3-control' 1 0  0 -1 0;
  contrast '4-control' 1 0 0 0 -1;
run;

proc mixed data=raw order=data;
  class group;
  model epoch=group/ddfm=satterth;
  repeated /group=group;
  lsmeans group/adjust=dunnett pdiff=control('control') ;
run;
 
Rick_SAS
SAS Super FREQ

Sorry for the delay. It was a busy week.

 

I don't have your data, so it is difficult to assess the problem.

I suspect the reason that you are not getting similar answers is because your CONTRAST statements are not consistent with DUNNETT's comparison. The DUNNETT comparison is between each treatment group and a control group. Your CONTRAST statements seem to be comparing the first level of the GROUP variable with the 2nd, 3rd, etc.  This is correct if the first level is the control group, so please verify that that is the case.

 

Here is some data and an analysis that we can all use.  When I run this program, the permutation p-values from PROC MULTTEST agree with the adjusted p-values from PROC MIXED to the second decimal place.

 

data raw;
call streaminit(1);
array g[5] $ ('control', '1','2','3','4');
array mean[5] (1 1.4 0.9 2 1);
array sd[5] (1 1.4 0.9 2 1);
length group $7;
do i = 1 to 5;
   group = g[i];
   do obs = 1 to 25;
      epoch = rand('Normal', mean[i], sd[i]);
      output;
   end;
end;
keep group epoch i obs;
run;

proc multtest data=raw permutation order=data out=OUT
                            nsample=100000 seed=1;
  class group;
  test mean(epoch / ddfm=sat);
  contrast '1-control' 1 -1  0 0 0;
  contrast '2-control' 1 0 -1 0 0;
  contrast '3-control' 1 0  0 -1 0;
  contrast '4-control' 1 0 0 0 -1;
  ods select Continuous pValues;
run;

proc mixed data=raw order=data;
  class group;
  model epoch=group/ddfm=satterth;
  repeated /group=group;
  lsmeans group/adjust=dunnett pdiff=control('control') ;
  ods select lsmeans diffs;
run;

sas-innovate-2024.png

Don't miss out on SAS Innovate - Register now for the FREE Livestream!

Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.

 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 3 replies
  • 852 views
  • 3 likes
  • 2 in conversation