topic Two Way ANOVA fitting Heteroskedasticity - Running out of Memory in SAS Procedures

Two Way ANOVA fitting Heteroskedasticity - Running out of Memory

qataqomes — Thu, 25 Feb 2016 14:18:15 GMT

Hello, all. I have a two way ANOVA testing the effects of Fungicide and (fungal) Isolate on a Log10EC50 response value (EC50 being the effective chemical concentration that inhibits growth halfway between a minimum and maximum response). I know that my groups of Fungicide*Isolate have very heterogeneous variance (variances range from 0.00073 to 0.87599), and I am trying to fit the model with hetereogeneous variance using the following code.

ANOVA Code

PROC mixed data=IMPORT; 
   TITLE3 "ANOVA Comparing LogEC50's, group=Isolate*Fungicide"; 
   class Trial Fungicide Plate Isolate;
   model LogEC50 = Fungicide|Isolate / ddfm=KR outp=outH;
   Random Plate(Trial) / group=Fungicide*Isolate;
run;

I am receiving a memory error. I have a large dataset (over 2,800 observations - the attached PDF shows Class Level Information; there will be about 390 groups that need to be fitted for heterogeneous variance). I have included the fullstimer option to aid in diagnosis.

LOG

ERROR: The SAS System stopped processing this step because of insufficient memory.
NOTE: The PROCEDURE MIXED printed pages 4-5.
NOTE: PROCEDURE MIXED used (Total process time):
real time 0.13 seconds
user cpu time 0.06 seconds
system cpu time 0.07 seconds
memory 5162.28k
OS Memory 36272.00k
Timestamp 02/25/2016 04:02:39 AM
Step Count 31 Switch Count 223
Page Faults 0
Page Reclaims 695
Page Swaps 0
Voluntary Context Switches 898
Involuntary Context Switches 719
Block Input Operations 0
Block Output Operations 40464

I am looking for a way to conduct the analysis with heterogeneous variance, or suggestions on perhaps transformations to assist with heterogeneity problems? I am running the code through SAS University Edition/SAS Studio, so I recognize that there may be memory problems associated with this platform? I have access to SAS 9.4 on library computers but a different memory error appears there (see below) that I cannot fix due to lacking administrator privileges to apply the hot fix.

ERROR: Integer overflow on computing amount of memory required. 
       A request to allocate the memory cannot be honored.

Any suggestions would be very welcome. I hope I provided enough detail, but would be more than willing to add any information that would be helpful! Thank you.

Re: Two Way ANOVA fitting Heteroskedasticity - Running out of Memory

lvm — Tue, 01 Mar 2016 17:14:06 GMT

You could switch to PROC HPMIXED, designed for large-scale problems.

However, I think you want the residual to have different groups, not the random effect.

repeated / group=fungicide*isolate;

In fact, based on the limited information, plate(trial) could be your residual, which would mean that you have an over-parameterized model (since the residual is already there). You might want to try:

random int / sub=trial;

repeated / group=fungicide*isolate;

With all that said, it is usually not good statistical practice to fit so many variances in a model. This usually reduces your power, and can lead to all kinds of estimation problems, even when there is enough memory.

Re: Two Way ANOVA fitting Heteroskedasticity - Running out of Memory

lvm — Tue, 01 Mar 2016 17:20:06 GMT

...One more thing.... Quoting from the HPMIXED User's Guide (applies also to MIXED):

"You should exercise caution in defining the GROUP effect, because strange covariance patterns can result with its misuse. Also, the GROUP effect can greatly increase the number of estimated covariance parameters, which can adversely affect the optimization process".