BookmarkSubscribeRSS Feed
qataqomes
Calcite | Level 5

Hello, all. I have a two way ANOVA testing the effects of Fungicide and (fungal) Isolate on a Log10EC50 response value (EC50 being the effective chemical concentration that inhibits growth halfway between a minimum and maximum response). I know that my groups of Fungicide*Isolate have very heterogeneous variance (variances range from 0.00073 to 0.87599), and I am trying to fit the model with hetereogeneous variance using the following code.

 

ANOVA Code

PROC mixed data=IMPORT; 
   TITLE3 "ANOVA Comparing LogEC50's, group=Isolate*Fungicide"; 
   class Trial Fungicide Plate Isolate;
   model LogEC50 = Fungicide|Isolate / ddfm=KR outp=outH;
   Random Plate(Trial) / group=Fungicide*Isolate;
run;

 I am receiving a memory error. I have a large dataset (over 2,800 observations - the attached PDF shows Class Level Information; there will be about 390 groups that need to be fitted for heterogeneous variance). I have included the fullstimer option to aid in diagnosis. 

 

LOG

ERROR: The SAS System stopped processing this step because of insufficient memory.
NOTE: The PROCEDURE MIXED printed pages 4-5.
NOTE: PROCEDURE MIXED used (Total process time):
real time 0.13 seconds
user cpu time 0.06 seconds
system cpu time 0.07 seconds
memory 5162.28k
OS Memory 36272.00k
Timestamp 02/25/2016 04:02:39 AM
Step Count 31 Switch Count 223
Page Faults 0
Page Reclaims 695
Page Swaps 0
Voluntary Context Switches 898
Involuntary Context Switches 719
Block Input Operations 0
Block Output Operations 40464
 
I am looking for a way to conduct the analysis with heterogeneous variance, or suggestions on perhaps transformations to assist with heterogeneity problems? I am running the code through SAS University Edition/SAS Studio, so I recognize that there may be memory problems associated with this platform? I have access to SAS 9.4 on library computers but a different memory error appears there (see below) that I cannot fix due to lacking administrator privileges to apply the hot fix.
ERROR: Integer overflow on computing amount of memory required. 
       A request to allocate the memory cannot be honored.
 
Any suggestions would be very welcome. I hope I provided enough detail, but would be more than willing to add any information that would be helpful! Thank you.
2 REPLIES 2
lvm
Rhodochrosite | Level 12 lvm
Rhodochrosite | Level 12

You could switch to PROC HPMIXED, designed for large-scale problems.

However, I think you want the residual to have different groups, not the random effect.

repeated / group=fungicide*isolate;

In fact, based on the limited information, plate(trial) could be your residual, which would mean that you have an over-parameterized model (since the residual is already there). You might want to try:

random int / sub=trial;

repeated / group=fungicide*isolate;

 

With all that said, it is usually not good statistical practice to fit so many variances in a model. This usually reduces your power, and can lead to all kinds of estimation problems, even when there is enough memory.

lvm
Rhodochrosite | Level 12 lvm
Rhodochrosite | Level 12

...One more thing.... Quoting from the HPMIXED User's Guide (applies also to MIXED):

"You should exercise caution in defining the GROUP effect, because strange covariance patterns can result with its misuse. Also, the GROUP effect can greatly increase the number of estimated covariance parameters, which can adversely affect the optimization process".

 

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 2 replies
  • 1288 views
  • 2 likes
  • 2 in conversation