topic Re: Insufficient Memory SAS OnDemand in Statistical Procedures

Insufficient Memory SAS OnDemand

mwilli20 — Mon, 27 Mar 2023 16:10:17 GMT

Hello all,

I am trying to run a repeated measures analysis on feeding behavior variables in a Randomized Complete Block Design. The measurements were repeated on animals weekly. I have ~1800 rows of data and want to run it all at once for proper analysis of the effect of treatment, week, and the interaction.

Here is the code I originally ran:

Data ELORA;
infile "/home/mwilli200/ELORA.PhD.2022/Feeding Behaviour Data.csv" dlm=","
firstobs=2;
input ID$ group block tmt$ week dintake wintake timefe visit
timevisit vsize rate meal timemeal msize;
run;
proc sort data=elora;
by id;
run;

*************************************************;
proc glimmix data=ELORA1 plots=studentpanel;
class block tmt ID week;
model dintake= tmt week tmt*week/ ddfm=kenwardroger;
random block block*tmt/ subject=id;
random week/ residual type=arh(1) subject=id;
lsmeans tmt tmt*week week/ pdiff adjust=tukey lines;
*output out=second predicted=pred residual=resid residual(noblup)=mresid
student=studentresid student(noblup)=smresid;
run;

I received the following error message:

ERROR: The SAS System stopped processing this step because of insufficient memory.

I ran the exact same program with only including weeks 1-4 to cut the number of rows down to 480. Below is the error message:

ERROR: The SAS System stopped processing this step because of insufficient memory.
NOTE: PROCEDURE GLIMMIX used (Total process time):
real time 2.83 seconds
user cpu time 0.48 seconds
system cpu time 2.35 seconds
memory 4115670.81k
OS Memory 4136412.00k
Timestamp 03/27/2023 04:00:10 PM
Step Count 51 Switch Count 7
Page Faults 0
Page Reclaims 958039
Page Swaps 0
Voluntary Context Switches 42
Involuntary Context Switches 5
Block Input Operations 0
Block Output Operations 544

I understand there are data limitations for this program (5 gb) but this is my only platform option currently and I have a Mac. Any help would be greatly appreciated.

Melissa,

Re: Insufficient Memory SAS OnDemand

PaigeMiller — Mon, 27 Mar 2023 16:57:27 GMT

SAS On Demand runs on a server, and the memory available is shared between all users. I don't think you can request more memory. The only thing I can think of for you is to try running your code at a different time of day when the server might have fewer users.

Re: Insufficient Memory SAS OnDemand

StatsMan — Tue, 28 Mar 2023 12:30:50 GMT

Do you want to treat block*id and block*tmt*id as random effects? That is what your RANDOM statement is doing. Do id's see different levels of block and block*tmt? If not, then take SUBJECT=ID off the RANDOM statement to fit block and block*tmt as random effects. RANDOM int tmt / subject=block will do that.

ARH(1) fits a covariance parameter for each week. It appears that you have 120 animals in the study, so the full data set has 15 weeks (120*15=1800)? If so, that number of covariance parameters should not pose a problem, unless demand on the server is high. You could try AR(1) to see if you can get that to fit.

Knowing the number of levels to each class effect will help get you a better answer from the community, too.

Re: Insufficient Memory SAS OnDemand

mwilli20 — Mon, 10 Apr 2023 13:31:39 GMT

Hello,

I have taken the subject out of my first random statement, and I think we are almost getting there. I can provide more information on the class levels:

block = 2 levels, was predetermined by weight at the beginning of the study

tmt = 4 different dietary treatments (1 per ID)

ID = 120 different animals (each see only one treatment, and one block)

week = 1-19 (***each animal has 1-X number of weeks of data, this is not the same for all animals as they were serially removed from the study).

This is my current code:

proc glimmix data=ELORA plots=studentpanel;
class block tmt ID week;
model dintake= tmt week tmt*week/ ddfm=kenwardroger;
random block block*tmt;
random week/ residual type=arh(1) subject=id;
lsmeans tmt tmt*week week/ pdiff adjust=tukey lines adjdfe=row;
output out=second predicted=pred residual=resid residual(noblup)=mresid
student=studentresid student(noblup)=smresid;
run;

This is my log error:

WARNING: The initial estimates did not yield a valid objective function.

Thank you for your continued support!

Re: Insufficient Memory SAS OnDemand

StatsMan — Wed, 12 Apr 2023 11:58:26 GMT

This paper provides some guidance when dealing with the warning message you receive with your model and data. You can modify the INITITER= option on the PROC GLIMMIX statement to use more than the default 4 iterations to find starting values for this model. You might try specifying your own starting values as well using the PARMS statement.

If you have not done so already, try PROC SGPLOT to plot the trajectories for each subject. Use week as your X variable and your response as the Y. You could group the id's by the treatment they received, too. This plot would give you some idea as to how the response varies over the weeks and if your model is appropriate for this data.

Re: Insufficient Memory SAS OnDemand

mwilli20 — Tue, 18 Apr 2023 17:05:14 GMT

Hello all,

I am still receiving a error warning when running the below code:

proc glimmix data=ELORA plots=studentpanel inititer=3000;
class block tmt ID week;
model dintake= tmt week tmt*week/ ddfm=kenwardroger;
random block block*tmt;
random week/ residual type=ar(1) subject=id;
lsmeans tmt tmt*week week/ pdiff adjust=tukey lines adjdfe=row;
run;

error:

WARNING: The initial estimates did not yield a valid objective function.

I am wondering if it is because not all subjects have an observation for every week of the study? How could I work with this?

Thanks,

Re: Insufficient Memory SAS OnDemand

mwilli20 — Tue, 18 Apr 2023 17:12:03 GMT

Hello again everyone,

Based on my previous reply, I decided to take out the second random statement, and the code seemed to run. What would I have lost in this analysis by removing the red section? Is this acceptable?

Thanks,

proc glimmix data=ELORA plots=studentpanel inititer=3000;
class block tmt ID week;
model dintake= tmt week tmt*week/ ddfm=kenwardroger;
random block block*tmt;
*random week/ residual type=ar(1) subject=id;
lsmeans tmt tmt*week week/ pdiff adjust=tukey lines adjdfe=row;
run;