Out of Memory Error

[Rasheed]

I am working on SAS program where I needed to create many data sets and I have successfully created data sets from CD_1 to CD_1001000 by using following sas command

CD='CD_1':'CD_1001000';                                                                               

do s=1 to 100;                                                                               

do k = 1 to 1000;                                                                               

sk=catt(s,k);                                                                               

skn=inputn(sk,'6.',6,0);                                                                               

call execute('create ' + CD[skn] +'  from ' + C[skn] + ';');                                                                               

call execute('append from ' + C[skn] + ';') ;                                                                               

call execute('close ' + CD[skn] + ';' ); 

After doing this when want to combine all these data sets I got out of memory error in log window, I am using following command to combine data sets

Data CCD;                                                                               

set CD_:;                                                                               

rename COL1=C;                                                                               

run; 

Please help

[LinusH]

Posting the vital part of the log would help.

Also, what are you trying to achieve?

Sounds tedious to create all those data sets, and the combine them again. Perhaps there's another less resource intense way to do this.

[Rasheed]

Actually It is a long SAS program that I am working on it, in short, I am doing 100 simulations and for each simulated data set I did 1000 bootstrap sampling (from higher order cross over designs for multivariate bioequivalence) there for each combination say simulation 1 and bootstrap sample 1 I need to create a data set I called it CD_1 therefore I needed to create data set CD_1 to CD_1001000.

Log error

497  Data CCD;

498  set CD_:;

ERROR: Out of memory.

ERROR: Out of memory.

ERROR: Out of memory.

ERROR: Out of memory.

[SASKiwi]

It appears your program is producing hundreds of datasets then trying to read them all in one DATA step. I'm not surprised you are running out of memory.

I suggest you consider a different approach that doesn't involve hundred's of datasets. I've never struck a problem that requires so many.

Unfortunately I've no idea what you are trying to do with your code as you haven't described your problem so I can't suggest a better way.

[Rasheed]

Actually It is a long SAS program that I am working on it, in short, I am doing 100 simulations and for each simulated data set I did 1000 bootstrap sampling (from higher order cross over designs for multivariate bioequivalence) there for each combination say simulation 1 and bootstrap sample 1 I need to create a data set I called it CD_1 therefore I needed to create data set CD_1 to CD_1001000.

Log error

497  Data CCD;

498  set CD_:;

ERROR: Out of memory.

ERROR: Out of memory.

ERROR: Out of memory.

ERROR: Out of memory.

Re: Out of Memory Error

[RW9]

I would suggest you post, in the statistical forum area, what you are attempting to do.  The people there may be able to help you define a proper model for the task.  Alternatively, post your code here, some sample data and an explanation of what you are trying to achieve.  Creating that number of datasets is obviously not the way to proceed.

[Ksharp]

Try proc append .

[Kurt_Bremser]

You're trying to feed 100.000 operations into the input buffer of the main interpreter, which causes an overflow.

Rethink your basic design, and make it less complicated.

[Astounding]

Also note, are you creating unique data sets?  Can your logic distinguish s=1 and k=11 from s=11 and k=1?  You might need to use something like sk=catt(put(s,z3.), put(k,z4.));