SAS IML Memory Issue

[Vivian]

Hi, all:

I have been a IML user for years.

However, I recently encountered a very weird problem when using IML on the linux system.

Basically I wrote a sas program for a new estimation method created with macros and IML, and then did some simulations on linux system.

The SAS IML was failed with the error message as follows:

ERROR: (execution) Unable to allocate sufficient memory. At least 3115064 more bytes required.

operation : CHOOSE at line 578 column 167

operands  : _TEM1001, *LIT1083, dwn

_TEM1001    624 rows    624 cols    (numeric)

*LIT1083      1 row       1 col     (numeric)

         0

dwn    624 rows    624 cols    (numeric)

For the above message, the corresponding codes in my program is "dwn=choose(wn=.,0,wn);" where wn is a 624*624 matrix.

Well, the point is that I did the same simulations (same codes on same server, I promise) last Nov, but everything was fine without memory problem.

Here is the information about the sas version and the linux system I used.

Last Nov. :

NOTE: SAS (r) Proprietary Software 9.3 (TS1M0)

NOTE: This session is executing on the Linux 2.6.18-238.9.1.el5 (LIN X64) platform.

Recently:

NOTE: SAS (r) Proprietary Software 9.3 (TS1M0)

NOTE: This session is executing on the Linux 2.6.18-274.7.1.el5 (LIN X64) platform.

Does the 'out of memory' issue really result from the system difference?

Any clue?

I have been struggling with this problem for a while.

I would appreciate anyone who can help out.

Thanks,

Vivian

[Rick_SAS]

Strange. SAS didn't change, but you are getting different behavior. I have no idea what has changed on your system,

but remember that memory is shared among all applications. If you have a big PDF file open, an email program running, or other applications, those are all using memory, too.

However, maybe I can solve the immediate problem of getting your program to run. Instead of

   dwn=choose(wn=.,0,wn);

try this:

   dwn = wn;  /* copy wn */

   idx = loc(dwn=.); /* find missing */

   if ncol(idx)>0 then dwn[idx]=0; /* replace with zeros */

The above code can be even more memory-efficient if you are willing to overwrite the original copy of wn:

   idx = loc(wn=.);

   if ncol(idx)>0 then wn[idx]=0; /* overwrite wn */

Of course, this might just move the out-of-memory error to another line of the program.

[Vivian]

Thanks a lot for your prompt reply.

As I have used SAS for many years, it is also my first time to have such problem.

I also have no idea what has been chenged in our system.

The staff at our school computer center in fact could not figure out what happened.

Regarding the out-of-memory problem, I did ever edited the codes and unfortunately, like you exppected, it moved the error to another line of the problem.

I even tried to release some memory and still, it didn't work.

That is why I have been struggling for such a log time.

Another interesting thing is that I submitted the same SAS program on a computer with window 7.

Everything works well without memory problem, even with the function CHOOSE.

So, no solution, right?

Anyway, I still appreciate your response.

Oh, by the way, as I have read through your book and started using IML Studio for a while, I have a question in mind.

Can IML Studio work together with some macro codes, like we ususally do in regular IML (PROC IML)?

Thank you very much!!

Vivian

[Rick_SAS]

You can contact technical support to see if they have any advice on the out-of-memory situation.

Regarding macros, SAS/IML Studio programs are not parsed by the macro facility, so it doesn't support macros in the usual way. However, you can

1) call macros within a SUBMIT/ENDSUBMIT block. These are executed in the SAS environment.

2) Use the SYMGET and SYMPUT family of functions to get or set macro variables from within a SAS/IML Studio program. For example, the following statements set a macro variable (The '@' operator is shorthand for 'submit this single SAS statement.'), then reads in the macro and assigns it to an IML variable:

@%let g=5; /* submit global stmt */

five = symgetn("g");

print five;

See p. 99 of my book for more examples.