Tips on making programs more efficient - server gives warning: physical memory usage too high

[chandler]

We use a server with operating system Windows Server 2008 Standard, 64 bit.  SAS for Windows is loaded as release 9.2,  64 bit version.

Our Information Technology department is not well versed on SAS and the needs of the SAS System.  There is a monitoring system in place and it keeps giving us these warnings:  "Spectrum ALARM - Physical Memory Usage Too High".   Has anybody ever dealt with this issue ?

Some have suggested that the problem may be a few programmers, new to SAS,  running code that is not efficient, thus, using up a lot of physical memory.   My question is What tips can I give to the new guys about how to write more efficient code, without sending them off to a SAS training course?

[Astounding]

There are a number of places that new users can use a lot of memory.  This is only a guess at what might apply in your shop.

In a DATA step, new users have to realize that these steps are different:

data new (keep=a list of final variables);

set old;

** calculations;

run;

data new (keep=a list of final variables);

set old (keep=just the variables I need to read in);

** calculations;

run;

The first step reads in all of the variables.  If you have a gazillion variables, that can consume memory.

In PROC FREQ, the software tracks all combinations.  So this table would require enormous amounts of memory:

proc freq data=purchases;

   tables credit_card_no * date / noprint out=counts;

run;

In PROC MEANS (or PROC SUMMARY), the CLASS statement tracks all levels at once.  That can also consume memory:

proc summary data=purchases;

   class credit_card_no;

   var amount;

   output ...

run;

But there's no guarantee that the fault lies with the users.  And here at the discussion forum, we don't really have much of a clue as to what the users are actually doing.  These are just some pitfalls using commonplace tools.

Good luck.

[FriedEgg]

You can setup restrictions to memory consumption by using SAS system options.

Inspect you current setup by running the following:

proc options group=memory; run;

you can limit the memory SAS will use by altering these options

Here are links to additional information about the memory options:

System administration: Memory	SORTSIZE= System Option	Specifies the amount of memory that is available to the SORT procedure.
	SUMSIZE= System Option	Specifies a limit on the amount of memory that is available for data summarization procedures when class variables are active.

MEMSIZE System Option under Windows, UNIX, OpenVMS, z/OS

REALMEMSIZE System Option under Windows, UNIX, OpenVMS, z/OS

MEMMAXSZ System Option: Windows

MEMLIB System Option: Windows

MEMCACHE System Option: Windows

LOADMEMSIZE System Option: Windows

MEMBLKSZ System Option: Windows

Be careful when setting the options as you can cause errors to occur by setting options too low and you can also dramatically increase disk operations and decrease performance.