topic Re: Memory issues in SAS Procedures

Memory issues

deleted_user — Mon, 07 Apr 2008 13:50:42 GMT

SAS computer - Win XP Pro, 2GB RAM, 2 Processors.

Procedure: Reads 2 1-2GB text data files (fixed record size). sorts each and then mergest the records.

When trying to reduce the run time of the job I noticed that Free RAM on the system never dropped below 1GB but there was a great deal of paging going on. Increasing the RAM (to 2.8GB) did not seem to increase the amount of RAM in use (just under 1GB) decreasing the RAM also did not seem to affect the amount of RAM in use.

What can I do to have SASS increase the amount of RAM it uses for this job and therefore reduce the paging ops (hopefully reducing the time the job takes to run)?

Re: Memory issues

deleted_user — Mon, 07 Apr 2008 16:15:34 GMT

something about buffers

Re: Memory issues

darrylovia — Mon, 07 Apr 2008 16:22:30 GMT

Bill
I believe that there is a SORTSIZE option that is used in the config file, that can be used to increase the amount of RAM used for sorting. I would start there and see if that helps PROC SORT.

-Darryl

Re: Memory issues

Doc_Duke — Mon, 07 Apr 2008 23:39:21 GMT

In SAS 9.1.3, there are some changes to how SAS does sorts in a multiprocessor environment. Read the what's new for SAS 9.1.3.

SAS uses a good bit of disk space for temporary storage during a SORT. You may just be observing that disk use, rather than OS paging.

You might also try re-coding the sort-merge into a PROC SQL procedure. SQL uses a totally different algorithm. I'm not sure that it is faster in your case, but it could be.

Re: Memory issues

deleted_user — Tue, 08 Apr 2008 16:02:13 GMT

I am a network/computer guy, not a SAS programmer (I am helping our SAS users out on this issue). I will forward the comments I get that deal with the coding piece to them.

Someone suggested that I was seeing normal disk io associated with the sort functionality. Just to be clear, I used windows performance monitor to look at disk IO and Paging. The pages/sec counter was very high.