topic Re: SAS System Options, Is MAX Too Much?? in SAS Programming

SAS System Options, Is MAX Too Much??

NKormanik — Fri, 09 Jun 2017 23:27:30 GMT

Using a new stand-alone computer that has 32 GB RAM, 6 core processors. Trying to tell SAS to go ahead and use as much of the resources as possible.

Trouble is, with the settings below, a job is SLOWER than with a previous average computer.

Please advise on which of these 'options' likely needs to be changed:

-JREOPTIONS=(
-Xms1g
-Xmx1g )

-BUFNO MAX
-BUFSIZE MAX
-CPUCOUNT 6
-DMSLOGSIZE = 999999
-IBUFNO MAX
-IBUFSIZE MAX
-LRECL = MAX
-MEMSIZE = MAX
-REALMEMSIZE = MAX
-SET SAS_NO_RANDOM_ACCESS = "0"
-SORTSIZE = MAX
-SUMSIZE = MAX
-THREADS
-UBUFNO MAX

Thanks very much!

Nicholas Kormanik

Re: SAS System Options, Is MAX Too Much??

Sven111 — Sat, 10 Jun 2017 00:19:43 GMT

I'm not an expert on this stuff, but if I recall you shouldn't set the REALMEMSIZE bigger than the amount of physical RAM your system has. I think there's diminishing returns on the various buffer related options and it's somewhat filesystem dependent on what's best. I think I just wrote a quick shell script that iterated over a range of values to try and find the sweet spot.

Re: SAS System Options, Is MAX Too Much??

Shmuel — Sat, 10 Jun 2017 00:18:49 GMT

Defining MAX for so many options makes system great overhead.

try change options to:

-BUFNO 3
-BUFSIZE MAX
-CPUCOUNT 6                         /* is your PC with 6 CPUs ? */
-DMSLOGSIZE = 999999
-IBUFNO 3
-IBUFSIZE MAX
-LRECL = MAX
-MEMSIZE = 0                        /* 0 means using MAX with HD if need */
-REALMEMSIZE =                  /* use default value */
-SET SAS_NO_RANDOM_ACCESS = "0"
-SORTSIZE = MAX
-SUMSIZE = MAX
-THREADS
-UBUFNO MAX

With so many options, you should try changing each time one option and check for performence.

Re: SAS System Options, Is MAX Too Much??

NKormanik — Sat, 10 Jun 2017 01:05:39 GMT

Changing one option at a time, through all possibilities for each option,
would take a year or so to complete.

Hoping for a more direct approach for finding the "sweet spot."

##- Please type your reply above this line. Simple formatting, no
attachments. -##

Re: SAS System Options, Is MAX Too Much??

Patrick — Sat, 10 Jun 2017 03:02:57 GMT

@NKormanik

Performance tuning can be quite intense. What's going to work best for you really depends on your environment and also how you're going to use SAS, i.e. are your processes mainly I/O bound or mainly CPU bound, are they often multithreaded or not.

http://support.sas.com/kb/51/659.html

SAS tables for example are up to SAS Viya single threaded so how many CPU's you've got won't make any difference there.

Your first step will be to implement valid test cases for your actual SAS usage. You will then also have to make sure that there aren't any other processes running while you're testing.

Changing one option at a time, through all possibilities for each option,would take a year or so to complete

You could write a script which batch submits your test cases. And you could implement a loop which passes in different parameter values to the SAS batch command and then just collect the stats. That would allow you to test a lot of cases in a fully automated fashion.

Re: SAS System Options, Is MAX Too Much??

SASKiwi — Sat, 10 Jun 2017 03:26:57 GMT

If I were you I would start with the SAS defaults for the options you listed. How does performance compare to your previous computer just with defaults? You need a benchmark before you start tweeking and the defaults will provide that. There is a good reason for the default values - they are generally pretty efficient for most types of usage.

Re: SAS System Options, Is MAX Too Much??

Ksharp — Sat, 10 Jun 2017 03:30:27 GMT

Sometime, if you could try SPDE engine.

libname xx spde 'c:\temp\';

options work=xx ;

Re: SAS System Options, Is MAX Too Much??

Kurt_Bremser — Sat, 10 Jun 2017 09:25:02 GMT

Before you start changing settings, get a clear picture of your performance issues. This means using tools to watch memory & cpu usage, and i/o throughput, and the "busy" states of the disks. Misconfigured storage can slow a supercomputer down to a crawl.

Allowing SAS to use all resources might deprive the operating system of the chance to optimise resource use itself. SAS provides whitepapers for all supported platforms that deal with optimising the settings for a given platform. In my experience, they have been very helpful (SAS on AIX/pSeries).

Re: SAS System Options, Is MAX Too Much??

NKormanik — Sun, 11 Jun 2017 23:29:39 GMT

Ksharp, tried SPDE on this stand-alone computer. Seemed everything took
longer. May have done it wrong, though.

Everyone else, thanks for your tips.

List below seems to work fine. Was just attempting to maximize resource
use. Any further suggestions, love to hear 'em.

-ALIGNSASIOFILES

-BUFNO 10

-BUFSIZE 64K

-CPUCOUNT 6

-DMSLOGSIZE 999999

-FULLSTIMER

-IBUFNO 10

-IBUFSIZE 32767

-LRECL MAX

-MEMSIZE 10G

-NOPRNGETLIST

-REALMEMSIZE 8G

-SET SAS_NO_RANDOM_ACCESS "0"

-SORTSIZE 8G

-SOURCE

-SOURCE2

-SUMSIZE 8G

-THREADS

-UBUFNO 10

-UBUFSIZE 64K

-VALIDVARNAME ANY

##- Please type your reply above this line. Simple formatting, no
attachments. -##

Re: SAS System Options, Is MAX Too Much??

ChrisNZ — Sun, 11 Jun 2017 23:51:42 GMT

libname XX spde 'c:\temp\' partsize=200g compress=binary;

options work=XX;

should work better:

- normally no need to split data files when there is only one path, I/O is usually the bottleneck, and you might as well keep it sequential

- much reduced I/O with SPDE's fantastic compression algorithm

Re: SAS System Options, Is MAX Too Much??

ChrisNZ — Mon, 12 Jun 2017 00:15:08 GMT

Here are two schemas taken from the book

High Performance SAS Coding

that summarise how the memory settings are nested.