Performance differences: SAS/Server vs SAS/PC

[deleted_user]

1) For many people to access the same data, it is much more efficient, user friendly, etc. to access the data from a central common database, not a flat file.


2) Data flow = data ----> SAS server --(subset of results)--> EG


3) Data, if a flat file, should be on the same server as the SAS server for maximum performance.

[deleted_user]

I wonder if part of the solution is also to tweak the configurations at the application tier, to ensure optimal IO. What do you guys normally set in sas*.cfg? is there a typically recommended value for -memsize & -sortsize? Thanks for your advice in advance

- Josh

[deleted_user]

-memsize 0 which should be the default. Let SAS use whatever it can get.

-threads
-asynchio
-bufsize 64k


Set -bufno to at least 2+ the number of IO paths available to the storage

memsize, maxmem..., realmem... should all be 0, the default, to allow SAS access to all the memory on the box.

[deleted_user]

I was under the impression that if the script needs more memory, you'd need to define -memsize in the script or to set it globally in sasv9.cfg. Are there situations whereby the sas application actually takes up too much memory or does sas ration out the available memory as it sees fit?

[deleted_user]

SAS is not a scripting language. It is a programming language/system. The SAS code is JIT compiled in blocks by the "sas" program/compiler. Code blocks are things like a proc or a data step or an SQL statement. SAS was orginally created when memory was scarce, and datasets were huge and had to be stored on tape; thus, SAS is very frugal and efficient with its use of memory. SAS requests memory from the OS, as it needs it, and then gives it back when it's done with it. The only thing on SAS that can be a memory pig is sorting or the use of CLASS variables instead of BY variables for very diverse datasets. It is generally best, just to let SAS have access to all the memory and let it work. Then if an out of memory condition occurrs due to multiple SAS processes running concurrently, it will probably be due to sorting, and sortsize can be adjusted accordingly.

EG is a java thing, and uses memory differently, attempting to restrict it may not be possible and is definitely ill advised.

A number of years ago, I set up a Unix SAS server on an AIX box and thought we were running out of memory, so I kept increasing the system memory until we had maxed out the box at 32 GB and it was still showing 100% memory utilization. Then I discovered that AIX clumped both file caching and process memory into the utilization numbers. So I found a way to determine maximum amount of memory all the 7 groups of > 20 SAS users were using. It was < 4 GB and we were processing > 50 GB of data regularly. Needless to say, I felt rather foolish. After some additional OS level memory and IO tuning, we ended up with < 40% memory utilization, as reported by AIX.

[prholland]

Chuck,

Small correction: EG is not a "java thing", but a ".NET thing", which is one of the reasons why it is only available for Windows platforms. However, I agree with you that attempting to restrict its memory usage is definitely ill-advised.

...........Phil Holland

[deleted_user]

Oops.

I guess the look and feel are unrecognizably the same to me.

[deleted_user]

That might explain why there arent any linux EG clients yet. which is a pity since alot of university undergraduates are probably running linux on their workstation. But does that mean something can or needs to be done to the workstation to make EG run optimally? Message was edited by: Joshua

[Kenny]

It seems to me that setting the memsize to zero would not be a good thing in a multi-user environment. And BTW, the default memsize is not 0 - at least not with our install. I was told by SAS tech support to increment memsize as needed until you find a good balance. Starting out at 0 could give multiple users access to ALL available memory, causing a shortage for subsequent users.

Perhaps I am wrong, but you might want to invetigate.

[deleted_user]

From the SAS documentation for Systems Options for Windows for MEMSIZE

Details


The operating system may use additional amounts of memory. The memory used by SAS includes virtual memory and is therefore not limited to RAM. If MEMSIZE is set to a value that is too low and could cause performance problems, the value of MEMSIZE is determined by SAS.

For optimal performance, use the default value of 0 for MEMSIZE.



I have used MEMSIZE=0 in the past with no ill effects for over 20 concurrent heavy users. memory is generally limited to BUFNO*BUFSIZE*(number of current/extent data sets in use). As I said earlier, the only break from this that I've seen is related to SORTSIZE and the resulting amount of memory used for sorting and classification variables. Message was edited by: Chuck

[deleted_user]

If you set it to memsize 0, isnt there a possibility that all the EG sessions will consume too much memory (given limited memory and many concurrent sessions), causing the box to crash? But I am also concerned that memory is wasted when sessions that need little memory are set by default to 216 or 512k. I couldnt find the reference to -memsize in http://support.sas.com/documentation/installcenter/913/hosts/index.html#f. do you have the online link to this piece of information? thanks for your advice

-Josh

[deleted_user]

How many EG sessions are you talking about?

If you set memsize=2G, and you start an EG session, SAS will NOT take 2 GB. You may be saying it's ok, but it won't.

In fact, I have yet to see a single SAS session go over 1 GB.
the largest consumption I have seen is about 700 MB, and that was due to a huge number (thousands) of classification variables.

http://support.sas.com/onlinedoc/913/docMainpage.jsp?_topic=lrcon.hlp/a002299817.htm

SAS Online documentation for 9.1.3
Base SAS
Operating Specific Information
SAS Companion for Microsoft Windows
Features of the SAS Language for Windows
System Options Under Windows

[deleted_user]

I'd probably assume 10 to 15. Its a small number, but the box is poorly dimensioned and every little misconfiguration is a stab at its very old heart. Thats why I am peering through all these arcane references (e.g. Paper 277-26 Windows NT Server Configuration and Tuning) to give an old box (2000 or 2003) as much a boost as I can afford
-
I wonder if sas 9 benefits from multi-threading. I'd assume that the more cpus, the faster the server is bound to run, though I am not sure if the application benefits in a proportional manner for single or concurrent jobs (x 2 cpu leading to x 2 work done per cpu cycle or per unit time)
-
I noted that in SAS 9.1 companion for windows, it says "For optimal performance, use the default value of 0 for MEMSIZE". But in "Configuration guidelines for Microsoft 2003 on the HP Integrity servers with SAS V9.1" that -memsize should ideally be set to (total physical size - 512mb) / # of concurrent SAS jobs. Hence, I wonder if there are other considerations to set -memsize for an environment that expects many concurrent sessions.

[deleted_user]

I believe I've stated this before, that I had over 20 concurrent heavy users and the total SAS memory consumption was < 4 GB on the SAS server.

I'm gotten a little confused here from our discussions.
Is the underlying server going to be Unix or Windows?

[deleted_user]

I think you are right. I am running both wintel and unix servers, so EG has to connect to both. I just wanted to put your observations in the context of what the various documents from SAS say about -memsize, but I've concluded there is no formula but a judgement call.