PC Ram usage cap at 7.8 Gig

[Reeza]

If it's University Edition it's on a virtual machine, using Oracle VirtualBox or is it a purchased copy from your University which is a different version, I think that's usually referred to as an Academic License but I could be wrong on that. Since you're referring to changing the config file you're likely using an academic license if you can modify the config file. 

I can't see the timings in your screenshots, they're way too small for me to read. 

[hhchenfx]

SLOWFAST

[ChrisNZ]

This thread is so confusing.

What was running when you had the circled ram usage? the data step or the proc sort?

It looks like it was the sort since the sort code is highlighted.

How are we then supposed to comment on the data step?

Also I notice that the "idle "memory usage is 3GB on one side, and 9GB on the other. Not the same config at all.

Please run  proc options group=memory;run;  on both machines and post the result.

[hhchenfx]

The slow one:

Group=MEMORY
SORTSIZE=268435456
Size parameter for sort
SUMSIZE=0 Upper limit for data-dependent memory usage during summarization
MAXMEMQUERY=0 Maximum amount of memory returned when inquiring as to available space
MEMBLKSZ=16777216 Size of memory blocks allocated to support MEMLIB and MEMCACHE options.
MEMMAXSZ=2147483648
Maximum amount of memory allocated to support MEMLIB and MEMCACHE options.
LOADMEMSIZE=0 Suggested memory limit for loaded SAS executables
MEMSIZE=2147483648
Specifies the limit on the total amount of memory to be used by the SAS
System
REALMEMSIZE=0 Limit on the total amount of real memory to be used by the SAS System
NOTE: PROCEDURE OPTIONS used (Total process time):
real time 0.00 seconds
cpu time 0.01 seconds

Fast one

Group=MEMORY
SORTSIZE=268435456
Size parameter for sort
SUMSIZE=0 Upper limit for data-dependent memory usage during summarization
MAXMEMQUERY=0 Maximum amount of memory returned when inquiring as to available space
MEMBLKSZ=16777216 Size of memory blocks allocated to support MEMLIB and MEMCACHE options.
MEMMAXSZ=2147483648
Maximum amount of memory allocated to support MEMLIB and MEMCACHE options.
LOADMEMSIZE=0 Suggested memory limit for loaded SAS executables
MEMSIZE=2147483648
Specifies the limit on the total amount of memory to be used by the SAS
System
REALMEMSIZE=0 Limit on the total amount of real memory to be used by the SAS System
NOTE: PROCEDURE OPTIONS used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

It seems there is no difference at all.

[Reeza]

Can you set the fullstimer option and run both and post the full log from each. 

options fullstimer;

[ChrisNZ]

Also runs this

  data _null_; FREERAM_MB=input(getoption('xmrlmem'),20.)/1024/1024; put FREERAM_MB= 8.; run;

And comment on this:

  What was running when you had the circled ram usage? the data step or the proc sort?

  Also I notice that the "idle "memory usage is 3GB on one side, and 9GB on the other. Why?

[hhchenfx]

I run the data step then sort. So the spike should assocaited with sort. I will run a clearer test later.

The idle memory issue, most likely, I open web, Doc, Excel on that fast PC.

Slow PC
FREERAM_MB=1633
NOTE: DATA statement used (Total process time):
real time 0.07 seconds
cpu time 0.01 seconds

Fast PC
FREERAM_MB=1622
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds

[ChrisNZ]

Mmm.

You have 32GB of RAM, 21 GB of MEMSIZE (MEMSIZE includes the paging space) but only 1.5 GB or RAM is available to SAS.

1. I can't see how SAS can use more than that 1.5 GB, so graph is bizarre.

2. Lets try to give more RAM to SAS. Set REALMEMSIZE to 18 GB instead of 0 to see what happens.

3. Don't forget to turn on fullstimer.

4. In any case, more RAM should not help the data step much, only the sort. The speed of the data step in your case is solely the speed to read and write to the disk. Maybe running inside a VM explains the drop in performance?

[ChrisNZ]

I wish there was a way to query the values set by SAS when the configuration values are zero.

[hhchenfx]

So you meant in the config, I should change to

/* Memlib Memory options */

-MEMLIB
-MEMCACHE 4
-MEMMAXSZ 32G 
-MEMSIZE MAX
-REALMEMSIZE 18G
-SORTSIZE 1G

[ChrisNZ]

@hhchenfx Yes that's what I mean.

You should see the difference when running 

data _null_; FREERAM_MB=input(getoption('xmrlmem'),20.)/1024/1024; put FREERAM_MB= 8.; run;

Wait, I notice you have

-MEMMAXSZ 32G 

Why?

This value must be lower than MEMSIZE (which 2GB or 21GB in your case, you have given us conflicting figures), and competes for RAM with REALMEMSIZE.

See my posts here and here.

About my comment on slow IO with a VM: Do you have SAS UE at home of the full desktop version?

[Reeza]

@ChrisNZ based on the screenshots it's a full SAS installation, not SAS UE. 

[ChrisNZ]

@Reeza Should we ask for proc setinit ? 

[ChrisNZ]

It looks like your (just a few seconds) faster sort might be using an internal (RAM-based) sort while the other sort uses external (disk-based) sort. If so, the value of option SORTSIZE is the reason.

Please show the values of SORTSIZE, REALMEMSIZE and MEMSIZE.