topic Re: Proc IML - show space strange behaviour in SAS Procedures

Proc IML - show space strange behaviour

yabwon — Tue, 24 Sep 2024 12:21:30 GMT

Hi SAS-C,

I've been playing with Proc IML recently. I've used SOAD server to run the following code because I wanted to test memory usage reporting (as described in the doc: https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.3/imlug/imlug_asstop_sect001.htm)

The code was:

proc iml;
  print "start session";
  SHOW space memory;
  
  print "x created";
  x = 1:1e6;
  x = shape(x,1000);
  SHOW space memory;
  
  print "y assigned";
  y=x;
  print (nrow(y)) (ncol(y));
  SHOW space memory;
quit;

proc iml;
  print "start session";
  SHOW space memory;
  
  print "x created";
  x = 1:1e6; /* <------------------- 6 */
  x = shape(x,1000);
  SHOW space memory;
  
  print "y assigned";
  y=x;
  print (nrow(y)) (ncol(y));
  SHOW space memory;
quit;

proc iml;
  print "start session";
  SHOW space memory;
  
  print "x created";
  x = 1:1e7; /* <------------------- 7 */
  x = shape(x,1000);
  SHOW space memory;
  
  print "y assigned";
  y=x;
  print (nrow(y)) (ncol(y));
  SHOW space memory;
quit;

proc iml;
  print "start session";
  SHOW ALL;
  
  print "x created";
  x = 1:1e8; /* <------------------- 8 */
  x = shape(x,1000);
  SHOW ALL;
  
  print "y assigned";
  y=x;
  print (nrow(y)) (ncol(y));
  SHOW All;
  
  print "z assigned";
  z=x+1;
  print (nrow(z)) (ncol(z));
  SHOW All;

  print "free y";
  FREE y;
  SHOW All;
quit;

The Log was:

1          proc iml;
 NOTE: IML Ready
 2            print "start session";
 3            SHOW space memory;
 4          
 5            print "x created";
 6            x = 1:1e6;
 7            x = shape(x,1000);
 8            SHOW space memory;
 9          
 10           print "y assigned";
 11           y=x;
 12           print (nrow(y)) (ncol(y));
 13           SHOW space memory;
 14         quit;
 NOTE: Exiting IML.
 NOTE: PROCEDURE IML used (Total process time):
       real time           0.02 seconds
       user cpu time       0.01 seconds
       system cpu time     0.01 seconds
       memory              16754.06k
       OS Memory           40632.00k
 
 15         
 16         proc iml;
 NOTE: IML Ready
 17           print "start session";
 18           SHOW space memory;
 19         
 20           print "x created";
 21           x = 1:1e6;
 21       !              /* <------------------- 6 */
 22           x = shape(x,1000);
 23           SHOW space memory;
 24         
 25           print "y assigned";
 26           y=x;
 27           print (nrow(y)) (ncol(y));
 28           SHOW space memory;
 29         quit;
 NOTE: Exiting IML.
 NOTE: PROCEDURE IML used (Total process time):
       real time           0.02 seconds
       user cpu time       0.02 seconds
       system cpu time     0.01 seconds
       memory              16180.68k
       OS Memory           40632.00k
 
 30         
 31         proc iml;
 NOTE: IML Ready
 32           print "start session";
 33           SHOW space memory;
 34         
 35           print "x created";
 36           x = 1:1e7;
 36       !              /* <------------------- 7 */
 37           x = shape(x,1000);
 38           SHOW space memory;
 39         
 40           print "y assigned";
 41           y=x;
 42           print (nrow(y)) (ncol(y));
 43           SHOW space memory;
 44         quit;
 NOTE: Exiting IML.
 NOTE: PROCEDURE IML used (Total process time):
       real time           0.08 seconds
       user cpu time       0.04 seconds
       system cpu time     0.04 seconds
       memory              156778.28k
       OS Memory           181256.00k
 
 45         
 46         proc iml;
 NOTE: IML Ready
 47           print "start session";
 48           SHOW ALL;
 NOTE: No modules are currently defined.
 NOTE: No files are currently open.
 NOTE: No data sets are currently open.
 49         
 50           print "x created";
 51           x = 1:1e8;
 51       !              /* <------------------- 8 */
 52           x = shape(x,1000);
 53           SHOW ALL;
 NOTE: No modules are currently defined.
 NOTE: No files are currently open.
 NOTE: No data sets are currently open.
 54         
 55           print "y assigned";
 56           y=x;
 57           print (nrow(y)) (ncol(y));
 58           SHOW All;
 NOTE: No modules are currently defined.
 NOTE: No files are currently open.
 NOTE: No data sets are currently open.
 59         
 60           print "z assigned";
 61           z=x+1;
 62           print (nrow(z)) (ncol(z));
 63           SHOW All;
 NOTE: No modules are currently defined.
 NOTE: No files are currently open.
 NOTE: No data sets are currently open.
 64         
 65           print "free y";
 66           FREE y;
 67           SHOW All;
 NOTE: No modules are currently defined.
 NOTE: No files are currently open.
 NOTE: No data sets are currently open.
 68         quit;
 NOTE: Exiting IML.
 NOTE: PROCEDURE IML used (Total process time):
       real time           1.03 seconds
       user cpu time       0.59 seconds
       system cpu time     0.54 seconds
       memory              2344295.78k
       OS Memory           2368756.00k

We can clearly see in the log that memory used size changes.

But the output shows everywhere the same values:

What am I doing wrong?

I bet $5 that it must be something trivial/obvious what I'm missing but I can't figure out what...

Bart

P.S. @Rick_SAS help.

Re: Proc IML - show space strange behaviour

Rick_SAS — Tue, 01 Oct 2024 15:48:01 GMT

Thank you for pointing to the documentation. The code you are running in SAS ODA uses a different system of memory management than is discussed in the documentation. What you are reading is out of date and needs to be updated. SAS IML no longer uses garbage collection, nor does it use "symbol space" and "workspace" the way it used to. Consequently, the output from the SHOW MEMORY statement is not useful or relevant.

Re: Proc IML - show space strange behaviour

yabwon — Tue, 01 Oct 2024 16:13:06 GMT

Is there any other way to get "memory usage"? Except using gigabytes-counting-module.

Bart

P.S. BTW my link was to SAS9.4 & viya3.3 but even the latest (2024.09) shows the same:

https://documentation.sas.com/doc/en/pgmsascdc/v_055/imlug/imlug_asstop_sect001.htm

Re: Proc IML - show space strange behaviour

Rick_SAS — Tue, 01 Oct 2024 17:23:13 GMT

As I said, the doc is out of date and needs to be updated.

What are you trying to accomplish? What information do you need about memory usage, and what do you intend to do with it?

Re: Proc IML - show space strange behaviour

yabwon — Tue, 01 Oct 2024 20:48:01 GMT

I was hoping to show my students how IML works with memory. For example how it's changing memory when FREE statement is used, how assigning "old" matrix to a "new" one changes memory usage, how it works with modules, etc.

All the best

Bart

Re: Proc IML - show space strange behaviour

Rick_SAS — Wed, 02 Oct 2024 08:53:29 GMT

You can use the SHOW NAMES statement (or SHOW ALLNAMES) to see which symbols have been defined. You can add the MODULES option if you want to see the names of the user-defined modules.

proc iml;
N = 10000;
print "--- 1 ---";
x = j(N, N);      /* allocate matrices and vectors */
z = 1:N;
w = z`;
show names;

start foo(x);
   return 1;
finish;
Q = x;  

print "--- 2 ---";
free x;           /* free memory for x */
show names modules;

print "--- 4 ---";
free / z w;      /* free all symbols except for z and w */
show names;

Re: Proc IML - show space strange behaviour

yabwon — Wed, 02 Oct 2024 09:14:40 GMT

SHOW NAMES MODULS; - that I know, I wanted to show/see how the memory usage is changing when I'm using/creating/reassigning matrices or modules 🙂

Bart

Re: Proc IML - show space strange behaviour

Rick_SAS — Wed, 02 Oct 2024 09:26:03 GMT

There is no option to output the current amount of memory in use while the IML procedure is running. The method by which IML allocates and frees memory is complicated and is not documented.

Re: Proc IML - show space strange behaviour

yabwon — Wed, 02 Oct 2024 09:36:33 GMT

😞 😞 😞