topic Re: How to convert existing dataset into Matrix and access each column/row in PROC IML?! in SAS Programming

How to convert existing dataset into Matrix and access each column/row in PROC IML?!

hellohere — Thu, 08 Sep 2022 22:12:37 GMT

Say, here is dataset, sashelp.cars.

Inside PROC IML, How to access it row, sashelp.cars.horsepower; or all row associated with BMW, like R does?!

Any learning manual for this?!

Re: How to convert existing dataset into Matrix and access each column/row in PROC IML?!

Quentin — Fri, 09 Sep 2022 01:22:07 GMT

There is documentation, and there are also papers written by users.

I googled "Introduction to SAS IML" and the first hit was a paper written by the developer of IML, that's probably a good place to start:

https://support.sas.com/resources/papers/proceedings13/144-2013.pdf

After scanning the paper it was easy to find an example I could adapt:

proc iml;
varNames = {"Make" "Model"};
use Sashelp.Cars;
read all var varNames into X; /* read data */
idx = loc(X[,1]="BMW"); /* row vector */
print (idx`)[label="Row"] (X[idx,])[c=varNames];
quit ;

I don't know R, so not sure if this is what you were hoping for...

Oops, I realized you want to see a numeric variable, subset by a character variable. Apparently an IML matrix can't have a mix of numeric in character vars. But with the help of this blog post (also by Rick): https://blogs.sas.com/content/iml/2015/05/11/loc-element-trick.html, I hacked together:

proc iml;
use sashelp.Cars;
read all var {"Make"} into M ;
read all var {"Horsepower"} into H;
Idx = loc(M="BMW");
print (M[Idx,]) (H[Idx,])  ;

But fair warning: I don't think today is actually the first time I've written PROC IML code, but it's could be only the second or third time. : )

Re: How to convert existing dataset into Matrix and access each column/row in PROC IML?!

hellohere — Fri, 09 Sep 2022 01:28:33 GMT

Thanks a huge. Let me read through.

Ye, I work with R/Matlab, in which easy to access column/row/element/subset.

Re: How to convert existing dataset into Matrix and access each column/row in PROC IML?!

Quentin — Fri, 09 Sep 2022 01:32:38 GMT

I'm sure you'll get better answers from someone who actually knows IML. : )

This also works:

proc iml;
use sashelp.Cars;
read all var {"Horsepower"} into H;
read all var {"Make"} into M ;
print (H[loc(M="BMW"),])  ;

Re: How to convert existing dataset into Matrix and access each column/row in PROC IML?!

Quentin — Fri, 09 Sep 2022 01:52:06 GMT

If you don't get better answers by tomorrow, I'd try posting in the IML board:

https://communities.sas.com/t5/SAS-IML-Software-and-Matrix/bd-p/sas_iml

Re: How to convert existing dataset into Matrix and access each column/row in PROC IML?!

hellohere — Fri, 09 Sep 2022 06:52:14 GMT

I read the PDF doc, tried out codes and got lots of senses.

STILL, how to get EMA() on one column?! I tried below, got

ERROR: (execution) Matrix has not been set to a value.

And tried to build own function with START/FINISH, still not efficient. Anywhere has the list of functions on

column/row operation?! THanks,

proc iml;
	ind=1:100;
	x1=ind+5*ranuni(i);
	ema1=ema(x1, 0.01);
	create _Matrix_Own_ema var {"ind" "ema1"};   
	append; 
	close  _Matrix_Own_ema;
run;quit;

Re: How to convert existing dataset into Matrix and access each column/row in PROC IML?!

hellohere — Fri, 09 Sep 2022 06:57:32 GMT

ownema <- function(x,win) {
   alpha <- 2/(win+1)
   filter(x, 1-alpha, method="recursive", side=1, init=x[1]/alpha)*alpha
}

R can use the above at ema. Does SAS/IML has similar?!

Re: How to convert existing dataset into Matrix and access each column/row in PROC IML?!

Ksharp — Fri, 09 Sep 2022 10:31:51 GMT

Calling @Rick_SAS

Re: How to convert existing dataset into Matrix and access each column/row in PROC IML?!

Quentin — Fri, 09 Sep 2022 10:41:30 GMT

Here’s a link to another of Rick’s blog posts:
https://blogs.sas.com/content/iml/2016/02/03/rolling-statistics-sasiml.html

Re: How to convert existing dataset into Matrix and access each column/row in PROC IML?!

PaigeMiller — Fri, 09 Sep 2022 12:29:12 GMT

@hellohere wrote:

I read the PDF doc, tried out codes and got lots of senses.

STILL, how to get EMA() on one column?! I tried below, got
ERROR: (execution) Matrix has not been set to a value.

Please, for this question and all questions from now on, when you get ERRORs or WARNINGS in a PROC or DATA step, show us the ENTIRE log for that specific PROC or DATA step where you get errors or warnings. Do not show us tiny parts of the log detached from the code as it appears in the log. Doing this will result in faster and better answers.

Re: How to convert existing dataset into Matrix and access each column/row in PROC IML?!

hellohere — Fri, 09 Sep 2022 23:24:23 GMT

Thanks all guys. I write own ema function. It is "identical" to the one from Rick's blog; it goes through

a column-count loop, rather than build-in mech. The computation cost might be same anyhow, but not eff. in coding.


proc iml;
	start ownema(xema, x, emawt);	
		xema=j(1,ncol(x));
		do i=1 to max(dimension(x));
			if i=1 then do;
				xema[,i]=x[,i];
			end;
			else do;
				xema[,i]=x[,i]*emawt+(1-emawt)*xema[,i-1];
			end;	
		end;
	finish;

	ind=1:100;
	x1=sin(ind);  /*dim=1 100*/
	emawt=0.1;
	
	run ownema(x1ema, x1,emawt);
	create _Matrix_Own_ema var {"ind" "x1" "x1ema"};   
	append; 
	close  _Matrix_Own_ema;
run;quit;

proc sgplot data=_Matrix_Own_ema;
series x=ind y=x1;
series x=ind y=x1ema;
run;quit;

proc iml;
use Sashelp.Air;  
   read all var "date" into t;
   read all var "air" into y;
close;
rct=nrow(t);
print rct;
dim=dimension(t);
print dim;
run;quit;

Re: How to convert existing dataset into Matrix and access each column/row in PROC IML?!

hellohere — Fri, 09 Sep 2022 23:29:08 GMT

By removing if i=1 in my own func, I bet computation cost is cut off in half.😀

Re: How to convert existing dataset into Matrix and access each column/row in PROC IML?!

hellohere — Fri, 09 Sep 2022 23:37:50 GMT

I try out code from another Rick's blog. A test for monotonic sequences and functions - The DO Loop (sas.com)

Somehow, get complains below. Any thoughts?

42601  proc iml;
NOTE: IML Ready
42602  /* Test whether a sequence of elements is monotonic increasing.
42603     Valid options are
42604     strict=0 : (Default) Return 1 if a sequence is nondecreasing
42605     strict=1 : Return 1 if a sequence is strictly increasing
42606  */
42607  start IsIncr(_x, strict=0);
42608     x = colvec(_x);
42609     if nrow(x)=1 then return(1);
42609!                                      /* one element is always monotonic! */
42610     d = dif(x,1,1);
42610!                                      /* lag=1; delete initial missing value */
42611     if strict then
42612        return( all(d > 0) );
42613     return( all(d >= 0) );
42614  finish;
ERROR: Too many arguments for function DIF.
ERROR: Module ISINCR was not defined due to resolution errors.
42615
42616  /* test whether sequences are increasing */
42617  x = {0,2,2,2,6,7,9};
42618  y = {0,1,3,4,6,7,9};
42619  z = {0,1,3,4,2,7,9};
42620  b1 = IsIncr(x);
ERROR: Invocation of unresolved module ISINCR.

 statement : ASSIGN at line 42620 column 1
42620!                     /* test weakly increasing */
42621  b2 = IsIncr(x, 1);
ERROR: Invocation of unresolved module ISINCR.

 statement : ASSIGN at line 42621 column 1
42621!                     /* test strictly increasing */
42622  b3 = IsIncr(y, 1);
ERROR: Invocation of unresolved module ISINCR.

 statement : ASSIGN at line 42622 column 1
42622!                     /* test strictly increasing */
42623  b4 = IsIncr(z);
ERROR: Invocation of unresolved module ISINCR.

 statement : ASSIGN at line 42623 column 1
42623!                     /* test weakly increasing */
42624
42625  print b1 b2 b3 b4;
ERROR: Matrix b1 has not been set to a value.

 statement : PRINT at line 42625 column 1
42626
42627  /* Test whether a sequence of elements is monotonic decreasing.
42628     strict=0 : (Default) Return 1 if a sequence is nonincreasing
42629     strict=1 : Return 1 if a sequence is strictly decreasing
42630  */
42631  start IsDecr(x, strict=0);
42632     return IsIncr(-x, strict);
42633  finish;
NOTE: Module ISDECR defined.
42634
42635  /* test whether sequence is increasing */
42636  u = {9,8,7,7,6,2,0};
42637  b5 = IsDecr(u);
ERROR: Invocation of unresolved module ISINCR.

 statement : RETURN at line 42632 column 4
 traceback : module ISDECR at line 42632 column 4

NOTE: Paused in module ISDECR.
42637!                     /* test weakly decreasing */
42638  b6 = IsDecr(u, 1);
ERROR: Module ISDECR called again before exit from prior call.

 statement : ASSIGN at line 42638 column 1
42638!                     /* test strictly decreasing */
42639  print b5 b6;
ERROR: Matrix b5 has not been set to a value.

 statement : PRINT at line 42639 column 1
42640
42641  start Func1(x);
42642     return (5 - 4*x)#log( x/(1-x) );
42643  finish;
NOTE: Module FUNC1 defined.
42644  start Func2(x);
42645     return (5 - 5.2*x)#log( x/(1-x) );
42646  finish;
NOTE: Module FUNC2 defined.
42647
42648  dt = 0.005;
42649  x = do(dt, 1-dt, dt);
42649!                        /* increasing sequence on a fine grid */
42650  y1 = Func1(x);
42650!                        /* image of sequence under F1 */
42651  y2 = Func2(x);
ERROR: (execution) Invalid argument to function.

 count     : number of occurrences is 7
 operation : LOG at line 42645 column 26
 operands  : _TEM1004

_TEM1004      7 rows      1 col     (numeric)

         0
        -2
        -2
        -2
      -1.2
 -1.166667
    -1.125

 statement : RETURN at line 42645 column 4
 traceback : module FUNC2 at line 42645 column 4

NOTE: Paused in module FUNC2.
42651!                        /* image of sequence under F2 */
42652  b1 = IsIncr(y1);
ERROR: Invocation of unresolved module ISINCR.

 statement : ASSIGN at line 42652 column 1
42653  b2 = IsIncr(y2);
ERROR: Invocation of unresolved module ISINCR.

 statement : ASSIGN at line 42653 column 1
42654  print b1 b2;
ERROR: Matrix b1 has not been set to a value.

 statement : PRINT at line 42654 column 1
42655
42656  run;
NOTE: Module MAIN is undefined in IML; cannot be RUN.
42656!     quit;
NOTE: Exiting IML.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: PROCEDURE IML used (Total process time):
      real time           0.03 seconds
      cpu time            0.03 seconds

Re: How to convert existing dataset into Matrix and access each column/row in PROC IML?!

Quentin — Sat, 10 Sep 2022 20:11:15 GMT

Strange, the code on that post works fine for me.

With SAS, you always start with the first error you see in the log. In this case:

ERROR: Too many arguments for function DIF.

That error doesn't make sense to me, unless you have created your own DIF function which doesn't take 3 arguments.

I would suggest starting a fresh session, and submitting only the code from the blog. If you still get errors, I would submit to tech support.