use Array in a proc fcmp function

[KachiM]

Hi Rhapsody,

Your desire:

Y=something(X)

can be achieved as a Function (something), only  if Y can take a SINGLE value where X can be an array. But you desire an array to be returned and so it can be achieved by a Subroutine only.

See the first part of my earlier solution. I reproduce that part:

data have;
input theta1-theta3;
datalines;
0.2  0.5  0
0.3  0.6  0.023
0.5  0.1  0.05
0.7  0.2  0.01 
;
run;

proc fcmp outlib = work.func.arr;
   subroutine add(t[*], tr[*]);
      outargs t, tr;
      tr[1] = t[1];
      do i = 2 to dim(t);
         tr[i] = t[i-1] + t[i];
      end;
   endsub;
quit;

options cmplib = work.func;

data want;
   array t[3] _temporary_;
   array trans[3] _temporary_;
   
   set have;
   array th theta1-theta3;
   do i = 1 to dim(t);
      t[i] = th[i];
   end;
   call add(t,trans);
   array tran trans1-trans3;
   do i = 1 to dim(t);
      tran[i] = trans[i];
   end;
   drop i th:;
run;

The Output:

Obs 	trans1 	trans2 	trans3
1 	0.2 	0.7 	0.500
2 	0.3 	0.9 	0.623
3 	0.5 	0.6 	0.150
4 	0.7 	0.9 	0.210

Since we want the Array, TRANS[ ], to be filled by FCMP program from the Array, T[ ], it is possible by a SUBROUTINE only. Hence, in Data Step we pass both the arrays by the subroutine (Subroutine ADD). This is done by passing one observation with theta1 to theta3 at a time where Array T[ ] holds them. Array TRANS[ ], while holding missing values on input, gets filled by the FCMP call routine:

call add(t,trans);

Note in FCMP parentheses ( ), are reserved for function arguments. Square brackets ([ ]) is a natural choice for Arrays.

Regards,

DATAsp

[rhapsody]

Oki.

The reason why I was looking into this option in the first place is to simplify the code. Instead of writing 30 lines of code you write just one function and everytime you wanna use that function you just write something like

data;

set input;

Y=something(X);

run;

And that gives you the output Array Y.

But since your solution implies about 30 lines of code I suppose that this kind of code simplification just isn't available in SAS?

[RW9]

Just re-assess your data modelling.  Normalised (i.e. data going down the page) is far simpler to work with than transposed (across, which is what you have).  It would save you all the hassel of array processing and lots of coding.  Alternatively if you have a lot of matrix data, then IML should be something you look at, that is built for matrix processing.

Simply put, if you find something is difficult, or requires lots of coding, then you likely have something else wrong which is causing you work later on, be that data modelling or choice of processing.

[KachiM]

Hello,

FCMP functions and call routines are written once, compiled and saved in a Library with special Dataset Name. You don't write the function every time to run it. Then, give the LibraryName.Dataset name as option and use the function.

The second part of the solution, I gave previously, is one way to achieve your need.

I am not aware of any other software doing it in one or two statements. I will be thankful to you if you can describe such a software.

Regards,

DATAsp

[rhapsody]

In your example you have to write quite a lot of lines ta call the function again which makes it pointless to work with functions in the first place. I'll give you an example in Matlab where you can write a function and then call it using only one row of code.

I have the input Array X

X

0,67	393,96	833,78	48,81	3,61	0,85	0,00	0,00	0,00	1,47
22,04	167,80	2189,82	8,60	1,97	2,44	0,00	0,00	0,00	0,04
8,83	39,58	1563,34	42,64	46,33	1,76	0,84	0,00	0,00	0,00
6,00	81,53	1039,95	12,26	25,77	0,76	0,94	0,00	0,00	0,00
1,88	87,12	386,61	11,52	19,83	2,85	0,03	0,00	0,00	0,00
6,12	61,22	195,15	47,00	0,14	0,92	0,00	0,00	0,00	0,00
0,38	109,71	649,68	93,17	13,05	0,00	0,70	0,00	0,00	0,00
2,31	17,99	1416,38	92,77	5,44	0,98	0,58	0,00	0,00	0,00
3,29	55,23	1542,51	35,90	19,83	1,21	0,00	0,00	0,00	0,88
4,04	130,80	1417,13	63,79	16,54	0,98	0,00	0,00	0,00	0,00
13,17	5,18	963,50	69,60	34,62	1,31	0,98	0,00	0,00	0,00
3,34	160,36	1341,45	135,73	31,93	0,00	0,04	0,00	0,00	0,00

And then I want a function which normalizes every row such that it sums to one. The function looks like

function Y = num2porcent(X)

[row col]=size(y);

sumY=sum(y,2);

Y = X./(sumY*ones(1,col));

end

Every time I wanna call it I just wirite

Y = num2porcent(X)

No more lines required.

The output is

Y

0,001	0,307	0,650	0,038	0,003	0,001	0,000	0,000	0,000	0,001
0,009	0,070	0,915	0,004	0,001	0,001	0,000	0,000	0,000	0,000
0,005	0,023	0,918	0,025	0,027	0,001	0,000	0,000	0,000	0,000
0,005	0,070	0,891	0,011	0,022	0,001	0,001	0,000	0,000	0,000
0,004	0,171	0,758	0,023	0,039	0,006	0,000	0,000	0,000	0,000
0,020	0,197	0,628	0,151	0,000	0,003	0,000	0,000	0,000	0,000
0,000	0,127	0,750	0,107	0,015	0,000	0,001	0,000	0,000	0,000
0,002	0,012	0,922	0,060	0,004	0,001	0,000	0,000	0,000	0,000
0,002	0,033	0,930	0,022	0,012	0,001	0,000	0,000	0,000	0,001
0,002	0,080	0,868	0,039	0,010	0,001	0,000	0,000	0,000	0,000
0,012	0,005	0,885	0,064	0,032	0,001	0,001	0,000	0,000	0,000
0,002	0,096	0,802	0,081	0,019	0,000	0,000	0,000	0,000	0,000

So this is a good example where I use an Array as input and get an Array as output and everytime I call it the only thing needed is one row of code.

[rhapsody]

I'm sorry, I made a small confusing error in the function. The corrected one is found below

function Y = num2porcent(X)

[row col]=size(X);

sumX=sum(X,2);

Y = X./(sumX*ones(1,col));

end

[KachiM]

Hi Rhapsody,

Thanks for showing the Matlab function. You simply use NUM2PORCENT() function to pass an input Array to get an output Array. The code of the function is hidden. Who knows how many statements it has.

In a similar way SAS provides hundreds of functions to use and we don't know its contents. For a specific need SAS provides Proc FCMP to build our own functions. Let us assume some programmer has written a function ARRAY2ARRAY() just to help you. You are supposed to use the compiled function saved into a Library  for infinite number of times. You just use one statement to pass your input Array as SAS Data Set to the function to get an output Array as a SAS Data Set. You write:

options cmplib = work.cmpar;                                     /* This is the programmer given Library */

%let rc = %sysfunc(Array2Array(have, want);            /* This is how the function is to be used */

passing the Data Set, HAVE to get an Output Data Set Want.

Now you can visualize the similarity between Matlab and FCMP Functions.

The programmer made his function as below. You assume it is hidden for the User like you.

proc fcmp outlib = work.cmpar.lib;
   function array2array(inds $, outds $);
   file log;
   array theta[1] / nosymbols; 
   array trans[1] / nosymbols; 

   rc = read_array(inds, theta); 
   
   rows = dim1(theta); cols = dim2(theta); 
   call dynamic_array(trans, rows, cols);    
   do i = 1 to rows;
      do j = 1 to cols - 1;
         trans[i,1] = theta[i,1]; 
         trans[i,j+1] = theta[i, j+1] + theta[i, j]; 
      end;
   end;
   rc = write_array(outds, trans); 
   put 'INFO: Created Data Set' outds  'with' rows 'Rows and' cols 'Cols.';
   return(1); 
   endsub;
quit;

Your Data Set (HAVE) is:

data have;
input theta1-theta3;
datalines;
0.2  0.5  0
0.3  0.6  0.023
0.5  0.1  0.05
0.7  0.2  0.01 
;
run;

You enter the following 2 statements:

options cmplib = work.cmpar;

%let rc = %sysfunc(array2array(have, want));

The output Data Set is:

Obs 	trans1 	trans2 	trans3
1 	0.2 	0.7 	0.500
2 	0.3 	0.9 	0.623
3 	0.5 	0.6 	0.150
4 	0.7 	0.9 	0.210

You use the first statement at the beginning of the program. The function can be used any number of times within the current SAS Session. The programmer can be requested to save to a permanent Library to use the function infinite number of times.

Kind regards,

DATAsp

[rhapsody]

I suppose the last lines are not complete?

options cmplib = work.cmpar;

%let rc = array2array(have, want);

They should be written in a datastep or something? All you do now is defining a macro variable using the %let statement

[Patrick]

@rhapsody 

I like your rational for using FCMP to create a function library in a DIS context.

I believe though that using FCMP is not the ideal approach for the problem at hand. The main reason is: You need to create a set of new variables and SAS variables need to be declared during compilation time. So I don't believe that's gonna work here.

In a DIS context: You could implement something dynamic via a SAS macro within a custom transformation. That would also allow for the re-usability and central code management you're after.

[KachiM · Accepted solution]

Hi Rhapsody

It is my bad. I did not use %sysfunc(). It must be:

%let rc = %sysfunc(array2array(have, want));

This is equivalent to your Y = f(X). Here I used rc = f(X,Y).

[rhapsody]

When I run your code I get the error

ERROR: Unable to open data set "WORK.have" in function READ_ARRAY.

ERROR: Second parameter passed to DYNAMIC_ARRAY must be a positive number.

ERROR: Error reported in function 'READ_ARRAY' in statement number 3 at line 8 column 1.

The statement was:

0 (8:1) rc = READ_ARRAY( inds="have", theta=(ndim=1, dim1=1 [.,...]) )

ERROR: An illegal argument is used in the function call in statement number 5 at line 10 column

1.

The statement was:

0 (10:1) cols = DIM2( theta=(ndim=1, dim1=1 [.,...]) )

ERROR: A TO value is missing in the DO statement in statement number 8 at line 13 column 1.

The statement was:

0 (13:1) do j = 1 to cols - 1;

INFO: Created Data Set want with 1 Rows and . Cols.

WARNING: Argument 2 to function PUTN referenced by the %SYSFUNC or %QSYSFUNC macro function is

out of range.

[KachiM]

ERROR: Unable to open data set "WORK.have" in function READ_ARRAY.

This ERROR may be due to the absence of HAVE in WORK Library. Please check it. All other Errors may be the result of the above Error.

[KachiM]

Rhapsody,

On Thursday you came with an ERROR message for not finding the WORK.HAVE Data Set. I replied you to check the presence of HAVE Data Set in WORK Library. What did you find? 

Please share your experience. 

DATAsp

[rhapsody]

I just ran your code again. I've attached an exact copy of that code to show you. It's this one

data have;

input theta1-theta3;

datalines;

0.2 0.5 0

0.3 0.6 0.023

0.5 0.1 0.05

0.7 0.2 0.01

;

run;

options cmplib = work.cmpar;

%let rc = %sysfunc(array2array(have, want));

Then I get the foloowing message in the log

1 data have;

2 input theta1-theta3;

3 datalines;

NOTE: The data set WORK.HAVE has 4 observations and 3 variables.

NOTE: Compressing data set WORK.HAVE increased size by 100.00 percent.

Compressed is 2 pages; un-compressed would require 1 pages.

NOTE: DATA statement used (Total process time):

real time 0.17 seconds

cpu time 0.00 seconds

　

8 ;

9 run;

10

11 options cmplib = work.cmpar;

12

13 %let rc = %sysfunc(array2array(have, want));

WARNING: No CMP or C functions found in library work.cmpar.

ERROR: The ARRAY2ARRAY function referenced in the %SYSFUNC or %QSYSFUNC macro function is not

found.

WARNING: Argument 2 to function PUTN referenced by the %SYSFUNC or %QSYSFUNC macro function is

out of range.

[rhapsody]

Hello! Sorry. It works now. and I just accepted your solution. Thank you very much 🙂