topic Re: Convolution of Two Vectors in SAS Programming

Convolution of Two Vectors

vioravis — Mon, 21 Nov 2011 12:05:26 GMT

Are there any functions in Base SAS (I do not have SAS enterprise miner) to compute the convolution of two vectors???

Please guide. Thank you.

Convolution of Two Vectors

art297 — Mon, 21 Nov 2011 13:20:41 GMT

I don't think there is in base sas, but there is if you license IML (e.g., http://support.sas.com/documentation/cdl/en/imlug/59656/HTML/default/viewer.htm#langref_sect138.htm ).

Re: Convolution of Two Vectors

vioravis — Mon, 21 Nov 2011 16:31:55 GMT

Thanks art297. I do not have IML. Looking for a way to implement it in Base SAS itself.

Is there a way to implement the vector operations shown below in Base SAS??

In the example below, data table test1 contains three variable Variable1, Variable2 and Variable3.

DATA test1;

set test1;

DO i = 1 to num;

set test1 nobs point = i;

variable1= sum(variable2[1:i-1]*variable3[(num-i+2)]:num)];

END;

RUN;

Thank you.

Re: Convolution of Two Vectors

art297 — Mon, 21 Nov 2011 16:44:07 GMT

Of course there is, but you would have to show a small example with some data and what should go into the calculations. e.g., when i eq 1 what value should the calculation sum(variable2[1:i-1] get?

What is the maximum number of records you will have? If it isn't too big, an easy solution might be to just preload the entire table into an array.

Re: Convolution of Two Vectors

vioravis — Mon, 21 Nov 2011 17:23:50 GMT

Hi art297, my dataset has close to 10,000 observations. The following is a sample dataset:

DATA test;

input X Y Z;

cards;

1 10 0

2 20 0

3 30 0

4 40 0

5 50 0

;

run;

I have also given some sample calculations below:

Iteration 1:

SUM(X[1:1]*Y[5–2+2:5]) = SUM(X[1:1]*Y[5:5]) = SUM([1]*[50]) = 50

Iteration 2:

SUM(X[1:2]*Y[5–3+2:5]) = SUM(X[1:2]*Y[4:5]) = SUM([1,2]*[40,50]) = SUM(40+100) = 140

Iteration 3:

SUM(X[1:3]*Y[5–4+2:5]) = SUM(X[1:3]*Y[3:5]) = SUM([1,2,3]*[30,40,50]) = SUM(30+80+150) = 260

… so on

The final dataset after all the calculations will look like this

DATA test;

input X Y Z;

cards;

1 10 0

2 20 50

3 30 140

4 40 260

5 50 400

;

run;

Thank you.

Convolution of Two Vectors

vioravis — Mon, 21 Nov 2011 17:31:34 GMT

In my previous post, Z corresponds to Variable 1, X corresponds to Variable 2 and Y corresponds to Variable 3.

Thanks.

Re: Convolution of Two Vectors

Tom — Mon, 21 Nov 2011 18:30:00 GMT

This may not be the fastest way, but it should work for any size vectors.

DATA test;

input X Y expZ;

cards;

1 10 0

2 20 50

3 30 140

4 40 260

5 50 400

;

run;

data want ;
  set test nobs=nobs ;
  z=0;
  i=_n_-1;
  do p1=1 to i;
    p2=nobs -i + p1 ;
    set test (keep=x rename=(x=x1)) point=p1;
    set test (keep=y rename=(y=y1)) point=p2;
    z1=(x1*y1);
    z=sum(z,z1);
   *  put (_n_ i p1 p2 x1 y1 z1 z) (=);
  end;
run;
proc print width=min;
run;

Obs X Y expZ z i x1 y1 z1

1 1 10 0 0 0 . . .

2 2 20 50 50 1 1 50 50

3 3 30 140 140 2 2 50 100

4 4 40 260 260 3 3 50 150

5 5 50 400 400 4 4 50 200

Re: Convolution of Two Vectors

art297 — Mon, 21 Nov 2011 19:16:37 GMT

The following doesn't look as tidy as Tom's suggested code but, with 10000 records, took 6 seconds on my machine, while Tom's approach took over a minute:

/*create some test data*/

DATA test (drop=i);

input X Y Z;

do i=1 to 2000;

output;

end;

cards;

1 10 0

2 20 0

3 30 0

4 40 0

5 50 0

;

data _null_;

if 0 then set test nobs=nobs;

CALL SYMPUT('NUMREC',nobs); /*** put # of records into NUMREC macro var ***/

stop; /*** stop, got number of records ***/

run;

/* now read values of x and y into arrays, and then

reread test and do calculations*/

data want (drop=_:);

array _xval(&numrec); /* create two arrays with same number of

elements as there are records in test

array _yval(&numrec);

i=0;

do until (eof1); /* load the array with x and y values */

set test end=eof1;

i+1;

_xval(i)=x;

_yval(i)=y;

end;

_rec=-1;

do until (eof2); /* read in each record separately */

set test end=eof2;

_rec+1;

z=0;

do _i=1 to _rec;

z+_xval(_i)*_yval(&numrec-_rec+_i);

end;

output;

end;

run;

Re: Convolution of Two Vectors

vioravis — Mon, 21 Nov 2011 22:01:32 GMT

Thanks a lot Tom and Art. I am looking at Art's solution since it is faster (For a record with 8000+ records it takes around 1.9 secs). Speed is very critical since this is going to be called multiple times for optimization. Any faster solution is welcome.

I am trying out one more enhancement to the code with an additional variable A as follows (A is just another variable in the dataset):

DO i = 1 to num;

set test nobs point = i;

Z = sum(X[1:i-1]*Y[(num-i+2)]:num)];

X = A + Z;

END;

RUN;

I tried the following but it isn't giving the expected solution:

data _null_;

if 0 then set test nobs=nobs;

CALL SYMPUT('NUMREC',nobs); /*** put # of records into NUMREC macro var ***/

stop; /*** stop, got number of records ***/

run;

/* now read values of x and y into arrays, and then

reread test and do calculations*/

data want (drop=_:);

array _xval(&numrec); /* create two arrays with same number of

elements as there are records in test

array _yval(&numrec);

array _Aval(&numrec);

i=0;

do until (eof1); /* load the array with x and y values */

set test end=eof1;

i+1;

_xval(i)=x;

_yval(i)=y;

_Aval(i) = A;

end;

_rec=-1;

do until (eof2); /* read in each record separately */

set test end=eof2;

_rec+1;

z=0;

do _i=1 to _rec;

z+_xval(_i)*_yval(&numrec-_rec+_i);

_xval(_i) = _Aval(_i) + z;

end;

output;

end;

run;

Can someone please point out the correct approach???

Thank you

Convolution of Two Vectors

art297 — Mon, 21 Nov 2011 22:09:49 GMT

I haven't looked at all of your code, but noticed that you never declared the new array. Make sure that you add one. e.g.

array _xval(&numrec); /* create THREE arrays with same number of

elements as there are records in test

array _yval(&numrec);

array _Aval(&numrec);

Convolution of Two Vectors

vioravis — Mon, 21 Nov 2011 22:15:08 GMT

Art,

That was a typo in the code I pasted here. In the actual code have declared it. It still doesn't work.

Thank you.

Re: Convolution of Two Vectors

art297 — Mon, 21 Nov 2011 22:21:24 GMT

Was this also just a typo?:

-xval(_i) = _Aval(_i) + z;

Or should it have been _xval(_i) = _Aval(_i) + z;

And, of course, that will use the new values in all of the later calculations. Is that what you want?

Convolution of Two Vectors

vioravis — Mon, 21 Nov 2011 22:28:36 GMT

Oops...sorry. That was another typo.

New values are to be used in the later calculation. But only the current i is replace in each iteration.

Thank you.

Re: Convolution of Two Vectors

art297 — Mon, 21 Nov 2011 22:44:04 GMT

Then, I suggest that you post an expansion of your original sample data, including the new variable, and what you want as an output.

Re: Convolution of Two Vectors

vioravis — Mon, 21 Nov 2011 23:00:56 GMT

Art,

Please find below the new data and the expected output.

DATA test;

input X Y A Z;

cards;

1 10 0.1 0

2 20 0.2 0

3 30 0.3 0

4 40 0.4 0

5 50 0.5 0

;

run;

DATA results;

input X Y A Z;

cards;

1.0 10 0.1 0

50.2 20 0.2 50

2550.3 30 0.3 2550

129553.4 40 0.4 129553

6581208.5 50 0.5 6581208

;

run;

Thank you

Re: Convolution of Two Vectors

Tom — Mon, 21 Nov 2011 23:33:40 GMT

You might improve performance by doing the multiplication only once.

proc transpose data=test out=x prefix=x ;

var x;

run;

proc transpose data=test out=y prefix=y;

var y;

run;

data _null_;

call symputx('NOBS',nobs);

if 0 then set test nobs=nobs;

run;

data convolute;

if _n_=1 then set x y;

array x x1-x&nobs ;

array y y1-y&nobs;

array xy (&nobs,&nobs) _temporary_;

if _n_=1 then do ;

do i=1 to &nobs; do j=1 to &nobs ;

xy(i,j)=x(i)*y(i);

end; end;

end;

***** now write summation code using indexes into xy 2-dimensional array ;

...

run;

Re: Convolution of Two Vectors

art297 — Tue, 22 Nov 2011 00:25:55 GMT

I don't see what you want. When, in the calculation and updates, should A be applied and how?

Re: Convolution of Two Vectors

vioravis — Tue, 22 Nov 2011 10:31:12 GMT

Hi Art,

I have given an example below. Please let me know if you need any other information. Thank you.

DATA test;

input X Y A Z;

cards;

1 10 0.1 0

2 20 0.2 0

3 30 0.3 0

4 40 0.4 0

5 50 0.5 0

;

run;

Iteration 1:

SUM(X[1:1]*Y[5–2+2:5]) = SUM(X[1:1]*Y[5:5]) = SUM([1]*[50]) = 50

X[2] = Z[2] + A[2] --> X[2] = 50 + 0.2 = 50.2

test dataset after iteration 1:

X Y A Z

1 10 0.1 0

50.2 20 0.2 50

3 30 0.3 0

4 40 0.4 0

5 50 0.5 0

This test will be used in the second iteration as follows

Iteration 2:

SUM(X[1:2]*Y[5–3+2:5]) = SUM(X[1:2]*Y[4:5]) = SUM([1,50.2]*[40,50]) = SUM(40+2510) = 2550

X[3] = Z[3] + A[3] --> X[3] = 2550 + 0.3 = 2550.3

test dataset after iteration 2:

X Y A Z

1 10 0.1 0

50.2 20 0.2 50

2550.3 30 0.3 2550

4 40 0.4 0

5 50 0.5 0

Re: Convolution of Two Vectors

vioravis — Tue, 22 Nov 2011 10:36:14 GMT

Thanks, Tom. I will try this out.

Convolution of Two Vectors

vioravis — Tue, 22 Nov 2011 11:42:44 GMT

Hi Tom,

The XY array you have defined holds the following matrix. Could you please explain what summation are you referring to on this XY array? Thank you.

[ ,1] [,2] [,3] [,4] [,5]

[1,] 10 10 10 10 10

[2,] 40 40 40 40 40

[3,] 90 90 90 90 90

[4,] 160 160 160 160 160

[5,] 250 250 250 250 250