topic Re: Proc IML Full Factorial Matrix in SAS/IML Software and Matrix Computations

Proc IML Full Factorial Matrix

Sanscrit_2prov — Fri, 25 Oct 2019 21:57:46 GMT

HI;

I would like to create a matrix reflecting a full factorial design. This would have the following structure

3 factors

0 0 1

0 1 0

1 0 0

0 1 1

1 1 0

1 0 1

1 1 1

I tried using allcomb but couldn't get a matrix like the above. Thanks

Re: Proc IML Full Factorial Matrix

PaigeMiller — Fri, 25 Oct 2019 23:12:14 GMT

You can generate a full factorial matrix in PROC PLAN and then read it into IML if you need it there. SAS has done the work to generate all sorts of designs, so you don't have to. It's also simple to do via looping in a DATA step, and of course you could do similar looping in IML.

Re: Proc IML Full Factorial Matrix

PeterClemmensen — Fri, 25 Oct 2019 23:02:39 GMT

May I ask, why do you want to do this? Seems like its a part of a larger problem?

Re: Proc IML Full Factorial Matrix

IanWakeling — Sat, 26 Oct 2019 07:18:49 GMT

Use the function EXPANDGRID it does exactly what you want, provided you have 15 factors or less.

  x = expandgrid(0:1, 0:1, 0:1);
  print x;

Re: Proc IML Full Factorial Matrix

Sanscrit_2prov — Mon, 28 Oct 2019 15:08:34 GMT

Unfortunately I have I can have more than 15 factors as they are combinations of covariate patterns. I did see Proc Factex and will check that out also. Thank you

Re: Proc IML Full Factorial Matrix

Sanscrit_2prov — Mon, 28 Oct 2019 15:38:42 GMT

Thank you I am looking at both Proc Plan and Proc Factex, but they have some limits in terms of number of factors, It would be nice to be able to do this with loops and I have played around with the program below, but instead of outputting observations in the data step I would like to work solely in IML and generate a matrix. instead, Below is what I have so far. Thanks. I have more work to do!

data design;
do i=1 to 4;
do j=1 to 4;
output;
end;
end;
run;
proc print data=design;
run;
PROC IML;
USE design;
read all var _num_ into m [colname=varnames];
l=rowvec(m);
print (l);
run;
quit;

Re: Proc IML Full Factorial Matrix

Sanscrit_2prov — Mon, 28 Oct 2019 15:39:30 GMT

I am recoding a Matlab program into SAS code. So I am learning IML for the first time.

Re: Proc IML Full Factorial Matrix

IanWakeling — Mon, 28 Oct 2019 15:49:35 GMT

If the factors all have 2 levels then the full factorial is nothing more than counting in binary. So you could use something like this:

x = j(8,3);
do i = 1 to 8;
  x[i, ] = band(i-1, {4 2 1}) > 0;
end;
print x;

Re: Proc IML Full Factorial Matrix

Rick_SAS — Mon, 28 Oct 2019 15:53:29 GMT

Do you know that you can get the missing value pattern matrix from PROC MI?

"Examine patterns of missing data in SAS"

If the data matrix is not too big, you can use heat maps to visualize the missing data patterns.

Re: Proc IML Full Factorial Matrix

PaigeMiller — Mon, 28 Oct 2019 19:06:30 GMT

@Sanscrit_2prov wrote:

Unfortunately I have I can have more than 15 factors as they are combinations of covariate patterns. I did see Proc Factex and will check that out also. Thank you

Wait a second ... full factorial with more than 15 factors? That will be huge, even if it is a 2-level design. And this is the first time you have mentioned covariates ... a full factorial with covariates, how are the covariates to be handled? Are they part of the full factorial?

Please explain this.

Re: Proc IML Full Factorial Matrix

PaigeMiller — Mon, 28 Oct 2019 19:12:11 GMT

@Sanscrit_2prov wrote:

Thank you I am looking at both Proc Plan and Proc Factex, but they have some limits in terms of number of factors,

I am not aware of a limitation on the number of factors in PROC PLAN. There's nothing in the documentation stating a limit. Why do you say this?

but instead of outputting observations in the data step I would like to work solely in IML and generate a matrix.

Yes, looping in IML can give you a full factorial, but the idea of not using SAS data steps (or did you mean not using SAS data sets?) is one that will make your life miserable. SAS is designed around the concept of a data step, and around data sets, and unless you are determined to do everything in IML, you can't (and shouldn't) avoid using data sets and data steps.

Re: Proc IML Full Factorial Matrix

Sanscrit_2prov — Mon, 28 Oct 2019 19:19:49 GMT

We are just using the design matrix to indicate the presence or absence of binary (present/absent) predictors (or covariates). I should have been clearer that this isn't an experiment. It is a program looking at average attributable risk over different patterns of predictors. The matrix tells us which predictors are present, It is then used in further matrix operations that multiply the beta coefficients for the predictors that are present, and the resulting sum is used in further calculations.

Re: Proc IML Full Factorial Matrix

Sanscrit_2prov — Mon, 28 Oct 2019 19:22:28 GMT

Yes I should have clarified. I use a SAS dataset read into an IML matrix.

Re: Proc IML Full Factorial Matrix

PaigeMiller — Mon, 28 Oct 2019 19:28:01 GMT

The matrix tells us which predictors are present, It is then used in further matrix operations that multiply the beta coefficients for the predictors that are present

This is what PROC SCORE does. SAS has done the hard work of programming this specific type of matrix calculation, so you don't have to. They have tested it thoroughly and debugged it.

Re: Proc IML Full Factorial Matrix

Rick_SAS — Mon, 28 Oct 2019 19:39:28 GMT

I don't understand what the data look like. Could you please provide an example data set (or matrix) that has three columns and five observations, and also supply the result that you expect to get?

Re: Proc IML Full Factorial Matrix

Sanscrit_2prov — Wed, 30 Oct 2019 16:59:21 GMT

Thank you so much. This is exactly what I needed and was unaware of the band function.

Re: Proc IML Full Factorial Matrix

Sanscrit_2prov — Wed, 30 Oct 2019 17:02:04 GMT

Thank you. Ian Wakeling solved my problem,

Re: Proc IML Full Factorial Matrix

Rick_SAS — Wed, 30 Oct 2019 18:24:36 GMT

If you do not need the individual elements but only the binary bit pattern, you can save quite a bit of memory by using a character string instead of a vector of 0/1 values. For example, compare BAND versus PUTN with a BINARYw. format:

proc iml;
n = 2##20;
x = band(n, 2##(20:0)) > 0;
print x;
y = putn(n, "binary20.");
print y;

For more examples and discussion, see "Creating a matrix with all combinations of zeros and ones."

Re: Proc IML Full Factorial Matrix

Sanscrit_2prov — Sat, 02 Nov 2019 18:24:37 GMT

I have a question.

Your code results in one row with 21 columns and the first column is a 1

How do I get all of the rows with the different columns equal to 1?

So let's say it was 4.

0000

0001

0010

etc

Re: Proc IML Full Factorial Matrix

IanWakeling — Sun, 03 Nov 2019 09:32:50 GMT

If you use the 'band' method, it only really makes sense to fill the matrix one row at a time as I have shown. I think Rick's point is mainly about storage space. As Paige has pointed out, the full-factorial matrix is going to be huge when you are looking at 2^16 or higher, and if you can manage with a character matrix, it will be reduce the memory requirement by a factor of 8. I have put both methods in to general modules below, so you can experiment.

proc iml;

start FullFactorial( n );
  r = 2 ## n;
  x = j(r, n);
  pot = 2 ## ( (n-1):0 );
  do i = 1 to r;
    x[i, ] = band( i-1, pot ) > 0;
  end;
  return( x );
finish;

start FFstr( n );
  fmt = cats("binary",char(n),".");
  return( putn( t( 0:(2##n - 1) ), fmt) );
finish;

a = FullFactorial( 4 );
print a;

b = FFstr( 4 );
print b;
  
quit;