About IanWakeling

IanWakeling · ‎05-29-2020

If you wish to keep the IML code, then the commented out part which 'uses a lot of space', could be made a lot more efficient. There is no need to keep the weights in a diagonal matrix, as it could be rewritten to use a vector of weights instead. This would save a lot of unnecessary multiplication too.

IanWakeling · ‎05-19-2020

My suggestion is to declare the storage immediately before the loop like so: b_storage = j(c_rows, 2, .); That is a matrix with as many rows as combinations, 2 columns and filled with missing values. Once you have the matrix b in the loop, then copy it to the ith row of the storage matrix as follows: b_storage[i, ] = b; The if/then/else statement can then be deleted, as can the definition of matrix b_storage_first. Incidentally the statements that define group_col and lag2cvrank_col can be moved outside of the loop, they are being overwritten on every iteration of the loop - this will save a bit more time. Finally, have you thought what happens if you have two data rows with identical pairs of x-values?

IanWakeling · ‎05-19-2020

You say the IML solution is taking a lot longer, but there are probably ways in which it could be made more efficient. It is generally a bad idea to 'grow' a matrix inside a loop with syntax like: b_storage=b_storage//b; as this makes a new matrix for each iteration, and moves data around in memory unnecessarily. It is better to create b_storage at it final size before the loop, and then write one row at a time. I am guessing the data step solution you mention uses a short cut to compute regression coefficients directly, since there are only two points and no error. You could do the same in IML to speed things up. For an even faster solution, it could be vectorized to eliminate the loop over the combinations.

IanWakeling · ‎05-13-2020

Indeed it is elementwise multiplication, but if you want the row sums of c, which was part of the original problem, then the whole process simplifies to normal matrix multiplication: a = {1 2 3}; b = {1 2 3, 4 5 6, 7 8 9}; d = b * a`;

IanWakeling · ‎05-12-2020

As Paige says you are overwritting the same data set in the loop. An alternative would be to build one data set with successive appends as follows: create MVN var {"N" "ID" "y0" "y1" }; do N=5 to 10 by 5; X=RandNormal(N*Numsamples,Mean2,Cov2); ID=colvec(repeat(T(1:Numsamples),1,N)); Z = j(nrow(X),1,N)||ID||X; append from Z; end; close MVN; I have added the loop variable N to the output data set which you can use in WHERE or BY statements in other SAS PROCs.

IanWakeling · ‎05-12-2020

You could apply the Kroneker product (the @ operator) and avoid the loop as follows: diff = Y @ j(nrow(X), 1) - j(nrow(Y), 1) @ X; Z = shapecol( diff[ ,+], nrow(X)); print Z; It is not as easy to see what is going on, so you may prefer Rick's solution if others need to use your code.

IanWakeling · ‎03-21-2020

Thanks for your reply Rick. I am still experimenting with Lists in IML, and I am using ListGetAllNames() which returns an empty matrix if there are no names associated with the list items. I am sure I will find a workaround.

IanWakeling · ‎03-21-2020

I was surprised recently to get a "Matrix has not been set to a value" error when attempting to assign an empty matrix using VALSET. If the syntax x = {}; is acceptable, should it not be possible to do the same thing indirectly? For example, call valset('x', {}); will generate the error referred to above. My actual code is more complex, where the 2nd argument to VALSET is a function that might occasionally return an empty matrix, so I have simplified things above to illustrate the point. I am using SAS/IML 15.1.

IanWakeling · ‎03-10-2020

I misunderstood what you were asking for. Looking at the last ANOVA table, things would be improved if you used rowcatc(CLC) instead of rowcat(CLC). The extra 'c' will compress the blanks and make things look more tidy. As you probably want different formats for different columns, it might be better not to concatenate everything into one matrix, so switch to something similar to the 2nd suggestion that I made earlier: print 'ANOVA table',CLC [format=$10.] DF [format=3.0] SS [format=8.3] MS [format=8.3] Fratio [format=6.2] pvalue [format=pvalue6.4];

IanWakeling · ‎03-10-2020

I am sorry, I don't have the time to check your program as it is quite large. What I suggest you do is validate what you have done in IML against a standard. You could export the design variables A B C D and response to a SAS dataset and then fit the model with main effects and 2-way interactions using PROC GLM. Best, Ian

IanWakeling · ‎03-06-2020

Here are some suggestions of ways to work around this problem. As it's only the 1st column that is character, then the simplest solution is just to use rownames - so form a numeric matrix called ANOVA by concatenation and then do print ANOVA [rowname=factors label='ANOVA table']; Note that it is possible to use commas in a print statement to move to the next line so something like print 'ANOVA table',fac DF [format=3.0] SS MS F p [format=pvalue6.4]; will give you something with a title above the table. The pvalue SAS format can be useful here as it will print "<.0001" for very small values. Alternatively you can investigate the Table data type in IML which allows for mixed columns. Rick has blogged about this here , but if your IML version is not up to date, you may not have this feature.

IanWakeling · ‎03-02-2020

Expandgrid can accept character arguments, so it could also be done like this. proc iml; start XAllLetterComb(n); if n=1 then return ( {'1','A'} ); cmd = "m = expandgrid("; do i = 1 to n; cmd = cmd + cat("{' ','", byte(65+n-i)," '}"); if i < n then cmd = cmd + ","; end; cmd = cmd + ");"; *print cmd; CALL EXECUTE(cmd); m[1]='1'; return ( compbl ( left( reverse( rowcat( m ))))); finish; c = XAllLetterComb(5); print c; To see the expandgrid syntax, then uncomment the print statement.

IanWakeling · ‎02-27-2020

Rick, an interesting idea to pre-compute the matrices and use a list. I have come up with a recursive direct product algorithm that is well suited to this. proc iml; n = 5; L = ListCreate( n ); L$1 = {0, 1}; do i = 1 to n - 1; L$(i+1) = ( L$1 @ j(2##i, 1) ) || ( {1,1} @ L$i ); end; do i = 1 to n; print (L$i); end; The code uses the list syntax that came in with SAS/IML 14.3.

IanWakeling · ‎02-26-2020

Like this? proc iml; start FullFactorial( n ); r = 2 ## n; x = j(r, n); pot = 2 ## ( 0:(n-1) ); /* reversed powers of 2 in this line */ do i = 1 to r; x[i, ] = band( i-1, pot ) > 0; end; return( x ); finish; a = FullFactorial( 4 ); print a [colname={A B C D}]; quit;

IanWakeling · ‎02-26-2020

If the order of the rows of the matrix is not important then the problem is equivalent to counting in binary. For example proc iml; start FullFactorial( n ); r = 2 ## n; x = j(r, n); pot = 2 ## ( (n-1):0 ); do i = 1 to r; x[i, ] = band( i-1, pot ) > 0; end; return( x ); finish; a = FullFactorial( 4 ); print a [colname={A B C D}]; quit;

Online Status	Offline
Date Last Visited	‎05-07-2025 04:11 AM

Re: Speed up function computation on a large matrix containing mostly ...

Re: How to generate all positive/negative permutations of an a one-row...

Re: Error using winsorize module: ERROR (execution) Matrix has not bee...

Re: Matrix exponential

Re: Matrix exponential

Re: What of my used syntax is the culprit for slowing down the perform...

Re: A SAS/IML Module to Extend the Series Subroutine

Re: Using IML executefile() to execute SAS code written to an external...

Re: Using IML executefile() to execute SAS code written to an external...

Re: SAS IML: how to create a T x T matrix from a T x 1 vector

Re: Error using winsorize module: ERROR (execution) Matrix has not bee...

A SAS/IML Module to Extend the Series Subroutine

Re: Multiple linear regression using proc iml

Re: Proc Mixed- Normality test for random effects

Re: Similar to LOC, but want index of nearest lesser value

Re: How to generate all positive/negative permutations of an a one-row...

Re: What of my used syntax is the culprit for slowing down the perform...

Re: Using IML executefile() to execute SAS code written to an external...

Re: SAS IML: how to create a T x T matrix from a T x 1 vector

Re: Multiple linear regression using proc iml

Sudoku Puzzle Generator

Four types of Latin square and an application to league scheduling

Re: PROC IML/ CASE SCHILLER/REPEAT SALES INDEX

Re: OLS with no intercept by pairs of observations

Re: OLS with no intercept by pairs of observations

Re: IML how to calculate new matrix based on values of two matrices

Re: Simulate bivariate normal data for multiple Sample size for two gr...

Re: IML how to calculate new matrix based on values of two matrices

Re: Indirect assignment of empty matrix

Indirect assignment of empty matrix

Re: How can I Concatenate more than two columns with different variabl...

Re: How can I Concatenate more than two columns with different variabl...

Re: How can I Concatenate more than two columns with different variabl...

Re: How can I find the Combinations of factor levels in SAS/IML

Re: How can I find the Combinations of factor levels in SAS/IML

Re: How can I find the Combinations of factor levels in SAS/IML

Re: How can I find the Combinations of factor levels in SAS/IML