BookmarkSubscribeRSS Feed
Venkibhu1
Calcite | Level 5

Dear All,

 

The below code is not executing successfully as the matrix size is large. The program is ending up with the below error

 

ERROR: (execution) Unable to allocate sufficient memory.

operation : LENGTH at line 1395 column 11
operands : _TEM1001
_TEM1001 33 rows 400000 cols (character, size 80)

 

Is there anyway to find and split the matrix based on the size?

 

dm "output" clear;
data one;
array cha{*} $18. c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 c11 c12 c13 c14 c15 c16 c17 c18 c19 c20 c21 c22 c23 c24 c25 c26 c27 c28 c29 c30 c31 c32 c33;
do j = 1 to 400000;
seed = 100;
do i = 1 to hbound(cha);
rannum = ranpoi(seed,50);
if rannum = 0 then rannum = 1;
cha{i} = repeat("a",rannum);
end;
output;
end;
run;

proc iml;
use one;
read all var _CHAR_ into y[colname=cNames];
close one;
c = length(y`);
k = cnames`||char(c[,<>]);
print k;
quit;

 

Thanks in Advance,

 

Venkat.

3 REPLIES 3
Rick_SAS
SAS Super FREQ

I'm on my way out the door, but look at the following links for dealing with big matrices: 

 

IanWakeling
Barite | Level 11

I think you are nearly running out of memory when you create y`, and then when you try to create c that is the final straw.  Since c needs less space than y`, I think it should work if you avoid forming y`,   For example:

 

proc iml;
use one;
read all var _CHAR_ into y[colname=cNames];
close one;
c = length(y);
print cnames;
k = t( cnames//char(c[<>,]) );
print k;
quit;
Rick_SAS
SAS Super FREQ

The size of y will depend on the length of the character variables in the data set.  A variable with length=256 takes twice as much RAM as one that has length=128. 

 

Look at the MEMSIZE option (second link) to allocate more memory. Since you are computing a maximum, you can also break the data into chunks, as shown in the third link. To build on Ian's program:

 


proc iml;
use one nObs N;
chunkSize = N/ 10;
max = .;
do data;
   read next (chunkSize) var _CHAR_ into y[colname=cNames];
   c = length(y);
   if max=. then max = c[<>, ];
   else max = (max // c[<>, ])[<>, ];
end;
close one;

k = t( cnames// char(max) );
print k;

 

 

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

Multiple Linear Regression in SAS

Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.

Find more tutorials on the SAS Users YouTube channel.

From The DO Loop
Want more? Visit our blog for more articles like these.
Discussion stats
  • 3 replies
  • 1322 views
  • 4 likes
  • 3 in conversation