BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
R_Fraser
Calcite | Level 5

Apparently SAS is not coping well with my very large matrix set-up here. Is there a way to do this computation more efficiently?

B = t(X)*diag(w)*X;

ERROR: (execution) Unable to allocate sufficient memory. At least 2147483647 more bytes required.

operation : DIAG at line 8841 column 14

operands  : w

w 365703 rows      1 col     (numeric)

Thanks,

Raphael

1 ACCEPTED SOLUTION

Accepted Solutions
Rick_SAS
SAS Super FREQ

Yes, see this tip on how to avoid multiplying by a large diagonal matrix. The second half of the article shows how to use ordinary elemtnwise multiplication to avoid forming the large matrix.

View solution in original post

4 REPLIES 4
jakarman
Barite | Level 11

The first challenge is understanding your computing environment. What machine type are you running on processors/memory OS version SAS version?

As SAS is having a lot of memory options, how are these being set? That are you environment limitation as set by you.. 

There is one row w: of 370k numeric (each is occupying 8 bytes) that is 2Mb depending of the size of X you are needing that size being multiplied.
How big is your resulting request?    The diag function will create a diagonal matric needing space for 2Mb*2Mb  = 4Tb. Well you do not need to wait another 7! days for that. 
http://blogs.sas.com/content/iml/2014/04/28/how-much-ram-do-i-need-to-store-that-matrix.html

---->-- ja karman --<-----
Rick_SAS
SAS Super FREQ

Yes, see this tip on how to avoid multiplying by a large diagonal matrix. The second half of the article shows how to use ordinary elemtnwise multiplication to avoid forming the large matrix.

R_Fraser
Calcite | Level 5

Thank you all! Just as I suspected, there is a more efficient way.

Ksharp
Super User

Using the following options to make your available memory bigger.

PROC IML   SYMSIZE=n1    WORKSIZE=(n2)  ;

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

Multiple Linear Regression in SAS

Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.

Find more tutorials on the SAS Users YouTube channel.

From The DO Loop
Want more? Visit our blog for more articles like these.
Discussion stats
  • 4 replies
  • 7634 views
  • 6 likes
  • 4 in conversation