I'm currently trying to run a hierarchical model on my dataset using a macro written by someone else that utilizes the Proc IML statement. However, I keep running into an error that I'm trying to figure out if it's my computer or SAS that is the problem.
The error I get is:
"Unable to allocate sufficient memory. At least 2147483647 more bytes required."
I've tried running it on my computer as well as a more powerful computer in our department (mine has 16GB ram, and the other has 32GB ram) and get the exact same error. I saw this post earlier in the year about limitations to the proc IML memory capacity and wanted to confirm that my understanding is that SAS is unable to work with larger datasets in the Proc IML statement (in 64-bit Windows 7) and that nothing can be done about it until, hopefully, SAS 9.4?
I have tried the memsize option and it doesn't make any difference. The code works since I tried using a small subset of my dataset to run the analyses and with about 5% of my observations, the macro works.
Do you have any recommendations on how best to split up the data? In a nutshell, the main dataset has about 225,000 observations (rows = people) with a comparison to approximately 106 variables. The second stage HLM takes the 106 and tries to group them. I'm not that familiar with SAS's ability to run logistic regressions on subsets and how to combine them.
A (225000 x 106) matrix isn't very large...only about 0.2 GB. You ought to have room for dozens of matrices of that size on your computer with 16GB of RAM.
Perhaps the macro uses many matrices of that size and does not free matrices that are no longer in use? If so, you might be able to tweak the macro.
If you want to try Steve's suggestion, here is an article that describes how to read a portion of the data at a time. Perhaps you will find it useful.
Or, if the macro is too complicated to pull apart, you can try using the FIRSTOBS= and OBS= data set options to subset the data within a macro loop.
Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.
If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website.
Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.
Find more tutorials on the SAS Users YouTube channel.