BookmarkSubscribeRSS Feed
MBI
Calcite | Level 5 MBI
Calcite | Level 5
Hi Everyone,

I'm running a PROC CORR on several thousand variables to do comparisons on product preferences. It works just fine if I keep the number of variables below 4500, but once I tell it to run PROC CORR on 5000 variables I immediately get an "Out of Memory" error.

I have tried changing OPTIONS to increase MEMSIZE with no luck. System memory is 2G, hard drive space has over 100G free, and I'm using Enterprise Guide on XP.

Is this a limitation on PROC CORR, a limitation on my system, or something else? Any advice on what I can do to get it to run? Should I be approaching this problem differently? Thanks for any advice!

-MBI
3 REPLIES 3
Doc_Duke
Rhodochrosite | Level 12
Go to the documentation for CORR and you will find this in the section under computer resources: "The only factor limiting the number of variables that you can analyze is the amount of available memory. The computer resources that PROC CORR requires depend on which statements and options you specify." as well as the formulas for computing the resources needed.

As far as work-arounds, you can break the correlations down. Do the 2500, do the other 2500, do the first 2500 WITH the second 2500. Then, you can combine your summary datasets and do the screening.

Doc Muhlbaier
Duke
MBI
Calcite | Level 5 MBI
Calcite | Level 5
Duke,

Thanks for the quick reply. I was able to find this link ( http://www.sfu.ca/sasdoc/sashtml/proc/zconcept.htm ) which describes the calculations you mentioned. Looks like I need a bigger hard drive!

-MBI
Doc_Duke
Rhodochrosite | Level 12
If you are doing the non-parametric correlations, then you may need the huge disk space. If you are doing Pearson correlation, then the need is for RAM.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 3 replies
  • 1247 views
  • 0 likes
  • 2 in conversation