I'm running a PROC CORR on several thousand variables to do comparisons on product preferences. It works just fine if I keep the number of variables below 4500, but once I tell it to run PROC CORR on 5000 variables I immediately get an "Out of Memory" error.
I have tried changing OPTIONS to increase MEMSIZE with no luck. System memory is 2G, hard drive space has over 100G free, and I'm using Enterprise Guide on XP.
Is this a limitation on PROC CORR, a limitation on my system, or something else? Any advice on what I can do to get it to run? Should I be approaching this problem differently? Thanks for any advice!
Go to the documentation for CORR and you will find this in the section under computer resources: "The only factor limiting the number of variables that you can analyze is the amount of available memory. The computer resources that PROC CORR requires depend on which statements and options you specify." as well as the formulas for computing the resources needed.
As far as work-arounds, you can break the correlations down. Do the 2500, do the other 2500, do the first 2500 WITH the second 2500. Then, you can combine your summary datasets and do the screening.