ERROR: Unable to allocate sufficient memory for the nonparametric correlation statistics.

[aiste]

Hello,

I am getting the following error 'ERROR: Unable to allocate sufficient memory for the nonparametric correlation statistics.' when I try to compute Spearman correlation coefficients. Anyone know how to overcome this problem?

Huge thanks in advance!

[SteveDenham]

First, check what memsize is set to in your SAS config file.  Increasing the value there may solve the problem.

I suspect you have a pretty large dataset (multiple gigabytes?).  One thing you might consider is using PROC MIANALYZE in a non-standard way.

First, use PROC SURVEYSELECT to subsample your data into multiple smaller datasets.  Then use PROC CORR separately on each of these, adding the variable _imputation_ to each output.  You could then run PROC MIANALYZE to get a model averaged result. See the example Combining Correlation Coefficients in the PROC MIANALYZE documentation for a way to get started..

SteveDenham

[Rick_SAS]

If your data set is really huge, I think Steve's suggestion can be simplified. You can simply use PROC SURVEYSELECT or the DATA step to take a random subset (a few million observations)  and the use PROC CORR to get estimates from the sample. 

[SteveDenham]

Not only is @Rick_SAS 's approach simpler, it will almost surely get around all of your memory problems.  If a couple million records still overwhelms memory, then there are other issues that need to be addressed first.

SteveDenham