Hello,
I am getting the following error 'ERROR: Unable to allocate sufficient memory for the nonparametric correlation statistics.' when I try to compute Spearman correlation coefficients. Anyone know how to overcome this problem?
Huge thanks in advance!
First, check what memsize is set to in your SAS config file. Increasing the value there may solve the problem.
I suspect you have a pretty large dataset (multiple gigabytes?). One thing you might consider is using PROC MIANALYZE in a non-standard way.
First, use PROC SURVEYSELECT to subsample your data into multiple smaller datasets. Then use PROC CORR separately on each of these, adding the variable _imputation_ to each output. You could then run PROC MIANALYZE to get a model averaged result. See the example Combining Correlation Coefficients in the PROC MIANALYZE documentation for a way to get started..
SteveDenham
If your data set is really huge, I think Steve's suggestion can be simplified. You can simply use PROC SURVEYSELECT or the DATA step to take a random subset (a few million observations) and the use PROC CORR to get estimates from the sample.
Not only is @Rick_SAS 's approach simpler, it will almost surely get around all of your memory problems. If a couple million records still overwhelms memory, then there are other issues that need to be addressed first.
SteveDenham
Build your skills. Make connections. Enjoy creative freedom. Maybe change the world. Registration is now open through August 30th. Visit the SAS Hackathon homepage.
Register today!ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.
Find more tutorials on the SAS Users YouTube channel.