topic Re: Insufficiency memory error in SAS Procedures

Insufficiency memory error

kirillschmidt — Fri, 20 Apr 2018 18:32:17 GMT

Hello!
I have a small problem with proc inbreed.
I need to calculate inbreeding coefficient for 1,700,000 individs. But my computer stopped processing because of insufficient memory. I tried to do it for 20,000 individs and SAS-file was about 3gb.Why does it take so much memory? Maybe I do something wrong. Are there any ideas how to fix/solve it? Thank you! And sorry for stupid questions.

Re: Insufficiency memory error

ballardw — Fri, 20 Apr 2018 19:17:07 GMT

You might try the NOPRINT option and direct the output to a dataset using the OUTCOV= option. Some times these memory issues come from massive amounts of output sent to a results window. Then use proc print or other option to examine the pieces you are interested in from the output set.

Also the specific options involved may increase memory needs. It might help to show your code for someone to suggest efficiency changes.

Re: Insufficiency memory error

kirillschmidt — Fri, 20 Apr 2018 20:20:50 GMT

Thanks for reply! Yes, I use NOPRINT in my code. And my code is really easy. So I don't know what to do. Seems this SAS procedure is the best solution for my need but final sas7bdat-file is unreally big for my pc.

My code is here.

proc inbreed data=example1 outcov=A covar matrix NOPRINT;
var individ mother father;
run;

/*example1 is 1.7M x 3 table*/

Re: Insufficiency memory error

ballardw — Fri, 20 Apr 2018 22:14:02 GMT

Did you happen to note how many variables are in the OUTCOV data when you ran this with 20000 records? Then consider the 1.7M.

You might consider, if practical, adding a generation or similar class variable to subset the data a bit.

If you have repeats of mother father reduce to a single one. Your current code is basically assuming everyone is in the same generation so repeats of family members in the same generation don't add much information while adding many more columns. The individuals would have the same relationship to external-to-the-family members.