Proc Freq Out of memory issue

[PravinMishra]

Hi Jaap,

Below is the error.

ERROR: Not enough memory for all variables. Variable RAR_ENTERED_ACCT removed after 3470836 levels.

ERROR: The SAS System stopped processing this step because of insufficient memory.

Below is the Code which I am running

proc freq data=SRS_AUTH_201302 noprint;

&num_freq;

run;

&num_freq is a macro variable having last of 45 variable with proper syntax.

[Reeza]

You should show what &num_freq looks like.

If it's something like

table var1*var2*var3*...*var45/out=want

Then you can try replacing that with a Proc SQL statement instead or partitioning the data set and running the code multiple times.

[Kurt_Bremser]

Variable RAR_ENTERED_ACCT removed after 3470836 levels.

This simply means that at the time of the error, 3470836 distinct values for RAR_ENTERED_ACCT had been encountered, which signals that this variable is not one for which freq makes ANY sense. Or do you like to read sheets with > 3 million lines?

Inspect the data first visually to determine which variables can sensibly be counted for distinct values!!

[Reeza]

Post your full code/log with error, on a server you should definitely not be running out of memory.

If you still are try something like the following. It's brute force and I would never use it in production:

*trim data set first;

data first_pass;

set have;

by col_name notsorted;

if first.col_name;

run;

*then run a proc freq on it;

proc freq data=first_pass nlevels;

table col_name;

run;

[jakarman]

This paragraph Base SAS(R) 9.4 Procedures Guide: Statistical Procedures, Second Edition explains the resource usage in proc print.

Giving a variable list all possible values must fit into memory.   That RAR_ENTERED_ACCT has a lot of them, wondering how that printed list would be.

Proc options will show all settings including the memory but you can limit that to group is memory. Base SAS(R) 9.4 Procedures Guide, Third Edition

Take all your variables in that list do those calculations and it will most likely showing you you are running out of memory

Normally with big datasets is doesn't make sense to print all variables in a dataset as there are some rather unique ones like accountnumbers  or record-id.

When there is a measurement a classification (using format) can be very useful.