BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
HyunJee
Fluorite | Level 6

I am trying to create an output dataset of the following variable comparisons using proc freq. I am able to do this with one subset of my population which is small, but when I run the same coding on the larger dataset, the error that the table is too large to process appears in the log. Is there a way to fix this in a setting or do I need to try and create the variable comparisons in a different program? Thank you for any help you can give.

proc freq data=hpv_single_reasons;

tables r_HPV_recode_1*r_HPV_recode_2*r_HPV_recode_3*r_HPV_recode_4*r_HPV_recode_5*r_HPV_recode_6*r_HPV_recode_7*r_HPV_recode_8*

r_HPV_recode_9*r_HPV_recode_10*r_HPV_recode_11*r_HPV_recode_12*r_HPV_recode_13*r_HPV_recode_14*r_HPV_recode_15*r_HPV_recode_16*

r_HPV_recode_17*r_HPV_recode_18*r_HPV_recode_19*r_HPV_recode_20*r_HPV_recode_21*r_HPV_recode_22*r_HPV_recode_23*r_HPV_recode_24*

r_HPV_recode_25*r_HPV_recode_26*r_HPV_recode_27/noprint out = hpv_single_reasons;

run;

1 ACCEPTED SOLUTION

Accepted Solutions
Alpay
Fluorite | Level 6

I think Proc Freq might be trying to utilize memory to fit all the combinations of class variables in it.

Have you tried Proc Summary instead? It will provide you the counts not the percentages which can be computed easily once you get the counts.

proc summary data=hpv_single_reasons nway missing;

    class  r_HPV_recode_1 r_HPV_recode_2 r_HPV_recode_3 r_HPV_recode_4 r_HPV_recode_5 r_HPV_recode_6

r_HPV_recode_7 r_HPV_recode_8 r_HPV_recode_9 r_HPV_recode_10 r_HPV_recode_11 r_HPV_recode_12 r_HPV_recode_13

r_HPV_recode_14 r_HPV_recode_15 r_HPV_recode_16 r_HPV_recode_17 r_HPV_recode_18 r_HPV_recode_19

r_HPV_recode_20 r_HPV_recode_21 r_HPV_recode_22 r_HPV_recode_23 r_HPV_recode_24

r_HPV_recode_25 r_HPV_recode_26 r_HPV_recode_27;

output out=hpv_single_reasons(rename=_freq_=Count drop=_type_);

run;

View solution in original post

2 REPLIES 2
Alpay
Fluorite | Level 6

I think Proc Freq might be trying to utilize memory to fit all the combinations of class variables in it.

Have you tried Proc Summary instead? It will provide you the counts not the percentages which can be computed easily once you get the counts.

proc summary data=hpv_single_reasons nway missing;

    class  r_HPV_recode_1 r_HPV_recode_2 r_HPV_recode_3 r_HPV_recode_4 r_HPV_recode_5 r_HPV_recode_6

r_HPV_recode_7 r_HPV_recode_8 r_HPV_recode_9 r_HPV_recode_10 r_HPV_recode_11 r_HPV_recode_12 r_HPV_recode_13

r_HPV_recode_14 r_HPV_recode_15 r_HPV_recode_16 r_HPV_recode_17 r_HPV_recode_18 r_HPV_recode_19

r_HPV_recode_20 r_HPV_recode_21 r_HPV_recode_22 r_HPV_recode_23 r_HPV_recode_24

r_HPV_recode_25 r_HPV_recode_26 r_HPV_recode_27;

output out=hpv_single_reasons(rename=_freq_=Count drop=_type_);

run;

Astounding
PROC Star

When the data requires too much memory to process, one approach to work around it is to use extra CPU time instead of memory.  In the extreme, you could sort your data by all 27 variables, and then use a DATA step with a BY statement to count observations.

Alternatively, if by some chance your data are already in order by one of the variables, you could use that as a BY variable in PROC FREQ (or PROC SUMMARY for that matter).  For example, if your data were in order by r_HPV_recode_1, but not in order by any of the other variables, you could use:

proc freq data=hpv_single_reasons;

  by r_HPV_recode_1;

  tables (same list, but remove r_HPV_recode_1) / out=hpv_single_reasons;

run;

The counts will be the same as if your original program had worked, although the percentages will be different.

There's no guarantee it will work, but it certainly has a good chance.  You might have to switch 2 or 3 variables to the BY statement instead of just 1, in which case the SORT + DATA step might be preferable.

Good luck.

sas-innovate-2024.png

Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 2 replies
  • 4226 views
  • 3 likes
  • 3 in conversation