topic A quicker way to summarise datasets? in SAS Enterprise Guide

A quicker way to summarise datasets?

deleted_user — Thu, 03 Dec 2009 16:51:11 GMT

Hi,
I currently use the Summary Tables node to summarise any data I have, which works fine with smaller datasets.

However, this takes several hours with larger datasets.

Would it be quicker to use proc sql or proc summary/means in a code node?

Thanks,

Re: A quicker way to summarise datasets?

ChrisHemedinger — Thu, 03 Dec 2009 17:59:02 GMT

Cat,

You can use the Query Builder node to create summary tables with aggregates/grouping. That uses PROC SQL.

Then you can use another task (List Data, List Report) to create a report, if you want.

Chris

Re: A quicker way to summarise datasets?

LinusH — Fri, 04 Dec 2009 10:03:10 GMT

By experience, the quickest way to summarize data with SAS is by using PROC SUMMARY/MEANS with CLASS.
Can't see that any EG tasks that is using them though.
My first guess would be that TABULATE (summary tables task) would use the same algorithm, but I'm not sure.
Apart from different grouping algorithms between SQL group by and SUMMARY CLASS, is the ability to specify ID columns,
in case where you have a set of columns that forms a hierarchy, no need to build summary groups for all, just for the level with finest granularity.

If your data is already sorted, PROC SUMMARY with BY is the most efficient way to summarize.

/Linus

Re: A quicker way to summarise datasets?

Cynthia_sas — Fri, 04 Dec 2009 14:01:15 GMT

Hi:
I believe the Summary Tables task uses PROC TABULATE and the Summary -Statistics- Task or Wizard uses PROC MEANS (which is the same as PROC SUMMARY).

To benchmark, you could compare the query method with the Summary Statistics method to see which produces the better results.

cynthia

Re: A quicker way to summarise datasets?

Peter_C — Fri, 11 Dec 2009 13:27:29 GMT

if tabulate is doing the summary, make sure the listing destination is not open!