topic Re: Sort Order and Merge By in SAS Enterprise Guide

Sort Order and Merge By

yudhishtirb — Tue, 28 Jun 2016 11:06:46 GMT

Hello Team,

Greetings

We have 2 datasets. On these 2 datasets we are performing some merge operations.

What will be impact if

1. input data sets are in Sorted order

2. Input data sets are not in sorted order

Thanks In advance

Re: Sort Order and Merge By

Kurt_Bremser — Tue, 28 Jun 2016 12:16:41 GMT

For a DATA step merge, sorting before the merge is mandatory. Exception: if the datasets are stored in SPDS, then the sort is done automatically (this does not reduce the overall resource consumption, you just don't need to write and execute the SORT step on your own)

If you use SAS SQL, no sorting is necessary, as SQL does this on the fly. But there is a big caution: with large datasets, the utilty file of the SQL step can grow out of proportion, and the random accesses to this file will be much less performant than the combined SORT/DATA steps. Perfomance penalties of orders of magnitude have been observed.

Re: Sort Order and Merge By

LinusH — Wed, 29 Jun 2016 06:33:33 GMT

Addition to @Kurt_Bremser exception list: implicit sort is also being done for SPDE and most SAS/ACCESS to RDBMS engines.

Re: Sort Order and Merge By

LinusH — Wed, 29 Jun 2016 06:33:50 GMT

Addition to @Kurt_Bremser exception list: implicit sort is also being done for SPDE and most SAS/ACCESS to RDBMS engines.