Creating a unique ID with CAS DATA Step

9 Likes

CAS is massively parallel, spreading its processing across multiple threads on multiple machines but this affects the way you need to write your CAS DATA Step code. Let's look at how we might choose to create a unique row identifier in CAS versus SAS.

.

SAS DATA Step

Creating a unique ID in SAS DATA Step is quite simple. We just use the _n_ automatic variable as shown below:

.

DATA tableWithUniqueID;
SET tableWithOutUniqueID; 

        uniqueID = _n_;

run;

.

CAS DATA Step

Creating a unique ID in CAS DATA Step is more complicated. Each thread maintains its own _n_. So if we just use _n_, we'll get duplicate IDs. Each thread will produce an uniqueID field value of 1. Each thread will produce an uniqueID field value of 2. And so on.... When the thread output is combined, we'll have a bunch of records with a uniqueID of 1 and a bunch with a uniqueID of 2.... This is not useful. To produce a truly unique ID, you need to augment _n_ with something else. CAS has its own set of automatic variables for such purposes. The _threadID_ automatic variable can help us get our unique ID as shown below:

.

DATA tableWithUniqueID;
SET tableWithOutUniqueID;

        uniqueID = put(_threadid_,8.) || '_' || Put(_n_,8.);

run;

While there are surely other ways of doing it, concatenating _threadID_ with the _n_ ensures uniqueness because the _threadID_ uniquely identifies a single thread and _n_ uniquely identifies a single row output by that thread.

For more information on threads and CAS DATA Step, see this blog post.

Bogdan_Teleuca · ‎07-21-2021

Very useful for CAS, thank you.

If you want the uniqueID to be a number, instead of a varchar, you can create it using a Cantor Function π(a,b)=1/2(a+b)(a+b+1)+b ;

The Cantor Function ensures a unique combination is obtained from two integers.

uniqueID= 1/2 * (put(_threadid_,8.) + Put(_n_,8.)) 
* ((put(_threadid_,8.) + Put(_n_,8.)) + 1 ) + Put(_n_,8.);