How to create SAS Scalable Performance Data Engine tables on the Hadoop Distributed File System

Allaluiah — Sun, 27 Dec 2015 20:01:39 GMT

How to join the 2 very large datasets with duplicate id keys in both the datasets

For example, I have the following datasets

table1 :

acct_no a b c d

a1 4 7 2 8

a3 5 32 8 12

a5 42 12 54 65

a1 5 2 17 23

acct_no is the key with duplicates because it is a transaction dataset.

table 2:

acct_no card_no

a4 54

a3 31

a4 43

a1 24

a8 12

a7 23

a8 45

acct_no in table 2 also has duplicates because the logic is that a single customer can have more than one card, one a primary card and the other being secondary card

Can anybody help me with the logic for the join?

The datasets contain 400 million records and are pretty wide and long, however i need to fetch only the card_no from table2, so table 1 would be my left table. Sorting will not help either

Re: How to create SAS Scalable Performance Data Engine tables on the Hadoop Distributed File System

LinusH — Sun, 27 Dec 2015 22:08:11 GMT

Contents of this post seem like a duplicate.

But the title is totally different from the question?

topic Re: How to create SAS Scalable Performance Data Engine tables on the Hadoop Distributed File System in SAS Programming

How to create SAS Scalable Performance Data Engine tables on the Hadoop Distributed File System

Re: How to create SAS Scalable Performance Data Engine tables on the Hadoop Distributed File System