topic Re: Select Distinct Duplicate values in SAS Enterprise Guide

Select Distinct Duplicate values

umar_milanzi — Tue, 30 May 2017 09:11:07 GMT

hi All,

i need help. I have a dataset, sample below:

CUSTOMER_NAME MOBILE_NUMBER

John 555-777

Sam 556-865

Joe 555-000

Sam+Joe 556-865

Tim 555-777

want to output customers with same mobile number but different names. in the data above we can see that Sam and Sam+Joe have the same number, this means Sam is Joe's partner so they can have the same number. but John and Tim are not connected and have the same number, those are the guys i want to output

thanks in advance

Re: Select Distinct Duplicate values

RW9 — Tue, 30 May 2017 09:44:27 GMT

Post test data in the form of a datastep.

Maybe something like (and can't test as not test data):

proc sql;
  create table WANT as
  select  distinct
          MOBILE_NUMBER,
          CUSTOMER_NAME
  from    HAVE
  group by MOBILE_NUMBER,
           CUSTOMER_NAME
  having count(*) gt 1;
quit;

Re: Select Distinct Duplicate values

umar_milanzi — Tue, 30 May 2017 10:03:10 GMT

data have;
input customer_name $ Mobile_Number ;

datalines;
john 555777
sam 556865
joe 555888
sam+joe 556865
tim 555777
; run;

Re: Select Distinct Duplicate values

Ksharp — Tue, 30 May 2017 10:21:11 GMT

"but John and Tim are not connected and have the same number, those are the guys i want to output"

You mean you do not want to output ?

Re: Select Distinct Duplicate values

Astounding — Tue, 30 May 2017 10:46:43 GMT

Names are often spelled differently, when entered more than once. As a result, I suggest you go about this as a two-step problem. First, find all the differences:

proc sql;

create table differences as

select a.*, b.customer_name as matching_name

from have a, have b

where a.mobile_number = b.mobile_number

and a.customer_name ne b.customer_name;

quit;

data differences2;

set differences;

if length(customer_name) le length(matching_name) then do;

dummy = customer_name;

customer_name = matching_name;

matching_name = dummy;

end;

drop dummy;

run;

In DIFFERENCES2, CUSTOMER_NAME and MATCHING_NAME will be different, but the DATA step guarantees that CUSTOMER_NAME will be the variation that contains more characters.

Then you can begin to automate the process of removing matches that should be removed. Due to variations in spelling, you may need more than one pass through the data. For example, does "Sam" match "Samuel"? Should those be output? Here is one pass through to eliminate the duplicates:

data want;

set differences2;

if index(customer_name, strip(matching_name)) = 0;

run;

Instead of the INDEX function, you may want to use FINDW. It really depends on the nature of the data, as to which function gives you better results.

Re: Select Distinct Duplicate values

umar_milanzi — Tue, 30 May 2017 10:49:09 GMT

Thanks @Astounding this looks like it is doing what i want.

Thanks everyone for the inputs.