topic Re: Seeing if two strings are "similar" enough in SAS Programming

Seeing if two strings are "similar" enough

cdubs — Thu, 26 Jul 2018 23:03:56 GMT

I have a column called "name" where people manually inputted a lot of people's names.

I am hoping to use SAS to look for duplicates, but there's a high chance there's high error rates, e.g. a few typos but a "normal human" would know that the two names are likely referring to the same person.

I was wondering what it might look like if I asked SAS to first sort names in alphabetical order, then for each entry, compare the string directly and below it -- if either of those entries has the same amount of characters +/- 2, and then the same letters but with 4 differences, then we will count that as a duplicate and then the new var duplicate = 1.

Not sure if people have ideas as to how I might compare strings this way?

Thank you!

Re: Seeing if two strings are "similar" enough

Patrick — Thu, 26 Jul 2018 23:08:37 GMT

@cdubs

Similar questions like yours come up from time to time. Have you already searched this forum with terms like "fuzzy match"?

You would find discussions like: https://communities.sas.com/t5/Base-SAS-Programming/Fuzzy-match-with-soundex-and-compged/m-p/295334/highlight/true#M61710

Re: Seeing if two strings are "similar" enough

cdubs — Thu, 26 Jul 2018 23:09:32 GMT

I haven't! I'll look at those now 🙂

Re: Seeing if two strings are "similar" enough

PeterClemmensen — Fri, 27 Jul 2018 01:11:53 GMT

Have a look at the COMPGED and COMPLEV functions are good places to start.