03-02-2018 08:14 PM
I have 2 datasets: main and mds
The mds dataset has pcp_names in 1 variable, and pcp first_name, last_name, and middle_initial also as 3 separate variables.
The main dataset also has a pcp variable but not necessarily in the same format as the pcp_names variable in the mds dataset (may have MD attached, may not have middle initial, etc.).
I want to flag variables in my main dataset who have a pcp in the mds dataset. We can translate this to mean that if there is some sort of combination of pcp first_name and pcp last_name in my mds dataset in the string variable pcp in my main dataset, then keep this or flag it 1. Any help on how to do this with 2 separate datasets would be really appreciated.
03-02-2018 08:32 PM
It’s a fuzzy join/merge that is data intensive because you essentially have to compare every single record in each file against every other record.
You can can look at some of the options for fuzzy lookups such as
compged, soundex, complev.
If you have another field you can join on as well, such as birth dates, age or facility that can signicantly reduce the number of comparisons.
Maybe the answer here from friedegg would be helpful.
03-02-2018 09:27 PM
Thank you - I am looking into those functions. I think it essentially needs to be something like an indexc embedded in a hash, but I am not sure if I can even do that.