Help in Jaro Wrinkler/Edit distance Similarity

Kalai2008 — Thu, 23 Jul 2020 03:48:13 GMT

Hi,

I was tasked to convert SAS programs to Hive queries in EAP. Since I am new to hive codes, I am not sure what is the equivalent code for the below SAS function in hive.

utl_match.jaro_winkler_similarity (REPLACE (UPPER(name1), ' ', ''), REPLACE (UPPER(name2), ' ', '')) AS j_score,

utl_match.edit_distance_similarity (REPLACE (UPPER(name1), ' ', ''), REPLACE (UPPER(name2), ' ', '')) AS e_d_score

I would appreciate if someone help me to provide the equivalent sas function code in Hive.

Thanks for checking.

Re: Help in Jaro Wrinkler/Edit distance Similarity

smantha — Thu, 23 Jul 2020 04:13:43 GMT

Unfortunately there are no exact equivalents in SAS. The closest that come to edit distances are compged and speeds functions. You have to understand that the main difference in edit distances functions is the weight that is given to operations. Unless you have SAS text miner or SAS Viya. If you have SAS Viya you can leverage python and py_stringmatching package. (AnHai Doan, Alon Halevy, Zachary Ives, “Principles of Data Integration”, Morgan Kaufmann, 2012. Chapter 4 “String Matching” (available on the package’s homepage).). You can develop these functions by yourself in SAS if you have time and patience to write your own algorithm.

topic Re: Help in Jaro Wrinkler/Edit distance Similarity in SAS Programming

Help in Jaro Wrinkler/Edit distance Similarity

Re: Help in Jaro Wrinkler/Edit distance Similarity