BookmarkSubscribeRSS Feed
juliajulia
Obsidian | Level 7

i have a dataset that has two ids: id1 and id2. i want to find out if there are any duplicated records for the combined id1 and id2. i created a code like this but it don't work. are there any easy way to do this in sas proc sql?

 

data combine;
set dataset;
combinedid=catx(id1, id2);
run;

PROC SQL;
SELECT id1, id2,
FREQ(combinedid) AS dupe
FROM dataset
GROUP BY combinedid
HAVING dupe GE 2;
quit;

4 REPLIES 4
Kurt_Bremser
Super User

Run a count and use HAVING:

proc sql;
select id1, id2, count(*) as count
from have
group by id1, id2
having calculated count > 1;
quit;

Or run a proc freq or proc summary and filter out the observations with a count > 1.

Reeza
Super User
Why use SQL at all? Why not use PROC SORT which has built in functions to help identify duplicates. Look at the NODUPKEY, NOUNIQUEKEY, DUPOUT, UNIQUEOUT options.
It can identify duplicates across multiple columns and easily separate them into their own data sets with no need to combine anything.

You can definitely roll your own via SQL but it's faster, easier and more efficient to use the developed procedures.

Reeza
Super User
And you may want to sort those IDs first as well.

ie should these be duplicates:

ID1 ID2
ABC DEF
DEF ABC

None of the posted solutions will deal with this scenario.
juliajulia
Obsidian | Level 7

Thank you all. Both proc sql and proc freq works with my case:

 

proc sql;
select id1, id2, count(*) as count
from have
group by id1, id2
having calculated count > 1;
quit;

 

PROC FREQ data=have;
TABLES id1*id2 / noprint out=duplist;
RUN;
PROC PRINT data=duplist;
WHERE count ge 2;
RUN;

sas-innovate-2024.png

Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 4 replies
  • 515 views
  • 0 likes
  • 3 in conversation