BookmarkSubscribeRSS Feed
juliajulia
Obsidian | Level 7

i have a dataset that has two ids: id1 and id2. i want to find out if there are any duplicated records for the combined id1 and id2. i created a code like this but it don't work. are there any easy way to do this in sas proc sql?

 

data combine;
set dataset;
combinedid=catx(id1, id2);
run;

PROC SQL;
SELECT id1, id2,
FREQ(combinedid) AS dupe
FROM dataset
GROUP BY combinedid
HAVING dupe GE 2;
quit;

4 REPLIES 4
Kurt_Bremser
Super User

Run a count and use HAVING:

proc sql;
select id1, id2, count(*) as count
from have
group by id1, id2
having calculated count > 1;
quit;

Or run a proc freq or proc summary and filter out the observations with a count > 1.

Reeza
Super User
Why use SQL at all? Why not use PROC SORT which has built in functions to help identify duplicates. Look at the NODUPKEY, NOUNIQUEKEY, DUPOUT, UNIQUEOUT options.
It can identify duplicates across multiple columns and easily separate them into their own data sets with no need to combine anything.

You can definitely roll your own via SQL but it's faster, easier and more efficient to use the developed procedures.

Reeza
Super User
And you may want to sort those IDs first as well.

ie should these be duplicates:

ID1 ID2
ABC DEF
DEF ABC

None of the posted solutions will deal with this scenario.
juliajulia
Obsidian | Level 7

Thank you all. Both proc sql and proc freq works with my case:

 

proc sql;
select id1, id2, count(*) as count
from have
group by id1, id2
having calculated count > 1;
quit;

 

PROC FREQ data=have;
TABLES id1*id2 / noprint out=duplist;
RUN;
PROC PRINT data=duplist;
WHERE count ge 2;
RUN;

hackathon24-white-horiz.png

The 2025 SAS Hackathon has begun!

It's finally time to hack! Remember to visit the SAS Hacker's Hub regularly for news and updates.

Latest Updates

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

SAS Training: Just a Click Away

 Ready to level-up your skills? Choose your own adventure.

Browse our catalog!

Discussion stats
  • 4 replies
  • 1345 views
  • 0 likes
  • 3 in conversation