BookmarkSubscribeRSS Feed
Obsidian | Level 7

Hi Everyone

I am merging two data sets with the below statement and when I do a few extra records are being created in the new data set. Is there any way I can either prevent this or identify the additional records in the new data set?


In the s0ssrp data set, there are 514,983. In the new s0ssrp1, there are 515,013.




data s0ssrp1 (rename=closed_reason=FtcOutcome);
merge s0ssrp (in=a) ssrpucms2 (keep=County Docket ReferralDate FtcStatus closed_reason ContractDate);
if a;
by County Docket;

if ReferralDate ge start and ReferralDate le stop then do;

else FtcReferred="N";

Opal | Level 21

This would happen if your second data set, ssrpucms2, occasionally contained more than one observation for a combination of COUNTY + DOCKET. 


Do you need help in locating where the duplicates are?

Obsidian | Level 7

I thought there were a lot of duplicates actually. But if this is the only scenario that would produce more than one record from the original data set, then I will take a look at the file tomorrow morning and check it further. If that is the case then that is an easier scenario than I thought.


I will get back. Thanks Astounding.



Jade | Level 19

Hi Paul,


You can find the duplicate combinations of County and Docket in dataset SSRPUCMS2 by submitting the following data step:

data dups;
set ssrpucms2;
by County Docket;
if ~(first.docket & last.docket);

If this resulted in more than 60 (=2*(515,013-514,983)) observations, it would seem that also dataset S0SSRP contains such duplicates. In this case the merge step would be questionable and should have created a message

NOTE: MERGE statement has more than one data set with repeats of BY values.

in the log. 


So, you could replace ssrpucms2 by s0ssrp in the above data step and should be alarmed if the resulting dataset DUPS is not empty.

Opal | Level 21

If County-Docket is a unique key in s0ssrp, you can choose to use the first or the last matching record from ssrpucms2 in the following way:


data s0ssrp1 (rename=closed_reason=FtcOutcome);
merge s0ssrp (in=a) ssrpucms2 (in=b keep=County Docket ReferralDate FtcStatus closed_reason ContractDate);
by County Docket;

/* Use the first record from ssrpucms2 in a County-Docket group */
if a and (not b or first.Docket);

/* Or... */

/* Use the last record from ssrpucms2 in a County-Docket group */
if a and (not b or last.Docket);

if ReferralDate ge start and ReferralDate le stop then FtcReferred="Y";
else FtcReferred="N";




Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.

If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website. 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Get the $99 certification deal.jpg



Back in the Classroom!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 4 replies
  • 1 like
  • 4 in conversation