I am merging two data sets with the below statement and when I do a few extra records are being created in the new data set. Is there any way I can either prevent this or identify the additional records in the new data set?
In the s0ssrp data set, there are 514,983. In the new s0ssrp1, there are 515,013.
data s0ssrp1 (rename=closed_reason=FtcOutcome);
merge s0ssrp (in=a) ssrpucms2 (keep=County Docket ReferralDate FtcStatus closed_reason ContractDate);
by County Docket;
if ReferralDate ge start and ReferralDate le stop then do;
I thought there were a lot of duplicates actually. But if this is the only scenario that would produce more than one record from the original data set, then I will take a look at the file tomorrow morning and check it further. If that is the case then that is an easier scenario than I thought.
I will get back. Thanks Astounding.
You can find the duplicate combinations of County and Docket in dataset SSRPUCMS2 by submitting the following data step:
data dups; set ssrpucms2; by County Docket; if ~(first.docket & last.docket); run;
If this resulted in more than 60 (=2*(515,013-514,983)) observations, it would seem that also dataset S0SSRP contains such duplicates. In this case the merge step would be questionable and should have created a message
NOTE: MERGE statement has more than one data set with repeats of BY values.
in the log.
So, you could replace ssrpucms2 by s0ssrp in the above data step and should be alarmed if the resulting dataset DUPS is not empty.
If County-Docket is a unique key in s0ssrp, you can choose to use the first or the last matching record from ssrpucms2 in the following way:
data s0ssrp1 (rename=closed_reason=FtcOutcome); merge s0ssrp (in=a) ssrpucms2 (in=b keep=County Docket ReferralDate FtcStatus closed_reason ContractDate); by County Docket; /* Use the first record from ssrpucms2 in a County-Docket group */ if a and (not b or first.Docket); /* Or... */ /* Use the last record from ssrpucms2 in a County-Docket group */ if a and (not b or last.Docket); if ReferralDate ge start and ReferralDate le stop then FtcReferred="Y"; else FtcReferred="N"; run;
Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.
If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website.
Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.
Find more tutorials on the SAS Users YouTube channel.