## Possible to merge by dates within a range?

Frequent Contributor
Posts: 78

# Possible to merge by dates within a range?

I have two datasets that need merged together and even though they have an ID number, I need some other variable that will match them since they can have multiple records. My only option is to use a date variable. The catch is that the date variable will not be the same exact date, but should fall within a range of +/- 30 days.

I have the following merge variables in each dataset:

id examdate;

id examdate examdate_p30 examdate_m30;

examdate matches in 70% of the population, but I need to match the other 30% allowing examdate in dataset 1 to fall between examdate_p30 and examdate_m30 from dataset 2.

Any thoughts?

Posts: 2,125

## Re: Possible to merge by dates within a range?

This problem is pretty straightforward in PROC SQL (and a real mess with a data step).

Look at the BETWEEN operator for comparison in the WHERE clause. You may need to do two steps, one for the exact matches and one for the inexact ones.
Frequent Contributor
Posts: 78

## Re: Possible to merge by dates within a range?

Ok, that sounds like a great idea. I was looking into something called fuzzy merges and couldn't get it working exactly right, so I will give this a try.

Thanks!
Posts: 3,167

## Re: Possible to merge by dates within a range?

Hi Doc,

quote:"and a real mess with a data step"

Well, not if hash() can be qualified as data step implementation.

Kindly Regards,

Haikuo

Contributor
Posts: 52

## Re: Possible to merge by dates within a range?

I'd do something like this. If there is no chance of multiple matches, stick with new_dataset_ver1 and order it however you want.

If there is a chance that there might be more than one match per ID, look at the code used to obtain new_dataset_ver2. I calculate a variable that rates the closeness of the match. Here I take the distance in days as an absolute value, then select the match with the closest value. I use the monotonic() function which can be tricky if you do more than use it to add row numbers. Here it is an example of how to rank potential matches.

This is untested code, but I do use the technique with time stamped dates. Be careful to set up decision rules (in the order by clause) carefully. The code sample below could force an arbitrary selection between two matches with the same time distance.

proc sql;

create table new_dataset_ver1 as

select

a.ID,

a.examdate as examdate_a,

b.examdate as examdate_b,

b.examdate_p30,

b.examdate_m30,

abs(a.examdate - b.examdate) as difference

from

dataset1 as a,

dataset2 as b

where

a.ID = b.ID and

b.examdate_p30 < a.examdate < b.examdate_m30

order by

ID,

difference;

create table monotonic as

select

*,

monotonic() as line_number

from

new_dataset_ver1;

create table min_linenumber as

select

ID,

min(line_number) as closest_date

from

 monotonic group by ID;

create table new_dataset_ver2 as

select

a.*

from

monotonic as a,

min_linenumber as b

where

a.ID = b.ID and

a.line_number = b.closest_date;

quit;

Posts: 5,541

## Re: Possible to merge by dates within a range?

You can try something like this (untested), it tries to deal properly with missing matches in dataset b :

proc sql;
create table temp as
select a.*, b.examdate as otherDate, b.var1, b.var2,
abs(intck("DAY", a.examdate, b.examdate)) as sepDays
from a left join b on a.id = b.id
where calculated sepDays <= 30
order by a.id, a.examdate, sepDays;
quit;

data want(drop=sepDays);
set temp;
by id examdate;
if first.examdate;
run;

PG

PG
Discussion stats
• 5 replies
• 6110 views
• 0 likes
• 5 in conversation