Why the merged data has more obs than before

[LarissaW]

I was trying to merge two datasets and I used proc compare to check before I merged them, below is the result from proc compare

The code I used to merge datasets is

proc sort data=LC.FAM;
by pid;
run;

proc sort data=hepa;
by PID;
run;

data combined1;
    merge LC.FAM  hepa ;
    by PID;
run;

I got a new dataset with 50273 rows, which contains more rows than hepa (50032 obs). Did anyone know why this happens?

[Quentin]

This can happen if there are duplicates in the data, or if there are mis-matches (e.g. values for pid in lc.fam that are not in hepa).

Running PROC COMPARE is an interesting idea.  You usually use PROC COMPARE to compare variables, but it will also tell you if you have duplicate values, or if you have mis-matches.  What do you get if you run:

proc compare base=hepa compare=lc.fam ;
  id pid ; *use an ID statement here, not a BY statement;
run ;

[Tom]

There is no way to know in advance how many observations the merge will generate.  Unless you know the values of PID in both datasets.

If HEPA has 100 observations that are each a distinct value of PID

And FAM has 10 observations that are each a distinct value of PID

Then merging them can result in between 100 observations (all of the values of PID in FAM already existed in HEPA) to 110 observations (none of the values of PID in FAM existed in HEPA).

And if either dataset has multiple observations for the same PID then even stranger things can happen.

[Kurt_Bremser]

This has nothing to do with graphics, so I moved this to the general Programming community.

[A_Kh]

In addition to abovementioned comments, if you need to merge based on PID.HEPA then use conditional merge, or SQL joins, then you get the 50032 obs. 

eg:

data combined1;
    merge LC.FAM (in=a)  hepa (in=b);
    by PID;
    if b;
run;