topic report number of abservation before and after removing the duplicates in New SAS User

report number of abservation before and after removing the duplicates

Smitha9 — Wed, 04 Nov 2020 18:59:30 GMT

Hi,

I have a dataset which has duplicates and I will be removing the duplicates and want to report the record of the observations before and after removing the duplicates.

Is there a code which I can do all together(before and after duplicates)

thank you in advance.

Re: report number of abservation before and after removing the duplicates

PeterClemmensen — Wed, 04 Nov 2020 19:15:54 GMT

What do you want to do with those numbers? Proc Sort Nodupkey puts them in the log ?

Re: report number of abservation before and after removing the duplicates

mady3 — Tue, 10 Nov 2020 21:26:05 GMT

Hi there,

So you can use the NODUPKEY option to remove the duplicate observations by using a BY statement with the keyword _ALL_. You can then also use the DUPOUT= option to then capture those removed observations so you can report the before and after. The following method also does not overwrite the original dataset.

PROC SORT

DATA = *original dataset*

NODUPKEY

OUT = *new dataset with removed duplications*

DUPOUT = *new dataset with removed observations*

BY _ALL_;

RUN;

I hope this helps!

Mady

Re: report number of abservation before and after removing the duplicates

mkeintz — Wed, 11 Nov 2020 04:35:50 GMT

proc sort data=old out=new nodupkey;
  by key1 key2 key3;
run;

data _null_;
  if 0 then set old  nobs=n_old;
  if 0 then set new nobs=n_new;
  put  (n_:) (=);
run;

The "if 0" conditions mean the corresponding then clauses are never executed, but the SAS complier nevertheless populates the n_old and n_new metadata values prior to execution stage.