topic Removing duplicates by satisfying a frequency condition in SAS Programming

Removing duplicates by satisfying a frequency condition

BenBrady — Mon, 16 Apr 2018 06:38:32 GMT

I have a data set which is much larger than my example below. I want to be able to remove the 'Person' duplicates by the 'type' variable. In particular, if they have the same 'Type', then can keep either. However, if they have different values for 'Type', i want to keep the record with the higher frequency. In this case it would be one with Type=2 (as there is 2 of them compared to only one record where Type=1). How would I go about doing this?

Person	Type
A	2
A	2
A	1
B	3
B	3

Re: Removing duplicates by satisfying a frequency condition

PeterClemmensen — Mon, 16 Apr 2018 06:55:31 GMT

like this?

data have;
input Person$ Type;
datalines;
A 2
A 2
A 1
B 3
B 3
;

proc sort data=have;
    by descending Person descending Type;
run;

data want;
    set have;
    by descending Person descending Type;
    if first.Person;
run;

Re: Removing duplicates by satisfying a frequency condition

s_lassen — Mon, 16 Apr 2018 08:02:58 GMT

Here is one way to do it, using SQL:

proc sql;
  create table want as select * from
   (select * from
     (select *,count(*) as n from have
      group  by person,type
     )
   group by person
   having n=max(n)
   )
  group by person
  having type=min(type);

quit;

Re: Removing duplicates by satisfying a frequency condition

shantanupl1 — Mon, 16 Apr 2018 10:08:59 GMT

Try this.

proc sql;
create table want1 as
select *, count(*) as n from have group by person, type order by person, calculated n desc;
run;

data want;
set want1;
by person descending n;
if first.person;
drop n;
run;