topic Re: Remove duplicates with condition in SAS Procedures

Remove duplicates with condition

umashankersaini — Fri, 25 Aug 2017 04:10:28 GMT

Hi Team,

I need to delete the duplicate IDs with certain conditions from following dataset.

Data R;
input Id sal flag;
cards;
1 100 1
1 100 0
2 200 0
2 200 1
3 300 0
4 400 0
4 500 0
;
Run;

1. ID should be unique

2. If flag has '1' / '0' on same ID then need to keep the row with '1'

3. If flag has only '0' on ID then need to keep that row

4. If flag has multiple '0' with respect to same ID then need to keep any one or SAL is different then any

The output could be :

1 100 1

2 200 1

3 300 0

4 400 0

Looking for kind support.

Regards,

Uma Shanker Saini

Re: Remove duplicates with condition

novinosrin — Fri, 25 Aug 2017 04:34:17 GMT

Data R;

input Id sal flag;

cards;

1 100 1

1 100 0

2 200 0

2 200 1

3 300 0

4 400 0

4 500 0

;

Run;

proc sort data=r out=temp;

by id sal descending flag;

run;

data want;

set temp;

by id sal;

if first.id and first.sal;

run;

Re: Remove duplicates with condition

umashankersaini — Fri, 25 Aug 2017 05:29:29 GMT

Hi Novin,

Thanks for your quick help and code works perfectly fine as per requirement.

It would be greatful if we can use it via proc sql because i need to implement it into approx 40 million obs and trying for SAS DI.

Regards,

Uma Shanker Saini

Re: Remove duplicates with condition

novinosrin — Fri, 25 Aug 2017 05:43:11 GMT

Try this:

proc sql;

create table want as

select *

from r

group by id

having sal=max(sal) and flag=max(flag);

quit;