topic Re: Delete duplicated rows defined by two variables in SAS Programming

Delete duplicated rows defined by two variables

imcbczm — Wed, 01 Mar 2017 19:11:35 GMT

Hi, in this example, row 1 and 4 are duplicates because if you swiched col1 and col2 for one of the rows, that row will be a duplicated row of the other based on col1 and col2. I only want to keep one of them. Thank you! Z

data have;

input col1 $ col2 $ col3;

datalines;

A B 1

C F 3

D H 5

B A 1

G H 6

;

run;

data want;

input col1 $ col2 col3;

datalines;

A B 1

C F 3

D H 5

G H 6

;

run;

Re: Delete duplicated rows defined by two variables

imcbczm — Wed, 01 Mar 2017 19:25:34 GMT

Ok, I figured it out, thank you anyway:

data temp;

set have;

new1=col1;

new2=col2;

call sortc(new1,new2);

run;

proc print data=temp;

run;

proc sort data=temp out=want2 nodupkey;

by new1 new2;

run;

proc print data=want2;

run;

Re: Delete duplicated rows defined by two variables

mkeintz — Wed, 01 Mar 2017 19:40:26 GMT

If the dataset is big, then this may offer some efficiencies. It passes through the data only once:

data need/ view=need;

set have;

new1=col1;new2=col2;

call sortc(new1,new2);

run;

proc sort data=need out=want nodupkey;

by new1 new2;

run;

It takes advantage of the fact that data set need is a data set VIEW, not a data set FILE. This means it is actualized only when called later (by the proc sort) its data is passed directly to proc sort without being written to disk.

Result, one pass of the data, producing sorted results with all duplicate combinations (not just duplicate permutations) of col1/col2 removed.