DATA Step, Macro, Functions and more

Delete duplicated rows defined by two variables

Reply
Contributor
Posts: 39

Delete duplicated rows defined by two variables

Hi, in this example, row 1 and 4 are duplicates because if you swiched col1 and col2 for one of the rows, that row will be a duplicated row of the other based on col1 and col2. I only want to keep one of them. Thank you! Z

 

data have;

input col1 $ col2 $ col3;

datalines;

A B 1

C F 3

D H 5

B A 1

G H 6

;

run;

 

data want;

input col1 $ col2 col3;

datalines;

A B 1

C F 3

D H 5

G H 6

;

run;

Contributor
Posts: 39

Re: Delete duplicated rows defined by two variables

Ok, I figured it out, thank you anyway:

 

data temp;

set have;

new1=col1;

new2=col2;

call sortc(new1,new2);

run;

proc print data=temp;

run;

proc sort data=temp out=want2 nodupkey;

by new1 new2;

run;

proc print data=want2;

run;

Valued Guide
Posts: 797

Re: Delete duplicated rows defined by two variables

If the dataset is big, then this may offer some efficiencies.  It passes through the data only once:

 

data need/ view=need;

  set have;

  new1=col1;new2=col2;

  call sortc(new1,new2);

run;

proc sort data=need out=want  nodupkey;

  by new1 new2;

run;

 

 

It takes advantage of the fact that data set need is a data set VIEW, not a data set FILE.  This means it is actualized only when called later (by the proc sort) its data is passed directly to proc sort without being written to disk.

 

Result, one pass of the data, producing sorted results with all duplicate combinations (not just duplicate permutations) of col1/col2 removed.

Ask a Question
Discussion stats
  • 2 replies
  • 100 views
  • 0 likes
  • 2 in conversation