topic Re: check duplicates in SAS Programming

check duplicates

michellel — Wed, 25 Feb 2015 03:45:33 GMT

Hi,

I have a data set looking like listed below. Every student has unique student id. For each student, there may different test id. For the same student and same test id, the variable line_id will list as 1,2,...,n. I am interested in the variable X. I want to check if contents in variable X are same for the same student and same test_id. If so for all of the data, which I think it should be, next step will be easily delete the observations for line_id not equal to 1 to drop the duplicates. Could anyone tell me how to program? Thanks a lot!

student_id test_id line_id X

1 2345 1

1 2345 2

1 3468 1

1 3468 2

1 3468 3

2 7896 1

2 1234 1

2 1234 2

2 3452 1

2 3452 2

Re: check duplicates

Reeza — Wed, 25 Feb 2015 04:58:56 GMT

Look into proc sort and the dupkey/duprecs and dupout options.

Re: check duplicates

pradeepalankar — Wed, 25 Feb 2015 08:07:28 GMT

Duplicates can be removed as suggested by Reeza.

data students;

input student_id test_id line_id;

cards;

1 2345 1

1 2345 2

1 3468 1

1 3468 2

1 3468 3

2 7896 1

2 1234 1

2 1234 2

2 3452 1

2 3452 2

;

proc sort data=students dupout=student_dups nodupkey;

by student_id test_id;

run;

but what you want in variable X is not clear, please provide example output dataset.

Re: check duplicates

Astounding — Wed, 25 Feb 2015 14:23:59 GMT

Here's one way;

data want;

output_all=0;

do until (last.test_id);

set have;

by student_id test_id;

if first.test_id then first_x=x;

if x ne first_x then output_all=1;

end;

do until (last.test_id);

set have;

by student_id test_id;

if output_all=1 then output;

else if line_id=1 then output;

end;

run;

Both DO loops read the same observations (all that exist for a student_id test_id combination). The top loop determines whether X changes within the group. The bottom loop outputs accordingly.

Good luck.