DATA Step, Macro, Functions and more

check duplicates

Reply
Contributor
Posts: 67

check duplicates

Hi,

I have a data set looking like listed below. Every student has unique student id. For each student, there may different test id. For the same student and same test id, the variable line_id will list as 1,2,...,n. I am interested in the variable X. I want to check if contents in variable X are same for the same student and same test_id. If so for all of the data, which I think it should be, next step will be easily delete the observations for line_id not equal to 1 to drop the duplicates. Could anyone tell me how to program? Thanks a lot!

student_id     test_id     line_id     X

1                    2345          1         

1                    2345          2

1                    3468          1

1                    3468          2

1                    3468          3

2                    7896          1

2                    1234          1

2                    1234          2

2                    3452          1

2                    3452          2

Super User
Posts: 19,772

Re: check duplicates

Posted in reply to michellel

Look into proc sort and the dupkey/duprecs and dupout options.

Frequent Contributor
Posts: 106

Re: check duplicates

Posted in reply to michellel

Duplicates can be removed as suggested by Reeza.

data students;

input student_id     test_id     line_id;

cards;

1                    2345          1

1                    2345          2

1                    3468          1

1                    3468          2

1                    3468          3

2                    7896          1

2                    1234          1

2                    1234          2

2                    3452          1

2                    3452          2

;

proc sort data=students dupout=student_dups nodupkey;

by student_id test_id;

run;

but what you want in variable X is not clear, please provide example output dataset.

Super User
Posts: 5,498

Re: check duplicates

Posted in reply to michellel

Here's one way;

data want;

   output_all=0;

   do until (last.test_id);

      set have;

      by student_id test_id;

      if first.test_id then first_x=x;

      if x ne first_x then output_all=1;

   end;

   do until (last.test_id);

      set have;

      by student_id test_id;

      if output_all=1 then output;

      else if line_id=1 then output;

   end;

run;

Both DO loops read the same observations (all that exist for a student_id test_id combination).  The top loop determines whether X changes within the group.  The bottom loop outputs accordingly.

Good luck.

Ask a Question
Discussion stats
  • 3 replies
  • 278 views
  • 0 likes
  • 4 in conversation