Programming the statistical procedures from SAS

Comparing multi-rater kappas

Reply
New Contributor
Posts: 2

Comparing multi-rater kappas

Hi all,

 

I have two data sets where multiple raters have rated multiple x-rays into 3 categories.  All raters have rated all x-rays and there is no missing data.

 

I've calculated overall kappas using magree for each of the two data sets.  Is anyone aware of a way to statistically compare these two kappas?  The macro produces a standard error for the overall kappa, which is very tiny (presumably because there are >200 raters in each group).  Would it be proper to use this SE to create confidence intervals?  For some reason the macro doesn't create confidence intervals, which makes me concerned I'm incorrect in doing this. Are there other ways to create these confidence intervals?  I've seen one example of bootstrapping to create CIs in this situation as well - not sure which is most appropriate.

 

Thanks!

SAS Super FREQ
Posts: 3,319

Re: Comparing multi-rater kappas

You mention a macro for computing the kappas. Is it the %MAGREE macro that is provided by SAS?  Can you provide a link or reference for the bootstrapping example that you are considering?

 

New Contributor
Posts: 2

Re: Comparing multi-rater kappas

Yes - the %magree macro from SAS is what I used.  

 

This is what I've read about using bootstrapping to generate CIs for kappas:

http://support.sas.com/resources/papers/proceedings12/323-2012.pdf

Ask a Question
Discussion stats
  • 2 replies
  • 121 views
  • 0 likes
  • 2 in conversation