BookmarkSubscribeRSS Feed
Kimberly09
Fluorite | Level 6

Hi Community,

 

I would like to test if 2 different researchers are rating different subjects in the same manner with respect to a dichotomous outcome. I know if both researchers had rated the same subject separately, I could determine the degree of agreement by the Kappa statistic. However, in this case, the 2 researchers conducted the same intervention on different subjects using the same assessment. Is there a statistical test I could use run in SAS to compare concordance between these 2 researchers? Thanks so much for your help!  

 

Example Data set (Actual sample size = 790)

Researcher 1                                                                 Researcher 2

subject id      completeness_rating                              subject id          completeness_rating

1001              Complete  (coded as 3)                          2001                  Complete (coded as 3)

1002              Partial  (coded as 2)                               2002                  Incomplete (coded as 1)

1003              Incomplete (coded as 1)                        2003                  Incomplete (coded as 1)

 

 

2 REPLIES 2
ballardw
Super User

Do you have pretty strong evidence that the two groups should have received a similar proportion of scores?

Chi-sq might be appropriate. But if you don't have reason to believe that the subjects distribution of scores should be similar the differences could be the subjects and not the researchers.

 

I would be tempted to run a quick Proc Freq requesting the chi-sq just to see.

Kimberly09
Fluorite | Level 6

Thank you for your reply. No, I don't have statistical evidence as to whether the 2 groups should have similar scores. However, the groups have similar demographic and clinical characteristics and the researchers seem to think the scores should be similar. I haven't done any statistical test to determine if this is the case (so far, I've only run the usual proc freq/univariate procedures). Here comes a basic question regarding the chi-square test.....Would I use the code below to determine if there's an association between the completeness variable and site (researchers), and if so, then what would I need to do?

 

 

proc freq data=check; 
tables complete_rate*site/chisq; /*the 2 researchers are at different sites/medical facilities*/
run; 

 

 

SAS Innovate 2025: Call for Content

Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 25. Read more here about why you should contribute and what is in it for you!

Submit your idea!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 2 replies
  • 541 views
  • 0 likes
  • 2 in conversation