Kappa coefficient

[darb]

So currently I am assessing agreement between two evaluators. They are assessing disease response and the table would like below:

                   Evaluater 2  CR     CRc      CRi        PR        SD       PD

Evaluater 1

CR

CRc

CRi

PR

SD

PD

To calculate a weighted Kappa is assigned an Ordinal value to each response: 1, 2, 3, 4, 5, 6. Which sas that uses the following weights as default: 1.00, 0.8, 0.6, 0.4, 0.2.

However, I would really like for CRc and CRi to have the same weights for distance from CR. Does anyone know how I should reorder the ordinal values if even possible, my issue is that they can't have the same ordinal value --- is the only way to combine CRc & CRi? I don't know if that would be clinically meaningful.

[ballardw]

What are you basing this statement on: "Which sas that uses the following weights as default: 1.00, 0.8, 0.6, 0.4, 0.2."

What does your current data actually look like is it something like:

Evaluator Value RecordID

There are a number of procedures that will do Kappa coefficients or analysis but we kind of need to know what the raw data looks like to make recommendations.

[darb]

I can print the weights Sas is using:

ods html;
proc freq data = test5 ;
table xrorres1*bestresp1/agree printkwts ;
test kappa;
weight count;
run;
ods html close;

Data looks like this:

xrorres1    bestresp1         count

CR                CR                   5

CR                CRi                  3

CR                CRc                 2

CR                PR                   0 

CR                SD                   0

CRi               PD                   0

etc                 etc                   etc