topic Re: A Macro to Calculate Kappa Statistics for Categorizations by Multiple Raters in SAS Programming

A Macro to Calculate Kappa Statistics for Categorizations by Multiple Raters

DanielDor — Sat, 29 Apr 2017 08:45:30 GMT

Greetings,

I would like to use A Macro to Calculate Kappa Statistics for Categorizations by Multiple Raters for a Delphi study. I need to assess the agreement on multi-category ratings by multiple raters.

I'm trying to use the MKAPPA Macro (http://www2.sas.com/proceedings/sugi30/155-30.pdf) but without success. I'm using SAS9.4, and i'm getting no output from the Macro.

Could anyone suggest me on what to do? Is there other Kappa Statistics Macros that I can use?

Thanks!

Re: A Macro to Calculate Kappa Statistics for Categorizations by Multiple Raters

Reeza — Sat, 29 Apr 2017 08:56:12 GMT

When you copy and paste from a PDF you likely introduced some invisible characters in your code. Sometimes typing it out or using a editor to remove these symbols is enough to get it working.

Why isn't it working for you?

Re: A Macro to Calculate Kappa Statistics for Categorizations by Multiple Raters

DanielDor — Sat, 29 Apr 2017 09:02:29 GMT

Hi Reeza,

1. Thanks for your prompt answer. I did copy-pasted from PDF. I will try to write it down and then to run it again.

2. I've also tried to use now this macro (http://support.sas.com/kb/25/006.html). It worked for the example with 5 raters and 10 subjects, but did'nt work for 35 raters and 35 subjects. I've got the following error:

WARNING: Apparent symbolic reference M not resolved.
ERROR: A character operand was found in the %EVAL function or %IF condition where a numeric operand is required. The condition was:
&m=1 or &n=1
WARNING: Apparent symbolic reference N not resolved.
ERROR: The macro MAGREE will stop executing.

3. Do you have an idea what can be the problem?

Thanks!

Re: A Macro to Calculate Kappa Statistics for Categorizations by Multiple Raters

Reeza — Sat, 29 Apr 2017 09:13:58 GMT

I can't run anything now, but I suspect with 35 raters and subjects you don't have enough information to calculate a kappa - degrees of freedom and inability to invert a matrix idea. You need more subjects than taters.

Re: A Macro to Calculate Kappa Statistics for Categorizations by Multiple Raters

vedatoguz — Tue, 12 Aug 2025 16:35:43 GMT

Hi Daniel,

Were you be able to run MKAPPA Macro (http://www2.sas.com/proceedings/sugi30/155-30.pdf)? I am having hard time to run it. It won't execute.

Would you be able to help me?

Thanks,

Vedat

Re: A Macro to Calculate Kappa Statistics for Categorizations by Multiple Raters

Tom — Tue, 12 Aug 2025 19:46:05 GMT

@vedatoguz wrote:

Hi Daniel,

Were you be able to run MKAPPA Macro (http://www2.sas.com/proceedings/sugi30/155-30.pdf)? I am having hard time to run it. It won't execute.

Would you be able to help me?

Thanks,

Vedat

Rather than posting on an eight year old thread it would be better to open a new thread.

Make sure to explain what you have tried and explain what problem you are having. What does you data look like? Did you get any error messages. etc.

Note: You probably do NOT want to try using the %DISPLAY and %WINDOW statements listed at the beginning of the appendix in that paper. Just edit and run the %LET statements to change the input macro variable values.

Re: A Macro to Calculate Kappa Statistics for Categorizations by Multiple Raters

Ksharp — Wed, 13 Aug 2025 01:21:14 GMT

Check this:
https://support.sas.com/kb/25/006.html
https://support.sas.com/kb/69/654.html