topic Determining measure intra-rater reliability/repeatably for ordinal categorical variables in Statistical Procedures

Determining measure intra-rater reliability/repeatably for ordinal categorical variables

massimon — Mon, 22 Jul 2019 11:33:43 GMT

Hello,

I have symptom data on 90 participants, which was captured weekly via survey for a total of 4 weeks. Each week the study participants rated their symptom severity as: none, mild moderate, severe or very severe.

We are interested in assessing how consistent a participants' symptom rating is across the 4 weeks. This will give us insight into weather we can conduct patient symptom assessments in a trial or in real world clinical practice setting less often than weekly (i.e. monthly).

What would be the best way to evaluate and measure intra-rater reliability and repeatably? Is there an analog to the intraclass correlation coefficient that can be used with ordinal categorical variables?

Thank you!

Re: Determining measure intra-rater reliability/repeatably for ordinal categorical variables

ballardw — Mon, 22 Jul 2019 19:55:04 GMT

Is there an assumption that the symptom should not change?

Or should only change in one direction (i.e. improvement)?

Without one of those assumptions I am not going to guess what "consistent" might be.

I know that when I was filling out symptoms related questions every 2 weeks for at least 6 months the expectation was that some of those symptoms could change and in either direction as the direction was used to consider for treatment modification.

Re: Determining measure intra-rater reliability/repeatably for ordinal categorical variables

massimon — Tue, 23 Jul 2019 11:16:57 GMT

The symptoms can remain the same, get worse, or get better.