Intra-rater reliability for continuous variable

[sms1891]

Hi all,

I need help in calculating intra-rater reliability for continuous variable measured by one person at two different times  for a dental instrument/ tool. Basically these force measurements at different times. The same dentist measured the forces twice (Fx_T1 and Fx_T2) and I would like to know if there is any reliability in the measurements.   The only issue with this measurement is we do not have individuals that we the measurements twice (at Time 1 and Time 2).  It is just the set of readings say like 500 force values at Time 1 and maybe like 600 force values at Time 2. I want to check if there is any consistency in the tool for measuring the forces. How best should I approach this situation. I have previously used the following SAS code but it was more so for a set of individuals with multiple measurements (by time).

Any help with sas code for this would be greatly appreciated.

Thanks,

SM

[sbxkoenk]

Moved post to "Statistical Procedures" board.

Koen

[sbxkoenk]

Do not use Cohen’s kappa for this
, but use the Intraclass kappa (Kraemer, Periyakoil, & Noda, 2002)
for measuring intrarater reliability ( = test-retest reliability).

Paper ###-2019
Measuring Test-Retest Reliability: The Intraclass Kappa
Dennis G. Fisher, Grace L. Reynolds, California State University, Long Beach
Eric Neri, Art Noda, Helena Chmura Kraemer, Stanford University
https://www.lexjansen.com/wuss/2019/65_Final_Paper_PDF.pdf

BR,
Koen

[mkeintz]

Without seeing the actual data structure, it is difficult to precisely understand how to advise you.  For instance, you've already stated that there are 500 time1 observations and 600 time2 observations, so there are at least 100 datapoints that won't contribute to assessing intra-rater reliabilty (unless the time2 observations are somehow repeated obs).  Please provide sample data, in the form of a working data step, which will help us to help you.  Otherwise, you will be stuck with nothing more than general advice whcih may not help advance your data analysis.

[sms1891]

Hi mkeintz,

Sorry I was wrong, the data set has 500 T1 values and 500 T2 values. How do I do the intraclass kappa in this situation? 

Thanks

SM 

[sms1891]

Thank you for the responses. I am attaching excel data set with Force variable (Fx) and Time variable with two different times T1 and T2. Same instrument was used to obtain the force values (n=3000) at time T1 and repeated by same person at time T2 to obtain a new set of force values (n=3000).

PROC IMPORT OUT= WORK.SAS 
            DATAFILE= "C:\Users\SAS Example.xlsx" 
            DBMS=EXCEL REPLACE;
     RANGE="Sheet1$"; 
     GETNAMES=YES;
     MIXED=NO;
     SCANTEXT=YES;
     USEDATE=YES;
     SCANTIME=YES;
RUN;

[ballardw]

Many users here don't want to download Excel files because of virus potential, others have such things blocked by security software. Also if you give us Excel we have to create a SAS data set and due to the non-existent constraints on Excel data cells the result we end up with may not have variables of the same type (numeric or character) and even values.

Instructions here: https://communities.sas.com/t5/SAS-Communities-Library/How-to-create-a-data-step-version-of-your-dat... will show how to turn an existing SAS data set into data step code that can be pasted into a forum code box using the </> icon or attached as text to show exactly what you have and that we can test code against.

[Ksharp · Accepted solution]

https://support.sas.com/kb/30/333.html#errinvar

[SteveDenham]

@Ksharp 's post hits the mark: intraclass correlation is a good measure of intra-rater reliability. It also points out that the kappa statistic is for categorical variables.  An example can be found at 

https://documentation.sas.com/doc/en/statug/15.2/statug_mixed_examples02.htm 

About two-thirds of the way through the example, it covers how to calculate an intraclass correlation.  And near the end, it fits a heterogeneous variances model, which can be used to calculate intraclass correlations for each level used in the GROUP= statement.

SteveDenham

[sms1891]

Thank you all for helping me out with this!