BookmarkSubscribeRSS Feed
☑ This topic is solved. Need further help from the community? Please sign in and ask a new question.
sms1891
Quartz | Level 8

Hi all,

I need help in calculating intra-rater reliability for continuous variable measured by one person at two different times  for a dental instrument/ tool. Basically these force measurements at different times. The same dentist measured the forces twice (Fx_T1 and Fx_T2) and I would like to know if there is any reliability in the measurements.   The only issue with this measurement is we do not have individuals that we the measurements twice (at Time 1 and Time 2).  It is just the set of readings say like 500 force values at Time 1 and maybe like 600 force values at Time 2. I want to check if there is any consistency in the tool for measuring the forces. How best should I approach this situation. I have previously used the following SAS code but it was more so for a set of individuals with multiple measurements (by time).

 

Any help with sas code for this would be greatly appreciated.

 

Thanks,

SM

 

1 ACCEPTED SOLUTION
9 REPLIES 9
sbxkoenk
SAS Super FREQ

Moved post to "Statistical Procedures" board.

Koen

sbxkoenk
SAS Super FREQ

Do not use Cohen’s kappa for this
, but use the Intraclass kappa (Kraemer, Periyakoil, & Noda, 2002)
for measuring intrarater reliability ( = test-retest reliability).

 

Paper ###-2019
Measuring Test-Retest Reliability: The Intraclass Kappa
Dennis G. Fisher, Grace L. Reynolds, California State University, Long Beach
Eric Neri, Art Noda, Helena Chmura Kraemer, Stanford University
https://www.lexjansen.com/wuss/2019/65_Final_Paper_PDF.pdf

 

BR,
Koen

mkeintz
PROC Star

Without seeing the actual data structure, it is difficult to precisely understand how to advise you.  For instance, you've already stated that there are 500 time1 observations and 600 time2 observations, so there are at least 100 datapoints that won't contribute to assessing intra-rater reliabilty (unless the time2 observations are somehow repeated obs).  Please provide sample data, in the form of a working data step, which will help us to help you.  Otherwise, you will be stuck with nothing more than general advice whcih may not help advance your data analysis.

 

 

--------------------------
The hash OUTPUT method will overwrite a SAS data set, but not append. That can be costly. Consider voting for Add a HASH object method which would append a hash object to an existing SAS data set

Would enabling PROC SORT to simultaneously output multiple datasets be useful? Then vote for
Allow PROC SORT to output multiple datasets

--------------------------
sms1891
Quartz | Level 8

Hi mkeintz,

Sorry I was wrong, the data set has 500 T1 values and 500 T2 values. How do I do the intraclass kappa in this situation? 

 

Thanks

SM 

sms1891
Quartz | Level 8

Thank you for the responses. I am attaching excel data set with Force variable (Fx) and Time variable with two different times T1 and T2. Same instrument was used to obtain the force values (n=3000) at time T1 and repeated by same person at time T2 to obtain a new set of force values (n=3000).

PROC IMPORT OUT= WORK.SAS 
            DATAFILE= "C:\Users\SAS Example.xlsx" 
            DBMS=EXCEL REPLACE;
     RANGE="Sheet1$"; 
     GETNAMES=YES;
     MIXED=NO;
     SCANTEXT=YES;
     USEDATE=YES;
     SCANTIME=YES;
RUN;

 

ballardw
Super User

Many users here don't want to download Excel files because of virus potential, others have such things blocked by security software. Also if you give us Excel we have to create a SAS data set and due to the non-existent constraints on Excel data cells the result we end up with may not have variables of the same type (numeric or character) and even values.

 

Instructions here: https://communities.sas.com/t5/SAS-Communities-Library/How-to-create-a-data-step-version-of-your-dat... will show how to turn an existing SAS data set into data step code that can be pasted into a forum code box using the </> icon or attached as text to show exactly what you have and that we can test code against.

SteveDenham
Jade | Level 19

@Ksharp 's post hits the mark: intraclass correlation is a good measure of intra-rater reliability. It also points out that the kappa statistic is for categorical variables.  An example can be found at 

https://documentation.sas.com/doc/en/statug/15.2/statug_mixed_examples02.htm 

 

About two-thirds of the way through the example, it covers how to calculate an intraclass correlation.  And near the end, it fits a heterogeneous variances model, which can be used to calculate intraclass correlations for each level used in the GROUP= statement.

 

SteveDenham

 

 

sms1891
Quartz | Level 8

Thank you all for helping me out with this!

sas-innovate-2024.png

Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 9 replies
  • 1019 views
  • 3 likes
  • 6 in conversation