Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- Home
- /
- Analytics
- /
- Stat Procs
- /
- Intra-rater reliability for continuous variable

Options

- RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page

☑ This topic is **solved**.
Need further help from the community? Please
sign in and ask a **new** question.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Posted 09-05-2023 06:04 PM
(1018 views)

Hi all,

I need help in calculating intra-rater reliability for continuous variable measured by one person at two different times for a dental instrument/ tool. Basically these force measurements at different times. The same dentist measured the forces twice (Fx_T1 and Fx_T2) and I would like to know if there is any reliability in the measurements. The only issue with this measurement is we do not have individuals that we the measurements twice (at Time 1 and Time 2). It is just the set of readings say like 500 force values at Time 1 and maybe like 600 force values at Time 2. I want to check if there is any consistency in the tool for measuring the forces. How best should I approach this situation. I have previously used the following SAS code but it was more so for a set of individuals with multiple measurements (by time).

Any help with sas code for this would be greatly appreciated.

Thanks,

SM

1 ACCEPTED SOLUTION

Accepted Solutions

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

9 REPLIES 9

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Moved post to "Statistical Procedures" board.

Koen

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Do not use Cohen’s kappa for this

, but use the Intraclass kappa (Kraemer, Periyakoil, & Noda, 2002)

for measuring intrarater reliability ( = test-retest reliability).

Paper ###-2019

Measuring Test-Retest Reliability: The Intraclass Kappa

Dennis G. Fisher, Grace L. Reynolds, California State University, Long Beach

Eric Neri, Art Noda, Helena Chmura Kraemer, Stanford University

https://www.lexjansen.com/wuss/2019/65_Final_Paper_PDF.pdf

BR,

Koen

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Without seeing the actual data structure, it is difficult to precisely understand how to advise you. For instance, you've already stated that there are 500 time1 observations and 600 time2 observations, so there are at least 100 datapoints that won't contribute to assessing intra-rater reliabilty (unless the time2 observations are somehow repeated obs). Please provide sample data, * in the form of a working data step,* which will help us to help you. Otherwise, you will be stuck with nothing more than general advice whcih may not help advance your data analysis.

--------------------------

The hash OUTPUT method will overwrite a SAS data set, but not append. That can be costly. Consider voting for Add a HASH object method which would append a hash object to an existing SAS data set

Would enabling PROC SORT to simultaneously output multiple datasets be useful? Then vote for

Allow PROC SORT to output multiple datasets

--------------------------

The hash OUTPUT method will overwrite a SAS data set, but not append. That can be costly. Consider voting for Add a HASH object method which would append a hash object to an existing SAS data set

Would enabling PROC SORT to simultaneously output multiple datasets be useful? Then vote for

Allow PROC SORT to output multiple datasets

--------------------------

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Hi mkeintz,

Sorry I was wrong, the data set has 500 T1 values and 500 T2 values. How do I do the intraclass kappa in this situation?

Thanks

SM

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Thank you for the responses. I am attaching excel data set with Force variable (Fx) and Time variable with two different times T1 and T2. Same instrument was used to obtain the force values (n=3000) at time T1 and repeated by same person at time T2 to obtain a new set of force values (n=3000).

```
PROC IMPORT OUT= WORK.SAS
DATAFILE= "C:\Users\SAS Example.xlsx"
DBMS=EXCEL REPLACE;
RANGE="Sheet1$";
GETNAMES=YES;
MIXED=NO;
SCANTEXT=YES;
USEDATE=YES;
SCANTIME=YES;
RUN;
```

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Many users here don't want to download Excel files because of virus potential, others have such things blocked by security software. Also if you give us Excel we have to create a SAS data set and due to the non-existent constraints on Excel data cells the result we end up with may not have variables of the same type (numeric or character) and even values.

Instructions here: https://communities.sas.com/t5/SAS-Communities-Library/How-to-create-a-data-step-version-of-your-dat... will show how to turn an existing SAS data set into data step code that can be pasted into a forum code box using the </> icon or attached as text to show exactly what you have and that we can test code against.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

@Ksharp 's post hits the mark: intraclass correlation is a good measure of intra-rater reliability. It also points out that the kappa statistic is for categorical variables. An example can be found at

https://documentation.sas.com/doc/en/statug/15.2/statug_mixed_examples02.htm

About two-thirds of the way through the example, it covers how to calculate an intraclass correlation. And near the end, it fits a heterogeneous variances model, which can be used to calculate intraclass correlations for each level used in the GROUP= statement.

SteveDenham

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Thank you all for helping me out with this!

**Available on demand!**

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.