Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- Home
- /
- Analytics
- /
- Stat Procs
- /
- Comparing two means, independent samples, non-normal distribution

Options

- RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page

🔒 This topic is **solved** and **locked**.
Need further help from the community? Please
sign in and ask a **new** question.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Posted 06-05-2018 02:40 PM
(3332 views)

Hi, all

I'm helping a colleague to evaluate some data. It pertains to two methods of capturing and correcting medical notes concerning patients:

1. The physician dictates the notes to an automated voice-to-text program, receives the text version of the notes, and corrects them manually.

2. The physician dictates the notes to a recording device, and the recording is then transcribed to a text file by a human at a keyboard. Again, the text file is then corrected manually by the physician.

Out of these processes come the word count of the resulting text file, and the time in seconds required by the physician to correct the errors in the file. The time is divided by the word count to result in a "seconds per word" statistic. We wish to compare the mean statistics for the two methods.

Although it is the same physician dictating both sets of notes, I'm treating them as independent samples. I'd appreciate feedback on whether this decision is correct.

Using PROC UNIVARIATE indicates that the results from the manual process aren't distributed normally. Based on what I've read, the independent samples and lack of normality indicate that a Wilcoxon rank-sum test (PROC NPAR1WAY) is the correct one to indicate whether the mean correction times are significantly different.

If I'm making any errors in this process, I would appreciate advice about how to approach it.

Thanks,

Tom

1 ACCEPTED SOLUTION

Accepted Solutions

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

The timings should be considered paired only if the physician is having the same notes transcribed by both methods. Otherwise there is no possible pairing.

The non-parametric comparison will protect you from the influence of extreme observations at almost no cost in terms of power. You should stratify the test by physician.

PG

8 REPLIES 8

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Although it is the same physician dictating both sets of notes, I'm treating them as independent samples. I'd appreciate feedback on whether this decision is correct.

These seem paired to me.

Using PROC UNIVARIATE indicates that the results from the manual process aren't distributed normally. Based on what I've read, the independent samples and lack of normality indicate that a Wilcoxon rank-sum test (PROC NPAR1WAY) is the correct one to indicate whether the mean correction times are significantly different.

This is one way to determine if the means are different. However if the data is really paired, then you want to look at the distribution of the differences between method 1 and method 2 to see if it is normally distributed. Even if the difference is not normally distributed, sometimes the Central Limit Theorem comes to the rescue if you have enough observations, and you can treat it as normal.

--

Paige Miller

Paige Miller

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

IIRC the t-test is pretty robust to deviations from normal - as long as they have the same distributions. There was a twitter debate on this a few weeks ago with some really good references (Yes, I'm that much of a data geek).

I would probably also be looking at grouping the analysis by physician and/or transcriber to ensure that there is no big deviations depending on who's doing the coding versus the machine. That's more EDA though.

We found some interesting physician and recorder counts that you can ask me about if we ever meet....one of the more memorable analysis projects of my career.

Fun project though 😄

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

The timings should be considered paired only if the physician is having the same notes transcribed by both methods. Otherwise there is no possible pairing.

The non-parametric comparison will protect you from the influence of extreme observations at almost no cost in terms of power. You should stratify the test by physician.

PG

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

The stimulus is the same. The treatment differs. This is paired.

--

Paige Miller

Paige Miller

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

I don't get it @PaigeMiller. Maybe our understanding of the nature of the data differs. How would the following obs be paired?:

physician caseId method timing John 1 manual 1.0 John 2 manual 1.1 John 3 manual 1.2 John 4 machine 0.9 John 5 machine 1.3 John 6 machine 1.0

PG

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Then I would say that @TomKari has not given us enough information to determine if it is paired or not.

This also fits the description he gave in his original post

physician caseId method timing John 1 manual 1.0 John 2 manual 1.1 John 3 manual 1.2 John 1 machine 0.9 John 2 machine 1.3 John 3 machine 1.0

--

Paige Miller

Paige Miller

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Hi, @PaigeMiller, @Reeza, @PGStats

Thank you all for your insightful comments. They are a great help for someone relatively inexperienced at this.

First of all, the details that I should have included.

1. All of the notes are being dictated by a single physician.

2. They are not duplicates; he dictated notes for some patients using the first method, and some notes using the second method.

3. We don't know who the transcriber is, and at this point have no way to find out.

Based on your comments, I believe that the observations are not paired. Apologies for leaving out the pertinent details.

Although Reeza indicates that a t-test is probably fine, I'm can't be sure if the distributions of the two samples are the same. I think I'll stick with the Wilcoxon rank-sum.

Thank you again for all of your help!

Tom

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

@TomKari wrote:

Hi, @PaigeMiller, @Reeza, @PGStats

Thank you all for your insightful comments. They are a great help for someone relatively inexperienced at this.

First of all, the details that I should have included.

1. All of the notes are being dictated by a single physician.

2. They are not duplicates; he dictated notes for some patients using the first method, and some notes using the second method.

3. We don't know who the transcriber is, and at this point have no way to find out.

Based on your comments, I believe that the observations are not paired. Apologies for leaving out the pertinent details.

Although Reeza indicates that a t-test is probably fine, I'm can't be sure if the distributions of the two samples are the same. I think I'll stick with the Wilcoxon rank-sum.

Thank you again for all of your help!

Tom

Yes, now I agree the data is not paired. In the future, if possible to do, setting this experiment up as paired would be a better statistical approach. To do this, the doctor would have to dictate the notes for a patient twice, using both methods.

--

Paige Miller

Paige Miller

Build your skills. Make connections. Enjoy creative freedom. Maybe change the world. **Registration is now open through August 30th**. Visit the SAS Hackathon homepage.

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.