BookmarkSubscribeRSS Feed
WayneReid
Calcite | Level 5

Hello,

For two sets of paired binary data (the before and after) I simply wish to compare how well the pairs match. I realize I could just calculate the percentage of matching pairs, but I wonder if a correlation coefficient would be a better approach due to the small sample size (n<10)? The phi & pearson coefficients seem to be the best choices for the given situation.

Does anyone have a better approach or method to analyze the data? Thanks.

Wayne

4 REPLIES 4
Ksharp
Super User

How about McNemar’s Test,which is used to paired binary contingency table .

And of course , don't forget corresponding analysis

Ksharp

SteveDenham
Jade | Level 19

Ksharp's mention of McNemar's test is right on.  I don't know if PROC FREQ gives an exact test for McNemar's test, but with small sample size, it is certainly worth investigating.

Steve Denham

WayneReid
Calcite | Level 5

Thanks for all the input gentlemen. I actually took a long look at McNemar before the initial post. Most of the info I found stated that it requires a sample size of at least 10 & that's why I was looking at correlation coefficients. After further research I did find some info that states the p value can be calculated using the binomial distribution for small sample sizes & a McNemar exact test.

Another additional concern is that after alterations to experiment parameters & data collection that at least 1 & as many as 2 of the contingency table values will be zero. There are 4 possible outcomes (T/T, T/F, F/T, F/F) but the results may only yield 2 or 3 outcomes. Not sure if that will require some type of correction.

I'm still looking but I somewhat feel I'm on the right track.

Wayne

Kayleah
Fluorite | Level 6

Hi Wayne, 

I am having the same issue you had. 

Small sample size.

I have a 4x4 table and thus used Bowker's Test for Symmetry.

I have also collapsed the 4 level response into a dichotomous response, where I used McNemars.

I have tried the Zeros option in the weight statement also.

 

I am curious as to what you ended up doing with your issue?

 

By the way, Steve Denham, SAS will compute the exact McNemars, by adding "AGREE" to the EXACTstatement in PROC FREQ.

 

Thank you,

Kayleah

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 4 replies
  • 3017 views
  • 2 likes
  • 4 in conversation