Small sample size of paired binary data

[WayneReid]

Hello,

For two sets of paired binary data (the before and after) I simply wish to compare how well the pairs match. I realize I could just calculate the percentage of matching pairs, but I wonder if a correlation coefficient would be a better approach due to the small sample size (n<10)? The phi & pearson coefficients seem to be the best choices for the given situation.

Does anyone have a better approach or method to analyze the data? Thanks.

Wayne

[Ksharp]

How about McNemar’s Test,which is used to paired binary contingency table .

And of course , don't forget corresponding analysis

Ksharp

[SteveDenham]

Ksharp's mention of McNemar's test is right on.  I don't know if PROC FREQ gives an exact test for McNemar's test, but with small sample size, it is certainly worth investigating.

Steve Denham

[WayneReid]

Thanks for all the input gentlemen. I actually took a long look at McNemar before the initial post. Most of the info I found stated that it requires a sample size of at least 10 & that's why I was looking at correlation coefficients. After further research I did find some info that states the p value can be calculated using the binomial distribution for small sample sizes & a McNemar exact test.

Another additional concern is that after alterations to experiment parameters & data collection that at least 1 & as many as 2 of the contingency table values will be zero. There are 4 possible outcomes (T/T, T/F, F/T, F/F) but the results may only yield 2 or 3 outcomes. Not sure if that will require some type of correction.

I'm still looking but I somewhat feel I'm on the right track.

Wayne

[Kayleah]

Hi Wayne, 

I am having the same issue you had. 

Small sample size.

I have a 4x4 table and thus used Bowker's Test for Symmetry.

I have also collapsed the 4 level response into a dichotomous response, where I used McNemars.

I have tried the Zeros option in the weight statement also.

I am curious as to what you ended up doing with your issue?

By the way, Steve Denham, SAS will compute the exact McNemars, by adding "AGREE" to the EXACTstatement in PROC FREQ.

Thank you,

Kayleah