topic Fisher's exact test: p-value or 95% confidence limits in Statistical Procedures

Fisher's exact test: p-value or 95% confidence limits

trungdungtran — Mon, 08 Apr 2019 07:31:36 GMT

Hi all,

I have run proc freq to test association between two binary variables using the Fisher's exact test. However, the outputs seem contradict to each other as Two-sided Pr <= P 0.0355 whereas the OR and the 95% Confidence Limits 0.1097 [0.0118, 1.0170] includes 1.

This means that p-value shows that they are significantly associated while the 95% CI does not indicate that.

My question is that based on the output from SAS, what should I use?

Below is the code I used:

Proc freq data=final;
	Table var1* var2 / fisher or;
Run;

Re: Fisher's exact test: p-value or 95% confidence limits

Rick_SAS — Tue, 09 Apr 2019 13:00:22 GMT

Please post the results of the Fisher's Exact Test and Odds Ratio tables. In particular, I am curious about the size of your sample, I am guessing that you have a small sample.

Strictly speaking, what you describe is possible, but probably rare. Fisher's exact test is an exact p-value based on a combinatorial computation. It is most often used for small samples. The OR and RR results are asymptotic results. For small samples, they can deviate from the exact computation. The sampling distribution in small samples might not be well-approximated by the asymptotic distribution that is used to compute the p-value.

Regarding "what to use," use the exact test. However, IMHO a wiser course of action would be to collect more data.

Re: Fisher's exact test: p-value or 95% confidence limits

trungdungtran — Thu, 23 May 2019 08:34:42 GMT

@Rick_SAS: Thank you for your reply and sorry for a late response from me.

I agree with you that I have a small sample size as the contingency table is matrix(data=c(17, 31, 5, 1), nrow=2, ncol=2, byrow=FALSE).

However, the numbers in my first post are the exact number rather than asymptotic approximation.

As you see that I take p-value from the first table (last row) and the exact confidence limits from the last two rows of the last table (see attachment please). They are all exact as I understand correctly from the output of SAS.

Please see here an article talking about this problem.

https://academic.oup.com/biostatistics/article/11/2/373/267728