topic Proc freq non-inferiority test result of 95%CI and P value is inconsistent in Statistical Procedures

Proc freq non-inferiority test result of 95%CI and P value is inconsistent

杨润煌 — Wed, 05 Jul 2023 13:51:53 GMT

here is my code:

proc freq data=table3_3_a;
    tables treat*DU1ORRES3 / riskdiff(margin=0.1 noninf) alpha=0.025;
run;

and here is the result of the freq

as i know, if the lower 95% cl is smaller than the non-inferiority margin(-0.1161<-0.1), then the p value will be bigger than 0.05 and the null hypothesis should be rejected, but the p value here is smaller than 0.05, i think i might missing something, could you help me understand better?

Re: Proc freq non-inferiority test result of 95%CI and P value is inconsistent

FreelanceReinh — Wed, 05 Jul 2023 16:24:13 GMT

Hello @杨润煌,

@杨润煌 wrote:

here is my code:
proc freq data=table3_3_a;
    tables treat*DU1ORRES3 / riskdiff(margin=0.1 noninf) alpha=0.025;
run;
(...)

as i know, if the lower 95% cl is smaller than the non-inferiority margin(-0.1161<-0.1), then the p value will be bigger than 0.05 and the null hypothesis should be rejected, but the p value here is smaller than 0.05, i think i might missing something, could you help me understand better?

You specified alpha=0.025 in your TABLES statement. Therefore, the correct conclusion is:

if the lower 95% cl is smaller than the non-inferiority margin(-0.1161<-0.1), then the p value will be bigger than 0.025 and the null hypothesis should be rejected (at that significance level).

The confidence coefficient for the test-based confidence limits is 100(1-2a)% according to the documentation.

Re: Proc freq non-inferiority test result of 95%CI and P value is inconsistent

杨润煌 — Thu, 06 Jul 2023 04:21:01 GMT

thanks for you information, I'd checked the document and it said the confidence limits of noninferiority is 100(1-2α)%, if i calculate the 95% CL in noninferiority test, i should compare p value with alpha=0.025 and in this case, the null hypothesis shouldn't be rejected, is this correct?

Re: Proc freq non-inferiority test result of 95%CI and P value is inconsistent

FreelanceReinh — Thu, 06 Jul 2023 07:09:22 GMT

@杨润煌 wrote:

thanks for you information, I'd checked the document and it said the confidence limits of noninferiority is 100(1-2α)%, if i calculate the 95% CL in noninferiority test, i should compare p value with alpha=0.025 and in this case, the null hypothesis shouldn't be rejected, is this correct?

In your example, p=0.0432>alpha, so this is correct. However, the default (and most commonly used) significance level is alpha=0.05. I assume you have good reasons to choose a different value.

Re: Proc freq non-inferiority test result of 95%CI and P value is inconsistent

杨润煌 — Thu, 06 Jul 2023 07:19:55 GMT

thanks for your help, i set alpha=0.025 since it is one-side test