topic Re: conflicting results between exact p value and confidence limits of OR in exact logistic in Statistical Procedures

conflicting results between exact p value and confidence limits of OR in exact logistic

amannelli — Tue, 06 Apr 2021 07:45:45 GMT

Good morning.

I used the EXACT statement in logistic regression to analyse data with quasi complete separation of results (none of the individuals in the unexposed group was a case). I obtained the median unbiased estimate of the logistic regression parameter. The exact p value was 0.062 but the parameter 95% CL's were 0.18; Infinity. When calculating the odds ratio, the lower limit was greater than 1 (1.2). Furhermore, very disturbingly, after including the ONESIDED option in the EXACT statement, while the p-value was reduce by half, the 95% CL's of the same OR were not changed.

Thank you very much for your help

Alessandro Mannelli

Re: conflicting results between exact p value and confidence limits of OR in exact logistic

ballardw — Tue, 06 Apr 2021 15:43:19 GMT

AS an absolute minimum you should show your code. Then we at least have a chance to see the possible effects of options.

Better would be to provide the diagnostic summaries and the actual output along with the code and possibly the log.

Code is needed because just Proc Logistic has 20+ statements that interact. Plus Proc Logistic is not the only procedure that can perform logistic regression.

Code and log entries are best posted into text boxes opened on the the forum with the </> icon to preserve text positions. The message windows will reformat text.

Conflicting 95% CL and exact p values in exact logistic regression

amannelli — Wed, 07 Apr 2021 06:51:16 GMT

Good morning. I used the EXACT statement in logistic regression to analyse data with quasi complete separation of results (none of the individuals in the unexposed group was a case). I obtained the median unbiased estimate of the logistic regression parameter. The exact p value was 0.055 but the parameter 95% CL's were 0.18; Infinity. When calculating the odds ratio, the lower limit was greater than 1 (1.24). Furhermore, very disturbingly, after including the ONESIDED option in the EXACT statement, while the p-value was reduce by half, the 95% CL's of the same OR were not changed.

I paste here codes including datalines: 95% CL and exact p value for riskfactor2 are conflicting, furthermore, CL are not affected by onesided option (p-value is)

data risk;
input riskfact1 riskfact2 tot pos;
datalines;
0 0 36 0
0 1 93 3
1 0 13 0
1 1 54 11
;
run;

proc logistic data=risk exactonly;
TITLE ' two-sided ';
title2 'Conflicting 95% CLs and exact p-value';
model pos / tot = riskfact1 riskfact2;
exact riskfact1 riskfact2 / estimate=both outdist = dist;
run;

proc logistic data=risk exactonly;
TITLE ' one-sided ';
title2 ' 95% CLs not affected by onesided option';
model pos / tot = riskfact1 riskfact2;
exact riskfact1 riskfact2 / estimate=both outdist = dist onesided;
run;

Thank you very much for your help Alessandro Mannelli

Re: conflicting results between exact p value and confidence limits of OR in exact logistic

amannelli — Wed, 07 Apr 2021 06:55:02 GMT

Good morning, I placed another post with datalines and codes, for the one
dataset where I can have data in binomial format (pos / tot), just in case,
I paste the same codes here. Sorry in case of double posting!! and thank
you very much. I have been a sas user for 30 years but never used this
support...

data risk;

input riskfact1 riskfact2 tot pos;

datalines;

0 0 36 0

0 1 93 3

1 0 13 0

1 1 54 11

;

run;

proc logistic data=risk exactonly;

TITLE ' two-sided ';

title2 'Conflicting 95% CLs and exact p-value';

model pos / tot = riskfact1 riskfact2;

exact riskfact1 riskfact2 / estimate=both outdist = dist;

run;

proc logistic data=risk exactonly;

TITLE ' one-sided ';

title2 ' 95% CLs not affected by onesided option';

model pos / tot = riskfact1 riskfact2;

exact riskfact1 riskfact2 / estimate=both outdist = dist
onesided;
run;

Re: conflicting results between exact p value and confidence limits of OR in exact logistic

sbxkoenk — Wed, 07 Apr 2021 09:10:03 GMT

Hello,

I haven't looked at your code (yet).

But next time you provide code (I tell it as you indicate you are new to this forum), use the 'Insert SAS Code' button (the running man in a square) on the toolbar beneath the string 'Body'.

That way your code preserves structure and formatting and there's some useful highlighting and colors. And it's much more dense.

Cheers,

Koen

Re: Conflicting 95% CL and exact p values in exact logistic regression

sbxkoenk — Wed, 07 Apr 2021 09:20:43 GMT

Hello,

Only the Exact Odds Ratios for 'riskfact2' are identical in both outputs (logistic1 and logistic2), and that makes sense as the first output clearly mentions:

Note: * indicates a median unbiased estimate and a one-sided p-value.

Also, have a look at this paper:

Paper 3018-2019
Predicting Inside the Dead Zone of Complete Separation in Logistic Regression
Robert Derr, SAS Institute Inc., Cary, NC

https://www.sas.com/content/dam/SAS/support/en/sas-global-forum-proceedings/2019/3018-2019.pdf

Cheers,

Koen

Re: Conflicting 95% CL and exact p values in exact logistic regression

sbxkoenk — Wed, 07 Apr 2021 09:32:43 GMT

On top of my previous response.

Also have a look at this communities entry:

Extrem odd ratio with firth logistic regression
https://communities.sas.com/t5/Statistical-Procedures/Extrem-odd-ratio-with-firth-logistic-regression/m-p/728253

Koen

Re: Conflicting 95% CL and exact p values in exact logistic regression

amannelli — Wed, 07 Apr 2021 09:37:25 GMT

Thank you very much. My problem, however, is that the exact p value is = 0.055 (p>0.05, accept the null hypothesis) whereas the 95% CI lower limit of the odds ratio is 1.24 (being > 1 I would reject the null hypothesis)

second, I realise that the ONESIDED option only affects the exact pa value, not the 95% CL's since they are the same as in the two-sided defalt for riskfactor2

Thank you and best regrds

Re: Conflicting 95% CL and exact p values in exact logistic regression

StatDave — Wed, 07 Apr 2021 16:05:15 GMT

You didn't say which release you are using, but in the current release, SAS 9.4 TS1M7, the p-value reported for riskfact2 is 0.0277 with confidence interval (0.2147, inf) and these are the same regardless of the ONESIDED option. This is because that estimate is a median unbiased estimate. The confidence interval for its odds ratio is (1.240, inf) and again is unchanged by the ONESIDED option.

You should carefully read the ending part of the "Details:Exact conditional logistic regression" in the documentation of the current SAS release. There it tells you that for a median unbiased estimate, the p-value is always a one-sided p-value. Since riskfact1 is not a median unbiased estimate, its p-value (0.0021) is a two-sided p-value, and with the ONESIDED option, the one-sided p-value is 0.0011.

Re: Conflicting 95% CL and exact p values in exact logistic regression

amannelli — Thu, 08 Apr 2021 08:31:11 GMT

Thank you very much StatDave_sas,

Now I understand that the ONESIDED option affected the p-value, but not the 95% CL, which is one-side even when such an option is not included.

Best regards

Alessandro Mannelli