topic Re: c-statistic (AUC) and Wilcoxon rank sum test P-value difference in New SAS User

c-statistic (AUC) and Wilcoxon rank sum test P-value difference

LAYMAN_YO — Wed, 04 Dec 2019 01:10:12 GMT

Could you elaborate on the difference of P-values (Chi-square) between AUC (ROC1-reference) with PROC LOGISTIC and Wilcoxon rank sum test with NPAR1WAY.

ROC curve area represents the estimate of p''=Pr(X1<X2)+Re(X1=X2) / 2 .

The WMW statistic can be used to test whether the AUC is significantly different from 0.5 (Bamber 1975). A convenient way to generate a 95% confidence interval for the ROC curve area [and hence Pr(X1< X2)] is to use the ROC option in PROC LOGISTIC.

Example

data abc;
input arm y;
cards;
2 11
1 12
2 13
1 14
2 15
1 16
1 17
1 18
1 19
1 20
;
run;
proc logistic data=abc;
class arm;
model arm(event='1')=y /outroc=troc;
roc;
roccontrast / estimate e;
run;

ROC Contrast Estimation and Testing Results by Row

Contrast Estimate StandardError 95% WaldConfidence Limits Chi-Square Pr > ChiSq

ROC1 - Model

-0.3571

0.1290

-0.6099

-0.1044

7.6705

0.0056

proc npar1way data=abc wilcoxon;
class arm;
var y;
exact wilcoxon ;
run;

Wilcoxon Two-Sample Test

9.0000

-1.5954

0.0553

0.1106

0.0725

0.1451

0.0583

0.1167

Kruskal-Wallis TestChi-Square DF Pr > ChiSq

2.9221

0.0874

What is the relationship between ROC1-Mo Chisq and K-W Chisq ?

Re: c-statistic (AUC) and Wilcoxon rank sum test P-value difference

Ksharp — Tue, 03 Dec 2019 11:34:19 GMT

Good question.

"AUC (ROC1-reference) with NPAR1WAY "

You mean K-S test ?

I think both test are non-parameter test,both are similar , but H0 is different .

K-S test H0: whether these two class/level are coming from the same polulation or not .

Wilcoxon rank sum test is more likt t-test (the non-parameter method of t-test).

its H0 : whether these two class/level's mean(expect value) are the same .

Better post it at Stat forum and calling @StatDave

Re: c-statistic (AUC) and Wilcoxon rank sum test P-value difference

PaigeMiller — Tue, 03 Dec 2019 13:02:42 GMT

@LAYMAN_YO wrote:

Could you elaborate on the difference of P-values (Chi-square) between AUC (ROC1-reference) with NPAR1WAY and Wilcoxon rank sum test with Logistic.

Could you be more specific and state exactly which commands or options in PROC NPAR1WAY you are referring to?

Re: c-statistic (AUC) and Wilcoxon rank sum test P-value difference

StatDave — Tue, 03 Dec 2019 14:10:06 GMT

As stated in the latter part of this note, the KS test assessed whether the event and nonevent populations differ, not just in location, but also in shape. Since interest usually is focused on a location shift (differing means), that test is a bit too broad.