topic Re: Why C statistic is different with and without plots=ROC in Proc Logistic in Statistical Procedures

Why C statistic is different with and without plots=ROC in Proc Logistic

richardxy — Wed, 28 Aug 2013 18:41:35 GMT

Hi all,

I'm relatively new to SAS　and the community, really appreciate the chance for getting help here. Thanks in advance for any help I may get.

I am running Proc Logistic to build a logistic regression model, and just found that I got two drastically different c statistic value with or without having the option plots=ROC in the Proc Logistic statement. It seems very confusing to me and I have no idea about the reason. A intense Google search didn't find anything.

My code is like this

Proc Logistic data=mydata desc;

weight wgt;

model target=&vars;

ouput out=myout p=p;

run;

The c statistic in the result is 0.552. Percent Concordant is 10.3, and Percent Tied is 89.6. Gamma is 0.996.

Then I turned plots=ROC on, and my code now looks like this.

Proc Logistic data=mydata desc plots=ROC;

weight wgt;

model target=&vars;

ouput out=myout p=p;

run;

The c statistic becomes 0.929, Percent Concordant is 92.4, Percent Tied is 0.9, and Gamma is 0.865. They all are very different from the previous set of values.

Anyone can share your insight into this? Why adding plots=ROC changes the value of results?

thanks,

Richard

Re: Why C statistic is different with and without plots=ROC in Proc Logistic

Reeza — Wed, 28 Aug 2013 18:57:00 GMT

It shouldn't but we can't see what &vars is.

If it truly is different with the exact same data you should post the output from the runs as well, and contact tech support.

Re: Why C statistic is different with and without plots=ROC in Proc Logistic

richardxy — Wed, 28 Aug 2013 19:02:06 GMT

Thanks Reeza for your reply.

&vars is just a string of variables, and it carries the same value for both runs. Will try the tech support.

Re: Why C statistic is different with and without plots=ROC in Proc Logistic

Reeza — Wed, 28 Aug 2013 19:11:51 GMT

It doesn't happen when I run with plots=roc vs without, not using weights.

I noticed you are using Weights, so you may want to consider this note:

Caution:PROC LOGISTIC does not compute the proper variance estimators if you are analyzing survey data and specifying the sampling weights through the WEIGHT statement. The SURVEYLOGISTIC procedure is designed to perform the necessary, and correct, computations.

Re: Why C statistic is different with and without plots=ROC in Proc Logistic

richardxy — Wed, 28 Aug 2013 19:37:05 GMT

I found another post which asked the same question and had an answer.

I need to specify binwidth=o and roceps=0 for Model statement, if plots=ROC is not used. The result proves that.