topic Re: Optimism corrected c-statistic in Statistical Procedures

Optimism corrected c-statistic

richa108 — Mon, 08 Jul 2019 12:43:48 GMT

Hi everyone,

I am new to SAS. I am trying to compute an optimism-corrected c statistic using the following code:

proc surveyselect data= mydata seed=12345

out =samples

method= urs

samprate=1

outhits

reps=1000;

run

proc logistic data=samples; by replicate;

model outcome= a b c d e;

ods output parameterestimates= out association= assoc;

run;

The result is a table of coefficients generated for each variable in each replicate (dataset out) and a table of concordance indices (dataset assoc).

How can I compute the optimism-corrected c statistic? Is there an efficient way to apply each replicate's coefficients to my original dataset to generate a C statistic and then subtract all of the bootstrapped c statistics from the original c statistic?

Thank you very much for your help.

Re: Optimism corrected c-statistic

Rick_SAS — Mon, 08 Jul 2019 19:58:00 GMT

I don't know what an optimism-corrected C statistic is, but I have a few comments and a question:

1. Be sure to use ODS to suppress the output from PROC LOGISTIC.

2. You might want to use the OUTEST= option to write the parameter estimates directly to a SAS data set in a form that can be read later by PROC SCORE or PROC LOGISTIC.

3. Yes, it is possible to score the original data by using the parameter estimates from each bootstrap sample. Are you going to try to do that in the Base SAS (primarily the DATA step) or do you have access to the SAS/IML matrix language?

Re: Optimism corrected c-statistic

StatDave — Mon, 08 Jul 2019 21:43:07 GMT

By "optimism corrected" I assume you mean an estimate that is adjusted for the bias that results from computing the area under the ROC curve (a.k.a. c statistic or AUC) to evaluate the model using the same observations as used to fit the model. This can be done by using separate validation data or by crossvalidation using an option. See this note.