BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
Xiaoningdemao
Quartz | Level 8

Hello All,

 

I have a data set named as 'modeldata'. I first use bootstrap to ge a bootstrap sample named 'out'. I fit the logistic model with the bootstrap sample 'out'. Now i want to evaluate the performance of the bootstrap sample model by apply the original data 'modeldat' to it, and check for the c-statistic or c-index. Is there a way to do so?

 

Thank you!!

 

I attached my code below, hope that helps.

 

PROC multtest DATA=modeldata

nsample= 1 OUTSAMP=OUT SEED=1

nocenter noprint BOOTSTRAP;

test mean(pass age height); run;

 

proc logistic data=out descending out=logisticest;

model pass=age height;

run;

 

 

1 ACCEPTED SOLUTION

Accepted Solutions
Rick_SAS
SAS Super FREQ

In the first call to PROC LOGISTIC you fit the model. In the second call you need to use the SCORE statement to evaluate the model on a new set of data. The FITSTAT option displays fit statistics for the model evaluated on the new data. The AUC column gives the area under the ROC curve, which is equal to the 'c' statistic in the association table.  

 

Here is an example:

ods graphics off;
proc logistic data=sashelp.class descending OUTmodel=LogiModel;
model sex = age weight height;
run;
 
data newdata;
set sashelp.class;
where 14 <= age <= 16;
run;

proc logistic descending INmodel=logiModel;
score data = newdata fitstat;
run;

View solution in original post

6 REPLIES 6
Reeza
Super User

It's called Scoring a model.

 

Look at SCORE within PROC LOGISTIC, I believe there's an example in the documentation, or look at PROC PLM.

Xiaoningdemao
Quartz | Level 8

Dear Reeza,

 

Thank you very much!! Following what you suggested, I find this at support. sas: https://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_logistic_se...

I think by using outmodle and then inmodel, I  can use the old model to fit the new data. But, the 'fitstat' option doesn't gave me the c-statistic I need.

Do I have to calculate it manunaly?

 

What I did before is, store the estimated coefficients of the model estimated by old sampe, assign it as the initial value of the new data, and set convergence criterion to huge number. By doing this will i get what i want? Cause this way i can get the c-statistic. I attached the code below.

 

Thanks again!!

 

Best wishes.

 

 

ods output Association=C_boot;

proc logistic data=boot descending out=boot_est;

model pass=age gender height;

run;

 

proc transpose data=C_boot (keep=nValue2) out=C_boot(keep=col4);

run;

 

ods output Association=C_test;

proc logistic data=newdata descending INEST=boot_est;

model pass=age gender height/GCONV=100000;

run;

 

Reeza
Super User

I don't know the answer...I've moved the question to the Stats Forum and hopefully one of the stats guru can answer your question. 

Perhaps @Rick_SAS?  Cat Happy

Xiaoningdemao
Quartz | Level 8
Dear Reeza,
Thanks a lot!!
Best wishes.
Rick_SAS
SAS Super FREQ

In the first call to PROC LOGISTIC you fit the model. In the second call you need to use the SCORE statement to evaluate the model on a new set of data. The FITSTAT option displays fit statistics for the model evaluated on the new data. The AUC column gives the area under the ROC curve, which is equal to the 'c' statistic in the association table.  

 

Here is an example:

ods graphics off;
proc logistic data=sashelp.class descending OUTmodel=LogiModel;
model sex = age weight height;
run;
 
data newdata;
set sashelp.class;
where 14 <= age <= 16;
run;

proc logistic descending INmodel=logiModel;
score data = newdata fitstat;
run;
Xiaoningdemao
Quartz | Level 8
Dear Rick,
Perfect!
Thank you very much~
PS: I read a few of your blogs on other topics lately. I found them very helpful~~

Best wishes.

Ready to join fellow brilliant minds for the SAS Hackathon?

Build your skills. Make connections. Enjoy creative freedom. Maybe change the world. Registration is now open through August 30th. Visit the SAS Hackathon homepage.

Register today!
What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 6 replies
  • 4742 views
  • 2 likes
  • 3 in conversation