turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

- Home
- /
- Analytics
- /
- Stat Procs
- /
- K fold CV - how to draw "average" ROC

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-10-2017 12:37 AM

I am running a 5 fold cross validation on my logistic model. The predicted probability of the outcome for each test group was generated using data from the training groups. My question is - how can I draw a "mean" ROC curve to summarize my results?

Accepted Solutions

Solution

07-10-2017
03:44 PM

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-10-2017 03:12 PM

All Replies

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-10-2017 01:26 AM

Are you asking how to do the calculation or how to create the graph?

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-10-2017 01:29 AM

How to create the graph, but it would also be useful to calculate the AUC. I essentially have output predicted probabilities from fitting my training data on each of my test data, I just dont know how to use this to create the ROC curve, ultimately get the AUC.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-10-2017 09:50 AM

What do you mean by 'average' ROC .

You can use ROC= option to get those K-fold roc dataset,

but you might not get the average roc, x could be different, you could try.

if you want plot all these 5 roc in the same graph. here is an example .

```
proc logistic data=test9 ;
model good_bad(event='good')= &varlist
/outroc=_roc lackfit scale=none aggregate rsquare firth;
output out=_output h=h c=c cbar=cbar;
run;
data plot_roc;
set roc(in=ina) _roc;
if ina then dsn='训练集';
else dsn='测试集';
run;
title ' ';
proc sgplot data=plot_roc aspect=1;
series x=_1MSPEC_ y=_SENSIT_ /group=dsn smoothconnect name='x';
lineparm x=0 y=0 slope=1/lineattrs=(color=verylightgray);
xaxis grid;
yaxis grid;
keylegend 'x' /title=' ' location=inside position=nw across=1;
run;
```

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-10-2017 12:29 PM

Well, I thought that since the training data was used to fit the test data, which is a different portion of the main dataset each time, those predicted probabilities could be used to get an "average" ROC (by basically combining each of the test datasets, each with predicted probabilities of the outcome, together). So, I basically have a dataset with my outcome, predictors, and predicted probabilities from my k fold CV and dont know how to construct the ROC curve.

Solution

07-10-2017
03:44 PM

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-10-2017 03:12 PM

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-10-2017 03:48 PM

Thanks so much!