Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

ROC curve not always showing up in model comparison

Accepted Solution Solved
Reply
Occasional Contributor
Posts: 8
Accepted Solution

ROC curve not always showing up in model comparison

Hello,

The ROC curve (and various other output) aren't always showing up when I open the results to the model comparison node.  Instead I only get Fit Statistics and Output.  Occasionally it shows up, but I can't seem to figure out why it does in some diagrams/flows but not in others.  Has anyone else had the same problem?  I'm working in EM 12.1.

Thanks!

Jo


Accepted Solutions
Solution
‎01-24-2017 03:56 PM
Occasional Contributor
Posts: 8

Re: ROC curve not always showing up in model comparison

Just to update this thread - I received help a while ago from SAS tech support.  Seems as though I shouldn't have been including segment variables - i.e., I should have been "rejecting" them at the data loading stage.  I was keeping them, but calling them "segment."  SAS tech also helped me use the "advanced" editor on data loading stages, which I wasn't trying before.  Anyway, problem solved - ROC curves are showing up consistently now!

View solution in original post


All Replies
SAS Super FREQ
Posts: 272

Re: ROC curve not always showing up in model comparison

When it's not showing up, do you have a binary target and the property ROC Chart set to Yes?

Occasional Contributor
Posts: 8

Re: ROC curve not always showing up in model comparison

Yes, those are always my settings.  I'm beginning to think that EM simply won't calculate ROC when the sample becomes too large.  I suppose that makes sense, as it involves running through the data repeatedly with many different cutoffs.  Still, I have obtained ROC curves in base SAS with far larger samples.  This won't seem to do it with ~3000.

SAS Super FREQ
Posts: 272

Re: ROC curve not always showing up in model comparison

Hmm, there are no sample size limitations that I'm aware of.  This sounds like one for SAS Tech Support.  Can you please  create a model package with the flow where you are seeing this issue (see 46764 - Creating a model package (SPK file) in SAS® Enterprise Miner(tm)) and send it to Tech Support for assistance: http://support.sas.com/ctx/supportform/createForm.

Occasional Contributor
Posts: 8

Re: ROC curve not always showing up in model comparison

Sounds good.  Thanks for your help!

New Contributor
Posts: 2

Re: ROC curve not always showing up in model comparison

perhaps you only need to set the role of target variable from "interval" to "binary". I found that when  target variable has value 0 and 1, the default role is not "binary", and I can't see roc before I set the role to "binary".

Occasional Contributor
Posts: 8

Re: ROC curve not always showing up in model comparison

Thanks, it is set it to binary, but I still find it not producing ROC curves.

Solution
‎01-24-2017 03:56 PM
Occasional Contributor
Posts: 8

Re: ROC curve not always showing up in model comparison

Just to update this thread - I received help a while ago from SAS tech support.  Seems as though I shouldn't have been including segment variables - i.e., I should have been "rejecting" them at the data loading stage.  I was keeping them, but calling them "segment."  SAS tech also helped me use the "advanced" editor on data loading stages, which I wasn't trying before.  Anyway, problem solved - ROC curves are showing up consistently now!

Super Contributor
Posts: 336

Re: ROC curve not always showing up in model comparison

Very useful update Joel!

Thank you!

Senior User
Posts: 1

Re: ROC curve not always showing up in model comparison

Thanks for the details and the update. 

☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 9 replies
  • 2783 views
  • 6 likes
  • 5 in conversation