ROC curve not always showing up in model comparison

[Joel_Wesley]

Hello,

The ROC curve (and various other output) aren't always showing up when I open the results to the model comparison node.  Instead I only get Fit Statistics and Output.  Occasionally it shows up, but I can't seem to figure out why it does in some diagrams/flows but not in others.  Has anyone else had the same problem?  I'm working in EM 12.1.

Thanks!

Jo

[WendyCzika]

When it's not showing up, do you have a binary target and the property ROC Chart set to Yes?

[Joel_Wesley]

Yes, those are always my settings.  I'm beginning to think that EM simply won't calculate ROC when the sample becomes too large.  I suppose that makes sense, as it involves running through the data repeatedly with many different cutoffs.  Still, I have obtained ROC curves in base SAS with far larger samples.  This won't seem to do it with ~3000.

[WendyCzika]

Hmm, there are no sample size limitations that I'm aware of.  This sounds like one for SAS Tech Support.  Can you please  create a model package with the flow where you are seeing this issue (see 46764 - Creating a model package (SPK file) in SAS® Enterprise Miner(tm)) and send it to Tech Support for assistance: http://support.sas.com/ctx/supportform/createForm.

[Joel_Wesley]

Sounds good.  Thanks for your help!

[YujunChiang]

perhaps you only need to set the role of target variable from "interval" to "binary". I found that when  target variable has value 0 and 1, the default role is not "binary", and I can't see roc before I set the role to "binary".

[Joel_Wesley]

Thanks, it is set it to binary, but I still find it not producing ROC curves.

[Joel_Wesley · Accepted solution]

Just to update this thread - I received help a while ago from SAS tech support.  Seems as though I shouldn't have been including segment variables - i.e., I should have been "rejecting" them at the data loading stage.  I was keeping them, but calling them "segment."  SAS tech also helped me use the "advanced" editor on data loading stages, which I wasn't trying before.  Anyway, problem solved - ROC curves are showing up consistently now!

[M_Maldonado]

Very useful update Joel!

Thank you!

[bcastillo10]

Thanks for the details and the update.