Hello,
The ROC curve (and various other output) aren't always showing up when I open the results to the model comparison node. Instead I only get Fit Statistics and Output. Occasionally it shows up, but I can't seem to figure out why it does in some diagrams/flows but not in others. Has anyone else had the same problem? I'm working in EM 12.1.
Thanks!
Jo
Just to update this thread - I received help a while ago from SAS tech support. Seems as though I shouldn't have been including segment variables - i.e., I should have been "rejecting" them at the data loading stage. I was keeping them, but calling them "segment." SAS tech also helped me use the "advanced" editor on data loading stages, which I wasn't trying before. Anyway, problem solved - ROC curves are showing up consistently now!
When it's not showing up, do you have a binary target and the property ROC Chart set to Yes?
Yes, those are always my settings. I'm beginning to think that EM simply won't calculate ROC when the sample becomes too large. I suppose that makes sense, as it involves running through the data repeatedly with many different cutoffs. Still, I have obtained ROC curves in base SAS with far larger samples. This won't seem to do it with ~3000.
Hmm, there are no sample size limitations that I'm aware of. This sounds like one for SAS Tech Support. Can you please create a model package with the flow where you are seeing this issue (see 46764 - Creating a model package (SPK file) in SAS® Enterprise Miner(tm)) and send it to Tech Support for assistance: http://support.sas.com/ctx/supportform/createForm.
Sounds good. Thanks for your help!
perhaps you only need to set the role of target variable from "interval" to "binary". I found that when target variable has value 0 and 1, the default role is not "binary", and I can't see roc before I set the role to "binary".
Thanks, it is set it to binary, but I still find it not producing ROC curves.
Just to update this thread - I received help a while ago from SAS tech support. Seems as though I shouldn't have been including segment variables - i.e., I should have been "rejecting" them at the data loading stage. I was keeping them, but calling them "segment." SAS tech also helped me use the "advanced" editor on data loading stages, which I wasn't trying before. Anyway, problem solved - ROC curves are showing up consistently now!
Very useful update Joel!
Thank you!
Thanks for the details and the update.
Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 25. Read more here about why you should contribute and what is in it for you!
Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.
Find more tutorials on the SAS Users YouTube channel.