Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

AUC value (Area Under Curve or ROC Index) in SAS Miner 9.3

Accepted Solution Solved
Reply
New Contributor
Posts: 4
Accepted Solution

AUC value (Area Under Curve or ROC Index) in SAS Miner 9.3

Good afternoon,

I am looking for the AUC value (Area Under the Curve or ROC Index) in SAS Enterprise Miner in SAS 9.3. I have explored all the options from the ROC curve graph (see pic) and from the nodes I am using (logistic regression, decision trees and neural nets) without any success.


Many thanks in advance.

Capture.JPG

 


Accepted Solutions
Solution
‎07-06-2017 01:31 PM
SAS Super FREQ
Posts: 306

Re: AUC value (Area Under Curve or ROC Index) in SAS Miner 9.3

You need to attach a Model Comparison node to the modeling node(s) that you are running, then in the Results for Model Comparison, if you look in the Statistics Comparison table, the row where Fit Statistics="_AUR_" corresponds to the area under the ROC curve.  The KS statistic is reported in that table as well.

Hope that helps!

View solution in original post


All Replies
PROC Star
Posts: 1,324

Re: AUC value (Area Under Curve or ROC Index) in SAS Miner 9.3

I'm don't use Enterprise Miner but I've calculated AUC using PROC LOGISTIC.  It looks like EM is generating SAS code in the background perhaps (similar to DI studio?).  If so, is there somwhere you can see PROC LOGISTIC code which you would be able to edit by hand?  Can you show the current PROC LOGISTIC code?

 

You would want to insert a ROC statement within the PROC LOGISTIC step. 

SAS Employee
Posts: 106

Re: AUC value (Area Under Curve or ROC Index) in SAS Miner 9.3

Hi. You could compute AUC from the dataset that EM uses to display the ROC chart in the Model Comparison node. 

 

Here's some basic code that uses the formula for the area of a trapezoid. Put it in a SAS Code node after your Model Comparison node and run it.  (disclaimer:  untested code.)

 

The code assumes you have a single model (i.e., just one modeling node is hooked up to the Model Compare node) and that the nodeID of your model comparison is 'MdlComp'. 

 

data roc;
retain AUC ;
set &em_lib..mdlcomp_emroc (where=(model ne " _Baseline_"));
avgHeight = (sensitivity + lag1(sensitivity)) / 2;
width = oneminusspecificity - lag1(oneminusspecificity);
smallArea = avgHeight * width;
AUC + smallArea;
run;

 

*last row contains (total) AUC;
proc print data = roc;
var sensitivity avgHeight width oneMinusSpecificity smallArea AUC;
run;

 

Hope this helps.

 

Ray

Solution
‎07-06-2017 01:31 PM
SAS Super FREQ
Posts: 306

Re: AUC value (Area Under Curve or ROC Index) in SAS Miner 9.3

You need to attach a Model Comparison node to the modeling node(s) that you are running, then in the Results for Model Comparison, if you look in the Statistics Comparison table, the row where Fit Statistics="_AUR_" corresponds to the area under the ROC curve.  The KS statistic is reported in that table as well.

Hope that helps!

☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 3 replies
  • 2937 views
  • 1 like
  • 4 in conversation