Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

Testing Confusion matrix in Enterprise Miner

Accepted Solution Solved
Reply
Contributor
Posts: 47
Accepted Solution

Testing Confusion matrix in Enterprise Miner

Hi, 

I wonder if someone can help me, I am using model comparison node and getting following kind of classification table for validation and training datasets. 

 

 

False       True            False       True

Negative    Negative    Positive    Positive

 

   364        4518         253         825

 

Can anyone please tell me how do we get these for testing dataset? 

In general do we need to find out for test datasets?

When we write these readings do we write aggregated values or for individual values?

 

 

Regards


Accepted Solutions
Solution
2 weeks ago
SAS Employee
Posts: 121

Re: Testing Confusion matrix in Enterprise Miner

SAS Enterprise Miner generates a ROC curve for the Train, Validate, and Test data set in the Model Comparison node when modeling a binary target.  It also generates a misclassification chart for the Train & Validate data sets but it does not generate a misclassification chart for the Test data set.  In the design of SAS Enterprise Miner, Test data sets are intended for a final unbiased evaluation of model performance so they are not used by default when a Validate data set is present.  

 

Please note that SAS Enterprise Miner always generates 

     F_<target variable name> : the target variable value  

     I _<target variable name> : the predicted target value (based on highest probability)

 

but it can also generates a D _<target variable name>  which contains the 'decision' outcome based on the decision weights and priors entered in the target profile when one is present.     For example, if the target variable is named 'BAD', SAS Enterprise Miner would create the variables F_BAD, I_BAD, and D_BAD.    

 

If a Test data set is available, you can add a SAS Code node after any modeling node and enter the following code in the Training code section.  This example assumes the target variable is named BAD.  

 

/*** BEGIN SAS CODE ***/

 

proc freq data=&em_import_test;
tables F_BAD*I_BAD;

tables F_BAD*I_BAD;  *only available if Decision profile has been created;
run;

 

/*** END SAS CODE ***/

 

The code above will generate both misclassification charts if the target profile is available. 

 

Hope this helps!

Doug

View solution in original post


All Replies
Solution
2 weeks ago
SAS Employee
Posts: 121

Re: Testing Confusion matrix in Enterprise Miner

SAS Enterprise Miner generates a ROC curve for the Train, Validate, and Test data set in the Model Comparison node when modeling a binary target.  It also generates a misclassification chart for the Train & Validate data sets but it does not generate a misclassification chart for the Test data set.  In the design of SAS Enterprise Miner, Test data sets are intended for a final unbiased evaluation of model performance so they are not used by default when a Validate data set is present.  

 

Please note that SAS Enterprise Miner always generates 

     F_<target variable name> : the target variable value  

     I _<target variable name> : the predicted target value (based on highest probability)

 

but it can also generates a D _<target variable name>  which contains the 'decision' outcome based on the decision weights and priors entered in the target profile when one is present.     For example, if the target variable is named 'BAD', SAS Enterprise Miner would create the variables F_BAD, I_BAD, and D_BAD.    

 

If a Test data set is available, you can add a SAS Code node after any modeling node and enter the following code in the Training code section.  This example assumes the target variable is named BAD.  

 

/*** BEGIN SAS CODE ***/

 

proc freq data=&em_import_test;
tables F_BAD*I_BAD;

tables F_BAD*I_BAD;  *only available if Decision profile has been created;
run;

 

/*** END SAS CODE ***/

 

The code above will generate both misclassification charts if the target profile is available. 

 

Hope this helps!

Doug

☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 1 reply
  • 229 views
  • 0 likes
  • 2 in conversation