Testing Confusion matrix in Enterprise Miner

[geniusgenie]

Hi, 

I wonder if someone can help me, I am using model comparison node and getting following kind of classification table for validation and training datasets. 

False       True            False       True

Negative    Negative    Positive    Positive

   364        4518         253         825

Can anyone please tell me how do we get these for testing dataset? 

In general do we need to find out for test datasets?

When we write these readings do we write aggregated values or for individual values?

Regards

[DougWielenga · Accepted solution]

SAS Enterprise Miner generates a ROC curve for the Train, Validate, and Test data set in the Model Comparison node when modeling a binary target.  It also generates a misclassification chart for the Train & Validate data sets but it does not generate a misclassification chart for the Test data set.  In the design of SAS Enterprise Miner, Test data sets are intended for a final unbiased evaluation of model performance so they are not used by default when a Validate data set is present.  

Please note that SAS Enterprise Miner always generates 

     F_<target variable name> : the target variable value  

     I _<target variable name> : the predicted target value (based on highest probability)

but it can also generates a D _<target variable name>  which contains the 'decision' outcome based on the decision weights and priors entered in the target profile when one is present.     For example, if the target variable is named 'BAD', SAS Enterprise Miner would create the variables F_BAD, I_BAD, and D_BAD.    

If a Test data set is available, you can add a SAS Code node after any modeling node and enter the following code in the Training code section.  This example assumes the target variable is named BAD.  

/*** BEGIN SAS CODE ***/

proc freq data=&em_import_test;
tables F_BAD*I_BAD;

tables F_BAD*I_BAD;  *only available if Decision profile has been created;
run;

/*** END SAS CODE ***/

The code above will generate both misclassification charts if the target profile is available. 

Hope this helps!

Doug