BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
geniusgenie
Obsidian | Level 7

Hi, 

I wonder if someone can help me, I am using model comparison node and getting following kind of classification table for validation and training datasets. 

 

 

False       True            False       True

Negative    Negative    Positive    Positive

 

   364        4518         253         825

 

Can anyone please tell me how do we get these for testing dataset? 

In general do we need to find out for test datasets?

When we write these readings do we write aggregated values or for individual values?

 

 

Regards

1 ACCEPTED SOLUTION

Accepted Solutions
DougWielenga
SAS Employee

SAS Enterprise Miner generates a ROC curve for the Train, Validate, and Test data set in the Model Comparison node when modeling a binary target.  It also generates a misclassification chart for the Train & Validate data sets but it does not generate a misclassification chart for the Test data set.  In the design of SAS Enterprise Miner, Test data sets are intended for a final unbiased evaluation of model performance so they are not used by default when a Validate data set is present.  

 

Please note that SAS Enterprise Miner always generates 

     F_<target variable name> : the target variable value  

     I _<target variable name> : the predicted target value (based on highest probability)

 

but it can also generates a D _<target variable name>  which contains the 'decision' outcome based on the decision weights and priors entered in the target profile when one is present.     For example, if the target variable is named 'BAD', SAS Enterprise Miner would create the variables F_BAD, I_BAD, and D_BAD.    

 

If a Test data set is available, you can add a SAS Code node after any modeling node and enter the following code in the Training code section.  This example assumes the target variable is named BAD.  

 

/*** BEGIN SAS CODE ***/

 

proc freq data=&em_import_test;
tables F_BAD*I_BAD;

tables F_BAD*I_BAD;  *only available if Decision profile has been created;
run;

 

/*** END SAS CODE ***/

 

The code above will generate both misclassification charts if the target profile is available. 

 

Hope this helps!

Doug

View solution in original post

1 REPLY 1
DougWielenga
SAS Employee

SAS Enterprise Miner generates a ROC curve for the Train, Validate, and Test data set in the Model Comparison node when modeling a binary target.  It also generates a misclassification chart for the Train & Validate data sets but it does not generate a misclassification chart for the Test data set.  In the design of SAS Enterprise Miner, Test data sets are intended for a final unbiased evaluation of model performance so they are not used by default when a Validate data set is present.  

 

Please note that SAS Enterprise Miner always generates 

     F_<target variable name> : the target variable value  

     I _<target variable name> : the predicted target value (based on highest probability)

 

but it can also generates a D _<target variable name>  which contains the 'decision' outcome based on the decision weights and priors entered in the target profile when one is present.     For example, if the target variable is named 'BAD', SAS Enterprise Miner would create the variables F_BAD, I_BAD, and D_BAD.    

 

If a Test data set is available, you can add a SAS Code node after any modeling node and enter the following code in the Training code section.  This example assumes the target variable is named BAD.  

 

/*** BEGIN SAS CODE ***/

 

proc freq data=&em_import_test;
tables F_BAD*I_BAD;

tables F_BAD*I_BAD;  *only available if Decision profile has been created;
run;

 

/*** END SAS CODE ***/

 

The code above will generate both misclassification charts if the target profile is available. 

 

Hope this helps!

Doug

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to choose a machine learning algorithm

Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 1 reply
  • 4834 views
  • 0 likes
  • 2 in conversation