Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

How to visualize and evaluate the results from the score node in EM 14.1?

Accepted Solution Solved
Reply
Occasional Contributor
Posts: 16
Accepted Solution

How to visualize and evaluate the results from the score node in EM 14.1?

Hello,

 

I am doing an analysis of imbalanced data for classification (2% to 98%) and evaluating different sampling techniques and classification algorithms. I use one dataset as a training/validation data and would like to use a separate dataset as a test / scoring data and asses / visualize the performance of a model on those data. However, when I use model comparison node after a score node, I still only see the model performance on training and validation dataset, and don't see any results for the separate dataset. I am attaching a screenshot of my workflow.

 

Screenshot 2017-02-03 18.36.10.png

 


Accepted Solutions
Solution
‎02-08-2017 03:50 PM
SAS Employee
Posts: 18

Re: How to visualize and evaluate the results from the score node in EM 14.1?

In order to see the statistics that are generated for a test data set, you need to make sure that your separate data set has been read in before the modeling nodes and has a Role of Test.  If you set the Role property in the data source to Test, then it will act like a Test data set and any of the fit statistics generated in the training and validation data sets will also be generated for the separate test data set.

 

There is a similar story for the Score data set.  You need to make sure that the Role property for that data set has been set to Score.  You will not get many fit statistics for it; however, you will get some summary statistics around the predicted probabilities or predicted values for the target variables.


test_statistics.JPG

View solution in original post


All Replies
Solution
‎02-08-2017 03:50 PM
SAS Employee
Posts: 18

Re: How to visualize and evaluate the results from the score node in EM 14.1?

In order to see the statistics that are generated for a test data set, you need to make sure that your separate data set has been read in before the modeling nodes and has a Role of Test.  If you set the Role property in the data source to Test, then it will act like a Test data set and any of the fit statistics generated in the training and validation data sets will also be generated for the separate test data set.

 

There is a similar story for the Score data set.  You need to make sure that the Role property for that data set has been set to Score.  You will not get many fit statistics for it; however, you will get some summary statistics around the predicted probabilities or predicted values for the target variables.


test_statistics.JPG
Occasional Contributor
Posts: 16

Re: How to visualize and evaluate the results from the score node in EM 14.1?

Thank you very much! Smiley Happy

☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 2 replies
  • 279 views
  • 0 likes
  • 2 in conversation