turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

- Home
- /
- Analytics
- /
- Data Mining
- /
- Scoring data and fit statistics in SAS E Miner

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-13-2017 01:18 AM - edited 07-13-2017 01:19 AM

Hi folks,

I am building a predictive model in SAS E Miner and through model comparison I choosed Neural Network model as best. I have also imported scoring data and scored new data with Neural Network model.I got predicted values and predicted probabilities in a new table. But requirement is also to get the fit statistics for the new scored data.

Please help!!

Accepted Solutions

Solution

a month ago

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

a month ago - last edited 3 weeks ago

Please note that to compute assessment statistics, you will need to have the outcome/target information for each observation. You cannot assess the fit if the outcome is unknown. In most cases, you are scoring new data where the outcome is unknown. If you later want to assess how the model performed, you can use a Model Import node to import a scored data set containing predicted values (or predicted probabilities) and compute the assessment statistics.

To do so, follow these steps:

1 - Add the scored data as a new Input data source and set the role to Training (even though it is a 'scored' data set)

Note: You will need a separate Model Import node flow for each scored data set you wish to import.

2 - Make sure the target variable has the same name as the one used in the SAS Enterprise Miner modeling flow

3 - Make sure at least one variable (even a dummy variable) is listed as an Input variable;

4 - Make sure any binary/nominal/ordinal target variable has the same level and is sorted in the same order as the one used in the SAS Enterprise Miner modeling flow

5 - In the Model Import node, map the probability of the target event to the correct level of the target variable (if using binary/nominal/ordinal target variable).

You can then view the Assessment information in the Model Comparison node.

As Wendy mentioned, if you have the outcome/target information and the input data, you can also specify the data source as Test (assuming you don't already have a Test data set) and pass it to the first node beyond the Data Partition node. If no Data Partition node is present, connect it to the same node the training and/or validation data set is connected to. SAS Enterprise Miner will then create assessment information for the 'Test' data (which is actually your scored data) as part of the flow.

All Replies

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-19-2017 12:49 PM

Can you just clarify for me what your exact steps are?

You have a flow with a Model Comparison node that chooses the Neural Network model.

Do you then attach the Model Comparison node and another Input Data node that has new data (but has the observed target in it) to score to a Score node? If so, what role is assigned to your Input Data?

Then you want assessment for that, correct?

Please let me know if I'm understanding correctly!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-20-2017 01:04 AM

Hi,

Thanks for reply!

Yes, Model Comparison node is used in the flow which chooses Neural Network as a best model.

Yes, I attached new input data with the Model Comparison node and used Score node to predict new cases.

Input role is assigned to the input data node.

Yes need a assessment of input data.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-20-2017 09:27 AM

There might be a better way to do this, but the only way I can figure is to set the role of this Input Data node to Test (this assumes you don't already have a Test partition), and connect that directly to the Neural Network node instead of the Score node, and you will get the assessment statistics there. The Score node is typically used for scoring new data where the target is unknown.

Solution

a month ago

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

a month ago - last edited 3 weeks ago

Please note that to compute assessment statistics, you will need to have the outcome/target information for each observation. You cannot assess the fit if the outcome is unknown. In most cases, you are scoring new data where the outcome is unknown. If you later want to assess how the model performed, you can use a Model Import node to import a scored data set containing predicted values (or predicted probabilities) and compute the assessment statistics.

To do so, follow these steps:

1 - Add the scored data as a new Input data source and set the role to Training (even though it is a 'scored' data set)

Note: You will need a separate Model Import node flow for each scored data set you wish to import.

2 - Make sure the target variable has the same name as the one used in the SAS Enterprise Miner modeling flow

3 - Make sure at least one variable (even a dummy variable) is listed as an Input variable;

4 - Make sure any binary/nominal/ordinal target variable has the same level and is sorted in the same order as the one used in the SAS Enterprise Miner modeling flow

5 - In the Model Import node, map the probability of the target event to the correct level of the target variable (if using binary/nominal/ordinal target variable).

You can then view the Assessment information in the Model Comparison node.

As Wendy mentioned, if you have the outcome/target information and the input data, you can also specify the data source as Test (assuming you don't already have a Test data set) and pass it to the first node beyond the Data Partition node. If no Data Partition node is present, connect it to the same node the training and/or validation data set is connected to. SAS Enterprise Miner will then create assessment information for the 'Test' data (which is actually your scored data) as part of the flow.