BookmarkSubscribeRSS Feed
mikepark
Obsidian | Level 7

I ran an ensemble model in SAS Enterprise Miner. In the results section of the Model Comparison Node, I got the confusion matrix and my false negative rate is higher than I would like. How do I specifically look at these rows/observations of data to see if there is a pattern in what was classified as false negative and false positive?

1 REPLY 1
fierceanalytics
Obsidian | Level 7

In EM, there is setting where you can find the default score cut above which your prediction is flagged as yes. Otherwise NO. I think that default is 0.5. Play with different cut-off values to see you can just manage away the problem. This is perfectly legal. Nobody says you have to accept 0.5 as cut-off. This is low-hanging fruit kind of approach. 

 

A bit harder route is when you exhaust your play above and the symptom remains. To find out, take the validation data set from EM. Now the prediction 1 and 0 based on your 'optimized' score cut above is the new prediction target. Run another model, preferably intuitive decision tree which allows you see the drivers in the background. You cannot do this with your original target because that means you simply are adding a separate stand alone model alongside your ensemble. 

 

Jia Xin

sas-innovate-2024.png

Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

 

Register now!

How to choose a machine learning algorithm

Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 1 reply
  • 783 views
  • 0 likes
  • 2 in conversation