Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

SAS EM Score node producing identical predictions on scoring set.

Accepted Solution Solved
Reply
Occasional Contributor
Posts: 5
Accepted Solution

SAS EM Score node producing identical predictions on scoring set.

I built a predictive model is SAS EM using the AutoNeural node along with a Training and Validation data set. I used the Score node to score the entries in both sets, and verified that the number of predicted entries match the confusion matrices produced by the AutoNeural node.

 

However, I tried to score a new data set with the score node, and it produces an identical Probability for level 1 of INDAI and Prediction for INDAI for each entry on the scored data. I have checked to make sure that the metadata for the new data set is set to Score and that all of the possible variables are identified. I don't know how else to trouble shoot the issue. Even internet searches run into dead ends.

 

Has anyone else encountered, and solved, this issue? I can provide resources as needed; I just don't know what to produce up front that would be helpful.


sas3.PNGsas2 - Copy.PNGsas1 - Copy.PNG

Accepted Solutions
Solution
‎02-17-2017 04:03 PM
Super User
Posts: 17,829

Re: SAS EM Score node producing identical predictions on scoring set.

Are all estimates in the data set the same or only this section?

Do you have all of the variables from the original data used in this data set you're using for scoring, or at least all the required variables? 

 

What happens if you extract the Scoring code and run that separately?

View solution in original post


All Replies
Super User
Posts: 17,829

Re: SAS EM Score node producing identical predictions on scoring set.

This has been asked before, I'm not sure I've seen a solution though. You may want to contact Tech Support instead.

 

Super User
Posts: 17,829

Re: SAS EM Score node producing identical predictions on scoring set.

I took a quick look at your screenshot, note that the variables all appear missing for those records. Which means for some reason your data isn't being imported or referenced properly.  I would start my investigations there and make sure I'm referencing the dataset to be scored correctly. 

Occasional Contributor
Posts: 5

Re: SAS EM Score node producing identical predictions on scoring set.

Thanks for the reply!

 

The data is acutally loaded in correctly. Those variables don't exist in this particular data set, so they are left blank. I could scroll over to the part where the variables are loaded, but it's that alone probably wouldn't help.

 

Unless there are any other ideas (unlikely since the previous question wasn't answered), I may have to contact tech support and report back with findings.

Solution
‎02-17-2017 04:03 PM
Super User
Posts: 17,829

Re: SAS EM Score node producing identical predictions on scoring set.

Are all estimates in the data set the same or only this section?

Do you have all of the variables from the original data used in this data set you're using for scoring, or at least all the required variables? 

 

What happens if you extract the Scoring code and run that separately?

Super User
Posts: 17,829

Re: SAS EM Score node producing identical predictions on scoring set.

https://communities.sas.com/t5/SAS-Data-Mining/SAS-EM-predicted-probabilities-in-score-data-set-are-...

 

So far from what I've found, it looks like each person was having issues with their input data set and it resulted in this issue.

Occasional Contributor
Posts: 5

Re: SAS EM Score node producing identical predictions on scoring set.

I apologize for the very late reply!

 

In case someone else finds this via a google search, the issue I had was that the scoring data set did not contain all of the variables that the training data set used in the neural network. I thought that the missing variables wouldn't matter, but it turns out that it did. Here's the SAS support response for good meaure:

 

Thanks for using SAS Enterprise Miner.  When you see that your scoring output contains only one value, this most likely means that one of the input variables in the score data set is missing for every observation.  The next step to verify if this is the root cause would be to run a PROC UNIVARIATE on the input variables to see if any of them have 100% missing.  

"

 

Thanks for your responses Reeza!

Super User
Posts: 17,829

Re: SAS EM Score node producing identical predictions on scoring set.

There was a post here earlier about how EM handles missing data when scoring, it isn't what you expect basically. Especially if there's a variable missing or the names don't match exactly. Check the data mining forum. 

☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 7 replies
  • 276 views
  • 1 like
  • 2 in conversation