topic Oversampling and Scoring in SAS Data Science

Oversampling and Scoring

jlevine — Fri, 16 Dec 2011 12:11:13 GMT

I am trying to use Enterprise miner 7.1 for over-sampling. I understand the concept, but am having trouble implementing it. How do I over-sample a dataset within the tool, and then have my Score node-created score code create the final predicted value based on the original target proportion.

In other words, my real data has a 2% success rate. I over sample using the Sample node, so my training data has a 50% success rate. I fit my model, and then use the score code to score. I want EM_EVENTPROBABILITY to have an average of .02, not an average of .50. What settings can I use to make that happen?

Re: Oversampling and Scoring

jmbnzhu — Wed, 03 Apr 2013 02:47:44 GMT

correct probabilities =

1/(1+(1/original fraction-1)/(1/oversampled fraction-1)*(1/scoring result-1));

Re: Oversampling and Scoring

eshtern — Wed, 03 Apr 2013 18:33:38 GMT

In the EM click on the input data node. Then go to the properties panel and click the '....' button next to the Decision option. Once the window pops up, go to the tab 'Prior Probabilities' and enter actual probabilities in the Adjusted Probabilities column. This will adjust the model results for oversampling in the modeling dataset.

Hope that helps.