BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
sathya66
Barite | Level 11

Hi ,

My data source  target variable has (~0.0019%) and overall there are 7,240,251 observations,please can you explain through oversample approach(step by step please) or is there any other method to build a model.

I am using sample node now but it seems not working for me

Thanks,

Sathya.

1 ACCEPTED SOLUTION

Accepted Solutions
WendyCzika
SAS Employee

So actually I think you want to be using the other approach for dealing with rare targets, which is to adjust the posterior probabilities instead of entering the decision weights (those only affect profit, not other fit statistics).  So do that, in the Decisions node, you would no longer use the inverse priors on the diagonal of the decision matrix but just revert those to 1's, then you want to click Refresh on the Targets tab, then on the Prior Probabilities tab, enter the original priors for your target (the very rare proportion for your event, e.g.).  Now this will apply an adjustment to your posterior probabilities - hopefully you will see better results this way.

 

To answer your other question, EM_CLASSIFICATION is the generically named variable containing the predictions based on your model.  Here are more details about those variables from the Score node:

 

EM_PROBABILITY

Probability of Classification

Posterior probability associated with the predicted classification. That is, it corresponds the maximum of the posterior probabilities, max(P1, P2, ..., Pk).

EM_EVENTPROBABILITY

Probability for level n of vnm

Posterior probability associated with target event.

EM_CLASSIFICATION

Prediction for vnm

I_variable, the prediction variable for a class target.

View solution in original post

6 REPLIES 6
WendyCzika
SAS Employee

Please take a look at this post: https://communities.sas.com/t5/SAS-Communities-Library/Tip-How-to-model-a-rare-target-using-an-overs....  This outlines one of the ways you can deal with rare targets.  There is also a section "Detecting Rare Classes" under Analytics>Predictive Modeling in the SAS Enterprise Miner Reference Help.

 

Hope this helps!

Wendy

sathya66
Barite | Level 11
Thanks Wendy,
I have been through those steps but no luck.
EM_EVENTPROBABILITY coulmn values are >0.50 for 2,41,000 rows ( means>50%).
but SAS eminer will give us less %'s (ex :-0.00678) I guess
please suggest me
Thanks,
sathya.
sathya66
Barite | Level 11

and also after scoring ,it is creating some variables ex:
EM_EVENTPROBABILITY
EM_PROBABILITY
.EM_CLASSIFICATION (in 0s and 1s),
which variable do I need to consider from them as a predicted column
Thanks,
Sathya.

WendyCzika
SAS Employee

So actually I think you want to be using the other approach for dealing with rare targets, which is to adjust the posterior probabilities instead of entering the decision weights (those only affect profit, not other fit statistics).  So do that, in the Decisions node, you would no longer use the inverse priors on the diagonal of the decision matrix but just revert those to 1's, then you want to click Refresh on the Targets tab, then on the Prior Probabilities tab, enter the original priors for your target (the very rare proportion for your event, e.g.).  Now this will apply an adjustment to your posterior probabilities - hopefully you will see better results this way.

 

To answer your other question, EM_CLASSIFICATION is the generically named variable containing the predictions based on your model.  Here are more details about those variables from the Score node:

 

EM_PROBABILITY

Probability of Classification

Posterior probability associated with the predicted classification. That is, it corresponds the maximum of the posterior probabilities, max(P1, P2, ..., Pk).

EM_EVENTPROBABILITY

Probability for level n of vnm

Posterior probability associated with target event.

EM_CLASSIFICATION

Prediction for vnm

I_variable, the prediction variable for a class target.

sathya66
Barite | Level 11
And now I built a model , I want to automate the SAS em project. Is that possible to automate the project either by batch job or by scheduler.(every month with new history table/input table and new score table )
Please help me with this .
Thanks,
Sathya .

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to choose a machine learning algorithm

Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 6 replies
  • 1848 views
  • 0 likes
  • 2 in conversation