topic Where to define prior probabilities? in SAS Data Science

Where to define prior probabilities?

badikidiki — Wed, 25 Feb 2015 12:15:02 GMT

Hello,

in an enterprise miner project we are analyzing a sample which has much more cases than reality.

Therefore we want to use prior probabilities.

In our project we are using the following nodes: sample, data partition, decision tree and model comparison.

Is it enough to define the prior probabilities in the first node, the sample node?

Does the decision tree node later on use the prior probabilities defined in the sample node?

Or do we have to define the prior probabilities in every node?

Thanks for your help!

badikidiki

Re: Where to define prior probabilities?

M_Maldonado — Wed, 25 Feb 2015 15:15:03 GMT

Hi,

Check out this tip, we do something similar to what you are trying to do.

https://communities.sas.com/docs/DOC-10220

I hope it helps,

Miguel

Re: Where to define prior probabilities?

badikidiki — Fri, 27 Feb 2015 16:11:48 GMT

Hi Miguel,

thank you very much for your answer!

I will try to use the decision node.

But I wonder what the Miner does without a decision node.

My first try was to define prior probabilities in my sample node.

After that I ran a data partition, different decision trees and a model comparison node.

Do prior probabilities defined in the sample node - without having a decision node - effect the following nodes or not?

badikidiki

Re: Where to define prior probabilities?

DougWielenga — Wed, 31 Jan 2018 17:26:49 GMT

I wonder what the Miner does without a decision node.

My first try was to define prior probabilities in my sample node.

After that I ran a data partition, different decision trees and a model comparison node.

Do prior probabilities defined in the sample node - without having a decision node - effect the following nodes or not?

badikidiki

@badikidiki,

I'm not sure what option you were using when you tried to adjust the probabilities using the Sample node. You can only define priors and weights using the Decisions property in the Input Data Source node or you can do so in the Decisions node itself. In general, I recommend doing so in the Input Data Source node. In either case, you can choose options whether to adjust the probabilities based on the priors you specified, and you can choose whether to use the decisions weights you specified. By default, the decision weights correspond to assigning each observation to the most likely outcome (all outcomes are equally weighted before it is changed).

If you are using a Sample node, I would avoid using the Sample node property to Adjust Frequency which could make the actual sample look much larger than it is. If you specified decision priors/weights in the Input Data Source node (or in the Decisions node), every subsequent node might be impacted. If you choose the option to Adjust Frequency in the Sampling node, you will likely see a different impact, but all adjustments are being considered together. I would recommend that if you plan on sampling, specify priors and weights that correspond to your anticipated sample.

Hope this helps!
Doug