Re: "Lesson 2: Fitting the Model -> Understanding the Offset" (Module "Predictive Modeling Using Logistic Regression") Would it be possible to clarify which method between offset and sampling weights is better? The above lesson seems to imply that offset is better, however, Appendix B1 (Sampling Weights) of the course notes, hints at the fact that in certain circumstances sampling weights may provide an advantage. As an aside, the Enterprise Miner Reference Help (p.189) strongly advises against using sampling weights for predictive modelling. ... View more

Lesson "What Is Separate Sampling?" under "Lesson 7: Model Assessment Using SAS Enterprise Miner" (module 3: Predictive Modeler using SAS Enterprise Miner) seems to imply that oversampling is done for efficiency reasons only, with minimal impact on the resulting model. However, based on some tests I have done using dataset "INQ2005", the use of oversampling vs the full population can have a significant impact on the final model. In particular, the comparison of two Decision Tree models (full dataset with primary proportion = 3.15% vs oversampling with primary proportion = 50%) has shown the following results:   Different optimal sub-trees are selected (based on Average Square Error); the model based on the full dataset results in a subtree with 26 leaves vs 16 from the model based on oversampling Difference in (some) splitting variables selected; this is also confirmed by differences in the list of variables reported under "Variable Important" More importantly, the performance (as measured by ASE) of the model based on the full sample shows a marked divergence between training and validation datasets (pointing to overfitting) compared to a more stable dynamic on the oversampled model   Overall, my taking on this is that oversampling is not just a matter of making the whole process more efficient but it leads to "better models" in the sense that, using a balanced sample (i.e. 50/50 split between primary and secondary outcome) seems to help the model to give equal importance to positive and negative cases resulting in more stable models (i.e. with less overfitting).   I would appreciate to hear other opinions on the above. Thanks   ... View more

In "Lesson 7: Model Assessment Using SAS Enterprise Miner" (of Module 3), topic "Calculating Expected Profit" seems to imply that using a Profit Matrix results in Enterprise Miner calculating a new threshold for making a prediction decision (text from the online lesson: "When the elements of the profit consequence matrix are constants, SAS Enterprise Miner makes prediction decisions solely based on the estimated probability of response and a constant decision threshold") However, based on some tests I have done with Decision Trees, from displaying the maximal tree and looking at field "Predicted Value" in each node (as displayed on the Tree window), it looks to me that the classification of a node (i.e. the assignment to either 1 or 0) is solely based on the posterior probabilities (adjusted for prior probabilities - if specified) and the default cut-off of 0.50. Said in other words, the definition of a Profit Matrix does not change the way nodes/leaves are classified, but it only affects the selection of the best sub-tree which is done based on metric Average Profit; such optimisation is carried out only if property Assessment Measure is set to "Decision" (and, of course, Method = Assessment). Is my understanding correct? ... View more