SAS Support Communities

With regard to chapter 7 of the course notes: 1. What is the difference between "SAS Code" and "Optimized SAS Code" (see page 7-8)? My Answer: SAS code: Base SAS code associated with the process flow diagram used in building the predictive model. Optimized SAS code: After excluding all redundant SAS codes associated with the variables not included in the final model, the optimized SAS code only includes BASE SAS code needed to score the new scoring data. My Answer: 2. Is it correct to say that when scoring a Score table, Enterprise Miner assumes the table is a true representation of the population in terms of proportions of events/non-events? My Answer: Yes this is  a correct assumption As such, no adjustments based on prior probabilities are applied; however, Decisions, in terms of Expected Profit and Decision classification, are based on the Decision Weights and Prior Probabilities specified for the data source used for training My Answer: This is incorrect.  Because if over or separate sampling was practiced during the model development, the SAS code for future scoring contains code for  proper posterior probability  adjustments, and computing expected and average profits.  3. With regard to Decisions, is it possible to use property Decision of the Input Data node related to the Score table (see page 7.7) to amend the decision weights to be applied during scoring? If so, would that require a target variable present on the Score dataset? My Answer: Yes, this step is automatically performed and this adjustment is included in the optimized Base SAS code. ... View more

I have a couple of questions on Cluster Analysis (chapter 8 of course notes): 1. In what scenarios should categorical variables, via dummy indicators, be used for Clustering? Or would it just be better to use interval variables as suggested by the course notes at page 8-9? ("An interval measurement level is recommended for k-means to produce non-trivial clusters") My Answers: For K-means and Hierarchical clustering  interval variables are recommended. SAS HP cluster node also can perform ABC clustering based on Manhattan distance. For this option you can also include dummy variables from a categorical var. 2. In what instances would a Range Standardisation (with reference to property "Internal Standardization") be recommend in place of the usual standardisation (i.e. subtracting the mean and dividing by the standard deviation)? My answer: For K-mean clustering and PCA , Z-standardization is preferred. For some special NN machine learning algorithm Range-normalization may be preferred. ... View more

1. Gain: this metric is reported in the Output window (under "Statistics Table") of the "Model Comparison" node (see page 6-7 of course notes); what is its formula/definition? Is this based on the definition given at page 256 of "Enterprise Miner 15.1: Reference Help": ((% of events in decile / random % of events in decile)-1). If so, what is its interpretation? MY ANSWER: Both LIFT and GAIN statistics are computed at the depth of 10th decile (by default) and Gain=Lift-1. The formula given above is correct for the Gain. 2. Gain Chart: this is the chart displayed as part of the "Score Ranking Overlay" output when selecting option "Gain"; below is a screenshot taken from the example/demonstration in "Lesson 7: Model Assessment Using SAS Enterprise Miner" (see also page 6-19 of the course notes); again, how are the values on the Y-axis calculated? My answer: The values Y-axis is Lift-1 3. Cumulative Gain: page 6-17 of the course notes states that "cumulative percent response" chart is more widely known as "cumulative gain" in the predictive modeling literature. It also adds that "[...] Plotting cumulative gain for all selection fractions yields a gains chart"; at page 6-20, it says "It is instructive to view the actual proportion of cases with the primary outcome (called gain or cumulative percent response) at each decile": (a) from other sources on internet (see this as an example), it seems that "cumulative gain" is related to the "percentage of the total possible positive responses (i.e. primary outcome events) at a given depth" (in the "Score Ranking Overlay" window, that is given by "Cumulative % Capture Response"); is this just an example of inconsistency in the use of the same term? (b) how does the "cumulative gain" differ from the Gain Chart in point (2) above? My answer: Cumulative gain is equal to Cumulative %  Response, Therefore SAS EM is only showing Cumulative %  Response. Please note that Cumulative % Capture Response = (Cumulative % of events in a decile / total number of events) is different from Cumulative %  Response = (Cumulative % of events in a decile). Please let me know if you have any further questions.   ... View more

Meaning of "Computed": This is the default setting for the replacement value. By default the replacement node identify outliers (mean +- 3 SD) for interval inputs and compute this truncated values (Top ceiling) and replaces globally for all inputs. If you want to apply this option only for the selected input variable then you need to click the replacement editor and apply the required setting there. Instead of using this computed value (Mean +- 3sd) you could also use a user-specified constant value in the global setting or in the replacement editor individually for the selected inputs. Similarly you can also choose 'Missing'. In the course demo, we want to replace all zeros only in DemMedIncome variable. Therefore using the replacement editor we selected DemMedianincome and  specified 1 as the lower replacement value to identify all the records with 0 income  and replaced with missing. I hope this explanation is clear. If you need further clarification please let me know. Also adding a screenshot of the replacement editor window.   ... View more

Thank you for asking this question and for further clarification. When computing for average profit, SAS EM always adjust the frequencies of primary and secondary outcome events for prior probabilities  when the prior probability adjustment is enabled in the decision setting (Please see page 6:24 course PDF). To illustrate this point I have modified the corresponding slide below: Hope this explanation is adequate. If not please let me know. ... View more

The surrogate model example is providing a solution to assess the variable importance of the neural net (black box model). If you want to make decision or prediction use the posterior probability values derived from the NN model directly.  Because the posterior Probability reported in the surrogate decision tree model is not adjusted for over-sample or priors. Therefore, in the course notes in case you need to use the posterior probabilities from the surrogate model, they provide the following solutions: 1) Use the scored data where event distribution reflects what is available in the reference population. 2) You could also use SAS code editor in EM and adjust for priors and decision weights (Not in the course notes) 3) In the transform node there is a rudimentary SAS code option where  you create a weight variable (based on prior probability values) an assign a role of frequency. That way you can adjust the posterior probability for priors. Please note this weight option is different from survey design weights and SAS EM is not meant for using survey data. It is  recommend for building predictive models. ... View more

If you choose the validation error but no validation data then the model selection is based on the selection method (Forward, backward or step wise settings) . If  the selection method specified is none then full model will be selected. ... View more

Many thanks for providing this page number and additional details. Currently no default setting is available to modify the default number of bins (10). Therefore at this point we have to manually adjust this number.    ... View more

I agree with your comments because adjusting for prior probability basically only shifting the intercept values. Therefore this should not affect the model selection. However, if you want the prior values affect your model decision you should consider the decision option and provide decision weights (Please refer Chapter 6 in the AAEM course notes) ... View more

I hate to provide rule of thump when optimal solutions for most options are data specific. However, for this case SAS Enterprise miner advanced metadata advisor is using 20 as the categorical levels threshold to reject the nominal variable. You could consider this default setting as the best rule of thump. ... View more