Hi, To the original question 1. if you are trying to score the tree on a new data set, you will not get a tree, even if you are scoring it inside EM. Because the new scoring operation does not automatically yield a target that can anchor a graph like we typically see in EM tree building process. 2. Sounds like you just want to replicate the tree graph in EM to EG environment. No, it does not happen automatically. A similar solution has been posted for proc HPSPLIT here. You can take the steps that consume the output from Proc HPSPLIT, attach to the end of your tree code to see if you can produce a graph. Some adjustments should be expected. https://communities.sas.com/t5/Mathematical-Optimization/Drawing-a-decision-tree-from-HPSPLIT/m-p/232799#U232799 Hope this helps? Jason Xin ... View more

Under ASSESS, there is a Decisions Node. If you have a multi-class target, it should be populated there for you to customize the proportions. I am not in front of EM, but recall there is a Custom Editor you can click. Jason Xin ... View more

Hi, I see what you are saying. Your original question was within the scope of EM. Since you did not indicate EM in your question, I thought you were asking about something else CALIS. Lift in the Association Node is based on the fact the unit of analysis there is transaction. Lift concept is to see how much 'stimulus' the presence of A provides to boost transactions of B. If you sell diapers alone, you sell 50 one night. Now you put beer by the diaper in your store, sales of diapers go up to 100. So the beer is lifting the diaper by 100%? See the lift of beer? The exercise at the Path Analysis node essentially is to count paths. One could observe along one specific path which path is most likely to follow next, and after that which path is to follow the second, like a tree branching out. Based on this weight of descriptive evidence, one can infer a little bit which are the most likely path combinations. But this exercise essentially is just count inference. In predictive modeling or analytics, wherever a lift concept is engaged, there is always a control/treatment situation. The 'tree branch' nature of the path analysis simply does not study that relation pattern. Therefore no lift is defined. In typical score models, you break down model universe by scores into, say 10 groups. Then you compare random performance against model treated performance by the groups. The lift defined there is genuine because each group being assessed lift is identical (and fully controlled); it is a real per-group measurement. In the Association node, the element of marginal lift is borrowed/stolen from the genuine lift concept to measure the impact from presence of A on purchase of B. But there is no way the group that purchases B is equal to the group of purchasing A and B. The group underlying the 'lift' is by definition not the same, therefore by definition of the association study, not controlled. So they have to put the word 'pseudo' in front of it.. Hope this helps? Thanks. Jason Xin ... View more

This helps a lot when customers have long list of predefined column features that they want to force into EM, from the sources such as Hive meta data store. One thing we have found that we should be careful about is the scope. Some are still getting used to what the word GLOBAL means here. But so far more benefits than harms. Thanks Jason Xin ... View more

By path analysis, you mean what procedure CALIS does or something else? Thanks. Jason Xin ... View more

Hi, The original question appears a bit vague 1. "the order in any display for SAS EM" : what does this mean? Does this mean under Edit Variable 'meta data' layer or elsewhere? 2. "in the orginal data set": if you are referring to the original input data set, the chance it will never be displayed the same as one moves along the EM flow. You mean the data set right before the node in action? I suspect you are talking when you input the data set into EM, you already have spent time settling the meta data, roles, types,... you don't want EM to act on it, to twist, to turn.. in ways that you (frankly speaking most people) don't know. If this is the case, it is first of all not a display issue, right? Please clarify. Jason Xin ... View more

Hi, 1. If you are into forecasting mind set, yes you can retain those variables in your model. We could call them autoregression. 2. Back to linear regression kind of predictive models, a 'taboo' in practice is NOT to include any variables used in defining the target. in most cases it means variables with the same names.Sometimes it refers essentially the same variables with different names. 3. After excluding these variables, variables, as described in the original questions, can be highly correlated to the defined target. That per se is not problem. Another subject is they are highly correlated among themselves. You can either pick one of them or load them into factors or components. Hope this helps? Jason Xin ... View more

Hi (I am sorry for delayed response. I don't come here often) I was not suggesting anything. I was just trying to help explain a symptom you described in your original post. I would concur boosting should work on over-sampled data. However, use frequency or weighting carefully, in building and judging the resulting model: it is unusual that one uses over-sampling without somehow 'putting it back', for various reasons. It is not obvious to many people that going from surrogating under regular decision tree modeling to surrogating under GB is actually a leap-forward, a leap that requires some fundamentally different expectations for surrogating. When building a regular DT, you are looking into a painted vivid tree graph, while GB kind of modeling where surrogating happens is where trees are not supposed to be 'SEEN'. In other words, in GB (and in random foresting), you are supposed to run trees in a 'black box'. It is meaningless to "can I take a look at one of trees?" in GB or RF , unlike how you normally should approach a regular DT . Against this general backdrop, how are we supposed to engage surrogating? This is one of the dim light on machine learning methods: you don't have diagnostic tools and facilities. We may have some new versions in the future on GB. ... View more

Hi, "Surrogate Rule=" interacts with Missing Value option in either DT or the GB node. When surrogate rule =0 (default in the GB node), Missing value's default kicks in, which is "Use in Search" for both DT and the GB nodes. "Use in Search" does not mean carrying the missing value forward. Instead, it imputes it, implicitly and legitimately, with rules like "grouping it with the branch that maximizes the worth of split". Now, when you set surrogate rule=2, in both DT and GB nodes, you tell EM to override the "Missing Value" default (="Use in Search"). When surrogating kicks in (data driven, no guarantee it will), AND it cannot find non-missing surrogates up to 2 levels, you get a missing value assigned to the non-leaf node (in the 'middle stream'). That missing status in DT does not create problem. If the DT gives you a terminal node that has missing value in it, so what? In (most) GB cases where max branches are relatively large (assuming other stopping rules do not stop spliting prematurely), say 100, a non-leaf node born at, say, 7th branch/level, carrying missing value, may get great chance to be re-surrogated-2 successfully at depth 78 for example (although carrying missing values down the path from branch 7 all the way down to 78 may very well be deemed as analytically unacceptable). However, the chance (see you only give us max depth=10) is the missing value gets to the very bottom layer of the trees. Again, this does not pose problem for DT. However, GB needs to build loss function to re-iterate. How are we supposed to build a loss function with missing input? We don't put rules in GB engine to tell it to drop the OBS due to missing loss function (that is sample violation). So the GB halts without a model. The software developer, however, cannot flag it as error because there is no error from design and quality of the product perspectives. Because you told it to surrogate 2. Surrogating should be applied often when the depth is deep where the surrogates are most similar to the missing spot for better info quality, or you have a great 'visual' of the surrounding of the missing. Surrogating rule= is retained today due to historical usage of GB. Historically GB was often used for universes with transparent /small research data, not industrial or noise as we often have today. --Jason Xin ... View more