topic Re: Partial Dependence Plot for boosting decision tree in SAS Data Science

Partial Dependence Plot for boosting decision tree

Art804 — Mon, 30 Nov 2015 04:07:15 GMT

Hi all

I'm wondering how I can have partial dependence plot when I'm using boosted decision tree?

Thanks

Re: Partial Dependence Plot for boosting decision tree

JasonXin — Tue, 01 Dec 2015 01:47:07 GMT

Hi,

By boosted decision tree, you mean the Gradient Boosting node in EM? Jason Xin

Re: Partial Dependence Plot for boosting decision tree

Art804 — Tue, 01 Dec 2015 02:30:55 GMT

Hi Jason

Thanks for reply. No, I'm developing my boosted tree using start and end group nodes in EM.

Re: Partial Dependence Plot for boosting decision tree

JasonXin — Tue, 01 Dec 2015 03:16:34 GMT

In that case, I am afraid you need to stick a SAS Code node to write your own programs. Best Regards
Jason Xin

Re: Partial Dependence Plot for boosting decision tree

JasonXin — Tue, 01 Dec 2015 18:26:48 GMT

Hi,

In case you have not seen, there have been some SAS programs that do boosting which I think similar to what you try to do. Here is a link found at Google.com.
http://www.sas-programming.com/2010/03/implement-boosting-algorithm-in-sas.html

Re: Partial Dependence Plot for boosting decision tree

Art804 — Tue, 01 Dec 2015 22:55:14 GMT

Hi Jason

Many thanks for that, however I could not find my answer in that link. I'm wondering I can extract partial dependence plot in R easily but in SAS ...

It is so frustrating for me that I'm using SAS EM to develop my models in my PhD thesis and now I have to come back to R.

Re: Partial Dependence Plot for boosting decision tree

M_Maldonado — Tue, 01 Dec 2015 23:27:53 GMT

Hi Art,

I looked into the partial dependence plot (2D and 3D versions) for gradient boosting and random forest about a year ago. I was not particularly impressed. It seemed useful when you have 2 or 3 variables, but I wasn't sure where that leads you when you have 4+ variables.

Since all partial dependence takes into account is "marginal effect of a variable on the class probability (classification) or response (regression)", I would much rather look at the variable importance coming out of the gradient boosting node.

If you have more insights about these plots, I will be happy to bring this up in our next development meeting. I am specially interested if these plots are something you would use in a real data set with 4 or more variables.

Thanks!

-Miguel

Re: Partial Dependence Plot for boosting decision tree

Art804 — Fri, 04 Dec 2015 04:38:12 GMT

Hi Miguel

Thanks for your reply. But Partial dependence plot can be used when you have more than 3 variables as well. Partial dependency assists in identifying interaction between different variable in model and have a better interpretation. For example in my study (traffic crash study) using importance variable shows that population density is a significant factor, however how I can find in flouncing of this variable on model. I mean, it is not clear increasing population density increased traffic crashes or decreased it. I know it is possible to find it in SAS model code but it is difficult and time consuming. (for instance

http://onlinepubs.trb.org/onlinepubs/conferences/2011/RSS/1/Chung,Y-S.pdf )

mnay thanks

Alireza

Re: Partial Dependence Plot for boosting decision tree

M_Maldonado — Fri, 04 Dec 2015 18:18:11 GMT

Thanks for the details, I will check out that paper and figure out if you can use a workaround to calculate them when you use Start/End group nodes.

Some input from one of my most tree-versed coworkers:

These plots are definitely useful although they might be misleading when the variable on the X-axis is strongly dpendent with other variables.
This video discusses the problems with them and an ICE method to resolve them. [I haven't looked at it entirely, but wanted to pass it along ASAP].

https://www.youtube.com/watch?v=f55onMzbmfY

Stay tuned and let's see if myself or someone from the community can come up with something.

Cheers,

Re: Partial Dependence Plot for boosting decision tree

RyDan — Thu, 20 Sep 2018 15:03:35 GMT

How can you get a partial dependence plot out of the Gradient Boosting node in EM ?

Re: Partial Dependence Plot for boosting decision tree

RyDan — Tue, 20 Nov 2018 15:58:38 GMT

Found out some good information on this here

https://blogs.sas.com/content/tag/ai-interpretability/?utm_source=LinkedIn&utm_medium=social-voicestorm&utm_content=6ee61e79-c47b-4192-a8c0-41465d86c3e1