Stochastic Gradient Boosting

[dodger]

Hi,

Is there any HP procedure for Stochastic Gradient Boosting? (ex: for proc logistic - there is proc hplogistic)
How can we check whether SAS license we have includes HPA or not?
With HPA capability, is there any improvement if SGB can run much faster?

Thanks a lot.

[ChrisHemedinger]

You probably want an answer that you can apply today, and I hope someone here will respond with that.  Have you seen this paper about ensemble models in SAS Enterprise Miner?  It includes a section about gradient boosting (PROC TREEBOOST).

In the near future, SAS Viya Data Mining and Machine Learning will offer PROC GRADBOOST. You can read about it here in this early preview group.  The SAS Viya platform supports in-memory processing across multiple cores.  Hope that gives you a glimpse of what's coming...

[WendyCzika]

Just want to follow up on Chris' reply... the paper he refers to mentions the Gradient Boosting node in SAS Enterprise Miner that you can use, but there is currrently no HP gradient boosting procedure or node.  

[JasonXin · Accepted solution]

Hi, The procedure that runs behind EM's non-HP GB node is proc treeboost. That is not a HP procedure. If you have EM license, your work at EM GUI is supported by SAS technical support. But your usage of proc treeboost is not officially supported. Depending on specific motivation to want HP version of GB, proc treeboost can still have some different benefits; some EM users have told they simply like 'line-coding' better than GUI. (i have a blog that has syntax of proc treeboost) No, SAS HPDM package does not have HP GB procedure or EM GUI node. Unlike random forest that has HPFOREST + a separate HP Forest node in HPDM. The Stochastic Gradient Boost node has been in regular, non-HP EM for many years, while random forest was added only several years ago. HP Forest was originated into EM in HP mode. Therefore, the evolution paths for GB and RF in EM are different. In the latest SAS high performance, in-memory offering Viya, a GB procedure is included to run in distributed mode on big data sets. Very scalable, if scalability is why you asked this question. Viya also supports calling SAS API from languages like Python, Lua or Java. That means Viya has potential (not guarantee) to support some custom GB implementations. Hope this helps? Thank you for using SAS. Jason Xin