topic Re: Extreme gradient boosting in SAS in SAS Data Science

Extreme gradient boosting in SAS

munitech4u — Mon, 18 Apr 2016 08:10:53 GMT

Is there a way we can tweak the GBM in sas EM to implement extreme gradient boosting algorithm?

Further, what is the best way to control overfitting in GBM using EM?

Re: Extreme gradient boosting in SAS

FriedEgg — Mon, 18 Apr 2016 19:15:29 GMT

It could be done using the Open Source Integration Node. Extreme Gradient Boosting is not something available from SAS, currently. It is certainly something I hope they add in the near future though.

EDIT:

I feel like pointing out here that the main appeal of XGBoost is it's performance from an engineering standpoint, rather than statistical. This is what really sets this pacakage apart from the GBM from SAS.

As far as overfitting is concerned, two traditional methods would be reducing the number of iterations as well as adjusting your subsample size.

Re: Extreme gradient boosting in SAS

rogerjdeangelis — Mon, 18 Apr 2016 15:12:53 GMT

There are pakages in R to do this, not sure about Python

If you have IML there is a interface to R, also WPS has an interface.

You might check xgboost.

I am out of my comfort zone on this reply

xgboost: eXtreme Gradient Boosting

However, I would switch to SAS when it is available, as long as SAS nakes it part of stat. The R packages are the wild west of programming and this is not

like the Atkinson and Whittaker functions in previous posts. You can examine the R source code.

Re: Extreme gradient boosting in SAS

FriedEgg — Thu, 21 Apr 2016 02:34:32 GMT

SAS Viya includes a distributed gradiant boosting proc which implements a very similar algorihm to xgboost

Re: Extreme gradient boosting in SAS

JasonXin — Thu, 21 Apr 2016 04:13:17 GMT

Hi, With Viya, it is possible for you to submit R or Python models to run in-memory by some API facilities, meaning you may not have to sample it down to test your GB or XGB. ~Q3 of 2016, you should see first batch of Viya ML released. That contains a Xboost. I don't think that has regularization. One better way to curtail over-fiting is actually just to score the model on external data sets and focus on 'prunning'. This is more analytical than rolling around regularization. Regularization, my personal opinion, is 'lazy man's trick. If you are worried about being replaced by machines in the future, then try to resort to regularization less. Regularization 'performance' on large simulated data should not prepare you towards believing the performance shall sustain onto wide, tall table that contains true complex (human) relationships. Hopefully, when you get on platforms like Viya, you will see for yourself. Return to EM. 1. EM does not have Xboost. 2. Recommend not to use OS integration node. If your true intention is to incorporate R GB models, score the R model, use Model Import Node. The node only requires that you provide the (same) target variable + the R model score. Then the Model Comparison node will include the R model in its performance grid.

Re: Extreme gradient boosting in SAS

munitech4u — Thu, 21 Apr 2016 09:58:40 GMT

hmm.. So SAS Viya seems to be the game changer. I was hoping that SAS release some open source thing in future.

Re: Extreme gradient boosting in SAS

RicardoGalante — Sun, 05 Mar 2017 20:25:09 GMT

Hello Experts,

Do you have an example of how running a R code with Extreme Gradient Boosting (xgboost) is SAS EMiner using the Open Source Integration node?

Thanks a lot!