Question about Gradient Boosting on large dataset_SAS EM

YG1992 — Tue, 30 Jan 2018 09:29:03 GMT

Hi everyone,

I have got several large populations with millions of observations and my task is two-class classification. If I apply gradient boosting directly on the whole dataset (train:validate = 70:30), then I always get 0.5 AUC and always the same predicted probabilities of class 1 and 2 for each observation; if I draw a sample of 100k or 200k first and apply gradient boosting with same hyper-parameter settings, the results are relatively normal with some AUCs higher than 0.5 and different probabilities of class 1 and 2 for each observation.

I would like to ask some SAS EM programmer here: could you please explain this situation? I guess the algorithm just stops updating any parameters at the very beginning but I don't no the exact and concrete reason. Last but not least: there is no error message when running GBDT for both large and small datasets.

Thank you very much.

topic Re: Question about Gradient Boosting on large dataset_SAS EM in SAS Data Science

Question about Gradient Boosting on large dataset_SAS EM

Re: Question about Gradient Boosting on large dataset_SAS EM