topic Assistance with SAS code for SMOTE and adaptive synthetic sampling algorithms in Statistical Procedures

Assistance with SAS code for SMOTE and adaptive synthetic sampling algorithms

mnasila — Thu, 17 Mar 2016 20:09:11 GMT

Dear all,

I am developing a predictive model for a data-set that has very imbalanced dependent variable. The ratio between the two categories of the dependent variable is 47500:1. I am exploring SMOTE sampling and adaptive synthetic sampling techniques before fitting these models to correct for the bias created by the imbalance. I mostly use SAS eguide but also comfortable with SAS enterprise miner. Has anyone used these sampling algorithms in SAS? I would appreciate assistance regarding coding these sampling techniques, I would also be happy if anyone would recommend any classification technique/s that would fit this problem. thanks in Advance.

regards

Re: Assistance with SAS code for SMOTE and adaptive synthetic sampling algorithms

sbxkoenk — Thu, 17 Mar 2016 22:03:52 GMT

SMOTE described here

http://support.sas.com/resources/papers/proceedings15/3282-2015.pdf

and ZIP containing SAS code

http://support.sas.com/resources/papers/proceedings15/3282-2015.zip

Two recent SAS papers from customers deal with / apply SMOTE.

SMOTE = Synthetic Minority Over-sampling TEchnique

Paper 3483-2015

Data sampling improvement by developing SMOTE technique in SAS

Lina Guzman, DIRECTV

http://support.sas.com/resources/papers/proceedings15/3483-2015.pdf

Paper 3282-2015

A Case Study: Improve Classification of Rare Events with SAS® Enterprise Miner™

Ruizhe Wang, GuideWell Connect; Novik Lee, GuideWell Connect; Yun Wei, GuideWell Connect

A rather novel technique called SMOTE (Synthetic Minority Over-sampling TEchnique), which has achieved the best result in our comparison, is discussed.

http://support.sas.com/resources/papers/proceedings15/3282-2015.pdf

Koen

Re: Assistance with SAS code for SMOTE and adaptive synthetic sampling algorithms

mnasila — Tue, 22 Mar 2016 20:20:20 GMT

thanks Koen for the papers, will go through them.

Re: Assistance with SAS code for SMOTE and adaptive synthetic sampling algorithms

mnasila — Wed, 23 Mar 2016 12:32:39 GMT

Hi Koen,

I am testing the SAS code on my dataset. when i get to the data _NULL_ step (Generating random cases with look up table) all the new cases generated from this step have missing observations. is this unusual? how do i go about fixing it? thanks.

Re: Assistance with SAS code for SMOTE and adaptive synthetic sampling algorithms

sbxkoenk — Thu, 24 Mar 2016 16:33:48 GMT

Hello,

I haven't tested the code accompanying the paper.

It's best to turn to the authors.

See the last page of the paper. It says:

CONTACT INFORMATION

Your comments and questions are valued and encouraged. Contact the author at:

Cheers,

Koen