topic Re: Solution for imbalanced data, categorical target, 98% weights on 0 outcomes. in SAS Data Science

Solution for imbalanced data, categorical target, 98% weights on 0 outcomes.

akmsharif — Mon, 10 Aug 2020 06:14:36 GMT

I am trying to do logistic regression, decision tree, KNN & neural network on a dataset where I have 9800 rows, the target is binary and 98% 0. I have 1000 interval predictors, all the variables have many 0s and not normal distributions. How should I approach to handle the imbalanced data in SAS Miner for each of the models? Can somebody pls help?

Re: Solution for imbalanced data, categorical target, 98% weights on 0 outcomes.

Ksharp — Mon, 10 Aug 2020 11:24:05 GMT

Oversample , make good:bad about 3:1 or 4:1 .

Re: Solution for imbalanced data, categorical target, 98% weights on 0 outcomes.

akmsharif — Mon, 10 Aug 2020 21:12:33 GMT

Can you please let me know a bit detail?

Re: Solution for imbalanced data, categorical target, 98% weights on 0 outcomes.

janex — Tue, 11 Aug 2020 03:56:18 GMT

Could you please take a look at the question of mine that I posted in my page @Ksharp ? Thank you very much.

Re: Solution for imbalanced data, categorical target, 98% weights on 0 outcomes.

Ksharp — Tue, 11 Aug 2020 11:54:18 GMT

Due to every small event probability , any model would not be trusted.

Oversample stands for enhancing event prob, if you have 1000 obs only 10 obs is 1,you need randomly sample 30 or 40 from the remain 990 obs which is 0 to form a train data to model . a.k.a 1:0 is about 1:3 or 1:4 .

if you are using PROC LOGISTIC ,don't forget to use PEVENT=0.01 to adjust predicted prob .

And @Rick_SAS maybe have good ideas.

Re: Solution for imbalanced data, categorical target, 98% weights on 0 outcomes.

akmsharif — Wed, 12 Aug 2020 04:06:40 GMT

@Ksharp, @Rick_SAS:
I am using SAS MIner.. How should I apply the PEVENT function there?
Thanks.

Re: Solution for imbalanced data, categorical target, 98% weights on 0 outcomes.

Ksharp — Wed, 12 Aug 2020 11:23:48 GMT

If I was right, left click the 'logistic' node ,and at left panel you could find an option PEVENT