BookmarkSubscribeRSS Feed
anu1999
Obsidian | Level 7

Hi,

I am working with a data with 50+ variables as input to the model. Most of the input variables have the right skewed distribution heavy at 0 value. I know log transformation is used for right skewed data but in my case when I have high concentration of zero, it wont help.

I was wondering if you have any suggestions based on your past experience on how you dealt with such data for modeling.

Thanks
A

3 REPLIES 3
Ksharp
Super User

Did your data confirm Possion distribution ? 

You could check Zero Inflation Model.

anu1999
Obsidian | Level 7

Hi Xia,

 

Thanks for suggestion. After reasearching, it looks like  Zero Inflation model is the right choice here. Do you know if this can be implemented in SAS enterprise miner. 

 

Thanks.

WendyCzika
SAS Employee

You can fit a GLM with the zero-inflated Poisson distribution in the HP GLM node in Enterprise Miner (in releases 13.1 and beyond). But that's for a target that has many 0's.  For inputs that are skewed, you could still use a Log transformation, just need to add a constant to the variables first to be able to log 0.  The Transform Variables node in EM can do the log transformation and will automatically add a constant.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to choose a machine learning algorithm

Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 3 replies
  • 2109 views
  • 0 likes
  • 3 in conversation