Hi,
I am working with a data with 50+ variables as input to the model. Most of the input variables have the right skewed distribution heavy at 0 value. I know log transformation is used for right skewed data but in my case when I have high concentration of zero, it wont help.
I was wondering if you have any suggestions based on your past experience on how you dealt with such data for modeling.
Thanks
A
Did your data confirm Possion distribution ?
You could check Zero Inflation Model.
Hi Xia,
Thanks for suggestion. After reasearching, it looks like Zero Inflation model is the right choice here. Do you know if this can be implemented in SAS enterprise miner.
Thanks.
You can fit a GLM with the zero-inflated Poisson distribution in the HP GLM node in Enterprise Miner (in releases 13.1 and beyond). But that's for a target that has many 0's. For inputs that are skewed, you could still use a Log transformation, just need to add a constant to the variables first to be able to log 0. The Transform Variables node in EM can do the log transformation and will automatically add a constant.
Don't miss out on SAS Innovate - Register now for the FREE Livestream!
Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.
Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.
Find more tutorials on the SAS Users YouTube channel.