I am building a logistic reg model and all is fine and I got a model which is predicting quite good.
My problem is that I used 40+ variables and model choose around 19 variable in the final outpout. Isnt it too much to have 19 variables in the final model? I know it varies by business but the models I built earlier never had more than 8-9 variables in the final output. The model is predicting quite well on 2 different types of Validation datasets so it dose not seem over fitted to me.
Another question: I have a variable Product in the final model which is highly predictive and it has three categories (three types of products ). I tried to break this model into three different models - one for each product. However, my overall model (where product is a variable) is more predictive than any of three individual models. Shall I keep only one model or it makes sense to break it into three different models - one for each product?
Many thanks as always.
With that many observations your probably fine. You can also consider reducing your significance level for less variables and see how it affects your accuracy. Typically it's 0.05, but because you have so much data many things will be statistically significant even though not practically.
Have you set prior probabilities since your event rate is so low?
How does your event rate change for the three levels that your considering stratifying on? Have you considered a stratified model instead of separate models?
Hi, you are rightI should increase the sig level as the sample size is huge. I havnt done the prior probabilities or Stratified Regression. I will read about them more.
If my model is tested on two different validation data sets and is predicting same output, shouldn't it be fine? I am getting .7 C statistics (70% concordant pairs) in trainging as well as validation dataset.
Thanks for all the replies.
Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.
If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website.
Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.
Find more tutorials on the SAS Users YouTube channel.