BookmarkSubscribeRSS Feed
sachin01663
Obsidian | Level 7

Hi,

 

I am building a logistic reg model and all is fine and I got a model which is predicting quite good.

 

My problem is that I used 40+ variables and model choose around 19 variable in the final outpout. Isnt it too much to have 19 variables in the final model? I know it varies by business but the models I built earlier never had more than 8-9 variables in the final output. The model is predicting quite well on 2 different types of Validation datasets so it dose not seem over fitted to me. 

 

Another question: I have a variable Product in the final model which is highly predictive and it has three categories (three types of products ). I tried to break this model into three different models - one for each product. However, my overall model (where product is a variable) is more predictive than any of three individual models.  Shall I keep only one model or it makes sense to break it into three different models - one for each product? 

 

Many thanks as always. 

 

Regards

Sachin 

7 REPLIES 7
Reeza
Super User

How many observations do you have?

Ksharp
Super User

1) You can use PROC HPGENSELECT to check how many variables you should retain again.

2)Maybe you should consider about Mixed Logistic Regression,Make product as a mixed effect.

Check PROC GEE or PROC GLIMMIX

sachin01663
Obsidian | Level 7
@Reeza : Around 500 K. Response variable is around 3%. For some variables, Response will be less.

@Ksharp: Let me check these Procs. Thanks
sachin01663
Obsidian | Level 7
@Reeza: Validation datasets had around 1 m each on which the model is working
Reeza
Super User

With that many observations your probably fine. You can also consider reducing your significance level for less variables and see how it affects your accuracy. Typically it's 0.05, but because you have so much data many things will be statistically significant even though not practically. 

 

Have you set prior probabilities since your event rate is so low? 

 

How does your event rate change for the three levels that your considering stratifying on? Have you considered a stratified model instead of separate models? 

sachin01663
Obsidian | Level 7

Hi, you are rightI should increase the sig level as the sample size is huge. I havnt done the prior probabilities or Stratified Regression. I will read about them more.

 

If my model is tested on two different validation data sets and is predicting same output, shouldn't it be fine? I am getting .7 C statistics (70% concordant pairs) in trainging as well as validation dataset. 

 

Thanks for all the replies. 

Sachin

sachin01663
Obsidian | Level 7

Yes, the event rate changes a lot. Product A has 2.5% , B has 4% and C has 3%. This is the reason I wanted to build different models becuase oneproduct has very high event rate so its highgly correlated. 

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

Multiple Linear Regression in SAS

Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 7 replies
  • 986 views
  • 2 likes
  • 3 in conversation