Hi All,
The model I am building has a low response rate, what are the characteristics of a Good Model in this case...It is clear that there is no point to look at the accuracy because it will be high anyway. AUC value,Gini,Lift Curve are they still indicators of a good model? Or is there any better way, to check the model perfomance in this case?
Many Thanks
How big is your dataset? You can resample your data so that the response rate is a better ratio, such as 50/50 and then establish priors to correct the parameter estimates.
Sadly I don't remember the term for this methodology.
If you are saying that 0.2% (1 out of 500) respondents actually respond, then I would first check to see if the non-response is random, or not. If it is not random, then there's no point in examining goodness of fit indicators, your model is worthless. If it is random, I'd still be very worried about using a model where you have such a high non-response rate, but who knows, if the sample size (not the non-response rate) is large enough, then maybe you might have something. It's hard to say without actually knowing the details of your situation and the data itself. I don't really think there is a canned answer in this case.
Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.
Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.
Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.
Find more tutorials on the SAS Users YouTube channel.