Model Performance with a low reponse rate : 0.2% Please Help. Thank you

[Kanyange]

Hi All,

The model I am building has a low response rate, what are the characteristics of a Good Model in this case...It is clear that there is no point to look at the accuracy because it will be high anyway. AUC value,Gini,Lift Curve are they still indicators of a good model? Or is there any better way, to check the model perfomance in this case?

Many Thanks

[Reeza]

How big is your dataset? You can resample your data so that the response rate is a better ratio, such as 50/50 and then establish priors to correct the parameter estimates.

Sadly I don't remember the term for this methodology.

[PaigeMiller]

If you are saying that 0.2% (1 out of 500) respondents actually respond, then I would first check to see if the non-response is random, or not. If it is not random, then there's no point in examining goodness of fit indicators, your model is worthless. If it is random, I'd still be very worried about using a model where you have such a high non-response rate, but who knows, if the sample size (not the non-response rate) is large enough, then maybe you might have something. It's hard to say without actually knowing the details of your situation and the data itself. I don't really think there is a canned answer in this case.