Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

Model Performance with a low reponse rate : 0.2% Please Help. Thank you

Reply
Frequent Contributor
Posts: 95

Model Performance with a low reponse rate : 0.2% Please Help. Thank you

Hi All,

The model I am building has a low response rate, what are the characteristics of a Good Model in this case...It is clear that there is no point to look at the accuracy because it will be high anyway. AUC value,Gini,Lift Curve are they still indicators of a good model? Or is there any better way, to check the model perfomance in this case?

Many Thanks

Super User
Posts: 19,768

Re: Model Performance with a low reponse rate : 0.2% Please Help. Thank you

How big is your dataset? You can resample your data so that the response rate is a better ratio, such as 50/50 and then establish priors to correct the parameter estimates.

Sadly I don't remember the term for this methodology.

Trusted Advisor
Posts: 1,908

Re: Model Performance with a low reponse rate : 0.2% Please Help. Thank you

If you are saying that 0.2% (1 out of 500) respondents actually respond, then I would first check to see if the non-response is random, or not. If it is not random, then there's no point in examining goodness of fit indicators, your model is worthless. If it is random, I'd still be very worried about using a model where you have such a high non-response rate, but who knows, if the sample size (not the non-response rate) is large enough, then maybe you might have something. It's hard to say without actually knowing the details of your situation and the data itself. I don't really think there is a canned answer in this case.

Ask a Question
Discussion stats
  • 2 replies
  • 257 views
  • 0 likes
  • 3 in conversation