BookmarkSubscribeRSS Feed
Kanyange
Fluorite | Level 6

Hi All,

The model I am building has a low response rate, what are the characteristics of a Good Model in this case...It is clear that there is no point to look at the accuracy because it will be high anyway. AUC value,Gini,Lift Curve are they still indicators of a good model? Or is there any better way, to check the model perfomance in this case?

Many Thanks

2 REPLIES 2
Reeza
Super User

How big is your dataset? You can resample your data so that the response rate is a better ratio, such as 50/50 and then establish priors to correct the parameter estimates.

Sadly I don't remember the term for this methodology.

PaigeMiller
Diamond | Level 26

If you are saying that 0.2% (1 out of 500) respondents actually respond, then I would first check to see if the non-response is random, or not. If it is not random, then there's no point in examining goodness of fit indicators, your model is worthless. If it is random, I'd still be very worried about using a model where you have such a high non-response rate, but who knows, if the sample size (not the non-response rate) is large enough, then maybe you might have something. It's hard to say without actually knowing the details of your situation and the data itself. I don't really think there is a canned answer in this case.

--
Paige Miller

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to choose a machine learning algorithm

Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 2 replies
  • 764 views
  • 0 likes
  • 3 in conversation