This is a very open-ended question. From how you describe it, different sampling methods will lead to better (or worse) models, depending on "the case". What are the different "cases" here you are referring to? What is the application? Are you training models for different customer segments for example? Do you have different problem statements/objectives where sometimes you are modeling rare events and sometimes it is more balanced? And then, how are you defining level of effectiveness for the different "sampling plans"? How are you validating your models?   Lots of questions. I think we need more details about what you are trying to do here, and what you are observing when you try one sampling method vs another. ... View more