topic Re: How to persuade clients to believe random sampling in All Things Community

How to persuade clients to believe random sampling

kbjedi — Wed, 20 Jun 2018 15:27:47 GMT

Hi, All,

For my project, we have to review clients' working records manually. Usually, there are over 30,000 records. So we use the simple random sample to review over 300 records. The sample size was calculated by an online sample size calculator. We extrapolated the over cost of all records from the review results of the sample. We have compared the random sample extrapolation and the censorship results for one case, the difference was really small.

My problem is some of my clients do not trust random sample results. They only want censorship. but the censorship may take years.

I tried to give them random sample rational and examples to explain, but they were still not buying it. One of my clients said, " I don't believe statistics."

Do you have any suggestion to make people believe the random sample, especially, when they don't believe statistics?

Thank you.

Re: How to persuade clients to believe random sampling

Reeza — Wed, 20 Jun 2018 15:31:57 GMT

Next time you go for a blood test, do you want them to take it all, or just a sample?

@kbjedi wrote:

Hi, All,

For my project, we have to review clients' working records manually. Usually, there are over 30,000 records. So we use the simple random sample to review over 300 records. The sample size was calculated by an online sample size calculator. We extrapolated the over cost of all records from the review results of the sample. We have compared the random sample extrapolation and the censorship results for one case, the difference was really small.

My problem is some of my clients do not trust random sample results. They only want censorship. but the censorship may take years.

I tried to give them random sample rational and examples to explain, but they were still not buying it. One of my clients said, " I don't believe statistics."

Do you have any suggestion to make people believe the random sample, especially, when they don't believe statistics?

Thank you.

Re: How to persuade clients to believe random sampling

kbjedi — Wed, 20 Jun 2018 15:35:31 GMT

Thank you, I tried to use some examples. The work records are about human behaviour and practice. So it is a little different. and your example sounds offensive actually. Thank you

Re: How to persuade clients to believe random sampling

Reeza — Wed, 20 Jun 2018 15:39:10 GMT

It's sarcasm, and you wouldn't actually use those words....but it definitely illustrates the point.

@kbjedi wrote:

Thank you, I tried to use some examples. The work records are about human behaviour and practice. So it is a little different. and your example sounds offensive actually. Thank you

I used to work on fraud detection and it was impossible to audit every single transaction, so we would audit a sample and extrapolate from there. This faced a court challenge by a physician but the methodology was upheld by a court of law.

I'm assuming you're also including some sort of confidence intervals or prediction intervals and examples of back testing that would verify your results to provide assurance that the estimates or conclusions are robust.

Re: How to persuade clients to believe random sampling

kbjedi — Wed, 20 Jun 2018 15:47:13 GMT

Thank you so much,

Yes, I have the confidence interval and margin of error included.

Can you please tell me more details about how does the court uphold your methodology?

Re: How to persuade clients to believe random sampling

Reeza — Wed, 20 Jun 2018 15:58:22 GMT

http://www.health-law.com/blogs-National-Leaders-in-Health-Law,Use-of-Statistical-Sampling-In-False-Claims-Act-Cases

There's judgments in either direction at this point, so sounds like the courts are still figuring it out. Our lawyers have indicated we can continue to use this methodology - most people pay rather than fight anyways.

@kbjedi wrote:

Thank you so much,

Yes, I have the confidence interval and margin of error included.

Can you please tell me more details about how does the court uphold your methodology?

Re: How to persuade clients to believe random sampling

ballardw — Wed, 20 Jun 2018 16:25:42 GMT

@kbjedi wrote:

Hi, All,

For my project, we have to review clients' working records manually. Usually, there are over 30,000 records. So we use the simple random sample to review over 300 records. The sample size was calculated by an online sample size calculator. We extrapolated the over cost of all records from the review results of the sample. We have compared the random sample extrapolation and the censorship results for one case, the difference was really small.

My problem is some of my clients do not trust random sample results. They only want censorship. but the censorship may take years.

I tried to give them random sample rational and examples to explain, but they were still not buying it. One of my clients said, " I don't believe statistics."

Do you have any suggestion to make people believe the random sample, especially, when they don't believe statistics?

Thank you.

If there are different types of records I might suggest stratifying by the document or record type. If 95% of the records are type X and 5% are type Y you may not see enough of type Y to have much of handle on type Y. There could be significantly different impacts if there are errors between the types. Suppose type X represented sales under $500 and the type Y include sales to $5,000,000. That could well warrant a higher rate of sampling for type Y.

Many simple tools do not take into effect such things as strata to calculate sample sizes.

Re: How to persuade clients to believe random sampling

kbjedi — Wed, 20 Jun 2018 17:40:48 GMT

Thank you, we also use strata sampling. Sometimes it is more work to explain to our clients.

Re: How to persuade clients to believe random sampling

ballardw — Wed, 20 Jun 2018 20:09:21 GMT

@kbjedi wrote:

Thank you, we also use strata sampling. Sometimes it is more work to explain to our clients.

That's why you get paid the big bucks!