topic Re: How to run Power analysis to capture sales lift? in Statistical Procedures

How to run Power analysis to capture sales lift?

t75wez1 — Sat, 05 Oct 2024 03:20:40 GMT

Hello,

I have a weekly product sale in EQ in time series data(total 88 weeks) with mean=15,000 and standard deviation=2,500.

I’m working on producing a matrix of confidence intervals and “power” analysis to determine what % lift (e.g. 5% or 6%) of sale do I need to observe to be confident the lift is statistically significant between the two test/control market pairs I identified.

The goal is to have a X%(e.g. 95%) power chance of obtaining a Y%(e.g. 90%) confidence interval whose half-width is at most from Z%(5% or 6%) lift of eq sales for next 26 weeks.

I need to complete this matrix below that will look something like this:

I've tried to use the code below but don't know how to set up the parameters in "proc power".

Can someone shed some lights here?

Thanks so much for your help in advance.

ODS Output PlotContent = want;
proc power;
onesamplemeans ci=t
SIDES=2
alpha = 0.05 ,0.1
halfwidth = 750 to 1500 by 150
stddev = 2500 
probwidth =.
ntotal = 8;
plot x=effect min=500 max=3000
XOPTS=( REF=1500 2000 2200 2500 CROSSREF=YES);
run;

Re: How to run Power analysis to capture sales lift?

Ksharp — Sat, 05 Oct 2024 07:06:24 GMT

It looks like you are doing Superiority Test. Check example:

Re: How to run Power analysis to capture sales lift?

t75wez1 — Sat, 05 Oct 2024 19:37:52 GMT

Thanks so much for your prompt response.

Still can't figure out how to set up "groupns" in the example you mentioned to complete the power analysis in my case.

I have the weekly product sale in EQ in time series data(total 88 weeks) for the test group with mean=15,000, standard deviation=2,500 cv=16.42 and the control group with mean=38,856 and standard deviation=4,700, cv=16.54.

Should I use "groupns=(88 88)" in below?

Thanks again for your insight.

***********************************************************

proc power;

twosamplemeans test=ratio

meanratio = 0.7 to 1.2 by 0.1

nullratio = 1.10

sides = L

alpha = 0.01

cv = 0.5 0.6

groupns = (300 180)

power = .;

plot x=effect step=0.05;

run;

Re: How to run Power analysis to capture sales lift?

Ksharp — Sun, 06 Oct 2024 03:33:33 GMT

I think you need side=U to do Superiority Test if SALE is bigger is better.

proc power;
ods output output=powdata;
   twosamplemeans  test=diff
      sides     = U      /*Superiority Test(upper one-size) --> Assuming SALE is bigger is better. */
      alpha     = 0.05 to 0.25 by 0.05  /*obtaining a Y%(95% 90% 85% 80% 75%) confidence interval*/
      groupns   = (88 88)  
	  stddev=3500   /* std of 176(=88+88) obs*/
      power     = 0.75 to 0.95 by 0.05
	  meandiff  = .;
run;

 %powtable (
Data = powdata,
Entries = meandiff,
Cols = alpha,
Rows = power
)

Re: How to run Power analysis to capture sales lift?

t75wez1 — Sun, 06 Oct 2024 16:51:03 GMT

Ksharp,

Thank Ksharp very much for the code with explanation and so much gleeful to have such a supportive environment.

However, I have 3 questions below:

1)What mathematical formula do you use to calculate the stddev=3500?

 stddev=3500   /* std of 176(=88+88) obs*/

2)How to download the %powtable to see the detailed SAS macro code?

3)Is it possible to export power table result from %powtable into Excel file?

Re: How to run Power analysis to capture sales lift?

Ksharp — Mon, 07 Oct 2024 00:49:59 GMT

1)According to Documentation ,option stddev= is the standard deviation in common to both groups.

You could try to use stddev of all the data(assuming these two group are from the same distribution).

Or could try PROC TTEST 's pooled method.

2)You could use " options mprint ;" to print the code from macro to LOG.

And @Tom have better way to get these code from build-in macro.

option mprint;
 %powtable (
Data = powdata,
Entries = meandiff,
Cols = alpha,
Rows = power
)

14   Entries = meandiff,
15   Cols = alpha,
16   Rows = power
17   )
MPRINT(POWTABLE):   options nonotes;
MPRINT(POWTABLE):   *---printdoc---*;
MPRINT(POWTABLE):   ods select none;
MPRINT(POWTABLE):   data _weightedData;
MPRINT(POWTABLE):   set powdata;
MPRINT(POWTABLE):   length _whichEntry $256;
MPRINT(POWTABLE):   _whichEntry = " ";
MPRINT(POWTABLE):   run;
MPRINT(POWTABLE):   proc contents data=_weightedData;
MPRINT(POWTABLE):   ods output Variables=_variables;
MPRINT(POWTABLE):   run;
MPRINT(POWTABLE):   data _null_;
MPRINT(POWTABLE):   set _variables;
MPRINT(POWTABLE):   call symput ("_NumVars", _N_);
MPRINT(POWTABLE):   run;

3) Yes. You could can, Just add ODS EXCEL in it.


ods excel file='c:\temp\temp.xlsx';
%powtable (
Data = powdata,
Entries = meandiff,
Cols = alpha,
Rows = power
)
ods excel close;

Re: How to run Power analysis to capture sales lift?

t75wez1 — Mon, 07 Oct 2024 02:22:31 GMT

Thanks @Ksharp again for your helps.

To build on my question (1)

I have the weekly product sale in EQ in time series data(total 88 weeks) for the test group with mean=15,000, standard deviation=2,500 and the control group with mean=38,856 and standard deviation=4,700.

The distribution between test and control group looks similar but not from the same distribution.

How to adjust those parameters in power analysis if it violates assumption you mentioned?

Re: How to run Power analysis to capture sales lift?

Ksharp — Mon, 07 Oct 2024 02:48:13 GMT

No idea about it. Here stddev= required the same value for both of group.
Maybe @StatDave could give you a hand.

Here is some thought:
1) you can firstly use PROC TTEST or PROC NPAR1WAY to do test the difference between these two groups. And see if it is statistical significant or not.

2)If it is Paried Sample,you need to try to use PARIEDMEANS statement of PROC POWER.

3)using data simulation skill by @Rick_SAS to get what you need.
https://blogs.sas.com/content/iml/2020/08/12/simulation-estimate-power-of-test.html
https://blogs.sas.com/content/iml/2020/08/17/power-curve-parallel-sas-viya.html

Re: How to run Power analysis to capture sales lift?

t75wez1 — Tue, 08 Oct 2024 14:46:53 GMT

Ksharp,

It appears to work fine. However, I 'd like to learn more about the superiority test you mentioned "Figure 3: Hypotheses for Noninferiority, Superiority and Equivalence Tests" from your reply.

Could you share the related documents?

Thanks for your help!

Re: How to run Power analysis to capture sales lift?

Ksharp — Wed, 09 Oct 2024 00:31:06 GMT

https://support.sas.com/resources/papers/proceedings15/SAS1911-2015.pdf
https://support.sas.com/kb/48/616.html