BookmarkSubscribeRSS Feed
RobertNYC
Obsidian | Level 7

Hi--

I'm putting together a new advertising creative campaign test  - the creatives I'm using have never been tested before and the company has never down and creative test campaign.  Normally, I would I would use proc power and base one proportion off a historical response rate and one proportion off an anticipated lift. So, like this

proc power;

  TwoSampleFreq

  Test=Fisher

  Alpha =0.01 0.05 0.1

  Sides = 1  

  GroupProportions = ( p1. p2.)

  Power =.9

  Npergroup  =.;

The issue that I have is that I have no historical benchmark  to base the sample size calculation off of; so,  no prior response rate based on creatives how the creative performed in the past. What I will know is what the total universe of people I can pull a sample from  will be. How should the sample size(s) be calculated when the creative test campaign is new? 

This is what my test cells look like:

CustomersTactic Sample Size (n=)
A. Hold Out GroupNo Tactic?
B. ControlEmail Only?
C. Test 1Direct Mail ?
D. Test 2Email + Direct Mail?
Total (n=)?

Any feedback/advice is greatly appreciated. Thanks! 

3 REPLIES 3
SteveDenham
Jade | Level 19

One can always appeal to a "worst case scenario"  Assume that your reference is 0.5, and generate sample sizes based on what you consider an important deviation from that value.  Then the matter becomes balancing cost of conducting the survey versus the expected return.

Steve Denham

RobertNYC
Obsidian | Level 7

Hi Steve

Thank you for feedback. This was very helpful.


I have one more question if you have time Smiley Happy

I would like to compute sample sizes based on these combinations:

A+B vs C+D

D vs A

D vs B

C vs A

C vs B


Should I use a different method/proc to calculate the sizes?   Any further suggestions/advice  of what to use is greatly

appreciated.

SteveDenham
Jade | Level 19

When in "worst case" mode where you have no prior estimate of any of the groups, it isn't going to make any difference (well except for the A+B vs C+D).  Any pairwise comparison is the same.  For instance if one group were at 0.5, and you wished to detect a shift of 0.05 with alpha=0.05 and power=0.8 using a two-sided Fisher's Exact test, you will need 1605 per group.

I would just modify your code to:

proc power;

  TwoSampleFreq

  Test=Fisher

  Alpha =0.01 0.05 0.1

  Sides = 1  

  GroupProportions = ( 0.5. 0.55.)

  Power =.9

  Npergroup  =.;

run;

I get n per group values of 2635, 1747 and 1350 for the three alpha values given.

Steve Denham

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 3 replies
  • 1458 views
  • 3 likes
  • 2 in conversation