topic A/B Test: determine if a lift measured sales is significant in Statistical Procedures

A/B Test: determine if a lift measured sales is significant

RobertNYC — Mon, 11 Jul 2016 15:02:22 GMT

Hi all,

I am no stats expert so any assistance will be greatly appreciated.

I’m working on a post-campaign analysis to determine if any measured sales lift (sales lift per audience) is significant between test and control treatment groups. From what I have been reading, I believe you could potentially use an Anova or Ancova to run a post-campaign analysis to determine if any measured sales lift is above or below a standard deviation or margin of error (but I'm not sure how to do that)?

My question is, does this make sense? I’m never run an ANOVA is SAS in there any sample code that one can share?

Thanks so mush!

Re: A/B Test: determine if a lift measured sales is significant

ballardw — Mon, 11 Jul 2016 17:55:49 GMT

I would start with Proc Anova if I wanted to run an Anova

If you look in the online help there are several examples and if you have a default BASE SAS install the online help has a section called "Learning to use SAS" that has a section on the Anova Procedure.

Re: A/B Test: determine if a lift measured sales is significant

Rick_SAS — Mon, 11 Jul 2016 18:13:16 GMT

However, be awar that PROC ANOVA is for balanced designs where the sample size is the same in both groups.

If that condition does not hold, use PROC GLM. You can also use the TTEST procedure to test for differences in the group means.

For a reference, you can browse the section "Introduction to Analysis of Variance Procedures" in the SAS/STAT User's Guide.