BookmarkSubscribeRSS Feed
wanderers
Calcite | Level 5

I work in the marketing analytics team of a bank and we work mainly on test/control testing. The test is the target population which was sent offers/campaigns and the control is left untreated. My team analyzes the difference in behavior between these two groups to gauge campaign performance.

 

Before we move into comparison, we ensure that the test and control are "like-to-like." This means that the test and control are similar in terms of a variable that is thought to be significant. For example, for a campaign that is aimed at improving checking-account balances of customers, we ensure that the test and control have equal mean checking-account balance i.e. they are "like-to-like." We have an iterative procedure that helps us do this:

 

(1) Run proc ttest on the test/control data with the checking-account balance as the "var" variable and "TEST/CONTROL" as the "class" variable.

(2) If p-value is not greater than 0.98, remove extreme observations from the appropriate group to bring about equal means.

(3) Follow (1),(2) till the p value is greater than 0.98

 

Please note that in our process of removal, we can't lose more that 2% of the test data.

 

I need help on two things:

 

(1) Is this procedure valid and is there a more effective way to ensure that the test/control populations are alike in terms of a single variable?

(2) Our superiors want us to take more than one variable into consideration in our "like-to-like" process. In this case, the proc ttest iterations cannot be used. How can we tackle this?

 

Please let me know if you need further elaboration on these questions. Thanks for reading.

 

 

 

 

2 REPLIES 2
PGStats
Opal | Level 21

If your significant variable really makes a difference in the behavior of your customers, then pairing would give you more power in your comparison. To enable a paired comparison, each tested customer should be paired with one or more controls on the basis of your significant variable(s).
If you want to involve more than one variable, one way is to define a distance that combines the variables into a single measure.

PG
StatDave
SAS Super FREQ

This falls in the general area of causal modeling of observational data.  SAS tools (macros) for this have been developed by the Rand Corp.  See their TWANG project page

sas-innovate-2024.png

Don't miss out on SAS Innovate - Register now for the FREE Livestream!

Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.

 

Register now!

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 2 replies
  • 1891 views
  • 0 likes
  • 3 in conversation