SAS Customer Intelligence 360: Email Subject Line Testing
There is a common marketing test referred to as A/B testing, in which different messages or promotional content are tested and results compared to establish a winner. The winner is used for subsequent communications because it statistically was proven to resonate better with the audience or target segment of customers (based on conversion rates).
SAS Customer Intelligence 360 offers various ways to conduct A/B tests, such as with web and mobile channels. However, a recent new feature has been released to conduct a similar test with email subject lines for bulk email tasks. “Bulk” email tasks deliver an email at one time to a selected target audience.
Email subject lines can be very important. How many emails have captured your attention – or not - based on the subject line content? It’s typically the first item scanned when viewing an email. When I scan my email quarantine list, I only look at the subject lines and the sender email addresses. If a subject line does not resonate with a viewer, there is a chance the email will be ignored and deleted. Marketing organizations are keen on testing email subject lines for effectiveness.
Subject Line Configuration
To use this feature, there is a wonderful toggle button aptly titled “Subject line A/B test” in the email task Header tab. This is where the subject lines are set up. The button can be toggled at any time during the development and configuration of the email task prior to publishing the task (which delivers the emails). Here is an example of an email task Header tab where the button was enabled and 2 subject lines A and B were filled in:
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
Note subject line B has some hard to comprehend code – those are merge tags, which allow the customers’ age to be inserted into the subject line (another topic for another blog). A conditional merge tag was set up so that anyone age 65 and higher will receive a subject line with their age number included in the text, and those under 65 will receive a different subject line text but without their specific age included.
Note the Distribution is by default set to 50% each – for a fair comparison for the test. Half the emails delivered include subject line A, and the other half include subject line B. The Duration can be selected among various numbers between 2 and 48 hours. This means, for example if 24 is chosen, the system will use the collected email performance data for only the first 24 hours after publishing the task for the test (it will continue to collect performance data, but data collected after 24 hours will not be used for the test). If enough data was captured it will determine a statistically proven winner based upon the selected Primary Metric and applied test parameters.
The Criteria for A/B Test section is where the type of conversion can be selected as the Primary Metric: “Open Rate” for opened emails, or “Click Rate” which means at least one link within the email needs to be clicked at least once.
You set the baseline conversion percentage rate, which should be the expected conversion rate based on the average open or click rate for all emails sent in the past 90 days (excluding bounced emails).
A/B tests use the default settings that are set in General Settings. These settings affect how winners are identified and the size of the recommended sample size. You can override the defaults and specify custom values for the task (these changes won’t impact other tasks).
As just alluded to, the recommended sample size is calculated based on the test settings and the applied Baseline conversion rate. Your target audience ideally should have a population at least equal to the recommended sample size.
As I mentioned, after the system reaches the set duration it will calculate a winning variant based on the applied test settings:
Winning Variant A can be used for any subsequent email campaigns or for a larger email campaign targeted at a much larger population. It also can be used to develop better subject lines for future emails.
More details on how to configure and set up this feature will be forthcoming soon in a revision to the course “Email Development and Delivery with SAS Customer Intelligence 360”, accessed via the SAS Customer Intelligence Learning Subscription.
Also, SAS R&D is planning to develop additional email A/B testing capabilities with regard to the email body content.
For more information on how to set up and deliver emails using SAS Customer Intelligence 360, please visit these sites below:
