SAS Intelligent Decisioning: A/B Test Node

[#1]

In SAS Intelligent Decisioning, you can use an A/B Test node to split incoming records between different paths so you can test what works best for your offers. These paths can include objects such as rules, models, and treatment groups.  You use the node to test or challenge a theory over time.  This node is not designed to give immediate results about which path is better or preferred; instead, it is meant for you to evaluate the results over time.

There are two types of A/B tests you can perform - Champion/Challenger and Challenger/Challenger.

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

Champion/Challenger

This setup has one champion path and one or more challenger paths. You decide what percentage of records go to each challenger. Whatever is left automatically goes to the champion. The champion will always get at least as much traffic as the smallest challenger, and everything together must add up to 100%. The records are split approximately according to the specified percentages among the paths using a RAND function.

Challenger/Challenger

In this option, all paths are challengers, and SAS Intelligent Decisioning splits records approximately evenly across each of the paths using a RAND function. There is no option to set percentages for the paths.

You can also use a numeric input variable as a path key. This ensures records with the same value (e.g. a Customer_ID column) always go down the same path. Without a path key, records are routed randomly to a path.

For tracking results, it’s suggested to add a Record Contacts node below your A/B test paths. When you do, SAS Intelligent Decisioning automatically captures details about the test such as which A/B test node executed and which path each record followed, so you can analyze the performance of the path later.  Then once you receive additional information from the campaign or tests, you would use the Subject Contacts API to help evaluate the various paths.

Potential Scenarios for Using the A/B Test Node

Here are some example scenarios of what you could do with the A/B Test node in SAS Intelligent Decisioning:

• Model testing: routing a certain percentage of transactions to an established ML mode vs a new one, to compare performance metrics and inspire the replacement of the Champion model in a project.
• Rule set optimization: checking how a rule does with a new threshold like a higher interest rate for specific demographic.  For example, evaluate if it increases profit margins without increasing churn rates significantly.
• Treatment experimentation: test different retention or incentive offers for your customers.

A/B testing is particularly valuable for organizations that mature their decisioning capabilities over time. Many organizations start with rule-based strategies and scorecards, then advance to statistical models and, ultimately, sophisticated machine learning approaches. At each phase, there is typically a proven strategy operating in production, alongside a need to safely introduce and evaluate a new alternative. The A/B Test node provides a disciplined, low-risk way to do this, offering clear governance, transparency, and performance insight. In effect, it enables live experimentation—allowing organizations to innovate with confidence while protecting the majority of their customers and outcomes.

Example Decision with an A/B Test Node

Scenario

We want to do a campaign to present qualified customers with an incentive to apply for a pre-approved personal loan.  Before we roll out the campaign, we first want to test on a subset of our customers if they respond better to an offer of a reduced interest rate for the first year or an offer of a gift card if they close on the personal loan.

Decision Flow

The first step of the decision flow is to use a rule set to filter for customers who are pre-approved for a personal loan up to $100,000.

For this scenario, we have two paths for the A/B Test node – lower interest rate for first year OR gift card.  You could have more paths, but for simplicity’s sake in this example we only have two.  The sum of all the percentages for the paths must equal 100%.

The first item on both paths is a Treatment Groups node. Treatments define offers that you might want to present to your customers or other entities. On the Lower Interest Rate path, top customers are offered a 1% interest rate reduction for the first year and the other pre-approved customers are offered a 0.5% interest rate reduction for the first year. On the Gift Card path, top customers are offered a $500 gift card and the other pre-approved customers are offered a $250 gift card.

Next, on each path is a DS2 Code node to read the resulting data grid from its above treatment group and write the offer code for the customer to a variable, so it will be included in the output of this decision.

The last object in each path is a Record Contacts node to record the results of the A/B Test path for future evaluation purposes.

This decision can be tested with the subset of customers chosen to be evaluated for this offer.  Viewing the decision path tracking shows that each path received approximately an equal number of records as specified in the A/B Test node. The numbers aren't exactly 50/50 due to the use of the RAND function for determining the path.

Here are the output results of the test run:

Finally, use the Subject Contacts API to help evaluate the results of the different paths once the test campaign is complete. In this example, that would be evaluating which incentive offer did more customers respond to the gift card or the lower interest rate one?

Conclusion

Summary

A/B Test nodes are useful when evaluating different theories and offers.  For more information, refer to the SAS Intelligent Decisioning documentation on Using A/B Test Nodes and the SAS Intelligent Decisioning: Beyond the Essentials course.

Acknowledgements

I would like to thank the following people for contributing to the material in this post:

• David Wiehle
• Ursula Polo
• Diana Maris
• Sandra Sorza Hernandez
• Brian Rowland.

Find more articles from SAS Global Enablement and Learning here.