Occasionally I am approached by a colleague to help figure out why an opimisation scenario in Marketing Optimisation, or CI360’s Engage Optimisation engine has failed. Of course there are multiple reasons for this, and I can’t answer them all here, but what I can highlight is the approach I take when both trouble shooting, but also for recommending the ‘best’ scenario. The strategy I use is consistent across business problems, whether it be Marketing, Risk, or Collections, or Banking, Telco or Retail, to name just nine combinations!
Here’s what I do, and apologies that some degree of knowledge of our optimisation tools is helpful in fully following the thread here:
Step 1. Find the Ceiling. This means maximising a non zero, non negative goal, (e.g. probability to respond) without applying any constraints, or contact rules. Essentially everyone gets everything they are eligible for. Now whilst this is unrealistic, it highlights a. the most that can be achieved for any option or set of options (like a channel). So when someone asks you for 100k leads, but there are only 90k eligible customers, you can quickly push back.
Step 2. Apply contact rules (and then row level constraints if these exist too). Here we are finding out the impact of your business policies. This again helps with pushing back on requirements that are simply not possible – but by doing step 2 one can identify whether it is the policy or the eligibility that is the challenge.
Side note 1: Contact Policy is one of the least analytically driven things out there it’s usually a finger in the air – and I may post some ideas about how to do these more analytically in the future
Side note 2: Eligibility should also be challenged – I often see a lot of ‘selection criteria’ appear as ‘eligiblity criteria’ for offers. Example “we excluded due to x as they don’t respond well to this offer”….one should always leave that type of decision to optimisation.
Step 3. Apply business constraints – but first apply the MAX constraints. Max constraints are always easier to meet, than min constraints. It’s like hair, you can easily cut some off, but it’s a lot harder to add it!
Step 4. Apply the min business constraints. This is where a lot of optimisation scenarios fail. The good news is that the optimisation tool will flag constraints that are likely to be causing the failing, but equally, looking at the results of step 3, and the constraints in step 4 will help a lot. Usually someone has asked for the impossible!
Step 5. Now start thinking about a different optimisation goal. Remember I said focus on a non zero, non negative goal function? That’s so that an optimisation engine will always pick it, as it wants to maximise that goal. However, if a revenue measure for example is the goal, it may be a negative value for some options for some customers (or some columns for some rows more generically!), which would mean it would not appear in selections in steps 1-4, which makes it harder to understand why the engine has done what it has.
Step 0. I know that 0 doesn’t usually come after 5, but I’m also hoping that you read all of this article before you start step 1! Step 0 is to ensure you are measuring the main three KPIs (offer volume, response or sales volume, and value (e.g. revenue or profit)), by each of the offers – especially those that have constraints. In SAS tools, this is achieved by creating “report only” constraints, which can help with trouble shooting, and building pragmatic scenarios.
Step 6. Figure out where BAU (business as usual) is……and beat it across all the KPIs. The reason I say this is that one of the biggest challenges of successful optimisation deployment is business buy in. It’s all well and good being pious and saying we should have the true optimisation solution, but in reality, if you halve the volume of the CMOs favourite campaign, or if you double the volume of the outbound call center’s files with one week’s notice, or if you double sales but halve revenue, you will annoy either the CMO, the COO, the CFO, or if you are very unlucky, all three….plus a few more!
However, you can achieve success by adding more constraints to ensure that for the important KPIs, you can sacrifice some performance to ensure that the important campaigns and channels improve across sales and revenue, and do not get a volume shock. Of course the definition of important and the granularity of detail you need will vary from business to business , but the principle remains the same.
Step 7. Explore. Now is when you can really start to unleash, and show the business how much more value can be generated by loosening constraint x, or investing in channel y, or stopping campaign z.
Side note: I have a bias towards optimising on sales (or response) rather than revenue in the early days of optimisation. There are a few reasons for this:
Sales and response are more easy to measure, and come in quicker than revenue, usually. Proving that optimisation “works” (and trust me, it “works”, it’s like, maths?) is easier this way.
Response models have been around longer than (predictive) revenue models, usually. As such I trust them to be more robust, and work (are more accurate) across a greater spread of the population.
I get scared about correlations of sales and revenue. I see a lot of businesses that have one team work on response models, and another on revenue…..and these teams don’t talk to one another…..but the customer behaviour will! What I mean by this is that one model says that the customer has high response, another model says they will have high revenue, but in reality high response customers have low revenue, and this has not been factored out of the estimates. (There are simple ways to combat this problem…….)
So in summary, if you follow the above steps towards optimisation, you will not only troubleshoot before the trouble has started, you will also be a long way towards being the toast of your management team!
... View more
