You tweaked something in SAS Marketing Optimization that seemed very trivial and now your objective value or eligible offers are suddenly different. What happened?
SAS Marketing Optimization is not an exact solver. In fact, no exact solver exists today that can solve problems on the scale of typical MO problems – often more than hundreds of millions of binary decision variables.
Rather, MO is a heuristic approach. This means that there are many ways to produce slightly different results for the exact same scenario. For example, MO is an iterative algorithm that uses stopping criteria to determine when to terminate iterations. Therefore, if you use the mo_opt_config.sas file to change the stopping tolerance or the number of iterations, the solution will be different. Or even if you change the order of the data in the customer table, the solution may be different. This is all part of the nature of a heuristic approach or any algorithm that uses approximations.
Yes, it is true that with an exact solver, you would not see a lower objective after removing two constraints. However, please keep in mind that with a heuristic approach, any number of changes to the problem or the data can result in a different path through the algorithm, and therefore a different solution. In this case, when you remove two constraints that aren’t binding, it’s easy to say that the objective should be better because the constraints are not binding. But that’s because you know after the fact that the constraints are not binding.
At the beginning of the optimization process, MO does not yet know that the constraints will not be binding, so it does need to use the constraint information in the early iterations – until it has enough information to conclude that the constraints are not important. So it takes a slightly different path through the algorithm than it does in the case where the constraints are not present. And because the algorithm is a heuristic approach, a slightly different path can result in a slightly different solution – meaning slightly better or slightly worse. A difference of 0.18% is well within the normal tolerance of what we would expect from a heuristic approach.
For most MO problems, there is no way to know the exact optimal solution (except perhaps in cases where the constraints are very simple and the optimal solution could be obtained by sorting and using a DATA step). Therefore, there is no way to quantify MO’s “inexactitude.” This is why we always try to encourage users to make comparisons based on the best possible alternative without MO rather than trying to compare against some theoretical optimum that may not be achievable.
For example, if using MO produces an X% increase over your best alternative solution (including possibly other marketing optimization software), then you should not be concerned about whether an exact solver would have been able to produce a Y% increase, unless you have access to an exact solver that can actually do so.
To recap: SAS Marketing Optimization is not an exact solver, and there are good reasons for that. What’s the most interesting thing this software enabled you to do? Share in the comments.
