The reason for slow uploads in SAS Marketing Optimization Input Data and what you can do about it

[#1]

Character customer attributes with a large number of distinct values are often the culprit when validation takes a very long time.

In the Input Data Wizard, you can select the customer attributes that you want to load into SAS Marketing Optimization. But we are finding that many people (myself included) tend to use the “Select All” since it’s easier than individually picking each attribute that you want to load.

If all of your attributes are numeric, or if the character attributes really are categorical attributes (like Gender or State, with not too many distinct values), then selecting more attributes than you really need shouldn’t affect the performance too much.

But if one of the attributes is something like address or social security number, with millions of distinct values, then the validation will take a *really* long time because all of the individual values need to be validated and loaded into the MO datastore.

Use numeric data or categorical attributes

Refresh will take even longer because even deleting the data from the datastore takes a while, and refresh actually does both (i.e., creates new data and then deletes the original). But since you would never really want to use any of these values in a filter (in other words, you would never create a constraint that applies only to customers where address=”123 Main St, Cary, NC”), you’ll get much better performance if you’re very careful when selecting customer attributes to only select those that are either numeric data or categorical attributes.

Delete old scenarios

If the initial validation is fast but the refresh process is slow, it probably means that you have a lot of scenarios using the input data. When the refresh runs, each of the scenarios must be recreated against the new data, and validation is performed at the scenario level to make sure that the scenario elements are still valid with the new data.

Delete old scenarios that you no longer need to speed up the refresh process.