A few weeks ago, we asked for your best data disaster stories. Those times when things just went terribly, terribly wrong. We were lucky to get some great stories as a result of the article (by the way, did you see Phil Simon’s recent post, Bad data management in a two-letter word?). Without further ado, here’s the roundup:
Community member ballardw chimed in with this gem:
“The biggest data disaster I ran into was related to a basic regression line issue. An electric utility had looked at a regression line of total usage over a period of years that included a 20 estimate trend line past the last year of data. Unfortunately no one had paid attention to the fact that most of the change was industry transitioning from their own "boiler" power to the grid and was much larger than the population growth. The regression was prepared at about the same time as 95% of the customers likely to make such a change was completed. So the company spent lots of $$$ to expand power but the actual growth changed from a nearly 10% increase projected per year to about 2.5%. Moral of the story: Look into the drivers of your trend lines, not just the lines themselves.”
Yikes! What an expensive way to learn a lesson about proper data management. Another great story came from Daniel Teachey, who witnessed this data disaster:
“About a month ago, a lawn care company left me a flyer pointing out the flaws in my yard. Trouble is, I’m an existing customer of this company, so they were criticizing their own handiwork. If the company used data quality tools to compare their potential customer route to their existing customers, this would never have happened. Know your customer … so you don’t lose your customer.”
Joe Dunlavy shared with a story and an important reminder for all data stewards:
“I was once asked to verify a data point of about +2.5 billion. It was the sum of a series of positive and negative numbers. I called up the data series and summed the row and got the 2.5 billion. BUT, on further investigation, the sum function was not working properly. The true value was more like +20 billion. Uh oh. I never discovered why the excel function did not work as usual. Formatting issue maybe? Double check your output, always.”
Have you seen similar errors in your workplace or daily life? After reviewing the stories, I remembered another good one:
On a recent trip, one of my good friends booked a hotel room for the vacation of a lifetime. Her credit card company flagged the international charge as possible fraud and declined the transaction. Then the hotel promptly cancelled her reservation due to non-payment. What’s more, no one told her until she arrived at the front desk! To make matters worse, the hotel was completely sold out. Fortunately, she squeezed into another friend’s room and was not kicked to the curb.
If the credit card company integrated data across different departments to provide a single view of the customer via Master Data Management, it could have accessed my friend’s profile and emailed her upon the hotel room cancellation. Crisis averted.
From silly to serious, data errors come in all shapes and sizes. Imagine how easily data management mistakes could turn your dream vacation into a nightmare.
If you’re itching for more stories, share yours below and encourage fellow community members chime in! We want to hear all the ways you’ve encountered errors or roadblocks along the way to your project’s success.
