Data Quality is the success key for analytical developments. Nowadays the amount of data that analytics works with makes difficult to identify errors, affecting to the accuracy and decisions based on their conclusions. One of the focus of the Innova-tsn approach for this use case, It’s to serialize and industrialize using analytical platforms to understand and recognize anomalous data and help to fix them.
With these data, another issue appears, not related to the quality of the data, but which affects the expected stable series. Historically, it has been observed that there are anomalous events that impact on the behaviour of energy time series.
Typically, such kind of events are manually identified. Some of them are known in advance due to the calendar effect, such as christmas holidays.
On the other hand, others are taken into acount on the basis of an ad hoc analysis, where deviations from the norm are observed. With hindsight, they are associated with an event or occurrence, such as a rainstorm or human actions, like political issues.
The aim of this work is to develop an automated and sophisticated algorithm to detect these anomalies, mainly based on autoencoder neural networks. In this way, time will be saved and hand-operated intervention of an analyst will be avoided.
Besides, we could use this automated events detection as a input of forecasting processes, and as a result, performance of the forecasting models will be increased.
Join us for SAS Innovate 2025, our biggest and most exciting global event of the year, in Orlando, FL, from May 6-9. Sign up by March 14 for just $795.
