Top 3 Learnings from International Symposium on Forecasting 2022
This year we had the pleasure to attend the 42nd International Symposium on Forecasting in Oxford, UK. SAS participants travelled from all across the globe to present their research, discuss their ideas and learn from some of the most reputable names in the forecasting space.
The presentations were captivating and the ongoing research that is taking place in forecasting got us very excited. While machine learning and deep learning techniques are becoming mainstream in the field of forecasting, there is still a lot of research around the more traditional statistical ways which are still well-utilized due to their robustness, their ability to achieve fast and accurate results, and their easiness of interpretation compared to the more complex ML-based methods.
Undoubtedly, it’s way too much to discuss in a single article but the presentations will be available on IIF’s YouTube channel so please keep an eye on that if you want to learn more about a particular topic. The learnings that stood out for us were the following:
1. Both ML and traditional statistical methods are now a part of the required toolkit of a forecaster
Machine learning methods have proven their worth in the latest forecasting competitions while more and more companies have started quickly adopting them in their regular forecasting activities.There is strong evidence that there are many cases when they fail to outperform robust statistical methods. Adebate between the two methods is equivalent of debating whether a fork or a knife is better to eat your dinner. ML techniques require special attention in data preparation and they work best when we are dealing with complex patterns and a large volume of interrelated series. Let’s keep in mind that in the M5 forecasting competition, 92.5% of the participating teams failed to outperform the benchmark which was based on exponential smoothing models.
2. Clustering before forecasting can enhance forecasting results
The problem arises when we try to decide which clustering method to use. Many presentations focused on which clustering method could lead to the most accurate forecasting results when models are applied (post-clustering) separately to each group. Youngjin Park, a principal researcher from the SAS R&D division, showcased some key clustering methods including similarity-based, component and k-means which can all be implemented using SAS procedure as well as an attribute clustering method that comes out of the box with SAS Visual Forecasting and classifies time-series data based on historical demand patterns. Her analysis showed that attributes-based clustering can enhance forecasting accuracy while similarity-based clustering is the most suitable among the first 3 methods and can be combined with the attributes-based segments to further improve the quality of forecasting models.
3. Are we using the right measure to assess models?
That’s a theme that has been going on since the inception of forecasting. This year a new criterion for forecast selection was presented, by prof. Fotios Petropoulos and prof. Enno Siemsen, that tries to quantify the representativeness of the forecasts. Until now we didn’t have a criterion to tell us if the forecasts produced make sense and we had to rely solely on human judgement to make this decision. The new criterion, called REP, aims to fill that gap by putting more emphasis on recent time windows and rejecting forecasts that look unrealistic. Using all data from the M1-M4 competitions, the authors of the paper proved that selecting forecasts by REP generally leads to better forecasts for most of the data frequencies tested. SAS is looking to incorporate this new criterion in SAS Visual Forecasting to give the option to practitioners to use it in their forecasting processes.
All in all, we learned loads and we can’t wait for next year’s conference in Charlottesville, Virginia USA, June 25-28!
