How does Forecast studio choose the best Model?

[jctran123]

Hi SAS Community, 

I am working on a forecasting project and would really appreciate if someone could shed some light on model selection. I understand that SAS selects the best model based on the selection criteria. My selection criteria is MAPE and I noticed that when I create a custom model, my model's MAPE beats the "best" model that SAS selects. My question is, does SAS consider all model types before spitting out the "best" model or do they only consider the defaults (ARIMA and exponential smoothing)? 

My current project has selected ARIMA as the best model. Are there other reasons why ARIMA could have been chosen aside from having the "best" MAPE? 

Thank you,

Jessica  

[alexchien · Accepted solution]

Hi Jessica,

Forecast Server uses an automatic time series diagnose process to come up with a list of candidate models, and then the model with the best selection criteria measure (e.g. MAPE) is selected as the winning model. By default only ARIMA and ESM class of models are considered in the diagnose process. You can also include UCM and combined models to be included in the candidate model list. Please note that it is possible that you can come up with a model that beats all the models the automatic diagnose process came up with. The diagnose process only looked at a subset of all possible ARIMA and ESM model classes based on the series characteristics it detected (lags, diffs, cross correlations, etc.). 

Hope this helps.

Alex

[jctran123]

Hi Alex,

This was very helpful! Thank you for explaining this to me! 

Jessica