I'm new to SAS and I was doing some forecasting in SAS Forecast Studio on sales data and was trying to figure out the difference between these two options:
Model Select -> Use holdout sample for model selection
Forecast -> Calculate statistics of fit over an out-of-sample range
It's my understanding that in using a holdout it does things in this order:
1) remove holdout from sample
2) calculate statistics of fit for multiple models
3) choose best model
4) re-introduce holdout data
5) calculate statistics of fit over all data
With the "Calculate statistics of fit over an out-of-sample range" option can someone explain to me how this process is different?
The holdout sample is used exactly as you describe. The out-of-sample calculates fit statistics for a period at the end of the series for just the selected model. In terms of usage I tend to use the holdout sample for large scale automatic model building and the out-of-sample feature when doing manual model building.
The holdout sample is used exactly as you describe. The out-of-sample calculates fit statistics for a period at the end of the series for just the selected model. In terms of usage I tend to use the holdout sample for large scale automatic model building and the out-of-sample feature when doing manual model building.
Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.
If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website.
Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.
Find more tutorials on the SAS Users YouTube channel.