I'm new to SAS and I was doing some forecasting in SAS Forecast Studio on sales data and was trying to figure out the difference between these two options:
Model Select -> Use holdout sample for model selection
Forecast -> Calculate statistics of fit over an out-of-sample range
It's my understanding that in using a holdout it does things in this order:
1) remove holdout from sample
2) calculate statistics of fit for multiple models
3) choose best model
4) re-introduce holdout data
5) calculate statistics of fit over all data
With the "Calculate statistics of fit over an out-of-sample range" option can someone explain to me how this process is different?
The holdout sample is used exactly as you describe. The out-of-sample calculates fit statistics for a period at the end of the series for just the selected model. In terms of usage I tend to use the holdout sample for large scale automatic model building and the out-of-sample feature when doing manual model building.
The holdout sample is used exactly as you describe. The out-of-sample calculates fit statistics for a period at the end of the series for just the selected model. In terms of usage I tend to use the holdout sample for large scale automatic model building and the out-of-sample feature when doing manual model building.
Don't miss out on SAS Innovate - Register now for the FREE Livestream!
Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.
Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.
Find more tutorials on the SAS Users YouTube channel.