07-15-2015 02:16 PM
I'm new to SAS and I was doing some forecasting in SAS Forecast Studio on sales data and was trying to figure out the difference between these two options:
Model Select -> Use holdout sample for model selection
Forecast -> Calculate statistics of fit over an out-of-sample range
It's my understanding that in using a holdout it does things in this order:
1) remove holdout from sample
2) calculate statistics of fit for multiple models
3) choose best model
4) re-introduce holdout data
5) calculate statistics of fit over all data
With the "Calculate statistics of fit over an out-of-sample range" option can someone explain to me how this process is different?
08-10-2015 05:37 AM
The holdout sample is used exactly as you describe. The out-of-sample calculates fit statistics for a period at the end of the series for just the selected model. In terms of usage I tend to use the holdout sample for large scale automatic model building and the out-of-sample feature when doing manual model building.