Out-of-sample range vs holdout sample

jbake — Wed, 15 Jul 2015 18:16:10 GMT

I'm new to SAS and I was doing some forecasting in SAS Forecast Studio on sales data and was trying to figure out the difference between these two options:

Model Select -> Use holdout sample for model selection

Forecast -> Calculate statistics of fit over an out-of-sample range

It's my understanding that in using a holdout it does things in this order:

1) remove holdout from sample

2) calculate statistics of fit for multiple models

3) choose best model

4) re-introduce holdout data

5) calculate statistics of fit over all data

With the "Calculate statistics of fit over an out-of-sample range" option can someone explain to me how this process is different?

Re: Out-of-sample range vs holdout sample

SnurreJensen — Mon, 10 Aug 2015 09:37:21 GMT

The holdout sample is used exactly as you describe. The out-of-sample calculates fit statistics for a period at the end of the series for just the selected model. In terms of usage I tend to use the holdout sample for large scale automatic model building and the out-of-sample feature when doing manual model building.

topic Out-of-sample range vs holdout sample in SAS Forecasting and Econometrics

Out-of-sample range vs holdout sample

Re: Out-of-sample range vs holdout sample