Hello Bonnie - First of all thanks for using SAS Forecast Server. The purpose of using out-of-sample data is to assess the predictive power of your champion model for a specific series _before_ the actual values arrive. The purpose of using holdout data is to pick the champion model on data which was not used for initial estimation of parameters. Here is an example - when using both out-of-sample and holdouts: let's say you have monthly data for 5 years (Jan 2010 - Dec 2014) and you want to forecast 12 periods (lead time = 12). specify out-of-sample range - this will always be considered first - if you specify an out-of-sample of 12, then Jan 2014 - Dec 2014 will be "set aside". The most recent data will be used for out-of-sample. specify hold out sample: if you specify 12, then Jan 2013 - Dec 2013 will be "set aside". initial fit region will be Jan 2010 - Dec 2012 - all parameters of all your models in your model repository will be estimate on initial fit region. Since lead time is 12, an initial forecast will be created into the hold out region. The best performing model will be picked using your preferred accuracy statistic based on the performance in your hold out region. After the champion is selected, the holdout region gets merged back to the initial fit region, now the fit range is Jan 2010 -Dec 2013. The parameters of the champion model will be re-estimated based on Jan 2010 - Dec 2013 data - a forecast will be generated for Jan 2014 - Dec 2014, which is your out-of-sample region. now can can assess how well your champion would perform "in real life" - as you have both actual and forecast values. if you are happy with the performance, you will have to turn off the out-of-sample region for production purposes, otherwise you will not predict the future (unless your lead time is longer than you out-of-sample region). by turning of out-of-sample, now your holdout region becomes Jan 2014 - Dec 2014 of course (since you don't have out-of-sample data anymore) - all forecast values will be for future periods. In order to assess performance you will have to wait until time moves on and actual values arrive. I'm hoping I'm addressing your question. Otherwise you may have to provide an example of what you are asking for. Thanks, Udo
... View more
