Hi:
The typical practice, followed for example by Forecast Server, is to estimate the model first with in-sample data, select a model, then reestimate the model parameters using all data.
Usually, you want a hold-out period that is at least as long as your forecasting horizon, and you want to leave enough data in your sample to estimate your parameters accurately. Exactly what that means depends a lot on your data. If you have weekly data and want to forecast one year head, obviously 54 data points is not even enough for a model with a weekly cycle, let alone a hold-out sample. If you have yearly data with low noise, no cycles, and a clearly defined trend, you most likely can use a hold-out sample of 20% of your data without impunity.
AIC has been often used for model selection when out-of-sample analysis is not practical because it is asymptotically equivalent to minimizing the one-step ahead MSE. You need to use some caution for comparing models using AIC but if you are limiting yourself to the class of UCM models you should be OK.
For more details about the use of AIC in time series forecasting you can look at these comments by Rob Hyndman .
http://robjhyndman.com/hyndsight/aic/
... View more
