All options for diagnose object determine how to choose best models from model families; all options for forecast objects determine how to choose the best model from model selection list. For example, in this example code, ARIMA and ESM model families are enabled, so the diagnose.setOption('holdout')  helps choose one model from ARIMA family and one model from ESM family. If you don't set this option explicitly, it will use the default value 0 for holdout. After you already have a list of models, the forecast object forecast.setOption('holdout') helps select the best model among this list. If you don't set this option explicitly, it will use the default value 0 for holdout when making this decision. So in your case, when you want to use non-zero holdout, you might want to specify this options in both objects to be consistent. ... View more