BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
jbake
Calcite | Level 5

I'm new to SAS and I was doing some forecasting in SAS Forecast Studio on sales data and was trying to figure out the difference between these two options:

Model Select -> Use holdout sample for model selection

Forecast -> Calculate statistics of fit over an out-of-sample range

It's my understanding that in using a holdout it does things in this order:

1) remove holdout from sample

2) calculate statistics of fit for multiple models

3) choose best model

4) re-introduce holdout data

5) calculate statistics of fit over all data

With the "Calculate statistics of fit over an out-of-sample range" option can someone explain to me how this process is different?

1 ACCEPTED SOLUTION

Accepted Solutions
SnurreJensen
SAS Employee

The holdout sample is used exactly as you describe. The out-of-sample calculates fit statistics for a period at the end of the series for just the selected model. In terms of usage I tend to use the holdout sample for large scale automatic model building and the out-of-sample feature when doing manual model building.

View solution in original post

1 REPLY 1
SnurreJensen
SAS Employee

The holdout sample is used exactly as you describe. The out-of-sample calculates fit statistics for a period at the end of the series for just the selected model. In terms of usage I tend to use the holdout sample for large scale automatic model building and the out-of-sample feature when doing manual model building.

sas-innovate-2024.png

Don't miss out on SAS Innovate - Register now for the FREE Livestream!

Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.

 

Register now!

Multiple Linear Regression in SAS

Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 1 reply
  • 2462 views
  • 0 likes
  • 2 in conversation