topic Re: Selecting the Correct Forecasting/Prediction Proc (Feb-Dec year 1, Jan-Mar year 2) in Statistical Procedures

Selecting the Correct Forecasting/Prediction Proc (Feb-Dec year 1, Jan-Mar year 2)

pdpat43 — Thu, 12 Jun 2014 00:47:42 GMT

I could use some guidance in selecting the most appropriate forecasting or prediction procedure given the following data, goals, and constraints:

Data:

Very large number of observations

Monthly data per person (Feb thru Dec in year 1; Jan thru Mar in year 2);

One continuous dependent variable

Goal:

Use Year 1 data to create a model

Apply early Year 2 data to predict December of Year 2

The monthly DV observations are not linear over time (simple downward slope - quadratic?)

My early attempts to predict have not been fruitful.

I have tried PROC LOESS, FORECAST, ARIMA, X12, and TRANSREG.

One of the above may be right, but the constrains of such large data mean long processing times and (commonly) insufficient memory.

I'd appreciate any guidance regarding the most suitable method so I can subset the data and try again.

Thanks!ss

Re: Selecting the Correct Forecasting/Prediction Proc (Feb-Dec year 1, Jan-Mar year 2)

Ksharp — Thu, 12 Jun 2014 12:45:43 GMT

DId you try proc esm ?

Re: Selecting the Correct Forecasting/Prediction Proc (Feb-Dec year 1, Jan-Mar year 2)

pdpat43 — Thu, 12 Jun 2014 21:14:52 GMT

No, I'll give PROC ESM a try as well.

At first glance, I don't see a SCORE option to predict on the Year 2 Jan-Mar values.

Am I missing something?

Thanks, Ksharp.

Re: Selecting the Correct Forecasting/Prediction Proc (Feb-Dec year 1, Jan-Mar year 2)

ets_kps — Mon, 16 Jun 2014 15:21:06 GMT

Hi Ksharp,

The ESM procedure uses a lead= option rather than a score statement. you can see the syntax here.

SAS/ETS(R) 13.1 User's Guide

Let us know if you need any help. -Ken

Re: Selecting the Correct Forecasting/Prediction Proc (Feb-Dec year 1, Jan-Mar year 2)

udo_sas — Tue, 17 Jun 2014 20:13:07 GMT

Hello -

If you are considering using time series techniques such as exponential smoothing, then your idea of: "Use Year 1 data to create a model Apply early Year 2 data to predict December of Year 2" will not work.

Time series models are closely tied to the data which is used to estimate parameters. This is very different to techniques like OLS regression.

For example: you can "train" a predictive model such as a logistic regression on training data, create a score file, and then apply this score file to new data.

This concept does not apply to statistical forecasting models - here you should use all history available to estimate the parameters of the model - usually the most recent data is the most relevant. Also, once you have estimated the parameters, these models are usually tied to the history which was used for estimation. For exponential smoothing models for example you can think of floating average with infinite memory but with exponentially falling weights.

In my opinion the question of whether to use a predictive model or a statistical forecasting model depends on your business question which you have in mind - note that for both areas very scalable algorithms are available.

If you can share a some example data and specify how you would the results to look like, we might be able to come up with some code snippet for you.

Thanks,

Udo

Re: Selecting the Correct Forecasting/Prediction Proc (Feb-Dec year 1, Jan-Mar year 2)

pdpat43 — Wed, 02 Jul 2014 22:18:03 GMT

Udo,

Thank you for the thorough response.

Here is a summarized version of what I have available:

Month	Year	Var1
1	2013
2	2013	94%
3	2013	93%
4	2013	92%
5	2013	91%
6	2013	91%
7	2013	80%
8	2013	90%
9	2013	89%
10	2013	88%
11	2013	87%
12	2013	87%
1	2014
2	2014	95%
3	2014	94%
4	2014	93%
5	2014	93%
6	2014
7	2014
8	2014
9	2014
10	2014
11	2014
12	2014

To make matters more complicated, Var1 is a rolling YTD average.

Does the "Accumulate=average" subcommand account for the heavier weight toward year's end?

Thank you!

Re: Selecting the Correct Forecasting/Prediction Proc (Feb-Dec year 1, Jan-Mar year 2)

udo_sas — Mon, 07 Jul 2014 18:15:45 GMT

Hello -

Many thanks for sharing an example - I don't think that statistical forecasting techniques such as exponential smoothing will be applicable for your situation - due to the lack of history. After plotting your data I was thinking that you might be better off using a curve fitting technique such as LOESS to your data and try to come up with a "profile" which can be applied to future points. However, again the lack of historic data will be an issue, unless you assume that 2013 is a strong representative of 2014.

This e-newsletter might give you some ideas: http://support.sas.com/community/newsletters/training/forecasting.html

Hope this makes sense.

Thanks,

Udo