Interpreting ARIMAX Models, Part 1
This post describes the essentials of how ARIMAX models work and illustrates how to interpret their interpretable parts. The intention is to help analysts better understand their project’s generated models so they can effectively communicate results and make informed choices in setting forecast model related options. This post is the first in a series. In this post and the next one, we’ll focus on interpreting ARIMAX models with an emphasis on automatically generated model results you’ll see in a SAS Visual Forecasting project. Subsequent posts in this series focus on additional diagnostics that augment and extend the interpretability of models generated a SAS Visual Forecasting project.
The ARIMAX model captures systematic variation in a dependent variable, and it’s useful to think about the systematic variation, or signal, in a time series in terms of components. For example, if a time series exhibits a seasonal pattern, one component of a reasonable forecast model controls for and extrapolates the seasonal component of variation. The focus of this series is on the more interpretable parts of ARIMAX models, and we’ll spend most of our time on components related to capturing systematic variation attributed to input variables. The input variable component is the X (for eXogenous) in ARIMAX. ARIMAX models quantify the relationship between an input and the dependent variable through a mechanism called a transfer function which sounds involved and tricky, but it’s not. If you’ve ever specified, fit and interpreted a linear regression model, you’ve modeled and explained a transfer function. The transfer function transfers (or transforms or filters) variation in the input into variation in the dependent variable. For simple models, one number, the estimated parameter, suffices to describe the relationship between the input and target. In ARIMAX models, variation in an input this period can impact the dependent variable this period and in subsequent periods, so a more general representation is needed to adequately model the relationship. We’ll begin with components that are called numerator orders in a transfer function.
Numerator order 0
Let’s start by considering a simple regression model in the context of time series.
Here, sales at time t are a function of the input price at time t plus an intercept term. An error term, delta, representing unexplained variation or noise is also added. A useful way to think of the error at time t is that it’s what’s left over after subtracting the intercept and the effect of price from sales at time t. Fitting this model to data yields the following.
One tip for interpreting time series regression models is to keep an eye on the time subscripts. Note that the t subscripts on the sales and price variables are the same. This model tells us that on average, when price increases by $1.00 in interval t, units of sales decrease by 1.2 in the same interval, and that’s the end of the story; changes in price this month, for example, don’t impact units of sales in following months. In a time series context, what we’ve described is contemporaneous, or lag 0, correlation. This estimated relationship is represented in the following plot.
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
Price and sales may be varying continuously, but to interpret this plot it helps to think about the relationship between price and sales as a pulse and response. Imagine price having a steady state level, but, every once in a while, it pulses or increases by $1.00. The plot represents the response or deviation (-1.2 units) of sales from its own steady state level to the unit pulse in price. While this plot is not currently produced by default in SAS Visual Forecasting, we introduce it here to build intuition and will use similar plots to extend model interpretability diagnostics in subsequent posts in this series.
This model, REGARIMA1_12, is listed in a Visual Forecasting project on the highlighted row below. The CONST term indicates that the model has an intercept.
Note, the numbers in the model name are for internal bookkeeping and don’t tell you anything about what’s in the model specification.
Numerator (0), Shift (1)
Next, let’s consider a situation where changes in price this interval don’t impact sales until the following interval. This could result from contracts that are in place, menu costs and so on. The specification representing this relationship would look like the following (keep an eye on the time subscripts).
Fitting this model yields;
This model tells us that, on average, when price increases by $1.00 in interval t, units of sales don’t change in the current interval, but they decrease by 2 units in the next interval. In a time series context, what we’ve described is a pure delay or shift 1correlation. This estimated relationship is represented in the following plot.
This model, REGARIMA1_134, is listed in a Visual Forecasting project on the highlighted row below.
Numerator (0, 1, 2)
Now, let’s consider a situation where an increase in price this month impacts sales this month and in the following two months. The specification representing this relationship is below.
Fitting this model yields a specification that looks like the following (see the note);
This model tells us that on average, when price increases by $1.00 in a given month, units of sales drop by 1.2 in that month, then drop 0.9 in the month following, and finally decline 0.5 two months later. The total impact on sales of the $1.00 increase in price is -2.6.
Note, the signs on the lag 1 (t-1) and lag 2 (t-2) price parameters in the specifications are correct for the described relationship. Numerator parameters after the first parameter are specified with a negative sign in front of them. You must interpret the listed estimates keeping the implied negative sign in mind.
This estimated relationship is represented in the following plot.
This model, REGARIMA1_135, is listed in a Visual Forecasting project on the highlighted row below.
Numerator (0, 1) AR (1)
Up to this point, the error term has represented random variation or noise, but this doesn’t need to be the case. There can be systematic patterns, or memory, in the time series regression model’s residuals. The error term, in this context the error series model, shown below captures systematic variation in the dependent variable that is not attributed to variation in price with an auto-regressive order 1 or AR(1) term. The lag 1 delta term captures the memory, and the epsilon term represents the noise. More detail on AR processes below.
Fitting this model yields a specification that looks like the following
The relationship between price and sales is given by their estimated transfer function.
This model, REGARIMA1_16, is listed in a Visual Forecasting project on the highlighted row below. Note, auto-regressive orders are denoted with a P.
More details on the error series model
One useful feature of ARIMAX models is that they can capture or accommodate all of the signal contained in the dependent variable’s variation. The input terms capture the impact of sources of systematic variation that are well measured and known via the transfer function. Other systematic patterns in the dependent variable can be attributed to sources like competitor activities, external events and policy changes that are not well measured and understood. The impact of some of these sources of variation can be captured through auto-regressive (AR) and moving average (MA) terms in the model and extrapolated into the future.
You can think of ARMA terms as an abstract representation of underlying sources of systematic variation that we can’t measure or are too expensive to measure and create transfer function features for. The variation that ARMA terms approximate needs to be stationary. This means that these signal or memory components are mean reverting and implies that they are fairly short lived. ARMA terms are not particularly interpretable, however, they can improve the model’s forecasts and they influence model interpretability. Diagnostics and statistical tests that are based on distributional properties assume that the model’s residuals are (white) noise. If we had fit the REGARIMA1_16 model without the AR(1) component, this assumption would have been violated; there would have been a pattern in the unexplained variation in the model’s residuals. In addition to this signal component being missing from the forecast, omitting the AR(1) term means that estimated standard errors and related statistics like p-values on transfer function terms would be suspect.
More details on how ARIMAX models are generated in SAS Visual Forecasting
When candidate input variables are available, the software will build the ARIMAX model two ways for each series. For the REGARIMA models, the transfer function is fit first and then the ARMA model is fit on this model’s residuals. Generated ARIMAX models do the process in reverse. The ARIMA model is identified and fit on the dependent variable series, and then transfer function is identified and fit on this model’s residuals. In applications, the order that these components are specified and fit can matter in terms of overall fit or validation performance. The screen shot below shows both of the generated ARIMAX models for a single time series in a project.
Both models contain the price input variable, but the two specifications are different in terms of the ARIMA components. In this case, the regression first model had a better overall fit to the data and was selected as champion.
Conclusion
There are more useful and interesting details about ARIMAX models to discuss and questions to answer. You may be wondering; what’s the deal with the I in ARIMAX? I’ve heard that transfer functions also contain denominator orders, so what do they good for? If AR and MA terms capture the same type of systematic variation, why do we need both? These questions will be answered in part 2 of this series, so stay tuned!
Find more articles from SAS Global Enablement and Learning here.
