Adventures with State Space Models: Introduction

[#1]

State Space Models (SSMs) are unique among sequence data modeling algorithms; they combine advantages of machine learning algorithms, like flexibility and scalability, with the interpretability and control associated with more traditional time series models. In SSMs, the various signal components in the data, like seasonality, trend, input variable effects and so on, can be modeled individually. This approach provides details and insight about how each component evolves over time. Component models are additive, so component predictions are easily combined into an overall forecast for the target. This post will focus on a univariate (in Y) example, but generalizing the SSM to model multivariate timeseries is straight-forward as we’ll see in subsequent posts in this series.

The purpose of this series of blogs is to introduce SSMs as a toolbox for applied analysts. The goal of this post is to introduce the modeling framework and build intuition. SSMs looks a little involved at first glance, so we’ll describe the general specification later and start with a scaled down version and a simple example. Once we’ve built a foundation, subsequent posts will discuss more general and interesting examples.

Fundamentally, SSMs can contain both static and dynamic components. Here, dynamic means that a component varies as a function of time. As an example of a static component, consider price as an ordinary input variable in a timeseries regression model that forecasts units of sales. The values of the price and sales variables both vary over time, but their relationship, usually denoted with the parameter beta (see below) does not. This means that the effect of a $1.00 increase in price on sales has the same impact in the model’s predictions, regardless of the value of the time index or time of year.

However, it’s not hard to think of an example where this static relationship is too restrictive; consumers tend to be much less price sensitive for some items in the weeks leading up to Christmas than in other, non-holiday, times of the year. A trajectory of price effects is needed to adequately model this relationship. We’ll see an example of estimating an SSM with dynamic regressors in a subsequent post.

To start, a simple SSM can be written as follows.

Equation 1 is called the Observation equation. It includes variables that we observe; y represents values of the dependent variable, and x represents values of the independent variables, both recorded over time. The relationship between y and listed input variables (x may be a vector) in the Observation equation is static, note that Beta does not have a time subscript. Other terms include Epsilon, a white noise error term, and the State, Alpha.

Equation 2 is the State equation. The State equation provides a rule for getting the model from one time interval to the next and regulates how the components in the model evolve over time. Eta is a random disturbance term. The State equation specifies the dynamic parts of the model.

To focus ideas, let’s replace y with sales and model it as function of a static input, promotions (promo). We’ll rename alpha and use the State equation to represent how the level of sales varies over time, independently of promotions.

According to equation 2; the level (of sales independent of promotional effects) next interval is equal to the level this interval plus some random variation generated by the disturbance term, Eta. The relationship shown is also known as a random walk. Eta is assumed to be normally distributed with mean zero. Notice that, if the variance of Eta is zero, then the level of the series is static; it’s just a flat line. What happens to the Level component of the model as the variance of Eta increases from zero?

Before answering this question, let’s look at the data we’ll use in the next demonstration. The plot shows weekly observations on Sales that start the first week of Feb. 2017 and run through the last week of May 2018. The input, promo, is a binary variable that flags three weeks in the data.

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

The model described above can be fit to the data using the State Space Model (SSM/ETS) procedure in SAS as follows.

• The procedure statement lists the input table that contains the Sales and Promo variables.
• The ID statement identifies the time ID variable, Date, and species the desired interval of the data.
• The TREND statement creates the Level component in the model; a random walk (rw) component named Level is specified. The LEVELVAR=0 option restricts the variance of the disturbance term on the State equation, named Eta above, to be zero. This restricts the Level component to be a static, best-fit-to-the-data flat line in the model.
• The IRREGULAR statement creates the white noise error term on the Observation equation, named Epsilon above, with no restrictions on its variance.
• The MODEL statement specifies the Observation equation.
• OUTPUT generates the table containing forecasts, standard errors, CI and so on for Sales and the model’s components.

Note that the TREND statement provides a convenient way to specify commonly used State elements. We’ll spend time discussing the specification of other State elements in subsequent posts in this series.

Both estimates listed below are significantly different from zero and both come from the Observation equation; the Model Parameter estimate represents the variance of the white noise error term, and the regression parameter estimate is the effect of the promotions input on sales.

Fit statistics provide a reference for subsequent model refinements.

The plot shows the best-fit static representation of the Level component (red line) along with the combined components (Level, Promo, WN) forecast for Sales (black line). Sales actuals are the dots. The forecast suggests that the model is too simple; forecasts are missing systematically high through late 2017 and then systematically low to about April, 2018.

Now, let’s generalize the model so that Level can be included as a dynamic component by commenting out the LEVELVAR=0 option.

There are now two estimated Model Parameters: the estimated variance of the Observation equation error, WN, and the estimate of Eta, the variance on the State equation disturbance. Both are significant using at t Value > 2 rule of thumb.

Note that an insignificant variance estimate for Level would not indicate that the Level is indistinguishable from zero. It would indicate that the Level component in the model is likely static, and that the previous model is appropriate.

Fit statistics improved relative to the model with the static Level component.

The plot of the forecast for the Level component shows its trajectory over time.

The plot below shows the forecast of the Level component (red line) along with the combined components (Level, Promo, WN) forecast for Sales (black dashed line) overlaid on the Sales actuals.

The goal of this post was to introduce SSMs to applied analysts and to highlight some of their main features. We described how the components of systematic variation in the data can be modeled individually and then combined to create a forecast for the dependent variable. A simple example using a dynamic level and a static regressor highlighted the SSMs flexibility in accommodating both static and dynamic components and then illustrated how the estimated components in SSMs can be assessed to provide insight into how these components evolve over time. In this series’ next post, we’ll describe how to add a seasonal component that evolves as a function of time to the model, so stay tuned for more SSM action!

Find more articles from SAS Global Enablement and Learning here.