Honing in on Troublesome Time Series: Interactive Modeling in SAS Visual Forecasting

[#1]

SAS Visual Forecasting lets you automate most models and focus the time of your expert forecasters on those models that need extra attention. The Interactive Modeling Node in SAS Visual Forecasting lets you easily drill down into a specific time series that needs some TLC from an expert. Although the Interactive Modeling Node in SAS Visual Forecasting has been around since Stable release 2020.1.3 (February 2021), it has undergone a number of enhancements over the last 16 months.

At its essence, the Interactive Modeling tab lets you lets you zoom in on a single time series to improve its forecast by:

• Exploring diagnostic plots
• Creating your own models
• Visually comparing multiple models
• Manually setting a champion model

This blog will demonstrate using the Interactive Modeling node in the latest LTS version (2022.1 – May 2022). I’ll also give you a sneak peek at the model download capabilities, just recently made available in stable release 2022.1.1 (May 2022). I catalog enhancements by stable release since the inception of the Interactive Modeling node in February 2021.

Using the Interactive Modeling Node

Once you run the automatic forecasting, you can add an Interactive Modeling node between a modeling node (or nodes) and the Model Comparison node.

Let’s say you are interested in electricity generation by source and state. Our example will use the ELECGENSUBSET data set, which includes a number of variables as shown below:

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

If you are unfamiliar with setting up a SAS Visual Forecasting project in SAS Model Studio, please see my course Using SAS Visual Forecasting on SAS Viya 4 https://eduvle.sas.com/course/view.php?id=1976, Chapter 3, Visual Interface for Visual Forecasting, which includes step-by-step instructions for building a pipeline with the ELECGENSUBSET data in exercise 03_021_Build_Pipeline. Recall that if you build a pipeline using the auto-forecasting template, the Auto-forecasting node will generate exponential smoothing (ESM), autoregressive integrated moving average with any significant inputs (ARIMAX) and intermittent demand (IDM) models. The Auto-forecasting node will then select the best model for each series using the mean absolute percent error (MAPE).

Three ways to add in interactive modeling node:

1. Open Nodes pane on left, select Postprocessing, Interactive Modeling, then drag to connector (in our example, the connector between the Auto-forecasting and Model Comparison nodes).
   
   

2. Select connector (in our example, the connector between the Auto-forecasting node and the Model Comparison node), right click, Insert, Postprocessing, Interactive Modeling.
 

3. Select Model Comparison node, right click, Add parent Node, Postprocessing, Interactive Modeling.

Once you have run the Interactive Modeling Node, right click and open it. There are three panes:

• Forecast
• Modeling
• Series Analysis

Forecast Pane

In the Forecast pane you can select individual time series (or multiple time series by holding down the Control key) in the right pane where all of the series are listed. Below I’ve selected Natural Gas for Pennsylvania (PA), Illinois (IL), and Maryland (MD).

Envelope plots including predicted and actual range, predicted and actual one and two standard deviations can be toggled off or on. As you see below, when they are toggled on a check mark appears next to “Actual values” and “Predicted values” under the envelope plot icon. Actual values (shown as dots), predicted values (shown as lines), and confidence limits for the forecasts (shaded areas) can also be turned on or off.

If you try to select more than 16 series, only 16 will be selected. A message will appear that “One or more of the selected series cannot be displayed on the graph. The maximum number of series that can be displayed is 16,” as illustrated below.

Modeling Pane

By default when you open the Interactive modeling node you will see the Modeling pane for all of the series, as shown below (269 of 269 series).

You can hone in on a specific series, such as Wind: CA, as shown below.

Or natural gas: PA, as shown below. Expand the graph using the outfacing arrows.

By default you will arrive in Graph view. Select table view.

This lets you see the full table including the selected _MODEL_, the Time ID values, Actual Values, Predicted Values, Prediction Standard Errors, Lower Confidence Limits, and Upper Confidence Limits.
|

Close the window. Now you can select the down arrow to the right of View diagnostic plot/table for a number of choices.

Explore these. Below see the White noise probability test (log scale).

Model fit includes parameter estimates and statistics of fit.

Below see parameter estimates.

Forecasts include:

For example, see Seasonal cycles below.

Basic error analysis includes Prediction errors and a Prediction error histogram. See the histogram below.

To download plots and tables select the down arrow.

Compare Models

Select the models you want to compare and then click the icon

to compare selected models.

As shown below, the best model is based on the selection criterion.

Select the icon

at the top right to customize which criteria are shown.
 

I’ve selected the Akaike Information Criterion (AIC), Amemiya’s prediction criterion (APC), mean absolute percent error (MAPE), and root mean square error (RMSE). The selection criterion that you use depends heavily on the domain in which you are working. For example, many economists use MAPE and many biologists commonly use RMSE. For a full description of the model selection criteria available see the SAS Model Studio: SAS Visual Forecasting User’s Guide .

Notice that the MAPE (the default selection criterion) selects a different model from the AIC, APC, or RMSE.

If you right click a model, you can set it as champion or created a copy of the selected model.

You will have the opportunity to confirm you want to commit to these changes.

Once you have set a champion, the champion icon

will appear next to the model that you set as champion.

To deselect the champion setting you can select the champion icon at the top right of your model table or right click the champion model and select Unselect champion model. You will need to commit again.

To see full model details roll over the model details or select the details icon.

To create your own model select the icon and choose exponential smoothing, ARIMA or subset (factored) ARIMA.

Errors or warnings are indicated in the Details column for the model:

— Error  

— Warning  

Right-click the model and select View status details. The Status Details window shows the error or warning messages generated by the model.

Series Analysis Pane

A number of series analysis graphs are available to you, such as seasonal cycles, percent change, histogram illustrating the distribution, etc.

To view details, select the vertical ellipsis (snowman) at the bottom right of the graph of interest.

You can turn on Graph view, Table view, or both.

Below see an example of a seasonal cycle graph.

Below is a close up view of a percent change graph.

The distribution of the data by the dependent variable electricity generation in megawatt-hours is illustrated below.

Advanced diagnostic graphs such as IACF and PACF are available by right clicking and selecting autocorrelation analysis.

For information on how to interpret these diagnostic graphs, see my previous post on interpreting results and diagnostic plots 

Downloading data

Notice the download arrow in top right of your Graph view. You can download the data for this series.

It will download to your downloads folder as a comma separated (.csv) file.

This functionality is available in LTS 2022.1.

Downloading Models: Sneak Peek (first available in stable 2022.1.1)

A new functionality available in stable 2022.1.1. is the ability to easily download your models from SAS Model Studio. From the Modeling tab in the Interactive Modeling node simply right click on the model that you want to download. You can download the system-generated models or your own custom-build models.

It will download locally as a CASL file.

You can access the file in your Downloads folder.
 

For more details see the Model Studio: SAS Visual Forecasting User’s Guide

You can also download the log for the Interactive node which provides a bit of information.

Summary

Many time series will get good forecasts simply through automated forecasting with SAS Model Studio. But for those that don’t, now you see how easy it is to diagnose, tweak and compare models using the Interactive Modeling Node to achieve high performance for even difficult to model time series.

Since its appearance in stable release 2020.1.3 (February 2021), the Interactive Modeling node has been continuously enhanced and improved.  I’ve detailed these improvements by version/date below.

Series Analysis tab

• Analyze individual series in the Series Analysis tab by generating plots for the dependent variable and any independent variables in the project.
  • Time series plots, histograms, and seasonal cycles plots available beginning with stable release 2020.1.3 (February 2021), enhanced in stable release 2020.1.4 (March 2021)
  • Plots for normal and standardized autocorrelation, partial autocorrelation, and inverse autocorrelation (available stable 2021.1.6 - October 2021)
  • Plot to display percent change between each time period for the series (available stable 2021.2.1 - November 2021)
  • Seasonal cycles plot lets you change the length of the season for each plot to investigate different seasonal patterns (available stable 2021.2.1 - November 2021)
  • Update settings for percent change and seasonal cycles plots on the enlarged “View Details” plot without having to return to the Series Analysis canvas (added in stable release 2021.2.2 - December 2021).

Modeling tab

• Compare models in the model selection list and choose a different model as the champion using the new Modeling tab (available beginning with stable release 2020.1.5 - April 2021)
• Additional plots
  • (added in stable release 2021.1.1 - May 2021):
    • Forecast region only
    • Historical and forecast region
    • Historical region only
    • Parameter estimates
    • Prediction error histogram
    • Prediction errors
    • Statistics of fit
    • White noise probability test
    • White noise probability test (log scale)
  • (added in stable release 2021.2.2 - December 2021):
    • Seasonal cycles
    • Standardized autocorrelation function
    • Partial autocorrelation function
    • Inverse autocorrelation function
    • Standardized inverse autocorrelation function
• Create new models for any time series
  • Exponential smoothing (ESM) models (available stable 2021.1.3 - July 2021)
    • Copy and edit ESM models (available stable 2021.1.5 - September 2021)
  • Autoregressive Integrated Moving Average (ARIMA) models (available stable 2021.1.4 - August 2021)
    • Add inputs to ARIMA models such as independent variables, events, or predefined variables (available stable 2021.1.6 - October 2021)
    • Copy and edit ARIMA models (available stable 2021.1.6 - October 2021)
    • Create subset ARIMA models for any time series (available stable 2021.2.1 - November 2021) SAS Help Center: Creating a Subset (Factored) ARIMA Model
• Champions:
  • Set Interactive Modeling node to maintain (reapply) any models that you committed as a champion (added in stable release 2021.2.2 - December 2021).
  • “Reset” the selected champions, i.e., revert to the champion selected by the preceding modeling node (available stable release 2021.2.3 – January 2022)
  • When models have been selected as champions using the Interactive Modeling node, a notification has been added to inform you if any of the champion models have failed after a project update. You can then make changes to correct the models or else select a different champion. (Added stable 2021.2.6 – April 2022)
• View the properties of a model that you select from the model selection list (stable 2021.2.5 – March 2022
  
  
  
• Download
  • Download the data for a time series and the CASL code for a model. This code could then be run in SAS Studio (stable 2022.1.1 – May 2022)

 
For More Information

• Joe Katz’s video on Interactive Modeling in SAS Visual Forecasting from May 2021
• Documentation Overview of Interactive Modeling
• Subset (factored) and seasonal ARMA models
• Subset (factored) ARIMA models

Find more articles from SAS Global Enablement and Learning here.

[SpirosP]

Great article Beth! Thanks for sharing