Using Recurrent Neural Networks in SAS Visual Forecasting

2 Likes

The RNNSPEC object now lets you easily compare RNN models to other time series models in SAS Visual Forecasting’s ATSM Package!

RNNS can give highly accurate results when working with sequential data such as time series data. They are used in many applications when order matters. Examples include natural language understanding, natural language generation, physical processes, and, of course, forecasting.

The network is called recurrent because the network feeds back into itself and makes decisions in several steps. Some RNN models can remember dependencies for long periods of time. For more background on RNNS, see my earlier post RNNs in SAS Viya which focuses on using RNNs with SAS Visual Data Mining and Machine Learning.

The RNNSPEC object generates time series recurrent neural network (RNN) model specifications. The RNNSPEC object first became available in SAS Viya in the 2021.2.2 stable release for use in the TSM object. It became easier to use with the ATSM package in LTS 2023.03 (May 2023).

ATSM Overview

Before we get into the RNNSPEC object, let’s do a brief review of the ATSM Package. Recall that we use the ATSM package to AUTOMATICALLY create time series models. The steps involved include:

Define time series variables to be used for modeling and forecasting
Diagnose and generate time series models
1. Set diagnose specifications for each model family in consideration
Forecast time series
1. Set model selection specifications
2. Forecast time series with the champion model
Collect results
Save the outputs for review or other analysis tasks

We have four main types of objects in the ATSM package:

Time series modeling and forecasting objects
Objects for controlling model identification
Collector objects
Repeater objects

Lets look more deeply at each of these object categories:

Time series modeling and forecasting objects. These include the main forecasting engine object, the FORENG object.

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

Objects for controlling model identification. These include the objects that control model identification.

Collector objects collect results. They essentially create a snapshot of results from the ATSM objects and save the results to a CAS table.

Repeater objects relay information. They make information from a CAS table (previously defined by a collector object) available to other ATSM objects.

So how do these all fit together? In this partial diagram, we see that the collector object OUTDIAG collects the parameters (diagnostic control options) from the DIAGSPEC object instance. The repeater object INDIAG replays (repeats) the diagnostic control options to the DIAGNOSE object instance.

Here see a full ATSM data flow example.

Source: https://go.documentation.sas.com/doc/en/pgmsascdc/v_037/castsp/castsp_atsm_sect002.htm

The FORENG (forecasting engine) object is the brains of the process. This is where the modeling happens. The addFrom method adds models to a FORENG instance. The model specification objects available in the addFrom method are shown in the table below.

How do we use these objects? To use ATSM objects within PROC TSMODEL your code must:

Require the ATSM package in the TSMODEL procedure
Declare each object referenced in your programs
Initialize the object to create an object instance
Use object methods to set options or configure inputs and outputs
Organize and customize workflows using scripting language statements

Using the RNNSPEC object in SAS Visual Forecasting

Now that we’ve had a brief background primer, let’s talk specifically about the RNNSPEC object.

Three RNN model architectures are available in SAS Visual Forecasting

Original recurrent neural network (RNN)
Long short-term memory unit network (LSTM)
Gate recurrent unit network (GRU)

These are available via PROC TSMODEL

Time Series Neural Network Forecasting Package
- standalone RNN forecasting package
- available beginning in stable release 2020.1.2
TSM package includes RNNSPEC
ATSM package now supports RNN forecasting models
- available beginning in LTS 2023.03 (May 2023)

The RNNSPEC object can now be used in the ATSM package to let you compare RNN models to your other forecasting models such as autoregressive integrated moving average models, exponential smoothing models, and unobserved components models. You now have the same options in TSM and ATSM. You must first define the RNNSPEC object using the TSM package and the ATSM package then honors all RNNSPEC object options.

You can add RNN models directly to a ATSM FORENG object, an ATSM SELSPEC object or a TSM FORENG object. The RNNSPEC must be initialized in the TSM Object. See a simple flow diagram below.

Source: SAS Viya documentation https://go.documentation.sas.com/doc/en/pgmsascdc/default/casecon/casecon_tsm_sect127.htm

As with your other forecasting models, you can include input and event variables in the model by using the TSDF object. Use inputs and events from the Time Series Data Frame (TSDF) object in an RNN model by using the RNNSPEC object’s AddTF method. The RNNSPEC model output can be viewed in the collector objects OUTFOR, OUTMODELINFO, OUTSELECT, and OUTSTAT.

Below is an example of code for running RNNSPEC with DIAGNOSE object SPECS courtesy of Taiyeong Lee.

/*******************************************/
/*	                                   */ 
/*  RNNSPEC with DIAGNOSE object specs     */
/*  based on example from Taiyeong Lee     */
/*  modified 20230622 by Beth Ebersole     */
/*                                         */
/*******************************************/

/* Create a CAS session and a CAS library */
cas mySession sessopts=(caslib=casuser timeout=1800 locale="en_US");
libname mycas cas sessref = mySession;

data mycas.air;
    set sashelp.air;
run;

proc tsmodel data=mycas.air outlog=mycas.outlog
     logcontrol=(error=keep warning=keep note=keep none=keep)
     outobj=(outstat=mycas.outstat modInfo=mycas.modInfo 
             airSelect=mycas.airSelect);
     id date interval=month;
     var air;
     require atsm tsm;

     SUBMIT;
	    
        declare object dataFrame(tsdf);
        rc = dataFrame.Initialize();
        rc = dataFrame.AddY(air);
        rc = dataFrame.SetOption('seasonality', 12);

        declare object diagSpec(diagspec);
        rc = diagSpec.Open();
        rc = diagSpec.SetARIMAX();
		rc = diagSpec.SetUCM();
        rc = diagSpec.Close();

        declare object diagnose(diagnose);
        rc = diagnose.Initialize(dataFrame);
        rc = diagnose.SetSpec(diagSpec);
        rc = diagnose.SetOption('holdout', 12);
        rc = diagnose.Run();

		declare object rnnspec_GRU(rnnspec);
		rc = rnnspec_GRU.Open( );
	    rc = rnnspec_GRU.SetOption(
							'RNNTYPE','GRU',
							'NINPUT', 12,
							'NVALIDATION',12,
							'NLAYER', 4,
							'NNEURONH',30,
							'NORMALIZE', 'STD',
							'BATCHSELECTION', 'SEQ',
							'POSTTRAIN', 'YES',
							'SEED', 12345
							);
	     rc = rnnspec_GRU.SetOptimizer(
							'ALGORITHM','ADAM',
							'LEARNINGRATE', 0.1,
							'BETA1', 0.8,
							'BETA2', 0.9,
							'LEARNINGPOLICY', 'STEP',
							'STEPSIZE', 5,
							'GAMMA', 0.5,
							'STAGNATION', 20,
							'MINIBATCHSIZE', 8,
							'WARMUPEPOCHS',15,
							'MAXEPOCHS', 100
							);
	 	rc = rnnspec_GRU.Close();
		
		declare object rnnspec_LSTM(rnnspec);
		rc = rnnspec_LSTM.Open( );		
		rc = rnnspec_LSTM.SetOption(
            'RNNTYPE','LSTM',
            'NINPUT', 12,
            'NVALIDATION', 0,
            'NLAYER', 4,
            'NNEURONH',30,
            'NORMALIZE', 'STD',
      		'BATCHSELECTION','SEQ',
            'POSTTRAIN', 'NO',
            'SEED', 12345
                    );
		 rc = rnnspec_LSTM.SetOptimizer(
         'ALGORITHM','ADAM',
         'LEARNINGRATE', 0.01,
         'BETA1', 0.8,
         'BETA2', 0.9,
         'LEARNINGPOLICY', 'STEP',
         'STEPSIZE', 5,
         'GAMMA', 0.5,
         'STAGNATION', 20,
         'MINIBATCHSIZE',8,
         'WARMUPEPOCHS',15,
         'MAXEPOCHS', 100
         );
		 rc = rnnspec_LSTM.close();

        declare object foreng(foreng);
		rc = foreng.Initialize(diagnose);
        rc = foreng.addFrom(rnnspec_GRU);
		rc = foreng.addFrom(rnnspec_LSTM);
        rc = foreng.SetOption('lead', 12);
		rc = foreng.SetOption('alpha',0.05);
		rc = foreng.setOption('SEASONTEST','NONE');
        rc = foreng.Run();

        declare object modInfo(outmodelinfo);
        rc = modInfo.Collect(foreng);
        declare object airSelect(outselect);
        rc = airSelect.Collect(foreng);
		declare object outstat(outstat);
		rc = outstat.Collect(foreng);

     ENDSUBMIT;

	  print outlog;
 quit;

 proc print data=mycas.modInfo;
 run;
 proc print data=mycas.airSelect;
 run;
 proc print data=mycas.outstat;
 run;

Example Output:

Example Results:

For More Information

SAS Viya documentation
Recurrent Neural Networks: An Essential Tool for Machine Learning 2018 by Yue Xi (with assistance from Doug Cairns)
2021 Post How to incorporate Recurrent Neural Networks in your SAS Visual Forecasting pipelines
General background on RNNs by Brandon Rohrer
To learn about Distributed and Parallel Hyperparameter Tuning of RNN Forecasting Models in PROC TSMODEL see this post 2023 by Taiyeong Lee.

Using Recurrent Neural Networks in SAS Visual Forecasting

Free course: Data Literacy Essentials

Get Started