Incorporating Generative AI models in SAS Visual Forecasting - Part 2

3 Likes

Step by step guide to create a GenAI node in SAS® Visual Forecasting

Over the last 12 months, more and more Foundation Forecasting Models have been entering the market. They are already showing some very impressive results and if you are involved in the forecasting space, you should keep an eye on their development over the next months and years. To give a very brief overview, these models are pre-trained on a large amount of publicly available, real-world and synthetic time-series data, in order to perform forecasts on unknown data sets. If you recognize this approach from other deep learning models - like LLMs - than that is no coincidence.

In this article, I will walk you through one use case of these models within the SAS® ecosystem. I developed a custom modeling node for SAS® Visual Forecasting (VF), which uses Amazon Science's Chronos Forecasting Model, to be used in a pipeline. If you want to learn more about this pre-trained model and how to deploy it yourself, check out their GitHub - amazon-science/chronos-forecasting.

The full and up to date code can be found in this GitHub repository.

Using the Node

First, let's talk about what we can achieve with this node. I used the sashelp.pricedata dataset to build a simple pipeline in SAS® Model Studio. If you are not familiar with this dataset, it provides monthly historic price and sales data for 17 products and multiple regions from January 1998 until December 2002. In this case I have grouped it by each unique product and I will predict the sales amount for each of those. The finished pipeline looks like this:

Pipeline Structure

As you can see, I don't use any state-of-the-art forecasting algorithms, but it should still show you some of the potential these models carry.

That being said, let's run the pipeline and see what results we get.

Champion Model

After a couple of seconds, the pipeline finishes successfully and the ensemble node is chosen as the champion model for this pipeline.

We can look the results to get an idea of how Chronos performed against the Auto-Forecasting node.

MAPE Comparison

You will notice that the foundation model has a bigger spread in its MAPE, reaching a higher accuracy for some series, as compared to the Auto-forecasting node.

Keeping that in mind, what does the Ensemble node considers to be the "better" model?

Ensemble Results

As you can see, the Foundation Forecasting Model generates a better fitting model on about 60% of the time-series in this dataset.

Now, if this piqued your interest and you want to see how to implement your own Foundation Model Node in a simple and reusable manner, then keep reading and I will explain how.

Building the Node

Prerequisites:

The node's code accesses the model by calling a REST endpoint where it is deployed. In this case it is a Python script which is hosted on the same Kubernetes Cluster as the SAS® Viya installation I am using. If you want to know how you can achieve this, check out part 1 of this series!
You will also need to install and set-up the EXTLANG package for PROC TSMODEL. Accessing a REST Endpoint in a node is only possible through Python or R code added in this way. If you would like to familiarize yourself with this package, you can read this documentation.
Lastly, your EXTLANG configuration will need to have two packages installed: numpy and requests.

Basics

A VF node consists of 4 necessary files:

metadata.json - Contains the build version of the node.
validation.xml - This will define how the user can interact with the node on the right hand panel. You can set options and their restrictions. Defaults are set in the next file.
template.json - Apart from setting default values for your options set in validation.xml, this file will define the name and description of your node, among other properties.
code.sas - This node contains the actual logic of your code. It's main part will be a PROC TSMODEL that you use to run the forecast model on the given data. You will be able to use multiple macro variables defined in VF as well as the options you defined in the validation.xml in your code. We will take a deeper look at this file in a later section.

For the sake of simplicity, I will be skipping an in-depth look at the first three files. What is important for our SAS® code, is that the user will be able to set options, which we can use as macro variables.

If you would like to get a better understanding of how these files work and ways to set them up, I would recommend you to take a look at one of these articles:

It can also be helpful to download and take a look at pre-existing nodes like Auto-Forecasting.

Implementing the model

The core of any node in VF will be the code.sas file. Let's walk our way through it from top to bottom:

Setup

First, let me tell you about any setup we need to complete before calling the model.

ods output OutInfo = _outInformation;

proc tsmodel data=&vf_libIn.."&vf_inData"n lead = &vf_lead.
            outobj=(	outfor  = &vf_libOut.."&vf_outFor"n
                        outStat = &vf_libOut.."&vf_outStat"n
                        outvarstatus=&vf_libOut..outvarstatus
                        pylog=&vf_libOut..pylog	)
            outarray = &vf_libOut..outarray
            outscalar = &vf_libOut..outscalar
            outlog = &vf_libOut.."&vf_outLog"n;
    id &vf_timeID interval = &vf_timeIDInterval setmissing = &vf_setMissing trimid = LEFT;
    %vf_varsTSMODEL;

    *define the by variables if exist;
    %if "&vf_byVars" ne "" %then %do;
       by &vf_byVars;
    %end;

    outarray ffm_fcst ffm_lfcst ffm_ufcst ffm_err ffm_stderr;
    outscalars _NAME_ $16 _MODEL_ $16 _MODELTYPE_ $16 _DEPTRANS_ $16 _SEASONAL_ _TREND_ _INPUTS_ _EVENTS_ _OUTLIERS_ _SOURCE_ $16;

The center part will be a TSMODEL procedure. To set it up, we will need to define some tables. These include the input table, the output forecast results and forecast statistics tables, and other tables and values.

For our use case, it will be important do declare the outarray and outscalar tables, as well as their columns, too. I will explain why this is necessary in a bit.

    require tsm extlang;

Next, we import the two important packages for this node. TSM is used create model objects from the python code's output which are then used to collect data for the necessary output tables. EXTLANG is the package that allows us to add Python and R code in the TSMODEL procedure. For any documentation regarding TSM , EXTLANG or PROC TSMODEL , refer to their respective documentations.

    submit;

    MODEL_SELECTION = '&_model_selection';

    CHRONOS_DOMAIN = '&_model_domain';          
    CHRONOS_ENDPOINT = '&_model_endpoint';      
    CHRONOS_URL = CHRONOS_DOMAIN || CHRONOS_ENDPOINT;

    NUM_SAMPLES = &_num_samples.;               
    TEMPERATURE = &_temperature.;               
    TOP_K = &_top_k.;                           
    TOP_P = &_top_p.;

Now, we need to start the model code which is wrapped in a submit-endsubmit-block. This is also where the options I talked about earlier come into play. As you can see, they are stored as macro variables and this is what they represent:

&_model_selection - This defines the type of model the data is supposed to be forecast with. As the model only supports Chronos right now, the only value this variable can contain is 'CHRONOS'.
&_model_domain and &_model_endpoint - These two variables set the REST endpoint, at which the Chronos model can be reached through a POST call.
&_num_samples - Now, we turn to model specific options. For this one, we can set the amount of times the distribution for each data-point in our prediction is sampled to generate the forecast values. Increasing this will generally improve the model accuracy and reliability, but also increase the execution time.
&_temperature - If you are already familiar with other types of foundation models, the last three options will be recognizable to you. Simply put, The temperature will change the shape of the probability distribution for the next value(s). A higher temperature will make the result "more random".
&_top_k and &_top_p - These values will also affect the prediction of the next value(s). While Top-K will set the absolute amount of predictions with the highest probability used in generating the value(s), Top-p will do the same with their sum of probabilities.

Writing Python code to access the model

The next step is to write the Python code that we will use to access Chronos and process the output.

    declare object py(PYTHON3);
    rc = py.Initialize();

	rc = py.AddVariable(&vf_depVar,'ALIAS','TARGET') ;
    rc = py.AddVariable(&vf_timeID, 'ALIAS', 'ds');
    rc = py.AddVariable(CHRONOS_URL);
    rc = py.AddVariable(_LEAD_); 
    rc = py.AddVariable(NUM_SAMPLES);
    rc = py.AddVariable(TEMPERATURE);
    rc = py.AddVariable(TOP_K);
    rc = py.AddVariable(TOP_P);
    rc = py.AddVariable(ffm_fcst,"READONLY","NO","ARRAYRESIZE","YES","ALIAS",'PREDICT');
	rc = py.AddVariable(ffm_lfcst,"READONLY","NO","ARRAYRESIZE","YES","ALIAS",'LOWER');
	rc = py.AddVariable(ffm_ufcst,"READONLY","NO","ARRAYRESIZE","YES","ALIAS",'UPPER');
	rc = py.AddVariable(ffm_err,"READONLY","NO","ARRAYRESIZE","YES","ALIAS",'ERR');
	rc = py.AddVariable(ffm_stderr,"READONLY","NO","ARRAYRESIZE","YES","ALIAS",'STDERR');
     * rc = py.AddEnvVariable('_TKMBPY_DEBUG_FILES_PATH', &log_folder);

We need to initialize a Python 3 object to which we can add variables and push Python code.

We will also need to assign the variables we just declared and the series we want to store out forecast results in to the object. This way, we can work with them in the Python code.

Writing the Python code to call the model endpoint and postprocess the results will be our next step. There are three different ways you can add code to the py object:

pushCodeFromTable() let's you define a CAS table you stored Python code in.
pushCodeFile() will let you define the path to a Python file.
pushCodeLine() is used to add Python code as strings line by line.

We will make use of the latter. Let's see what this looks like.

    rc = py.pushCodeLine("import json");
    rc = py.pushCodeLine("import math");
    rc = py.pushCodeLine("import requests as req");
    rc = py.pushCodeLine("import numpy as np");
	
	rc = py.pushCodeLine("url = CHRONOS_URL");
    rc = py.pushCodeLine("prediction_length = int(_LEAD_)");
    rc = py.pushCodeLine("num_samples = int(NUM_SAMPLES)");
    rc = py.pushCodeLine("temperature = float(TEMPERATURE)");
    rc = py.pushCodeLine("top_k = int(TOP_K)");
    rc = py.pushCodeLine("top_p = float(TOP_P)");

The first step will be to import any needed Python packages. We will also parse the variables we added to make sure we have valid data-types.

    rc = py.pushCodeLine("payload = json.dumps({");
    rc = py.pushCodeLine("	'prediction_length': prediction_length,");
    rc = py.pushCodeLine("	'num_samples': num_samples,");
    rc = py.pushCodeLine("	'temperature': temperature,");
    rc = py.pushCodeLine("	'top_k': top_k,");
    rc = py.pushCodeLine("	'top_p': top_p,");
    rc = py.pushCodeLine("	'data': TARGET.tolist()[:-prediction_length]");
    rc = py.pushCodeLine("})");
    rc = py.pushCodeLine("headers = {'Content-Type': 'application/json'}");

    rc = py.pushCodeLine("resp = req.post(url, headers=headers, data=payload, verify=False)");
    
    rc = py.pushCodeLine("forecast_validation = resp.text.splitlines()");
    rc = py.pushCodeLine("forecast_validation_values = np.genfromtxt(forecast_validation, delimiter=',', skip_header=1)");

Due to its architecture, Chronos will only be providing us with future forecast values. This creates a problem when we want to use the model as a node, as we will need to generate some form of forecast statistics to compare the results to other nodes. The simplest solution to this will be to run the model using all but the last x values as input data, where x is the forecast length.

The way that this endpoint is set up, we will receive a string containing out forecast values. This string represents a data set containing 3 by x values, one column for each the median prediction as well as the lower and upper 80-% prediction interval borders. We convert this string to numeric 2-dimensional arrays for the next steps.

    rc = py.pushCodeLine("payload = json.dumps({");
    rc = py.pushCodeLine("	'prediction_length': prediction_length,");
    rc = py.pushCodeLine("	'num_samples': num_samples,");
    rc = py.pushCodeLine("	'temperature': temperature,");
    rc = py.pushCodeLine("	'top_k': top_k,");
    rc = py.pushCodeLine("	'top_p': top_p,");
    rc = py.pushCodeLine("	'data': TARGET.tolist()");
    rc = py.pushCodeLine("})");
    rc = py.pushCodeLine("headers = {'Content-Type': 'application/json'}");

    rc = py.pushCodeLine("resp = req.post(url, headers=headers, data=payload, verify=False)");
    
    rc = py.pushCodeLine("forecast = resp.text.splitlines()");
    rc = py.pushCodeLine("forecast_header = forecast[0].split(',')");
    rc = py.pushCodeLine("forecast_values = np.genfromtxt(forecast, delimiter=',', skip_header=1)");

This step is then repeated for the actual forecast prediction.

    rc = py.pushCodeLine("prediction = np.empty(TARGET.shape[0] - (2 * prediction_length))");
    rc = py.pushCodeLine("prediction[:] = np.nan");


    rc = py.pushCodeLine("lower = np.empty(TARGET.shape[0] - (2 * prediction_length))");
    rc = py.pushCodeLine("lower[:] = np.nan");
	rc = py.pushCodeLine("upper = np.empty(TARGET.shape[0] - (2 * prediction_length))");
    rc = py.pushCodeLine("upper[:] = np.nan");

    
    rc = py.pushCodeLine("prediction = np.concatenate((prediction, forecast_validation_values[:, forecast_header.index('median')]))");
    rc = py.pushCodeLine("prediction = np.concatenate((prediction, forecast_values[:, forecast_header.index('median')]))");

    
    rc = py.pushCodeLine("lower = np.concatenate((lower, forecast_validation_values[:, forecast_header.index('low')]))");
    rc = py.pushCodeLine("lower = np.concatenate((lower, forecast_values[:, forecast_header.index('low')]))");
	
    
    rc = py.pushCodeLine("upper = np.concatenate((upper, forecast_validation_values[:, forecast_header.index('high')]))");
    rc = py.pushCodeLine("upper = np.concatenate((upper, forecast_values[:, forecast_header.index('high')]))");

We continue our post-processing by turning this array into three separate series that have the values placed at the correct index. That means that for each time-stamp we do not predict, that value needs to be set to np.nan for it to show up as a missing numeric value (.) in the final tables.

    rc = py.pushCodeLine("err = TARGET - prediction");

	rc = py.pushCodeLine("interval_width = upper - lower");
	rc = py.pushCodeLine("z_score = 1.28");
	rc = py.pushCodeLine("stderr = interval_width / (2 * z_score)");

	rc = py.pushCodeLine("PREDICT = prediction");
	rc = py.pushCodeLine("LOWER = lower");
	rc = py.pushCodeLine("UPPER = upper");
	rc = py.pushCodeLine("ERR = err");
	rc = py.pushCodeLine("STDERR = stderr");

    rc = py.Run();

As a last step, we create the prediction error and standard error series and assign our newly created series to the variables we added to the object at the beginning of the python block. We run the code by calling the Run() method.

Generating the Model Objects

Part of the node workflow is defining the following objects.

	declare object pylog(OUTEXTLOG);
	rc = pylog.Collect(py, 'EXECUTION');

	declare object outvarstatus(OUTEXTVARSTATUS);
	rc = outvarstatus.Collect(py);

	declare object pyExmSpec(EXMSPEC);
	rc = pyExmSpec.open();
	rc = pyExmSpec.setOption('METHOD', 'PERFECT');
	rc = pyExmSpec.setOption('NLAGPCT', 0);
	rc = pyExmSpec.setOption('PREDICT', 'ffm_fcst');
	rc = pyExmSpec.setOption('LOWER', 'ffm_lfcst');
	rc = pyExmSpec.setOption('UPPER', 'ffm_ufcst');
	rc = pyExmSpec.setOption('STDERR', 'ffm_stderr');
	rc = pyExmSpec.close();

    declare object tsm(tsm);

	
    rc = tsm.Initialize(pyExmSpec);
    rc = tsm.AddExternal(ffm_fcst, 'PREDICT');
	rc = tsm.AddExternal(ffm_lfcst, 'LOWER');
	rc = tsm.AddExternal(ffm_ufcst, 'UPPER');
    rc = tsm.AddExternal(ffm_err, 'ERROR');
	rc = tsm.AddExternal(ffm_stderr, 'STDERR');
	rc = tsm.SetY(&vf_depVar);
	rc = tsm.SetOption('LEAD', &vf_lead);
	rc = tsm.SetOption('ALPHA', 0.2);

    rc = tsm.Run();

	declare object outFor(tsmFor);
	declare object outStat(tsmStat);

	rc = outFor.collect(tsm);
	rc = outFor.SetOption('MODELNAME', 'Chronos');
	rc = outStat.collect(tsm);
	rc = outStat.SetOption('MODELNAME', 'Chronos');

Now that we have our forecast values, we need to make sure that they, along with any additional model information, are stored in the right output tables. As you may remember, we declared some of those tables under the outobj statement when opening the TSMODEL procedure. We will work on populating them in this step.

The first two objects are relatively self explanatory. They will store any log lines that were collected during the execution of the external language code and show the status of any shared variables we used in it.

Next, we will declare an external model specification. This will allow us to define how each series is supposed to be mapped for the output forecast table. We use this object to declare a time-series model instance. Doing it this way will allow us to add the values we generated in Python as external series for the model. we also declare, which forecast lead was used and the alpha value, meaning the prediction interval level. We can now finally set the OUTFOR and OUTSTAT objects to collect the necessary information from the model and map it to the output tables.

    _NAME_ = vname(&vf_depVar);
	_MODEL_ = "ffmModel";
	_MODELTYPE_ = MODEL_SELECTION;
	_DEPTRANS_ = "NONE";
	_SEASONAL_ = 1;
	_TREND_ = 1;
	_INPUTS_ = 0;
	_EVENTS_ = 0;
	_OUTLIERS_ = 0;
	_SOURCE_ = "TSM.EXMSPEC";

	endsubmit;
run;

This final step in the TSMODEL procedure will be necessary since we use the simpler TSM package instead of ATSM. Due to this, we are missing some values for the node to execute without any issues. Namely, we have the OUTMODELINFO table that is missing, and a row that we need to add to the OUTSTAT table. We will be using the OUTSCALAR table to keep track of the values and create / modify the tables in separate data steps. That being done, we can close the submit block and end the TSMODEL procedure.

Fill in what's Missing

All that is left to do is some minor housekeeping to make sure the node runs without errors.

data &vf_libOut.."&vf_outModelInfo"n;
	retain _NAME_ _MODEL_ _MODELTYPE_ _DEPTRANS_ _SEASONAL_ _TREND_ _INPUTS_ _EVENTS_ _OUTLIERS_ _SOURCE_ _STATUS_;
	set &vf_libOut..outscalar;
run;

data &vf_libOut.."&vf_outStat"n;
    set &vf_libOut.."&vf_outStat"n;
    _SELECT_ = 'forecast';
run;

data &vf_libOut.."&vf_outInformation"n;
    set work._outInformation;
run;

This is where we will create and modify the tables I just talked about.

Using the Node

I've already told you about the node's performance in the beginning. Let's wrap back around so that you can achieve these results as well.

Importing

First, you will need to zip the files. Make sure that the 4 files we created are at the top level of the zipped archive to avoid any issues while importing.

Head to Model Studio on your SAS® Viya Instance. In the Exchange, which you can access on the left-hand side, you will be able to upload the node using the hamburger menu at the top-right of the window. If the import finishes successfully, you will find your node under Forecasting Modeling as "Foundation Forecasting Model".

Open it with a left click to change options, like the model URL, globally. Now is also a good time to check if all of the options and default parameters you set show up how you want them to.

Node Options

Project-Use

I will plug the node into the pipeline I created in the beginning and take a closer look at its results.

First, some raw forecast values. You will see that we will get 24 forecast values. The lead for monthly data is 12, so we get 12 predictions on existing data, plus 12 forecast values. Everything is working as we expect it to.

Forecast Results (Raw)

Of course, this will also be reflected when we open the Forecast viewer to get a better overview.

Forecast Results (Viewer)

Conclusion

Now that you have seen how we can implement these exciting new models into SAS® Visual Forecasting, I hope that I have sparked your interest. Feel free to use this node to run Chronos models in your own projects. Alternatively, you can also adapt the code to use different foundation forecasting models or any other model you can access through an API.

Potential Hurdles in this Process

I've run into my fair share of problems while creating this node, so here are some of those and how I resolved them:

Import fails
- "Failed to deserialize validation model from xml" - as you might expect, this comes from having an invalid validation.xml.
- "Failed to read component template" - same idea, just with the template.json file.
Cannot see node options
- More than likely, there is an issue with you validation.xml file. It might not be malformed, but parameters have the wrong values, constraints are applied to the wrong property type, or any such mistakes.
Log shows invalid EXTLANG configuration / user is not allowed to run external language programs.
- This is an issue that your administrator will have to fix for you.
- You can also start by isolating this issue. Create a script that you can run in SAS Studio based on the node's code to make sure the error stems from the configuration.
- Refer to the EXTLANG documentation for more information on this topic.
Log shows HTTP call finishing with status code 500
- I've seen this one show up in the last few lines of the node's execution log.
- The log line will not very helpful in this case. More than likely, it is connected to the OUTMODELINFO and OUTSTAT tables.
- A good start would be to compare the tables that are created during the pipeline runtime.
  In the log, you should see a library being assigned. It will start with "Analytics_Project...". Simply place it into the following code and run it:
```
cas mysess;
 
libname test cas caslib='Analytics_Project...';
```
- Then, see if you are missing any tables or columns.
- Also, see if the MERGED_ATTRIBUTES table is present.
OUTMODELINFO shows ESMBEST, forecast value for every timestamp.
- This problems points to an invalid EXMSPEC or TSM object definition.
- Refer to each object's documentation to find out more.

Acknowledgements

I want to thank Arpit Jain and David Weik for giving me the opportunity to work on this project and helping me with any questions I had. I also want to give thanks to Spiros Potamitis for connecting me to the Forecasting Community as well as Javier Delgado, Tammy Jackson and Thiago Quirino for helping me with questions regarding the TSMODEL procedure.

Atabarezz · ‎09-09-2024

Great example and best to compare LLM based models with ML and stat models altogether...

Looking forward for this to be built-in to VF