Incorporating Generative AI models in SAS Visual Forecasting - Part 1

3 Likes

Incorporating Generative AI models in SAS Visual Forecasting - Part 1

Introduction & Deploying Chronos

This is the first of a two part blog post about pretrained forecasting models and how to integrate them with SAS Visual Forecasting. The second part can be found here.

Introduction to Pretrained Forecasting Models

LLM models such as the GPT models behind ChatGPT are pretrained. In a recent article which was published in Foresight, SAS forecasting experts discussed how will Generative AI influence forecasting software and the challenges that come up in terms of available and domain specific data to develop powerful pretrained models that could outperform traditional techniques in various use-cases. However, pretrained models which are based on neural network architectures, originally designed for natural language processing, are earning ground in the field of forecasting and have shown potential in efficiently predicting time series data across various domains. Even though, we don't expect that those models will replace the traditional forecasting techniques any time soon, pretrained models will keep evolving, becoming more accurate, cheaper and faster as the time goes by. This chapter will explore the fundamentals of pretrained, transformer-based models, their application to forecasting and how to deploy them in SAS Viya.

The Chronos Model

Among the cutting-edge forecasting tools available today, the Chronos model stands out for its performance and versatility. Designed specifically for time series prediction, Chronos leverages the power of transformer architecture to capture complex patterns and dependencies in data. This section will give you a quick introduction to Chronos, but if you want to dive deeper check out the original paper on it here. The great thing about Chronos, which was developed by Amazon Science, is that it is licensed under Apache 2.0, so that we can build on top of it.

The key innovation lies in its time series tokenization process, which maps real-valued observations to a finite set of tokens through scaling and quantization. This allows Chronos to utilize off-the-shelf language model architectures like T5 or GPT-2 with minimal modifications, primarily adjusting the vocabulary size to accommodate the tokenized time series. The model is trained using a standard categorical cross-entropy loss, eschewing distance-aware objectives in favor of allowing the model to learn relationships between nearby tokens implicitly. By treating time series as sequences of tokens without explicit time or frequency information, Chronos demonstrates the general-purpose sequence models in capturing temporal dependencies. The simplicity of this approach, combined with its ability to leverage existing language model training pipelines, makes Chronos a promising framework for developing powerful and adaptable time series forecasting models.

Deploying Chronos as a Container

Now let us get to the meat and potatoes of this post: deploying the Chronos model so that we can integrate with SAS Viya. You can find the full code for this deployment on GitHub.

For the integration the idea is to make Chronos callable via a REST-API endpoint so that we have an easy time of integrating in multiple tools. The API endpoint should have two main properties: 1. Take in the time series data and 2. be able to receive options to tune the model.

A REST-API endpoint is chosen so that we are flexible with how we can call it: Proc HTTP, Proc Python, Open Source Code nodes, SAS Visual Analytics Data Driven Content objects, etc., etc.. In addition creating a containrized workload that can be scaled independant of SAS Viya ensures that your current workloads are not impacted by this additional model.

For that we have written a short Python script which will serve as the point of entry and running the inference on the Chronos model:

import torch
from chronos import ChronosPipeline
from fastapi import FastAPI, HTTPException, Request
from pydantic import BaseModel
import pandas as pd
import os
from fastapi.responses import StreamingResponse
import io
from typing import List, Dict

app = FastAPI()

class Item(BaseModel):
    prediction_length: int
    num_samples: int
    temperature: float
    top_k: int
    top_p: float
    data: List[float]

chronos_model = os.getenv('CHRONOS_MODEL')
if chronos_model is None:
    chronos_model = "amazon/chronos-t5-small"

print(f"Running using the {chronos_model} model")
pipeline = ChronosPipeline.from_pretrained(
  chronos_model,
)

# Actual prediction process
@app.post("/predict/")
async def predict(item: Item):
    try:
        prediction_length = item.prediction_length
        num_samples = item.num_samples
        temperature = item.temperature
        top_k = item.top_k
        top_p = item.top_p
        context = torch.tensor(item.data)

    except Exception as e:
        raise HTTPException(status_code=400, detail=f"Error processing input data: {e}")

    try:
        forecast = pipeline.predict(
            context,
            prediction_length,
            num_samples=num_samples,
            temperature=temperature,
            top_k=top_k,
            top_p=top_p,
            )
        low, median, high = np.quantile(forecast[0].numpy(), [0.1, 0.5, 0.9], axis=0)
        dfOut = pd.DataFrame(data={"low": low, "median": median, "high": high})
        stream = io.StringIO()
        dfOut.to_csv(stream, index = False)
        response = StreamingResponse(iter([stream.getvalue()]),
                                    media_type="text/csv"
                                    )
        response.headers["Content-Disposition"] = "attachment; filename=predictions.csv"
        return response
    except Exception as e:
        raise HTTPException(status_code=500, detail=f"Error during prediction: {e}")

Next we will create a Docker file which we can then run:

FROM python:3.12-slim

WORKDIR /chronos

COPY ./app /chronos/app

RUN apt-get update && apt-get install -y --no-install-recommends git

RUN pip install git+https://github.com/amazon-science/chronos-forecasting.git

RUN pip install pandas fastapi uvicorn pydantic

EXPOSE 8000

ARG UID=1001
ARG GID=1001

RUN groupadd -g "${GID}" sas \
  && useradd --create-home --no-log-init -u "${UID}" -g "${GID}" sas

USER python

ENV CHRONOS_MODEL "amazon/chronos-t5-small"

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Calling from SAS Studio

Just to test the waters let us quickly build a SAS script to call the Chronos API with a SAS dataset, note that a deeper integration with SAS Visual Forecasting pipelines will be showcased in the next article - here is the script to call it:

* Get the Viya Host URL if you deployed chronos here;
%let viyaHost=%sysfunc(getoption(SERVICESBASEURL)); 
* Specify the chronos endpoint;
%let chronosEndpoint = /forecast/predict/;

filename _ffmIn temp;

* Define the structure for the call;
data _null_;
	set sashelp.air(keep=air) end=EoF;

	file _ffmIn;

	if _n_ eq 1 then do;
		predictionLength = '{"prediction_length": ' || put(10, 8.) || ',';
		num_samples =  '"num_samples": ' || put(20, 8.) || ',';
		temperature = '"temperature": ' || put(1, 8.) || ',';
		top_k = '"top_k": ' || put(50, 8.) || ',';
		top_p = '"top_p": ' || put(1, 8.) || ',';
		dataBase = '"data": [';
		put predictionLength;
		put num_samples;
		put temperature;
		put top_k;
		put top_p;
		put dataBase;
	end;
            
	if not EoF then do;
		currentIteration = compress(put(air, 8.) || ',');
		put currentIteration;
	end;
	else if EoF then do;
		currentIteration = compress(put(air, 8.));
		dataClose = ']}';
		put currentIteration;
		put dataClose;
	end;
run;

filename _ffmOut temp;

* Call the chronos model;
proc http
	method = 'Post'
	url = "&viyaHost.&chronosEndpoint."
	in = _ffmIn
	out = _ffmOut;
	headers 'Content-Type' = 'application/json'
	'Accept' = 'text/csv';
quit;

* Import the returned CSV file;
proc import
	file = _ffmOut
	dbms = csv
	out = work.results
	replace;
run; quit;

* Clean up;
filename _ffmIn clear;
filename _ffmOut clear;
%symdel viyaHost chronosEndpoint;

This gives us the following results:

Conclusion and Next Steps

Now to recap we quickly introduced you two pretrained and transformer-based forecasting models and specifically the Chronos model family. Then we took a look at how to deploy it as an API and a container and finally called it in a basic fashion using 'proc http' from SAS Studio. In the next article we will dive deep into the topic of how to integrate Chronos with SAS Visual Forecasting step by step - find it here.