Creating an AI Assistant for SAS Viya in 5 steps (@sassoftware/viya-assistantjs) - Part I

2 Likes

Introduction

Since you opened this article, you must take the next step and visit the SAS Innovate 2024 site to learn about SAS approach to providing a world class AI and Analytic experience for SAS users.

Like most application developers in the world, I was curious to learn about LLM and how one can put the LLM to work for specific domains.

My first attempt was to build a copilot for the drag and drop app builder (with react components) I was working on. While I did get an initial prototype working, I was left with the uneasy feeling that I was using a hammer when the job called for a screwdriver - I could build an entire app using drag and drop and minimal coding, so why build a layer with LLM? I shelved the project with a note to build @sassoftware/restaf library copilot for building REST API based Viya apps.

Around this time, OpenAI released the first version of their AI Assistant API for writing AI Assistants. The description from OpenAI is:

Create an Assistant by defining its custom instructions and picking a model. If helpful, add files and enable tools like Code Interpreter, File Search, and Function calling.
Create a Thread when a user starts a conversation.
Add Messages to the Thread as the user asks questions.
Run the Assistant on the Thread to generate a response by calling the model and the tools.

There are a few more steps involved, but I was able to build assistants using this. The tools mentioned in step 1 allow developers to integrate their backend with the assistant. For Viya app developers this meant using the published REST API to access SAS Viya. In my case, I used the @sassoftware/restaf libraries to make the calls to SAS Viya, but one could use any http library to make the calls.

Recently OpenAI released V2 beta that added support for vector store. The additional operations allow the developer to add to the vector store which could be used by the files_search tool. Please note that Azureai Assistant does not support vector store at this time.

As with any project I work on, I always try to create reusable code (the lazy programmer approach!). This project is no different. I created a library @sassoftware/viya-assistantjs to simplify writing assistants. This library is available under Apache-2 license. Details on the Api and usage is here.

Use this library to build an Assistant in 5 steps

Decide on the tools you need for your domain and develop them.
1. The library comes with a default set of tools as an example and to jump start your journey.
2. The source code for these tools are available in the repository.
3. Additional examples are here.
Call the setupAssistant with configuration to initialize the assistant.
1. Pass in information to logon to Viya. The assistant will manage both CAS and SAS sessions.
  1. See this link for details
2. Pass in information for creating and managing the assistant.
Accept prompt from user and call the runAssistant with the prompt.
1. The OpenAI assistant maintains a thread of the conversation
Process the response
Repeat steps 3 and 4.

The library has a uploadFile method for uploading documents to the vector store associated with the assistant. This method can also be used in the tools.

Key point of the samples tools

The purpose of the sample tools is to show how easy it is to invoke SAS Viya from the tools and use the results in your response. While the mechanism to call Viya to resolve user prompts is necessary, it is not sufficient. One must have a well defined domain that is of interest to the user. The tools must be designed to access data and programs that are relevant to the domain. In SAS this usually is a collection of SAS programs and data that can resolve user questions about the domain.

Build an Assistant Step-by-Step

Import the necessary modules. Note the two entries imported from @sassoftware/viya-assistantjs

// Step 0: Import the necessary modules
import * as readline from "node:readline/promises";
import fss from "fs/promises";
import { stdin as input, stdout as output } from "node:process";
import getToken from "./getToken.js";
import { setupAssistant, runAssistant } from "@sassoftware/viya-assistantjs";
let { host, token } = getToken();

Set up the tools.

const tools = [
  {
    type: "function",
    function: {
      name: "listTables",
      description: `for a given library for  either sas or cas source, get the list of available tables.
      (ex: list tables in cas library samples, list tables in sas library sashelp)`,
      parameters: {
        properties: {
          library: {
            type: "string",
            description: "A SAS library like casuser, sashelp, samples",
          },
          start: {
            type: "integer",
            description: "Start at lookup at this index. Default is 0.",
          },
          limit: {
            type: "integer",
            description:
              "Return only this many tables. If not specified, then return 10 tables.",
          },
          source: {
            type: "string",
            description: "The source of the data. cas or sas",
            enum: ["cas", "sas"],
          },
        },
        type: "object",
        required: ["library"],
      },
    },
  },
];

Set up the function for the tool.

async function listTables(params, userData, appControl) {
  let { library, source, start, limit } = params;
  // get session information
  let appEnv = await appControl.getViyaSession(source);
  // setup filters
  let p = {
    qs: {
      limit: limit == null ? 10 : limit,
      start: start == null ? 0 : start,
    },
  };

  // use restafedit library to get the list of tables.
  // Note you can make use your favorite library to make this call. see documentation
  let r = await appEnv.restafedit.getTableList(library, appEnv, p);
  // return the retrieved list
  return JSON.stringify(r);
}

Write the chat program.

// run a chat session
chat(config)
  .then((r) => console.log("done"))
  .catch((err) => console.log(err));

async function chat(config) {
  //Setup assistant
  let appControl = await setupAssistant(config);

  // create readline interface and chat with user
  const rl = readline.createInterface({ input, output });

  // process user input in a loop
  while (true) {
    let prompt = await rl.question(">");
    // exit session
    if (prompt.toLowerCase() === "exit" || prompt.toLowerCase() === "quit") {
      rl.close();
      break;
    }
    // let assistant process the prompt
    let promptInstructions = " ";
    try {
      // run prompt
      let response = await runAssistant(appControl, prompt, promptInstructions);
      console.log(response[0].content);
    } catch (err) {
      console.log(err);
    }
  }
}

Here is a short video showing the assistant in action.

(view in My Videos)

Below is a recording of an assistant using the builtin tools.

(view in My Videos)

Conclusion

I hope you find this article useful. Feel free to clone the library and modify it to suit your needs. Since the library is based on an SDK that is in beta, I expect to enhance the library and publish updates as the OpenAI assistant evolves and I learn more.

All comments are welcomed.