SAS Visual Investigator Copilot – Image Description Generation

1 Like

SAS Viya Copilot is a set of SaaS features that utilizes LLMs to provide end users with a more intuitive and accessible way to work within the SAS Viya ecosystem. This post is part two of a series exploring SAS Viya Copilot for Visual Investigator, with a deep dive into its image description generation feature.

SAS Viya Copilot for Visual Investigator

For a more in-depth introduction to SAS Viya Copilot, please see my previous post: SAS Visual Investigator Copilot – Assisted Search.

There are several ways to leverage Copilot to enhance your SAS Visual Investigator usage. You can use the Copilot pane to learn how to perform a task in SAS Visual Investigator or use it to answer questions about your particular SAS Visual Investigator implementation. Through conversational chats, the former gives you general SAS Visual Investigator knowledge while the latter gives you tailored responses specific to your organization’s configuration.

For example, ask Copilot how to apply bulk dispositions to all the alerts within a queue or ask Copilot to list the object types and relationships as configured by your administrator.

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

Copilot can help you with a wide range of inquiries about your SAS Visual Investigator configuration, including but not limited to the following:

Object types: The types of entities available in your system.
- Example: Claims, Policies, Vehicles, Alerts, etc.
Relationships: The types of relationships that exist between the different entities in your system.
- Example: Claims to Policy (many to one), Vehicles to Claims (many to many), etc.
Workflows: The workflows associated with a particular entity.
Tasks: Which tasks are included in the workflow.
Reference data: The reference data created by your administrator.

Copilot is unable to do the following:

Directly modify your configuration
Display sensitive or restricted information
Provide an export of your configuration settings

In addition to surfacing insights about your SAS Visual Investigator instance through conversational chat within the Copilot pane, there are three in-context experiences that have been directly embedded:

Assisted search: Use natural, conversational language to search in SAS Visual Investigator.
Image description generation: Use GenAI to create descriptions for images that appear on a page.
Prompt-based text generation: Use GenAI to create text based on a preconfigured prompt.

In the last post, I explored the Assisted Search feature with multiple examples. In this post, I will explore the image description generation feature.

Feature Enablement

Depending on your software version, the abovementioned features may first need to be toggled on within SAS Environment Manager. Note that this is only relevant for software releases older than 2026.04, as the features become generally available in the 2026.04 release.

The relevant features are:

GenAI Assisted Search
GenAI Image Description
GenAI Text Generation
GenAI Conversational Agent

Capability

Additionally, in order for a user to perform an assisted search, they must belong to a group with the “Perform searches assisted by generative AI” capability (svi.search.assisted).

GenAI Image Description Generation

Configuration

Before the image description generation feature can be used, a few configurations need to be put in place in the administrator’s UI (Manage Investigate and Search). Firstly, you need to create an image category if it doesn’t already exist. This can be done on the entity’s configuration page, found under Data Objects.

Once the image category has been defined for the entity of interest, go to that entity’s desktop page template. On the page template:

Add an Image Carousel control to the canvas
In the Image Carousel Configuration Pane, define the Image Source. This should be set to the Image Category created earlier.
Check the Enable description generation checkbox to enable the GenAI feature.
Write a relevant prompt within the Prompt text box. An example could be “Generate a description based on the picture”.
Optionally, check the Automatically generate check box to generate the description automatically. If this box is left unchecked, the image description will be generated on command with the press of a button by the user.

Usage Example

Now, let’s navigate back to the Investigator’s UI (Investigate and Search Data) to see how this feature works.

In the example below, I’m creating a new Case object. The newly added Image Carousel control can be seen on the Create Object page. Use the Add image button to add new images from your file system.

After selecting the desired image from my file system, I’m led to the Image Properties window. A Generate button can be found on top of the image description box. Click on this button to automatically to generate a description of the uploaded image. As you can see in the example below, the AI generated image description is incredibly thorough, covering the type of the vehicles present, their physical appearance (color, label), as well as their parking configuration and what’s in the background.

At this point, it’s worthwhile to repeat the adage that AI can and do make mistakes. As such, it’s important to validate the image description to ensure its accuracy. If needed, you can manually amend the description before clicking OK.

The image description will appear under the carousel of images on the Case object’s page.

Summary

In this post, I provided some general usage note on the Viya Copilot pane, as well as diving more in-depth into the image description generation feature of Viya Copilot for Visual Investigator. Stay tuned for more!