Using SAS® Decisions to Generate Batch Responses from Azure OpenAI

3 Likes

This presentation showcased how to invoke an Azure OpenAI model from within a SAS decision, enabling you to unleash the capabilities of GPT. Learn how to pass a series of prompts in batch to the Azure OpenAI REST API and witness the results being stored in a table effortlessly. Discover how you can dynamically control input parameters by retrieving them from a table, ensuring flexibility and customization. Before sending the prompt, there was a demonstration of the unique ability to calculate the number of tokens and the cost, empowering you to make informed decisions and optimize resources.

Presentation slides are attached to this post.

Recorded Presentation and Demonstrations

Watch the recorded presentation, including several short demonstrations:

GPT-4 content generation and entity extraction from the content
GPT-4 chat example, answering questions about SAS Intelligent Decisioning.
Retrieval Augmented Generation (RAG) with text-davinci-003, answering questions about SAS Intelligent Decisioning, from the SAS official documentation.
Python code generation for Python nodes in SAS Intelligent Decisioning, using Azure AI Studio.

(view in My Videos)

Components

Azure

Azure subscription.
You must request OpenAI in Azure. Currently, there is a long waiting list… for GPT-3 models and …an even longer list for GPT-4!
After you get the access, you must deploy an OpenAI resource.
An access key.
The endpoint.
Deploy a model: we used GPT-4 for all examples, except for RAG, where we used a text-davinci-003.

SAS Viya

Python configured in SAS Viya with SAS Micro Analytics Service, CAS, SAS Compute.
Python OpenAI package installed.
A Python code calling the Azure OpenAI endpoint. The code file must be structured as a function. The outputs have to be declared in a certain way, as they are used by the decision variables.

Python Packages

You can install these using pip:

pip install openai

Specifically for SAS Viya, if you use the SAS Configurator for Open Source (which creates and executes a sas-pyconfig job), you would need to add the new packages in the change-configuration.yaml.

Python Program - Entities Extraction

Python Code File

import openai
def execute (pr):
   'Output:prompt,text,cost'
   'DependentPackages:os,openai'
   # Variables
   openai.api_type = "azure"
   openai.api_base =  "REPLACE_WITH_YOUR_ENDPOINT_HERE" # your endpoint should look like the following https://YOUR_RESOURCE_NAME.openai.azure.com/
   openai.api_version = "2023-05-15" # your model deployment will tell you what version you need to use
   openai.api_key = "REPLACE_WITH_YOUR_API_KEY_HERE"
prompt = str(pr)
   response = openai.ChatCompletion.create(
      # engine = "deployment_name"
      engine="gpt-4",
      messages=[{"role":"system","content":"You are an assistant designed to extract entities from text. Users will paste in a string of text and you will respond with entities you've extracted from the text as a JSON object. Here's an example of your output format:\n{\n   \"name\": \"\",\n   \"company\": \"\",\n   \"phone_number\": \"\"\n}"},
                {"role":"user","content":"My name is Robert Smith. I'm calling from Contoso Insurance, Delaware. My colleague mentioned that you are interested in learning about our comprehensive benefits policy. Could you give me a call back at (555) 346-9322 when you get a chance so we can go over the benefits?"},
                {"role":"assistant","content":"{\n   \"name\": \"Robert Smith\",\n   \"company\": \"Contoso Insurance\",\n   \"phone_number\": \"(555) 346-9322\"\n}"},
                {"role": "user", "content": pr}
				]
                                          )
   print(response)
   print(response['choices'][0]['message']['content'])
   text = response['choices'][0]['message']['content']
   text_clean = text.replace("\n", "")
   print (text_clean)


   # cost section https://azure.microsoft.com/en-au/pricing/details/cognitive-services/openai-service/
   cost_per_prompt_token = 0.03 / 1000 # usd per 1000 tokens
   cost_per_completion_token = 0.06 / 1000 # usd per 1000 tokens
   completion_tokens = response['usage']['completion_tokens']
   prompt_tokens = response['usage']['prompt_tokens']
   cost = prompt_tokens * cost_per_prompt_token + completion_tokens * cost_per_completion_token
   print ('Cost for prompt and answer in USD: ', round (cost, 4))
   # limit max-tokens https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/chatgpt?source=recommendations&pivots=programming-language-chat-completions

   return prompt, text_clean, cost


print ( execute ("Hello. My name is Robert Smith. I'm calling from Contoso Insurance, Delaware. My colleague mentioned that you are interested in learning about our comprehensive benefits policy. Could you give me a call back at (555) 346-9322 when you get a chance so we can go over the benefits?") )
print ( execute ("My name is John Mackane. I'm calling from Contoso Bank, in Delaware. My colleague mentioned that you are interested in learning about our home loan. Could you give me a call back at (555) 876-9322 when you get a chance so we can go over the benefits? ") )
print ( execute ("Hi, here Alice Springer. I'm calling from Regional Coop, in Alice Springs. Could you give me a call back at (08) 9876 9322? Thanks ") )

Output

Sample prompts and generated output:

Azure_OpenAI_in_SAS_Intelligent_Decisioning_GPT-4_Entity_Extraction.png

Python Program - Chat Function

Python Code File

import openai
def execute (pr):
   'Output:prompt,text,cost'
   'DependentPackages:os,openai'
   # Variables
   openai.api_type = "azure"
   openai.api_base =  "REPLACE_WITH_YOUR_ENDPOINT_HERE" # your endpoint should look like the following https://YOUR_RESOURCE_NAME.openai.azure.com/
   openai.api_version = "2023-05-15" # your model deployment will tell you what version you need to use
   openai.api_key = "REPLACE_WITH_YOUR_API_KEY_HERE"

   prompt = str(pr)
   response = openai.ChatCompletion.create(
      engine="gpt-4", # engine = "deployment_name".
      messages=[
         {"role": "system", "content": "You are a helpful assistant. You help answer questions about SAS Intelligent Decisioning."},
         {"role": "user", "content": "Can you use Python code in SAS Intelligent Decisioning?"},
         {"role": "assistant", "content": "Yes, Python code files are supported in SAS decisions."},
         {"role": "user", "content": "Can you publish decisions from SAS Intelligent Decisioning to Azure?"},
 		 {"role": "assistant", "content": "Yes, decisions can be from SAS Intelligent Decisioning to Azure by using SAS Container Runtime (SCR)."},
		 {"role": "user", "content": prompt}

      ]
   )

   print(response)
   print(response['choices'][0]['message']['content'])
   text = response['choices'][0]['message']['content']
   # cost section https://azure.microsoft.com/en-au/pricing/details/cognitive-services/openai-service/
   cost_per_prompt_token = 0.03 / 1000 # usd per 1000 tokens
   cost_per_completion_token = 0.06 / 1000 # usd per 1000 tokens
   completion_tokens = response['usage']['completion_tokens']
   prompt_tokens = response['usage']['prompt_tokens']
   cost = prompt_tokens * cost_per_prompt_token + completion_tokens * cost_per_completion_token
   print ('Cost for prompt and answer in USD: ', round (cost, 4))
   # limit max-tokens https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/chatgpt?source=recommendations&pivots=programming-language-chat-completions

   return prompt, text, cost

# sample questions to test the code - remove them after a test
execute('Does Azure OpenAI support customer managed keys? ')
execute ('Can you publish decisions from SAS Intelligent Decisioning to Azure? ')
execute ('What about rule sets? To which destinations can you publish rule sets from SAS Intelligent Decisioning?')

Output

Sample prompts and generated output:

Azure_OpenAI_in_SAS_Intelligent_Decisioning_GPT-4_Chat.png

Retrieval Augmented Generation

A retrieval augmented generation (RAG) methodology can be used to refine the answers received. You can use your own data as a source of information. You can direct the Azure OpenAI model to use only search results from the document to construct the answer.

In other words, you can ask questions on your own unstructured data.

Output

We have used the SAS Intelligent Decisioning documentation as a source. As you can see, when you use RAG, the answer is much more precise.

Conclusions

SAS Intelligent Decisioning is delivering great capabilities, when it comes to combining models, rule sets, data queries, decision points, code files.

Azure OpenAI can help you fulfill many / additional use cases. Mentioned four: Content generation, Summarization, Code Generation and Semantic Search (using RAG).
You can embed Azure OpenAI in your SAS decisions by using a simple Python code file, calling the Azure OpenAI REST API.
Faster time to Value: By using Azure OpenAI and SAS Intelligent Decisioning, you can take decisions to a next level.