Using SAS® Decisions to Generate Batch Responses from Azure OpenAI
- Article History
- RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Printer Friendly Page
- Report Inappropriate Content
This presentation showcased how to invoke an Azure OpenAI model from within a SAS decision, enabling you to unleash the capabilities of GPT. Learn how to pass a series of prompts in batch to the Azure OpenAI REST API and witness the results being stored in a table effortlessly. Discover how you can dynamically control input parameters by retrieving them from a table, ensuring flexibility and customization. Before sending the prompt, there was a demonstration of the unique ability to calculate the number of tokens and the cost, empowering you to make informed decisions and optimize resources.
Presentation slides are attached to this post.
Recorded Presentation and Demonstrations
Watch the recorded presentation, including several short demonstrations:
- GPT-4 content generation and entity extraction from the content
- GPT-4 chat example, answering questions about SAS Intelligent Decisioning.
- Retrieval Augmented Generation (RAG) with text-davinci-003, answering questions about SAS Intelligent Decisioning, from the SAS official documentation.
- Python code generation for Python nodes in SAS Intelligent Decisioning, using Azure AI Studio.
- Chapters
- descriptions off, selected
- captions settings, opens captions settings dialog
- captions off, selected
This is a modal window.
Beginning of dialog window. Escape will cancel and close the window.
End of dialog window.
This is a modal window. This modal can be closed by pressing the Escape key or activating the close button.
Components
Azure
- Azure subscription.
- You must request OpenAI in Azure. Currently, there is a long waiting list… for GPT-3 models and …an even longer list for GPT-4!
- After you get the access, you must deploy an OpenAI resource.
- An access key.
- The endpoint.
- Deploy a model: we used GPT-4 for all examples, except for RAG, where we used a text-davinci-003.
SAS Viya
- Python configured in SAS Viya with SAS Micro Analytics Service, CAS, SAS Compute.
- Python OpenAI package installed.
- A Python code calling the Azure OpenAI endpoint. The code file must be structured as a function. The outputs have to be declared in a certain way, as they are used by the decision variables.
Python Packages
You can install these using pip:
pip install openai
Specifically for SAS Viya, if you use the SAS Configurator for Open Source (which creates and executes a sas-pyconfig job), you would need to add the new packages in the change-configuration.yaml.
Python Program - Entities Extraction
Python Code File
import openai
def execute (pr):
'Output:prompt,text,cost'
'DependentPackages:os,openai'
# Variables
openai.api_type = "azure"
openai.api_base = "REPLACE_WITH_YOUR_ENDPOINT_HERE" # your endpoint should look like the following https://YOUR_RESOURCE_NAME.openai.azure.com/
openai.api_version = "2023-05-15" # your model deployment will tell you what version you need to use
openai.api_key = "REPLACE_WITH_YOUR_API_KEY_HERE"
prompt = str(pr)
response = openai.ChatCompletion.create(
# engine = "deployment_name"
engine="gpt-4",
messages=[{"role":"system","content":"You are an assistant designed to extract entities from text. Users will paste in a string of text and you will respond with entities you've extracted from the text as a JSON object. Here's an example of your output format:\n{\n \"name\": \"\",\n \"company\": \"\",\n \"phone_number\": \"\"\n}"},
{"role":"user","content":"My name is Robert Smith. I'm calling from Contoso Insurance, Delaware. My colleague mentioned that you are interested in learning about our comprehensive benefits policy. Could you give me a call back at (555) 346-9322 when you get a chance so we can go over the benefits?"},
{"role":"assistant","content":"{\n \"name\": \"Robert Smith\",\n \"company\": \"Contoso Insurance\",\n \"phone_number\": \"(555) 346-9322\"\n}"},
{"role": "user", "content": pr}
]
)
print(response)
print(response['choices'][0]['message']['content'])
text = response['choices'][0]['message']['content']
text_clean = text.replace("\n", "")
print (text_clean)
# cost section https://azure.microsoft.com/en-au/pricing/details/cognitive-services/openai-service/
cost_per_prompt_token = 0.03 / 1000 # usd per 1000 tokens
cost_per_completion_token = 0.06 / 1000 # usd per 1000 tokens
completion_tokens = response['usage']['completion_tokens']
prompt_tokens = response['usage']['prompt_tokens']
cost = prompt_tokens * cost_per_prompt_token + completion_tokens * cost_per_completion_token
print ('Cost for prompt and answer in USD: ', round (cost, 4))
# limit max-tokens https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/chatgpt?source=recommendations&pivots=programming-language-chat-completions
return prompt, text_clean, cost
print ( execute ("Hello. My name is Robert Smith. I'm calling from Contoso Insurance, Delaware. My colleague mentioned that you are interested in learning about our comprehensive benefits policy. Could you give me a call back at (555) 346-9322 when you get a chance so we can go over the benefits?") )
print ( execute ("My name is John Mackane. I'm calling from Contoso Bank, in Delaware. My colleague mentioned that you are interested in learning about our home loan. Could you give me a call back at (555) 876-9322 when you get a chance so we can go over the benefits? ") )
print ( execute ("Hi, here Alice Springer. I'm calling from Regional Coop, in Alice Springs. Could you give me a call back at (08) 9876 9322? Thanks ") )
Output
Sample prompts and generated output:
Python Program - Chat Function
Python Code File
import openai
def execute (pr):
'Output:prompt,text,cost'
'DependentPackages:os,openai'
# Variables
openai.api_type = "azure"
openai.api_base = "REPLACE_WITH_YOUR_ENDPOINT_HERE" # your endpoint should look like the following https://YOUR_RESOURCE_NAME.openai.azure.com/
openai.api_version = "2023-05-15" # your model deployment will tell you what version you need to use
openai.api_key = "REPLACE_WITH_YOUR_API_KEY_HERE"
prompt = str(pr)
response = openai.ChatCompletion.create(
engine="gpt-4", # engine = "deployment_name".
messages=[
{"role": "system", "content": "You are a helpful assistant. You help answer questions about SAS Intelligent Decisioning."},
{"role": "user", "content": "Can you use Python code in SAS Intelligent Decisioning?"},
{"role": "assistant", "content": "Yes, Python code files are supported in SAS decisions."},
{"role": "user", "content": "Can you publish decisions from SAS Intelligent Decisioning to Azure?"},
{"role": "assistant", "content": "Yes, decisions can be from SAS Intelligent Decisioning to Azure by using SAS Container Runtime (SCR)."},
{"role": "user", "content": prompt}
]
)
print(response)
print(response['choices'][0]['message']['content'])
text = response['choices'][0]['message']['content']
# cost section https://azure.microsoft.com/en-au/pricing/details/cognitive-services/openai-service/
cost_per_prompt_token = 0.03 / 1000 # usd per 1000 tokens
cost_per_completion_token = 0.06 / 1000 # usd per 1000 tokens
completion_tokens = response['usage']['completion_tokens']
prompt_tokens = response['usage']['prompt_tokens']
cost = prompt_tokens * cost_per_prompt_token + completion_tokens * cost_per_completion_token
print ('Cost for prompt and answer in USD: ', round (cost, 4))
# limit max-tokens https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/chatgpt?source=recommendations&pivots=programming-language-chat-completions
return prompt, text, cost
# sample questions to test the code - remove them after a test
execute('Does Azure OpenAI support customer managed keys? ')
execute ('Can you publish decisions from SAS Intelligent Decisioning to Azure? ')
execute ('What about rule sets? To which destinations can you publish rule sets from SAS Intelligent Decisioning?')
Output
Sample prompts and generated output:
Retrieval Augmented Generation
A retrieval augmented generation (RAG) methodology can be used to refine the answers received. You can use your own data as a source of information. You can direct the Azure OpenAI model to use only search results from the document to construct the answer.
In other words, you can ask questions on your own unstructured data.
Output
We have used the SAS Intelligent Decisioning documentation as a source. As you can see, when you use RAG, the answer is much more precise.
Conclusions
SAS Intelligent Decisioning is delivering great capabilities, when it comes to combining models, rule sets, data queries, decision points, code files.
- Azure OpenAI can help you fulfill many / additional use cases. Mentioned four: Content generation, Summarization, Code Generation and Semantic Search (using RAG).
You can embed Azure OpenAI in your SAS decisions by using a simple Python code file, calling the Azure OpenAI REST API. - Faster time to Value: By using Azure OpenAI and SAS Intelligent Decisioning, you can take decisions to a next level.