SAS Explore Presentations

Re-experience the presentations and tutorials from SAS Explore
BookmarkSubscribeRSS Feed

Using SAS® Decisions to Generate Batch Responses from Azure OpenAI

Started ‎09-29-2023 by
Modified ‎10-30-2023 by
Views 2,234

This presentation showcased how to invoke an Azure OpenAI model from within a SAS decision, enabling you to unleash the capabilities of GPT. Learn how to pass a series of prompts in batch to the Azure OpenAI REST API and witness the results being stored in a table effortlessly. Discover how you can dynamically control input parameters by retrieving them from a table, ensuring flexibility and customization. Before sending the prompt, there was a demonstration of the unique ability to calculate the number of tokens and the cost, empowering you to make informed decisions and optimize resources.

 

Presentation slides are attached to this post.

 

Recorded Presentation and Demonstrations

Watch the recorded presentation, including several short demonstrations:

  • GPT-4 content generation and entity extraction from the content
  • GPT-4 chat example, answering questions about SAS Intelligent Decisioning.
  • Retrieval Augmented Generation (RAG) with text-davinci-003, answering questions about SAS Intelligent Decisioning, from the SAS official documentation.
  • Python code generation for Python nodes in SAS Intelligent Decisioning, using Azure AI Studio.

 

Using SAS Decisions with Azure OpenAI.mp4
Video Player is loading.
Current Time 0:00
Duration 0:00
Loaded: 0%
Stream Type LIVE
Remaining Time 0:00
 
1x
    • Chapters
    • descriptions off, selected
    • captions off, selected
      (view in My Videos)

       

      Components

      Azure

      • Azure subscription.
      • You must request OpenAI in Azure. Currently, there is a long waiting list… for GPT-3 models and …an even longer list for GPT-4!
      • After you get the access, you must deploy an OpenAI resource.
      • An access key.
      • The endpoint.
      • Deploy a model: we used GPT-4 for all examples, except for RAG, where we used a text-davinci-003.


      SAS Viya

      • Python configured in SAS Viya with SAS Micro Analytics Service, CAS, SAS Compute.
      • Python OpenAI package installed.
      • A Python code calling the Azure OpenAI endpoint. The code file must be structured as a function. The outputs have to be declared in a certain way, as they are used by the decision variables.

      Python Packages

      You can install these using pip:

      pip install openai
      

      Specifically for SAS Viya, if you use the SAS Configurator for Open Source (which creates and executes a sas-pyconfig job), you would need to add the new packages in the change-configuration.yaml.

       

      Python Program - Entities Extraction

      Python Code File

       

      import openai
      def execute (pr):
         'Output:prompt,text,cost'
         'DependentPackages:os,openai'
         # Variables
         openai.api_type = "azure"
         openai.api_base =  "REPLACE_WITH_YOUR_ENDPOINT_HERE" # your endpoint should look like the following https://YOUR_RESOURCE_NAME.openai.azure.com/
         openai.api_version = "2023-05-15" # your model deployment will tell you what version you need to use
         openai.api_key = "REPLACE_WITH_YOUR_API_KEY_HERE"
      prompt = str(pr)
         response = openai.ChatCompletion.create(
            # engine = "deployment_name"
            engine="gpt-4",
            messages=[{"role":"system","content":"You are an assistant designed to extract entities from text. Users will paste in a string of text and you will respond with entities you've extracted from the text as a JSON object. Here's an example of your output format:\n{\n   \"name\": \"\",\n   \"company\": \"\",\n   \"phone_number\": \"\"\n}"},
                      {"role":"user","content":"My name is Robert Smith. I'm calling from Contoso Insurance, Delaware. My colleague mentioned that you are interested in learning about our comprehensive benefits policy. Could you give me a call back at (555) 346-9322 when you get a chance so we can go over the benefits?"},
                      {"role":"assistant","content":"{\n   \"name\": \"Robert Smith\",\n   \"company\": \"Contoso Insurance\",\n   \"phone_number\": \"(555) 346-9322\"\n}"},
                      {"role": "user", "content": pr}
      				]
                                                )
         print(response)
         print(response['choices'][0]['message']['content'])
         text = response['choices'][0]['message']['content']
         text_clean = text.replace("\n", "")
         print (text_clean)
      
      
         # cost section https://azure.microsoft.com/en-au/pricing/details/cognitive-services/openai-service/
         cost_per_prompt_token = 0.03 / 1000 # usd per 1000 tokens
         cost_per_completion_token = 0.06 / 1000 # usd per 1000 tokens
         completion_tokens = response['usage']['completion_tokens']
         prompt_tokens = response['usage']['prompt_tokens']
         cost = prompt_tokens * cost_per_prompt_token + completion_tokens * cost_per_completion_token
         print ('Cost for prompt and answer in USD: ', round (cost, 4))
         # limit max-tokens https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/chatgpt?source=recommendations&pivots=programming-language-chat-completions
      
         return prompt, text_clean, cost
      
      
      print ( execute ("Hello. My name is Robert Smith. I'm calling from Contoso Insurance, Delaware. My colleague mentioned that you are interested in learning about our comprehensive benefits policy. Could you give me a call back at (555) 346-9322 when you get a chance so we can go over the benefits?") )
      print ( execute ("My name is John Mackane. I'm calling from Contoso Bank, in Delaware. My colleague mentioned that you are interested in learning about our home loan. Could you give me a call back at (555) 876-9322 when you get a chance so we can go over the benefits? ") )
      print ( execute ("Hi, here Alice Springer. I'm calling from Regional Coop, in Alice Springs. Could you give me a call back at (08) 9876 9322? Thanks ") )
      
      

      Output

      Sample prompts and generated output:


      Azure_OpenAI_in_SAS_Intelligent_Decisioning_GPT-4_Entity_Extraction.png

       

       

      Python Program - Chat Function

      Python Code File

       

      import openai
      def execute (pr):
         'Output:prompt,text,cost'
         'DependentPackages:os,openai'
         # Variables
         openai.api_type = "azure"
         openai.api_base =  "REPLACE_WITH_YOUR_ENDPOINT_HERE" # your endpoint should look like the following https://YOUR_RESOURCE_NAME.openai.azure.com/
         openai.api_version = "2023-05-15" # your model deployment will tell you what version you need to use
         openai.api_key = "REPLACE_WITH_YOUR_API_KEY_HERE"
      
         prompt = str(pr)
         response = openai.ChatCompletion.create(
            engine="gpt-4", # engine = "deployment_name".
            messages=[
               {"role": "system", "content": "You are a helpful assistant. You help answer questions about SAS Intelligent Decisioning."},
               {"role": "user", "content": "Can you use Python code in SAS Intelligent Decisioning?"},
               {"role": "assistant", "content": "Yes, Python code files are supported in SAS decisions."},
               {"role": "user", "content": "Can you publish decisions from SAS Intelligent Decisioning to Azure?"},
       		 {"role": "assistant", "content": "Yes, decisions can be from SAS Intelligent Decisioning to Azure by using SAS Container Runtime (SCR)."},
      		 {"role": "user", "content": prompt}
      
            ]
         )
      
         print(response)
         print(response['choices'][0]['message']['content'])
         text = response['choices'][0]['message']['content']
         # cost section https://azure.microsoft.com/en-au/pricing/details/cognitive-services/openai-service/
         cost_per_prompt_token = 0.03 / 1000 # usd per 1000 tokens
         cost_per_completion_token = 0.06 / 1000 # usd per 1000 tokens
         completion_tokens = response['usage']['completion_tokens']
         prompt_tokens = response['usage']['prompt_tokens']
         cost = prompt_tokens * cost_per_prompt_token + completion_tokens * cost_per_completion_token
         print ('Cost for prompt and answer in USD: ', round (cost, 4))
         # limit max-tokens https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/chatgpt?source=recommendations&pivots=programming-language-chat-completions
      
         return prompt, text, cost
      
      # sample questions to test the code - remove them after a test
      execute('Does Azure OpenAI support customer managed keys? ')
      execute ('Can you publish decisions from SAS Intelligent Decisioning to Azure? ')
      execute ('What about rule sets? To which destinations can you publish rule sets from SAS Intelligent Decisioning?')

       

      Output

      Sample prompts and generated output:

       

      Azure_OpenAI_in_SAS_Intelligent_Decisioning_GPT-4_Chat.png

       

       

      Retrieval Augmented Generation

      A retrieval augmented generation (RAG) methodology can be used to refine the answers received. You can use your own data as a source of information. You can direct the Azure OpenAI model to use only search results from the document to construct the answer.

      In other words, you can ask questions on your own unstructured data.

       

      Azure_OpenAI_RAG_Methodology.png

       

      Output

      We have used the SAS Intelligent Decisioning documentation as a source. As you can see, when you use RAG, the answer is much more precise.

      Azure_OpenAI_RAG_Methodology_Sample_Answer.png

       


      Conclusions

      SAS Intelligent Decisioning is delivering great capabilities, when it comes to combining models, rule sets, data queries, decision points, code files.

      • Azure OpenAI can help you fulfill many / additional use cases. Mentioned four: Content generation, Summarization, Code Generation and Semantic Search (using RAG).
        You can embed Azure OpenAI in your SAS decisions by using a simple Python code file, calling the Azure OpenAI REST API.
      • Faster time to Value: By using Azure OpenAI and SAS Intelligent Decisioning, you can take decisions to a next level.

       

      Version history
      Last update:
      ‎10-30-2023 01:38 AM
      Updated by:
      Contributors

      sas-innovate-white.png

      Our biggest data and AI event of the year.

      Don’t miss the livestream kicking off May 7. It’s free. It’s easy. And it’s the best seat in the house.

      Join us virtually with our complimentary SAS Innovate Digital Pass. Watch live or on-demand in multiple languages, with translations available to help you get the most out of every session.

       

      Register now!

      SAS Explore 2023 presentations are now available! (Also indexed for search at lexjansen.com!)

      View all available SAS Explore content by category: