BookmarkSubscribeRSS Feed

APIs in action: build an automated profiler with SAS Viya

Started ‎03-27-2020 by
Modified ‎04-07-2021 by
Views 5,609

Customer Profiler applicationCustomer Profiler applicationRecently we were approached by a customer to tackle the following business challenge in their SAS Viya environment.

 

The challenge 

The customer’s marketing operations team needed an easy-to-use tool allowing them to compare two different populations:

  1. a targeted subset of their customer base
  2. the entire population

This exercise would provide value for internal teams, exposing how particular audiences differed from the whole (e.g. responded to an email offer vs. no response). The solution required flexibility and allow the end-user to quickly assign and select different attributes to compare (such as acquisition channel and credit worthiness). The other requirement was for the solution to be automated and therefore shorten the time for the marketing team to take action.

 

We recognized by creating the application for this requirement we'd cover many other use cases for automation (simple to complex). Our SAS team designed a solution combining the ease and customization of a SAS Visual Analytics front-end with the automation and flexibility of SAS Job Execution Web Application. This was developed in SAS Viya version 3.5 on a cloud instance.

 

Solution overview

 

Customer Requirements

  • Easy-to-use
    • The end-user uploads a sample list of customer IDs in the form of a CSV file that would filter a dashboard containing visualizations based on the entire list of customer ID’s, to reflect only the records in the uploaded sample
  • Automated
    • The report refreshes automatically upon uploading the CSV
  • Intuitive
    • Easily understand and compare the differences in populations
  • Scalable
    • Support data volumes approaching 26 GB

Solution Components

  • SAS Viya:
    • The primary component of this solution is SAS Viya (CAS), for its ability to support the large data volumes
  • CAS engine:
    • CAS processes the filter requests very quickly because of its distributed, in-memory architecture, designed for fast performance on large volumes of data
  • Visual Analytics:
    • Web-based interface will house the dashboard and visualizations
    • Provides a breadth of dynamic, analytical graphics to present the data in a compelling way to uncover actionable insights
  • SAS Job Execution Web Application:
    • Responsible for executing user-initiated or scheduled scripts comprised of SAS code
  • Additional features:
    • This GUI allows users to manage and run custom SAS code behind the scenes to automate tasks with little manual effort
    • Jobs are repeatable – build once, run many times
    • An embedded HTML input form provides a user interface to the job that, combined with the job definition, executes in real time
    • Build the reports once and update when the data does

End- User Experience

  • The user is presented an HTML form embedded within the Visual Analytics (VA) dashboard to upload a customer list (in the context of this project, it is the desired CSV file)
  • The HTML form prompts the user to upload a file to CAS
  • This triggers a backend script to automatically run the SAS Job that uploads the CSV and filters the report
  • The result is a 2-page VA report containing the HTML form to upload the customer list CSV file and another page comparing 2 sets of visualizations side-by-side– one set that reflects the uploaded customer list CSV and one that reflects the entire data set. Below is a video outlining a representation of the report and how it works.

 

Application development

 

Build the report

We’ll start by building a simple report with static data. Afterward, we’ll embed a script to enable automatic updates when the user provides new data.

 

Let’s begin in SAS Visual Analytics. From a blank report, we can drag and drop visualizations into the workspace and assign data. Our simple report design includes:

  • A sample profile of the full customer population (right side)
  • A sample profile of the user-selected customer list (left side)

This video shows how to build this basic report and begin to populate data variables.

 

 

 

Did you know…Visual Analytics allows you to join multiple tables directly in the report editor? It’s easy. Watch the video below for details.

 

 

Build the front-end

Now that we’ve set up the profiler comparison dashboard let’s set up the user-input page. Create a new tab and drag in the Web Content object.

 Web Content objectWeb Content object

 

 

 

On the Options tab, the Web Content section is where we’ll insert our automated script. We’ll come back to this once our script is created.

 

Web Content URLWeb Content URL

 

 

 

You can include more customized design features and align with your corporate style guides to increase user adoption. For tips on designing interactive and intuitive reports in SAS Visual Analytics refer to the following resources:

 

Automation and HTML creation

Next, let’s look at how to create the automated script and HTML.

 

Automated script

Why re-create the wheel? We’ll use the out-of-the-box example scripts provided in SAS Job Execution as a starting point to accelerate our development.

 

Access the sample scripts at http://<host>/SASJobExecution/samples

 

For this application, we used “Upload a CSV File” sample. You can select the check box next to it and then use the Copy To button in the top right to copy the sample to your location. Watch the video below for a demonstration.

 

 

Once you’ve copied the sample, you can navigate back to Content and open your file.

 

We made a few enhancements to the sample source code in “Upload a CSV File.”

  1. Connect to CAS in-memory engine.
    /*  Set the options necessary for creating a connection to a CAS server.     */
    /*  Once the options are set, the cas command connects the default session   */ 
    /*  to the specified CAS server and CAS port, for example the default value  */
    /*  is 5570.                                                                 */
    /*****************************************************************************/
    
    cas mySession host='&lt;server&gt;' port=5570;

    NOTE: added this section to the file; replace <server> with your CAS host

     

  2. Import the new user-selected file.
    proc import datafile= upload
      out=work.Customer_List_in
      dbms=csv
      replace;
      getnames=yes;
    run; quit;

    NOTE: replaced the ‘out’ value

     

  3. Verify the join variable (customer_id) is formatted correctly.
    data work.Customer_List_stg (drop=customer_id_old);
    	set work.Customer_List_in (rename=(&iuml;&raquo;&iquest;Customer_ID = customer_id_old));
    	length Customer_ID $16.;
    	Customer_ID = put(customer_id_old, 16.);
    run;

    NOTE: Added this section to the file. This is important for the data join. If your VA report does not load correctly, check that the variable name matches correctly.

     

  4. Load the data to the Public CAS library.
    *****************************************************************************/
    /*  Drop Customer_List from Public and load/promote imported file            */
    /*****************************************************************************/
    
     proc casutil; 
         droptable casdata="Customer_List_new" incaslib="Public" quiet; 
         *promote casdata="Customer_List_stg" casout="Customer_List" incaslib="Public" outcaslib="Public"; 
         load data=work.customer_list_stg casout='customer_list_new' outcaslib='Public' promote;
     quit;

    NOTE: added this section to the file

     

  5. Finally, the code below generates a simple HTML response after the file is loaded to specify that the file is available.
    data _null_; 
     length filename $1024; 
     filename = htmlencode(strip("&amp;_WEBIN_FILENAME")); 
     call symputx('_WEBIN_FILENAME', filename); 
    /*  if (upcase("&amp;_WEBIN_FILEEXT") ne 'CSV') then do;  */
    /*    rc = dosubl('ods _all_ close;');  */
       file _webout; 
       put '&lt;!DOCTYPE html&gt;'; 
       put '&lt;html&gt;'; 
       put '&lt;head&gt;&lt;title&gt;File Loaded&lt;/title&gt;&lt;/head&gt;'; 
       put '&lt;body&gt;'; 
       put '&lt;h2&gt;Uploaded file "' filename +(-1) '" is now available as Customer_List.&lt;/h2&gt;'; 
       put '&lt;h2&gt;You may now refresh your report and close this window.&lt;/h2&gt;';
       put '&lt;/body&gt;'; 
       put '&lt;/html&gt;'; 
       abort cancel; 
    /*  end;  */
     run;

    NOTE: added this section to the file

 

Customize the HTML file

Modify the HTML form as needed. Access the HTML by right clicking on your Job Definition and going to Edit > HTML form as shown here:

 

Access HTML generated fileAccess HTML generated file

 

NOTE: In this example we are using SAS Viya version 3.5. If you are using Viya 3.4, you need to copy both the Job Definition and the Job Form from the Sample page and you’ll see them as two separate elements.

 

The following highlights key updates in the example script.

 

  1. You’ll notice several pieces of display text; change these as needed for your job to instruct the end user. 
    <body role="main">
    
    <div>SAS<sup>&#174;</sup> Job Execution</div>
    
    <h1>Upload a CSV File</h1>
    
    <p>
    Use this page to upload a CSV file from your local
    machine to the SAS server machine.
    </p>

     

  2. In the following code, the hidden input types serve as job parameters

    1. If your program has a different name or location than your HTML form, change the _program parameter. If your HTML and definition have the same name keep this as $PROGRAM$.
    2. If you are using the form within a VA report as we are in this example, remove the target= “_tab” parameter. In the following code, the hidden input types serve as job parameters.

       

    <form action="/SASJobExecution/" method="post"
      enctype="multipart/form-data">
    <input type="hidden" name="_program"     value="$PROGRAM$">
    <input type="hidden" name="_action"      value="execute"/>
    <input type="hidden" name="_output_type" value="ods_html5"/>
    <input type="hidden" name="_csrf"        value="$CSRF$">
    <input type="hidden" name="_odsstyle"    value="HTMLBlue">

     

  3. Below is the code for the “Submit” button. Change the command name on the button if desired. You can also remove the “Show SAS Log” checkbox.
    <input type="submit" value="Upload Data" class="pointer"/>
    <input type="checkbox" name="_debug" id="_debug" value="log" class="pointer"/><label for="_debug">Show SAS Log</label>​

 

Add endpoint to the report

Once we have the Job Execution SAS code and the HTML form, locate the endpoint to embed in the front-end report.

 

Right-click on your Job Execution project, then select Properties, and open the Details tab.

 

Job Execution property detailsJob Execution property details

 

Copy the Job submit action and paste this back to your Visual Analytic web form.

 

Finish building the application

Let’s return to Visual Analytics to complete the application. In the web content form, go to Options and then enter the job submit URL under Web Content.

 

Web Content URL entry fieldWeb Content URL entry field

 

 

Save the report and select View Report to test functionality.

 

View Report optionView Report option

 

Check out the application

Congratulations! If you’ve followed along through this article, you now have access to the final application.

 

Final Customer Profile reportFinal Customer Profile report

 

Want a quick-start?

Clone the assets used in this article from the sas-viya-job-execution-service-automated-profiler GitHub repository.

 

You’ll need the Upload_a_CSV_File_definition.json  file to configure the automated script in SAS Job Execution. As discussed previously, in Viya 3.5, the job definition and the job HTML are contained in the one file.

 

Import the json file and the VA report via the SAS Environment Manager > Content > Import button.

 

Import the JSON file & VA reportImport the JSON file & VA report

 

Make the following required modifications.

  1. In SAS Job Execution, you need to enter your server name in the CAS connection
  2. In the Visual Analytics Report, edit the Web Content link to your server

 

That's a wrap

We've demonstrated how to build an application using a prepackaged sample in the SAS Viya Job Execution Web Application. The customer was quite pleased with the POC we created. In fact, they were impressed at how easy the process was to complete. They immediately realized the benefits and took this approach to create several other applications.

 

Related posts

Job Execution 2.1 User Guide (PDF)

Job Execution 2.1 User Guide (Web-based)

SAS communities: Creating a Job in VA 8.3 via Job Execution Web Application

Data Entry in SAS Visual Analytics 8.3 Blog Post

  • Part 1 - The Basics
  • Part 2 - Write a Viya Job with a form
  • Part 3 - Integrate SAS Viya Jobs with SAS Visual Analytics
Comments

Amazing product, team!

This is a really cool solution! Great job!

Version history
Last update:
‎04-07-2021 11:40 AM
Updated by:

sas-innovate-2024.png

Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

 

Register now!

Free course: Data Literacy Essentials

Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning  and boost your career prospects.

Get Started

Article Tags