An Overview of SAS ESP Deployment on Windows

SAS ESP continues to make its mark on the IOT and event processing landscape. In the early years SAS ESP was a mostly standalone product with basic integration into the SAS platform. As it matured it has steadily converged with the platform and now is an integral part of several solutions.

Although SAS ESP is typically deployed on Linux hosts it can also be deployed on Windows. The deployment process on Windows has changed over time and is now closely aligned with a standard SAS Viya 3.4 deployment on Windows. Several significant and notable changes occurred between SAS ESP 5.1 and 5.2. In this article we will look at the deployment process when SAS ESP 5.2 is ordered as a single product (i.e. no other SAS Viya products) for Windows and compare it to the steps require for a SAS VA on SAS Viya deployment on Windows.

Prerequisite reading

You will find that deploying SAS Viya solutions on Windows is quite different from deploying on Linux. I suggest reading an article written by my colleague Raphaël Poumarede about the new process of deploying on Microsoft Windows.

Differences

In his article Raphaël mentions that a limited number of products are supported on Windows. That list of products includes SAS ESP, but it should be noted that if only SAS ESP (stand-alone) is selected the deployment process varies somewhat from an order with additional solutions. So let's walk through the deployment phases for SAS ESP 5.2 and capture those differences at a high level, as well as a few SAS ESP-specific changes that should be revealed.

It should first be noted that the ESP order options on Windows are limited to just the ESP server. SAS Event Stream Manager, SAS ESP for CAS and SAS ESP for Edge are not available on Windows.

A SAS ESP-only order does not contain the full stack of microservices that a SAS VA or SAS VDMML order would. It requires a minimal number of services to provide authentication and core services. More on this later.

Requirements

It stands to reason that some of the requirements would vary from a full deployment of just SAS Visual Analytics. Here we capture the key items.

Hardware

The minimum number of cores for SAS ESP-only is four, compared to at least eight if SAS Visual Analytics is included in the order. The minimum memory required for SAS ESP is noted as variable, 32-64GB, compared to a minimum of 48GB for SAS Visual Analytics. To provide a reference point our deployment in RACE required roughly 5GB of RAM. So even at 32GB there is still plenty of memory for processing large projects and event volumes.

User Accounts

For SAS Visual Analytics the administrator must define the installer, cas and postgres accounts. For SAS ESP-only the installer and postgres accounts are required. CAS is not delivered as part of SAS ESP and therefore the related account is not required.

Security

SAS Visual Analytics requires Kerberos as the network authentication protocol on Windows. An LDAP is required for logging on to SAS ESP Studio or using SAS Logon to authenticate to an ESP engine, but it does not require Kerberos for authentication. Even if the customer chooses not to use SAS ESP Studio, which optionally can authenticate users via an LDAP server, LDAP is required by several critical services.

It is not a requirement to enable authentication when connecting clients to an ESP engine, but if the customer wants to authenticate clients, which is highly recommended, they have four options: oauth_local, kerberos, SASLogon and oauth2.

SAS/ACCESS engines

Releases prior SAS ESP 5.2 included the DataDirect ODBC drivers to enable DB connectors and adapters access to data sources. The DataDirect ODBC drivers are no longer included with ESP orders. Therefore Windows for x64 orders requiring access to data sources may include one or more following:

SAS/ACCESS Interface to ODBC
SAS/ACCESS Interface to PC Files
SAS/ACCESS Interface to PostgreSQL

One optional SAS/ACCESS interface is included with SAS ESP 5.2. Additional engines must be purchased.

Pre-installation

Most of the steps required for pre-installation are similar to a SAS VA deployment.

Mirror

Like deployments on Linux it is highly recommended that you use Mirror Manager to create a local mirror repository. The local repository will contain the MSI files used for installation. For a standalone ESP order on Windows the repository will require slightly more than 2GB. This is substantially smaller than 21GB required for an example order with VA, VS, VDMML, CONNECT, three ACCESS engines and two text mining languages. The VA repository alone is 12GB because it contains all of the required microservices included with Viya.

The following screen shot shows the packages included with an ESP-only order.

Ports

As you can imagine the number of ports and default port numbers for a VA deployment on Windows will be similar to that of a Linux deployment. However, an ESP-only order requires a subset of ports that a VA deployment would require. Earlier it was noted that the number of services included with an ESP-only order is limited. As are result it stands to reason that fewer services means fewer ports.

The list of ports can be found in the SAS Event Stream Processing 5.2 on Windows: Deployment Guide. However, it appears the list is missing the Postgres ports for SAS Infrastructure Data Server (ports 5430-5439). In addition to the standard ports for Viya, it is critically important that the ports used for ESP servers should also be made opened where necessary (i.e. pub/sub and http).

Deployment Scripts

Both a VA deployment and an ESP-only deployment on Windows require that the SAS orchestration CLI utility be used to create the deployment scripts.

Encrypt Postgres Credentials

Like a full deployment of VA an ESP-only deployment requires that a script be run to encrypt the Postgres account credentials.

Installation

There are only a few steps required for SAS ESP installation and they are similar to a SAS VA deployment with a few exceptions.

CAS

A SAS VA deployment requires that the credentials for the CAS account be encrypted. However a SAS ESP-only order does not contain CAS and therefore does not require a cas account to be defined.

LDAP/Kerberos

SAS ESP Studio users require authentication. Therefore you will need to configure your deployment to use an LDAP provider. Likewise if the desire is to enforce authentication on an ESP server to use SASLogon then an LDAP server must be available to the deployment and configured to use it. This configuration is accomplished via the sitedefault.yml file, very much like it would be done on a Linux deployment.

It is possible to deploy the software without configuring an LDAP provider. Like Viya deployments on Linux, the sasboot account is created at deployment time and can be used to authentication when logging on to SAS ESP Studio. However, this is not recommended! It may be okay for a workstation but should not be used for dev/test/prod deployments.

Post-Install

Required Microsoft Packages

A deployment of ESP requires that the Microsoft Visual C++ Redistributable Package for Visual Studio 2013 (64-bit version) be installed in order to run SAS ESP on Windows. This package is made available during the installation. Once the installation is complete the installer can run the executable file to install the required software. This is in contrast to a VA deployment which requires both the Visual Studio 2013 and 2015 packages. In addition the OpenSSL libraries are required if you intend to encrypt traffic between the ESP clients and servers.

Set Environment Variables

In order to take start ESP servers and access various features, such as the command line interface, it is necessary to define the DFESP_HOME environment variable and add several paths to the PATH environment variable.

Enable Metering

SAS ESP licensing is based on the number of events processed in a production environment (i.e. consumption). Therefore the metering service is required and must be started to measure activity. Obviously this is not the case for a SAS VA on Viya deployment, as its license is based on the number of cores.

Additional Info

Services

One of the key differences noted earlier is that an ESP-only deployment contains a subset of the services required for a SAS VA on Viya deployment. In this screen shot we see that there are 12 services running. Technically there are 13, but sas-viya-all-services is a service used to start/stop other services.

Issue with sas-viya-all-services

A deployment of SAS ESP from an order in early October contained a defect where sas-viya-all-services would only start three of the remaining 12 services. The issue is related to the startup scripts expecting CAS be part of the environment, but obviously CAS is not part of an ESP-only deployment. The script is expecting a specific path under C:\Program Files\SAS\Viya, but the expected path does not exist. This can be easily resolved by running the following code from an administrator PowerShell session to create the directory.

# set $ViyaScriptRoot to the root of your Viya scripting directory
# generally this is something like C:\Program Files\SAS\Viya\bin
if ($ViyaScriptRoot -eq $null) {
    $ViyaScriptRoot = “C:\Program Files\SAS\Viya\bin”
    }
$casBinPath = Join-Path $ViyaScriptRoot “..\SASFoundation\utilities\bin\”
if ((Test-Path $casBinPath) -eq $false) {
    New-Item -Path $casBinPath -ItemType “Directory”
    }

Once the directory is created, the sas-viya-all-services service should start all services as expected. Thanks to R&D for their assistance with this issue.

Updating the software

The process to update a deployment using the latest repositories requires a few simple steps.

If using a mirror, sync the mirror with the repositories at SAS. You can use the "diff" option of mirrormgr to view those packages that will be updated.
Although not noted in the doc, the services were stopped in our test environment.
Run the setup.bat command with the -update option.

The update process log will show which packages will be updated with the installed and updated versions. Here is an example of the updates that occurred from a mirror of an order created in early October and a mirror refresh of the order in early December.

Updates found:
Package                 Installed Version ->       Update Version
sas-esp                        10.2.0.217 ->             10.2.1.0
sas-espstrmvwr                 10.2.0.173 ->             10.2.1.0
sas-datasvrc                     0.3.7.12 ->              0.3.8.1
sas-eseastore                  1.15.0.110 ->             1.15.1.0
Do you wish to download updates? [yes/no]

If you reply "yes" the update will continue with the update process followed by starting the services.

Final Thoughts

Although an ESP-only deployment on Windows is similar to a VA on Viya deployment, there are several differences of which to be aware. Hopefully these highlights provided some perspective on those differences.

References

SAS® Viya® 3.4 on Windows: Deployment Guide

SAS® Event Stream Processing 5.2 on Windows: Deployment Guide