Enabling SAS Event Stream Processing Authentication via Oauth

If you have paid attention to SAS Event Stream Processing news recently you are aware that interest in processing and analyzing data that has just been generated is growing rapidly. This interest will lead to customers implementing new software designed to analyze data that is typically sensitive in nature. With an ever-increasing attention paid to data security, these customers will likely want to ensure that access to events stored in memory within the ESP engine is secure. To that end a new (ESP 3.2) capability has been added to the latest version of ESP that builds on existing capabilities to shore up security of sensitive data.

The 3.1 release of SAS Event Stream Processing announced the encryption of traffic between client and server. This capability was among the first of numerous ESP security enhancements that are either available or announced. One of the new features of SAS Event Stream Processing 3.2 was the introduction of optional authentication between clients and servers. Authentication was added to the product's network interfaces such as the XML server API and the Java/C Pub/Sub APIs and adapters. It was also added to the SAS ESP Streamviewer, a tool that allows the user to display events streaming through a model. However, authentication has not been added to SAS Event Stream Processing Studio, as that tool is intended for development and testing new models before they are deployed to production.

Enabling ESP Authentication

The high-level steps for enabling authentication are available in Chapter 26 of the SAS Event Stream Processing 3.2 User's Guide, but the details required to implement these steps may not be so obvious. The following diagram provides an overview of the example used in the blog by showing the steps required for a user to authenticate a LASR adapter to an ESP XML server. In this blog we highlight each step with a brief description and then at the end is a link to a document with greater detail. The diagram may not make sense at this point, but hopefully by the end of the blog the pieces will come together.

Cloud Foundry UAA

Authentication between a client and server in SAS Event Stream Processing requires a token obtained from a supported OAuth2 Authorization server. The only supported OAuth2 server at present is Cloud Foundry (CF) User Account and Authentication (UAA). These tokens are based on the JSON Web Token and are digitally signed by UAA. UAA is an identity management service from Cloud Foundry, but it does not require a full deployment of Cloud Foundry's Platform as a Service (PaaS) to access its services. As a result CF UAA may be deployed as a standalone service. Cloud Foundry services are Open Source Software and therefore are readily available for download and deployment.

Generating and Making Use of the Token

As noted earlier, the SAS Event Stream Processing 3.2 User's Guide identifies the major steps required to generate a token. However, these steps are fairly broadly defined to meet the needs of a diverse customer base. This section organizes those steps and provides a brief description of each.

Define users for authentication

It is first necessary to define those accounts that will be used to authenticate ESP clients to ESP servers. Enabling authentication on an ESP server is a global and permanent setting. Therefore all clients connecting via TCP or pub/sub APIs to that server must use the authentication token. In order to generate a token a user account must be defined.

Install GIT (if not installed) - required for CF UAA download

Git is a widely-used source code management system for software development. It is a distributed revision control system with an emphasis on speed, data integrity, and support for distributed, non-linear workflows. The source for CF UAA is stored in an open source Git repository and therefore Git is required to download the software. Install Git on the same machine you expect to install CF UAA.

Install OpenSSL (if not installed) - required for generating keys

OpenSSL is required to generate private/public key pair for verifying and signing the token. In most cases the OpenSSL package will be installed.

If SAS Event Stream Processing Authentication and Encryption was selected and installed along with the ESP engine and ESP studio, then openssl will be installed in the /bin directory of the ESP engine (i.e. $SASHOME/SASEventStreamProcessingEngine/3.2.0/bin).

Install JDK 1.8 (if not installed) - required by the CF UAA web server

CF UAA requires JDK 1.8. CF UAA will fail to build the web server using versions prior to 1.8. Therefore if it doesn't exist, it will be necessary to download it and install it on the same machine as CF UAA.

Download and install CF UAA using GIT

Using the Git package that was downloaded in step 2, download (clone) the CF UAA software to a directory of your choice. In this example the software was downloaded to /opt/sas/uaa.

Generate SSL keys

SSL keys are necessary to verify and sign tokens. Generate tokens to be used in subsequent configuration steps.

Modify UAA port number (if currently in use)

CF UAA uses a Tomcat server to create the UAA server. The default port used by this server is 8080. If you install CF UAA on a system where an existing service is already using port 8080, such as SASServer1_1, then it will be necessary to modify the port.

Define location for UAA configuration updates and set associated environment variable

UAA configuration settings are defined in the YAML language. The configuration file for the CF UAA server is uaa.yml. A default uaa.yml file is laid down by the deployment, but it is recommended that it not be changed. In this example the configuration file was stored in /opt/sas/resources.

Configure SSL keys in UAA configuration file

In order to include the SSL keys in the UAA configuration, add the private and public keys to the uaa.yml file. The public key will become the verification key and the private key will be the signing key.

Configure CF client ID in UAA configuration file

It is necessary to define a client ID that will be referenced when generating a token. The same client ID is also used when starting an ESP server to enable authentication. An example of a client ID is espadmin.

Configure CF users in UAA configuration file

In this example Cloud Foundry users are defined in the uaa.yml configuration file. Users can also be added via the command line if available. User account data and encrypted password and secret values are stored in an internal database, hsqldb, or UAA can be configured to store them in a PostgreSQL or MySQL database. CF UAA can also be configured to connect to external user stores through LDAP and SAML.

Update oauth-clients.xml with client ID

Although not noted in the documentation, it is necessary to update the Spring configuration to reflect the client ID configured in UAA as well as token expiration values.

Start CF UAA web service (using Java 1.8)

Once environment variables are defined and configuration is complete, it is time to start the UAA service. Starting the service is simple. The “gradlew run” command to build the web applications and start the web service.

Generate and save an OAuth token

Now that the UAA web service is running it is possible to generate a token. Token generation is accomplished via the curl command that specifies the clientID, user name and password within the REST call, while the token is returned to the user’s shell. Once the access token is available, it should be saved to a secure file.

Configure public SSL key in the ESP Server

In order for the ESP server to authenticate users, the public SSL key created in an earlier step used to sign the token must be stored in a newly-created directory within the SAS ESP installation. Place this key file in $DFESP_HOME/etc/oauth/pubkey.pem. Saving the key in this location makes the public key available to all ESP servers, but is only used when authentication is enabled at ESP server start.

Start the ESP server with authentication

Starting an ESP XML server with authentication requires that the -auth clientID parameter be specified at startup.

Pass token as part of client call to the ESP server to authenticate

Finally, the last step is to invoke a client connection to the server. The location of the token is specified by using the -O option of the adapter. If an invalid token is specified the connection will fail. If successful, the adapter functions as it would without going through authentication.

As you can see there are many steps required to establish authentication between ESP clients and servers. Most steps are straight-forward exercises that require minimal time to execute, but it may take some time to digest the purpose of each step. Review the SAS Event Stream Processing User's Guide for additional options and information. Additional references can be found at the end of this blog.

Streamviewer

Earlier it was noted that authentication does not exist for SAS ESP Studio. However, authentication is enabled for SAS ESP Streamviewer. Although there is no predetermined setting to enter the OAuth token, a pop-up window is displayed when connecting to the HTTP pubsub port. The following screen shot shows the window presented for entering the OAuth token.

The text value of the token should be entered here or the URL that returns the token. Once the token is entered, the subsequent window allows the user to subscribe to existing windows in a model.

More details...

You were hoping for more? Well you are in luck. The detailed version of this blog is available as an attachment. This document contains commands required to execute these steps on a Red Hat machine with Visual Analytics 7.3 and Event Stream Processing 3.2 deployed.

Wrapping Up

In this blog we touched on the key items required to enable authentication between an ESP client and server. Since CF UAA is the tool of choice for generating OAuth tokens, it is worthwhile spending time gaining a better understanding of the capabilities of the software. In this example we manually entered key components in the uaa.yml configuration file. A more likely scenario is using the UAA command line to manage users and client IDs.

It is obvious that this blog was orginially written for SAS ESP 3.2. However, the content for OAuth authentication still applies to SAS ESP 4.3. It should be noted that SAS ESP 4.3 also supports additional authentication mechanisms. For more information, click on the SAS ESP 4.3 link in the Resources section below.