We’re smarter together. Learn from this collection of community knowledge and add your expertise.

Yes, you can secure ESP !

by SAS Employee RPoumarede on ‎12-04-2017 03:31 AM - edited on ‎02-16-2018 05:04 PM by Community Manager (1,345 Views)

 

Encryption, authentication and access control are standard features for an “Enterprise Class” software. Although security was originally not the primary focus of ESP, those features have been progressively added or extended all along the ESP Software life cycle.

Today SAS® Event Stream Processing 4.3 comes with a solid set of security capabilities.
However, your customer might be running a previous version of ESP.

So, here we will first provide a short history of this aspect for the major ESP releases, then delve a little bit more into the new ESP 4.3 capabilities and finally walk through a few examples with the new SASLogon Authentication feature. (Access controls and encryption examples could be the topic for a future blog :))

1.png


The History

 

ESP version Added feature Commment
3.1 Encryption
  • Encryption support for pub/sub API (C and java) and adapters
  • Encryption support applies for Streamviewer (as it relies on pub/sub API)
support for ESP client-server communication (using SAS approved TLSv1.2 ciphers)
3.2 Authentication
  • oAuth2 token Authentication for pub/sub API (C and Java) and adapters
  • Auth2 token Authentication for the XML Server API (includes REST administration API)
Authentication was not added to ESP Studio. This tool is intended to create and test new models in developer’s environment before they are deployed to production.
Authentication support applies for Streamviewer (as it relies on pub/sub API), it will prompt the user for a token.
4.2 Authentication
  • SASLogon user/password Authentication for pub/sub API (C and Java) and adapters
  • SASLogon user/password Authentication for the XML Server API (includes REST administration API)
SASLogon authentication when enabled, allows ESP to use the SASLogon Viya microservice to validate the user authentication.
The user and password is transmitted from the ESP client to the ESP Server, which itself submits those credentials to the SASLogon service.
If the service accepts the authentication with the provided credentials, then the ESP server to client communication is enabled.
4.3 (current) Authentication
  • Kerberos (keytab) Authentication support added for pub/sub API (C and Java) and adapters
  • Kerberos (keytab) Authentication support for the XML Server API (includes REST administration API)
Encryption
  • SSL support for ESP Studio to ESP Server communications
  • SSL support for Event Stream Manager to ESP Server communications
User Access Control
  • Explicit read/write permissions on engine, project, query, and window objects based on the user.
Explicit permissions are defined for ESP users in a ".yml" file.
5.1 Authentication
  • SASLogon authentication for ESP Studio
 

 

 

The capabilities matrix (for ESP 4.3)


Based on tests, documentation, and RnD or Product Management feedback I have tried to summarize available security capabilities depending on the ESP version and ESP component in the table below.

Security Features in ESP 4.3 Type: Pub/sub API Adapters ESP Client, REST API ESP Streamviewer ESP Studio Event Stream Manager service
  Ports used : pubsub(5555) ESP Client, REST API http-admin (5556), http-pubsub (5557) http-pubsub (5557), pubsub (5555) http-admin (5556), http-pubsub (5557) http-admin (5556), http-pubsub (5557)
Authentication oAuth2 YES YES YES[1] YES[2] YES
  SASLogon credentials YES[3] YES[3] YES[3] NO[4] NO
  Kerberos[5] YES YES YES NO[4] NO
User Access Control Permissions[6] YES YES YES NO[4] NO
Encryption(*) TLSv1.2 YES YES YES YES[7] YES[7]

 

[1] Authentication from Streamviewer to ESP server. User will be prompted for a token or userid/password.
[2] ESP Studio does not authenticate users. Studio user can supply OAuth token or credentials if authentication is turned on at ESP server.
[3]SASLogon microservice requires that the SAS Viya platform be available (VI, VA, VDMML, VS, ...).
[4] ESP Studio integration with Viya services is planned to be available beginning at ESP 5.1. The standalone (no authentication) will remain possible in ESP Studio.
[5]The XML server does not support Kerberos authentication on the Microsoft Windows platform.
[6]In ESP 4.3, Access control is managed via a permissions file. You cannot use ACLs on 64-bits ARM platforms.
[7]"SSL" checkbox is available in the UI.

 

(*) Here we are NOT talking about client encryption between the Web Browser and the ESP Web Application, but we are talking about the communications between the ESP Web Application and the ESP Server. Note that with 5.1, it is planned that ESP Studio can use HTTPS from the browser when it is installed with SAS Viya.

 

Note: Most of the cases have been tested. However as there are a lot of possible combinations and ESP clients, the table might not be 100% accurate or complete enough, feel free to contact me if you detected anything wrong !

Event Stream Manager


Event Stream Manager (ESM) is a completely new with ESP 4.3. It is a web-based client that enables you to manage your SAS Event Stream Processing environment (Administer ESP deployments and projects, monitor deployment health, etc...).

As ESM is a SAS Viya web application, it integrates with all SAS Viya components and Microservices (including authentication).

If deployed in stand-alone mode, it comes with its own set of Viya Microservices. As other SAS Viya application, ESM can use full Viya capabilities for authentication (user/password, Kerberos, etc...) and TLS can be configured for the ESM User Interface.

As you can see below, the end user is authenticated in the GUI (it stills not the case for ESP Studio). 2.png

However, when working with an ESP server, ESM supports "encryption-enabled" ESP server but does not support ESP server SASLogon Authentication.

3.png

 

The Authinfo file


In many cases, you won’t have the possibility to provide a user/password in an "interactive mode" (prompt or command line), or even if you can provide a user/password, maybe you don’t want to expose this password in plain-text.

In such cases, you can use an "authinfo" file to store the credentials that you need for the authentication. The Viya documentation provides a nice "How to" section, explaining Authinfo File creation. Basically, it is a simple text file called .netrc or .authinfo, usually located in user's home directory, secured to be read only by its owner and containing user credentials in a specific format, such as :

host host-name port 5570 user your-userid password your-password


However a simplified format, such as :

default user your-userid password your-password

can work too.

I found out the hard way that ESP can be, kind of "capricious" with the format of the .netrc file content... for example:

  • For dfesp_client, use "user" field, do not use the "port" field
  • For Streamviewer, you need to use the "username" field (instead of "user")
  • For Streamviewer do not use "host" or "port", just default, user and password fields

Note that you can also use "authinfo" file in ESP adapters when connecting to thirdparty data/event providers (such as CAS).

Finally, an interesting thing about the "authinfo" file authentication is that you can use the SAS PROC PWENCODE to encode your password using one of the available encoding method (for example sas004 which uses a 256-bit key plus 64-bit salt value to encode passwords).


Example with the SASLogon authentication in ESP


First you need to start the ESP Server with the SASLogon authentication enabled, for example:

[sas@sashdp04 ~]$ dfesp_xml_server -pubsub 5555 -http-pubsub 5557 -http-admin 5556 -auth saslogon://sashdp01.race.sas.com:80/SASLogon -model file://vwapeng.xml


In the server log, you should see :

2017-05-31T03:48:57,385; INFO ; 00000017; DF.ESP; (dfESPengine.cpp:655); dfESPengine::initialize() for engine esp: Pub/Sub services are enabled for port 5555, encryption disabled, client authentication enabled 2017-05-31T03:48:57,385; INFO ; 00000018; DF.ESP; (dfESPpermissions.cpp:26); dfESPpermissions::init(): Access control disabled (could not open permissions.yml, error: file not found) 2017-05-31T03:48:57,386; INFO ; 00000019; DF.ESP; (dfESPengine.cpp:675); dfESPengine::initialize() dfESPengine version 4.3 completed initialization 2017-05-31T03:48:57,392; INFO ; 00000020; DF.ESP; (Esp.cpp:636); esp engine started, version 4.3, pubsub: 5555 2017-05-31T03:48:57,392; INFO ; 00000021; DF.ESP; (Esp.cpp:651); attempting to communicate with saslogon url 'http://sashdp01.race.sas.com:0/SASLogon' 2017-05-31T03:48:58,398; INFO ; 00000022; DF.ESP; (Esp.cpp:667); communication with saslogon url 'http://sashdp01.race.sas.com:0/SASLogon' succeeded


Now your ESP server is running a project defined in vwapeng.xml and will only accept access to the project windows content from authenticated users.

Note: Do not forget to specify the SASLogon Viya service port (80 by default). Otherwise your ESP server will start without error but you will never be able to authenticate.

It is time to test the access to the ESP server:

[sasdemo01@sashdp04 ~]$ dfesp_xml_client –url "http://sashdp04:5557/SASESP/windows?count=true" <authenticate scheme='basic'/> http://sashdp04:5557/SASESP/windows?count=true : status 401 (requires authentication)


As we can see, credentials are required, now let’s create an authinfo file with the sasdemo credential and try again:

[sasdemo01@sashdp04 ~]$ echo 'host sashdp04 user sasdemo01 password lnxsas' > $HOME/.netrc [sasdemo01@sashdp04 ~]$ chmod 600 $HOME/.netrc [sasdemo01@sashdp04 ~]$ dfesp_xml_client -url "http://sashdp04:5556/SASESP/windows?count=true" -auth "saslogon://sasdemo01?secure=off" <windows> <window-aggregate contquery='contQuery' count='3608' name='aggW1hourRet' project='vwap_CAS'/> <window-aggregate contquery='contQuery' count='3608' name='aggW24hourRet' project='vwap_CAS'/> <window-aggregate contquery='contQuery' count='3608' name='aggW5minRet' project='vwap_CAS'/> ...


Now it works !

 


Streamviewer SASLogon authentication

As it is using two communication channels (http-pubsub and pubsub), the Streamviewer configuration for SASLogon is a bit specific, in order to use the SASLogon authentication you’ll need to provide the user password at two levels:

  • UI for communications on http-pubsub port 5557,
  • Authinfo file for communications on pubsub port 5555

Let’s see what it means with an example:
First we need to start the Streamviewer service with an authinfo file containing the user credentials.

[sas@sashdp04 ~] echo 'default username sasdemo01 password lnxsas' > .netrc [sas@sashdp04 ~]$ nohup $DFESP_HOME/bin/streamviewer.sh -product h2 -http 8100 -h2file /opt/sas/viya/config/strmvw/streamviewer_h2 > /opt/sas/viya/config/strmvw/streamviewer.log & [1] 4047 [sas@sashdp04 ~]$ nohup: ignoring input and redirecting stderr to stdout


After starting the ESP XML server with SASLogon authentication enabled, in Streamviewer we can define and test the connection to the server.

When prompted we just provide the sasdemo user and password.

4.png

We can now subscribe to the ESP windows to display the real-time dashboards.

5.png


Conclusion

On the security side, ESP 4.3 brings very important new capabilities, securing his position as a true Enterprise Class software.

Thanks to an increased integration with Viya, ESP 4.3 brings the capability to almost all the ESP tools/clients to authenticate through user/password or Kerberos mechanisms.

With ESP 4.3 a new tool is also appearing, "Event Stream manager" which brings and can leverage the full Viya capabilities for authentication.

However, do not underestimate the required time to set up and test authentication, encryption and permission policies as many combinations are possible with the various ESP tools/clients and ways to access ESP.

Finally, please note that all "5.1+" mentions are road-map information, so subject to changes.

Many thanks to Ravi Sunchu, Chris M West, Robert Levey, Vince Deters for their invaluable help to cover this ESP topic.

Your turn
Sign In!

Want to write an article? Sign in with your profile.


Looking for the Ask the Expert series? Find it in its new home: communities.sas.com/askexpert.