Editor's Note: Adapter Manager is now known as Cluster Manager. The text below has been changed, but the video still references Adapter Manager.
SAS Event Stream Processing version 4.2 brings a lot of new features. Among them is the Cluster Manager (formerly known as Adapter Manager.) The ESP Cluster Manager is a way to process streaming events on a grid of ESP engine instances and make ESP massively parallel. In addition, the Cluster Manager provides elasticity which is very interesting when deploying ESP in the cloud. Admins can then decide to scale ESP dynamically across multiple hosts, adding and removing ESP engines as demand increases and decreases.
The purpose of ESP Cluster Manager
Let’s have a look to the following video to understand what the goal of the Cluster Manager is.
Configuration of a Cluster Manager
An ESP Cluster Manager is defined via an XML file. In the XML file, you have to define several things:
Here is an example of a Cluster Manager configuration:
Regarding the routing policy, we have 3 different choices:
The Hash policy requires you to define fields that are used in the hashing mechanism.
“Venue” is the field to be hashed. The hash value is an integer between 0 and the number of engine instances minus one. The router uses the hash value to determine which engine instance the event is sent to. Value hashing is useful to ensure that related events are processed together. For example, in the broker surveillance model, you need all events relating to a specific broker sent to the same engine to accurately detect broker malpractice like front running.
How does it work behind the scenes?
Watch the following video to learn how the Cluster Manager works when it starts.
A few commands to know
The Cluster Manager has its own executable called dfesp_am_server. To start a Cluster Manager with its corresponding configuration file, run the following command:
No HTTP pubsub port is required. Note that the Cluster Manager will NOT automatically start ESP XML Factory servers that are defined in the Cluster Manager configuration file (esp-cluster). The ESP XML Factory servers must be provisioned and started before running the Cluster Manager. In order to dynamically add a running ESP XML Factory server to the control of an Cluster Manager, the following command needs to be executed:
curl is used here (instead of dfesp_xml_client) because we can provide the request body via standard input (stdin). The request body contains the definition of the new ESP XML Factory server to add to the Cluster Manager’s control. This ESP instance must be running first. The REST API URL means that we want to add an engine called “srv02esp3” (must be aligned with the request body) to the router called “espmap1” (the router takes the name of the esp-map we defined in the Cluster Manager’s configuration file). If we no longer need to publish events to a running ESP XML Factory server, then we can remove it from the Cluster Manager’s control, by running this command:
Want to see it in action?
Watch this video to view a demo of the ESP Cluster Manager.
Thanks for reading.
Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.
If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website.
Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning and boost your career prospects.