SAS Event Stream Processing version 4.2 brings a lot of new features. Among them is the Adapter Manager. The ESP Adapter Manager is a way to process streaming events on a grid of ESP engine instances and make ESP massively parallel. In addition, the Adapter Manager provides elasticity which is very interesting when deploying ESP in the cloud. Admins can then decide to scale ESP dynamically across multiple hosts, adding and removing ESP engines as demand increases and decreases. The ESP Adapter Manager acts as an ESP cluster manager.
The purpose of ESP Adapter Manager
Let’s have a look to the following video to understand what the goal of the Adapter Manager is.
Configuration of an Adapter Manager
An ESP Adapter Manager is defined via an XML file. In the XML file, you have to define several things:
Here is an example of an Adapter Manager configuration:
Regarding the routing policy, we have 3 different choices:
The Hash policy requires you to define fields that are used in the hashing mechanism.
“Venue” is the field to be hashed. The hash value is an integer between 0 and the number of engine instances minus one. The router uses the hash value to determine which engine instance the event is sent to. Value hashing is useful to ensure that related events are processed together. For example, in the broker surveillance model, you need all events relating to a specific broker sent to the same engine to accurately detect broker malpractice like front running.
How does it work behind the scenes?
Watch the following video to learn how the Adapter Manager works when it starts.
A few commands to know
The Adapter Manager has its own executable called dfesp_am_server. To start an Adapter Manager with its corresponding configuration file, run the following command:
No HTTP pubsub port is required. Note that the Adapter Manager will NOT automatically start ESP XML Factory servers that are defined in the Adapter Manager configuration file (esp-cluster). The ESP XML Factory servers must be provisioned and started before running the Adapter Manager. In order to dynamically add a running ESP XML Factory server to the control of an Adapter Manager, the following command needs to be executed:
curl is used here (instead of dfesp_xml_client) because we can provide the request body via standard input (stdin). The request body contains the definition of the new ESP XML Factory server to add to the Adapter Manager’s control. This ESP instance must be running first. The REST API URL means that we want to add an engine called “srv02esp3” (must be aligned with the request body) to the router called “espmap1” (the router takes the name of the esp-map we defined in the Adapter Manager’s configuration file). If we no longer need to publish events to a running ESP XML Factory server, then we can remove it from the Adapter Manager’s control, by running this command:
Want to see it in action?
Watch this video to view a demo of the ESP Adapter Manager.
Thanks for reading.