Demystifying ESP 5.2 Streaming Analytics
Streaming Analytics has been a major initiative in the last ESP versions development. It started with version 4.3 last year (see Steve Foerster’s article for an introduction to the ESP Analytics Windows), and has grown with 5.1 and 5.2 to provide today various analytical capabilities. Let’s review the ESP 5.2 Streaming Analytics features and their integration points with SAS and the rest of the world.
This post will be articulated into 2 big categories that clearly identify how you will design your ESP project: Streaming Analytics with Online models and with Offline models.
Here is the definition of online and offline models, extracted from the documentation:
“Online models use algorithms that are packaged with SAS Event Stream Processing Analytics and that are trained in SAS Event Stream Processing projects.
Models are referred to as offline when they are specified, developed, trained, and stored separately from the ESP server.”
Online Models
The use of online models implies that ESP relies on its embedded analytical capabilities to provide streaming analytics. It does not require any other external components.
1 – Algorithms
SAS ESP includes many pre-packaged analytical algorithms. Depending on which algorithm you want to use, you will have to use the right ESP window.
Why are there 2 types of ESP windows for running online models? Well the 2 categories of online algorithms work differently and require different inputs:
- The “Score” window (and the “Score algorithms”) requires an online trained model and this will be provided on the fly by the “Train” window.
Typical Online “Train” / ”Score” Windows design
- The “Calculate” window (and the “Calculate algorithms”) only requires some properties to be set.
Typical Online “Calculate” Window design
2 – Real-Time Reconfiguration
Some online algorithms can be reconfigured in real-time. What does that mean?
It means that you can change some algorithm properties while your ESP project is running and while your model is trained or your statistics are computed.
Indeed, you don’t want to stop your ESP project just because you want to change a little bit how the model is trained. As of now, only a few properties of a few algorithms support “reconfig” requests, but many more will be available in the future.
Properties are changed in real-time with “request” events:
3 – Model Supervision
You can also supervise your online trained models using a “Model Supervisor” window. A “Model Supervisor” window can receive and store numerous models as inputs and send them appropriately to the “Score” window either immediately (as they come) or on-demand using requests events. Here is an example:
Offline Models
The use of offline models means that ESP will integrate analytical capabilities coming from other SAS solutions (for example Viya) or third-party languages (for example Python).
1 – ASTORE models
What is an ASTORE file?
Definition from a SAS communities article:
“The ASTORE is a binary file that contains information about the state of an analytic object and it is transportable -- that is, the file can be produced on one host and consumed on others without the need of traditional SAS export or import. This universal and compact file form was developed to overcome the shortcomings of encoding a large number of rules from hundreds of models (typical in ensembles) into SAS DATA Step code.”
ESP 5.2 supports ASTORE files created using the methods referenced here in SAS Viya. It supports various algorithms like Bayesian Network, Gradient Boosting Tree, Random Forest, etc.
ESP also supports Deep Learning models (DNNs, CNNs, RNNs) through the same concept of ASTORE files.
How do I score events using an ASTORE file? Again, using a “Score” window.
The “Score” window not only works with online models but also with offline ASTORE models.
The ASTORE file requires to be read through a “Model Reader” window. The ASTORE file is either pre-loaded or loaded on-the-fly using a “request” event.
Typical Offline ASTORE “Model Reader” / “Score” Windows design
As Steve mentioned in his article, multiple ASTORE models can be accumulated and supervised in the “Model Supervisor” window. At any time, on-demand, one can decide to use a specific ASTORE file in the scoring window because it better fits the current situation.
Typical Supervised Offline ASTORE “Model Reader” / “Score” Windows design
2 – Recommender system
From the documentation, “A recommender system tries to predict the rating or preference that someone would give an item such as a book, article, or any product in general.”
A recommender system can be built using the “recommend” CAS Action Set in SAS Viya, and it can’t be transported using an ASTORE file. Instead, 3 specific CSV files, generated by the “recommend” CAS action set, require to be injected in the ESP flow.
The use of an “offline” recommender system is similar to the use of an ASTORE file:
- It needs a “Model Reader” window
- It could be supervised in a “Model Supervisor” window and changed on-demand using a request
- It is used in a “Score” window
Typical Offline Recommender “Model Reader” / “Score” Windows design
3 – User-specified
Here, we are talking about:
- The “Calculate” window only
- Integration with SAS Model Manager through the concept of SAS Micro Analytic Service modules (MAS modules) in ESP
- Integration with other languages using the same MAS module concept
The “Calculate” window provides a final way to perform scoring on streaming events.
First, you can import the champion model from the current Viya version of SAS Model Manager (15.2) if it’s part of the same Viya environment (SAS ESP Studio 5.2 and SAS Model Manager 15.2 are deployed all together).
This import will result in a MAS module creation in the ESP project. Depending on the source algorithm, the MAS module will be created as a standalone DS2 package or as an ASTORE file along with a DS2 wrapper.
What is interesting is that there is a tight integration between SAS Model Manager 15.2 and SAS ESP Studio 5.2. If the champion model changes, the ESP Studio user will be notified of that change and can decide to update the model immediately or not (at design time, not at run time).
Secondly, you can import a model from SAS Model Manager 14.2 (latest SAS 9.4 version of SAS Model Manager) through a ZIP file.
Finally, you can create a custom MAS module using either C or DS2 or Python code, containing customized scoring or computing functions, callable from the “Calculate” window.
4 – Benefits of using the SAS Model Manager direct integration (in a "Calculate" Window) vs an offline ASTORE-based model (in a "Score" Window)
Special thanks to Steve Jenisch, from the IoT division, for providing the following insights:
If any predictor variable transformations are developed in the modeling pipeline (that is, new feature variables that are derived from event variables that are present in the input schema of the ESP window the model is to be deployed to), then VDMML will generate those calculations into the scoring model. In the case of a DS2 package, the new features are generated inline with the scoring code. In the case of an ASTORE-based model, the new features are generated in the DS2 wrapper, which in turn invokes the ASTORE scoring entry, and these are deployed by SAS Model Manager as a single unit (thus the “multi-type model”).
This is an important point, since any other ASTORE-based models that are deployed to ESP (Score Window) will require that derived variable transformations be separately identified and implemented in an upstream window (and passed through on the Score Window input schema).
Also, with regard to MAS modules associated with ASTORE-based models, MAS will load the model file directly and transparently. No additional Model Reader with a load request is required.
With the MAS approach - in combination with the Model Studio integration with Model Manager - the objective is to not require the user to know or care if the model implementation is DS2 or ASTORE.
Wrap-up
As a wrap-up, please find below a chart representing the various capabilities behind ESP Streaming Analytics.
Thanks for reading.
Hi Nocolas,
I came across that page when I was doing my research online, and I have tried the method mentioned in the documentation. However, it doesn't work for me. It seems that ESP couldn't recognize the relevant information in the zip file.
For example, it couldn't find the Score Code. And nothing is registered after clicking "OK".
It might be due to the point that you have mentioned. It's okay, I'll keep searching for solutions. Thank you very much for your suggestions!
Have a nice day!
Thanks,
Yiping
