SAS Event Stream Processing Kubernetes Operator Framework
With the growing amount of data, with the attention Internet of Things (IoT) is drawing, and with the strong requirement of data analytics, we continue to make SAS Event Stream Processing (ESP) more robust and powerful with cutting-edge technologies.
SAS ESP Kubernetes leverages all the capabilities and features provided by the Kubernetes. ESP Kubernetes Architecture is independent of the underlying platform, therefore, can be deployed in any cloud environment, such as Amazon AWS, Microsoft Azure, and others, or on-premises. ESP 2020.x and later version will be exclusively container-based.
Watch our demo video on SAS ESP Kubernetes Operator Framework to learn how the ESP Operator works behind the scene. It explains the communication path from SAS Event Stream Processing Studio and Event Stream Manager to run the ESP server/(s) in the Kubernetes pods with the help of the ESP Operator. The video demonstrates how easy it is to create and test the ESP models right from the ESP Studio. Using ESM, we can deploy and manage these ESP models with their configured deployment settings in the Kubernetes cluster. The video also exhibits how an auto-scaling model can be easily configured and monitored right from the ESM.
Benefits of SAS ESP-Kubernetes:
- Horizontal and vertical runtime elasticity
- Automatic and optimized resources utilization
- Resources isolation
- A highly available, reliable, and fault-tolerant environment
- Self-healing, quick auto-recovery from failures
- Provisions of creating multi-user, multi-tenant deployments
- Auto-scaling of ESP Servers using Load Balancer
- Secure access to web-clients, i.e., ESP Studio, Event Stream Manager, and ESP Streamviewer using User Account and Authentication (UAA) and OAuth2 Proxy
- Integrated PostgresSQL for metadata management
- Container support for all cloud platforms
- Full configuration management
Components of SAS ESP Kubernetes Operator Framework
The figure below illustrates the different components of SAS ESP Kubernetes Operator Framework Analytics. Let’s briefly discuss how they work together in the Kubernetes environment where all the components run in pods.
- ESP Operator
The ESP Operator is the first point of contact and is responsible for starting, stopping, updating, deleting, and monitoring of the ESP server pods in the Kubernetes cluster.
It is built using the Kubernetes Operator APIs and allows the cluster administrators or maintainers to define their Kubernetes objects. The ESP operator primarily executes the custom resources that the ESP server can run. Custom resource embeds the definition and the specifications of the ESP model that the ESP server runs. Whenever there is a new custom resource, the ESP operator makes sure to read it, deploy it, and spin up a corresponding pod in the Kubernetes nodes.
- Web Clients (SAS ESP Studio, SAS Event Stream Manager, SAS ESP Steamviewer)
They also run in the pods in the same cluster and have seamless access to all the ESP servers running in the Kubernetes pods. ESP Studio creates and tests the ESP Models, and ESM deploys the ESP Models and manages all the deployments under one roof, and Streamviewer visualizes the data in the ESP server pods.
SAS ESP Meter is also deployed that bills the number of events processed by the ESP servers over a month or year. It is accessible from the ESM.
- User Account and Authentication (UAA)
The UAA is a multi-tenant identity management service used as a standalone OAuth2 server. Its primary role is to issue tokens for all the web client applications and authenticates the users with their credentials. We use UAA to provide multi-user, multi-tenant deployments.
- PostgresSQL DB Persistent Volume
A persistent volume is configured in the cluster to allow the ESP server pods to read or write files that might be required by the ESP model. For example, if an ESP model needs to write some output, then this volume is used. We provide manifests to configure.
- Filebrowser
The Filebrowser is a standalone or middleware 3rd party application that provides a file management interface within a defined directory structure. It allows access to multiple users where each user can have its directory structure. It is used to upload, delete, preview, rename, and edit files and folders. With this tool, you can access the configured persistent store used by the Kubernetes pods in the cluster.
With SAS ESP Kubernetes, leverage the capabilities of cloud-enabled IoT for real-time processing, data analytics, and make intelligent decisions.
