Observability Solutions for the SAS Viya Platform
Within the Viya Platform, SAS® provides administrators with two different observability solutions: SAS® Enterprise Session Monitor and SAS® Viya® Monitoring for Kubernetes.
Why two SAS® observability solutions? Can they co-exist in a Viya Platform deployment? A lot of questions from the field regarding this topic...
In this post, I will provide you with information regarding these two observability solutions and will compare them.
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
SAS Enterprise Session Monitor (SAS ESM) and SAS Viya Monitoring for Kubernetes (SAS V4M) have some overlapping capabilities, but each offers some important features that the other does not.
Both tools...
- ... let you observe Kubernetes cluster items (nodes, pods...), but SAS ESM is limited to only SAS Viya components (pods and nodes where these pods are running). Many of the metrics and workload-related events that SAS ESM gathers come from parsing SAS Viya components logs.
- ... have an alerting capability.
How can these two observability solutions help you as an administrator?
Looking at the differences:
- SAS ESM is more business user-oriented and provides good visibility of session workload and resource usage from Programming and CAS sessions.
- SAS V4M is more I.T. user-oriented or infrastructure management oriented. Monitoring programming and CAS session workload is so difficult as to be practically impossible in SAS Viya Monitoring.
| SAS Enterprise Session Monitor (SAS ESM) | SAS Viya Monitoring for Kubernetes (SAS V4M) | |
|---|---|---|
| What is observed? | All SAS deployments (SAS 9.4, Viya 3.x, Viya Platform) where the SAS ESM agents are deployed on (Windows, Unix, Kubernetes). Can observe all kinds of supported SAS deployments from a single SAS ESM server. |
The Kubernetes cluster where the SAS V4M solution is deployed on. Can observe single or multiple Viya deployments in a single Kubernetes cluster. |
| What cluster means? | Set of nodes/machines where SAS components are deployed and running. | Kubernetes cluster. |
| May I have a full view of my cluster resources consumption? | Yes, the metrics come from all processes or pods, not only the SAS Viya Platform ones, that are running on machines where the SAS ESM agents are deployed. Using the Dashboard, the CPU Performance and Session Count portlet provides an aggregated value of CPU consumption at the cluster level (Cpu load histogram). |
Yes, for each cluster where SAS V4M is deployed. The metrics are gotten at the container level, for all pods not only the SAS Viya Platform ones, and then aggregated at pods, nodes, and cluster levels. |
| What kinds of metrics are gotten and stored? | All metrics from the SAS components (servers, services, workloads) and the nodes/machines running processes. On Kubernetes, all metrics from the nodes where the SAS ESM agent is deployed (nodes, pods, containers). |
All metrics from the Kubernetes cluster (nodes, pods, containers). |
| What kinds of logs are gotten and stored? | Events that are gotten from all SAS programming and CAS sessions’ logs parsing. | All logs from SAS Viya platform pods. |
| Where is the data stored? | Into the SAS ESM PostgreSQL database. Depending on where the SAS ESM server is deployed, its PostgreSQL database server could be on Windows, Linux, or in a Kubernetes dedicated namespace. |
The data is stored in Kubernetes persistent volumes.
|
| May I control the volume of data that is stored? | Yes, the data that is stored in the ESM PostgreSQL database is controlled by retention periods depending on their type. These parameters can be adjusted anytime using the Admin settings / Database Maintenance interface. |
Yes, but the data retention settings are defined only during the SAS V4M solution deployment. The retention settings are by databases (Monitoring: Prometheus - Logging: OpenSearch) It is required to redeploy the SAS V4M solution to modify these settings, this impacts the existing stored data. |
| May I monitor and analyze SAS Programming Runtime Environment servers and sessions workload? | Yes, SAS ESM is designed to monitor and analyze users' workload sessions. Depending on SAS deployments integration with SAS ESM, it is possible to get information about used Procs and Data. This information is available by session, so by user too. |
Yes, it is possible session by session, but complex at the workload level. For each SAS compute session, we know the user that owned it. But it is complex when it is required to associate multiple SAS Compute sessions to have aggregated information at the workload level, and for a specific user. |
| May I monitor and analyze CAS server and sessions workload? | Yes, SAS ESM is designed to monitor and analyze users' workload sessions. Depending on SAS deployments integration with SAS ESM, it is possible to get information about used CAS actions and Data. This information is available by session, so by user too. |
Yes, it is possible session by session, but complex at the workload level. It is difficult to know which user owns really a CAS session. And it is complex to associate specific workload CAS sessions between them and to parents’ SAS compute sessions currently. As a result, it is complex, if not impossible, to have a view of the users' workload. |
| May I monitor and analyze a full user workflow? | Yes, SAS ESM was mainly designed to monitor and analyze users' workload sessions. Each time a new session is started, a unique identifier is created and propagated to sub-sessions as an environment variable for the purposes of reconciliation with parent jobs/sessions. This information is stored and can be used to analyze user workload sessions after they are ended. |
Depends. It is currently limited. Difficult to associate sessions and sub-session live, and much more complex after these sessions ended. SAS V4M is great to get information about each process, but not great for having a full view of user workload sessions. |
| What are the limitations in term of reporting? | The reporting is limited. Reports/applications are provided through the SAS ESM web interface, and they can only be modified using filtering. It is possible to export the data from PostgreSQL, inject them into CAS tables, and then use SAS Visual analytics to create custom reports. |
A lot of dashboards are provided with the SAS V4M solution (Monitoring: Grafana - Logging: OpenSearch), but there is no limitation since it is possible to develop custom dashboards, modify the provided ones, or use dashboards available on the web. It is possible to export the data from both Prometheus and OpenSearch, inject them into CAS tables, and then use SAS Visual analytics to create custom reports. |
However, SAS Viya Monitoring for Kubernetes has a couple of features ESM does not have. It gathers Kubernetes events and lets you search and filter them just like log messages. SAS Viya Monitoring for Kubernetes project also includes samples and guidance for how to monitor SAS Viya deployments using the main cloud service providers’ own observability suites: Azure Monitor, Amazon CloudWatch, and Google Kubernetes Engine Cloud Logging and Cloud Monitoring.
So, it is useful to have both observability solutions deployed.
The technologies in the background
For those who are interested in the more technical aspect of these SAS observability solutions...
| SAS Enterprise Session Monitor (SAS ESM) | SAS Viya Monitoring for Kubernetes (SAS V4M) |
|---|---|
Server and agent developed in Java using third-party technologies.
The SAS ESM database is managed using PostgreSQL |
Based on market standard technologies. Available on Kubernetes and OpenShift
|
I hope this article has been helpful to you.
Special thanks to my SAS Colleagues, Christopher Blake (SAS ESM Product Director), Prasad Poozhikalayil (Product Manager), Greg Smith (And the R&D WLM Team), and David Stern (GEL).
References:
- SAS Viya Monitoring for Kubernetes (SAS V4M) documentation:
- SAS Enterprise Session Monitor (SAS ESM) documentation:
Find more articles from SAS Global Enablement and Learning here.
