SAS Enterprise Session Monitor Usage Series Part 2 This is the second post in a series for how to use SAS Enterprise Session Monitor. I’ve frequently referred people who were interested in learning practical information about Enterprise Session Monitor usage to two blog posts on the old ESM website. But as all good things on the internet do over time, the old host disappeared. So, I’ve reached out to the Enterprise Session Monitor team and received permission to bring the information back, in the hopes that you too will find it useful. -Erik Pearsall   Links SAS Enterprise Session Monitor Usage Series Part 1 Unlocking SAS Performance with Enterprise Session Monitor Heatmaps Obsessing over Observability SAS Enterprise Session Monitor - Obsessing over Observability - SAS Users   ESM Deployment Using Ansible SAS Enterprise Session Monitor – Deployment Using Ansible   Custom Process Monitoring with filterSpec in Enterprise Session Monitor Taking Control: Custom Process Monitoring with filterSpec in Enterprise Session Monitor   Tag! You're it! Mastering ESM Tags. Tag, You’re It!: Mastering ESM Tags Analytic Workloads through the Lens of SAS Enterprise Session Monitor A version of this article originally appeared on boemskats.com In July of 2020 as “How Analytic Workloads Work” by Nikola Markovic. This has been modified and reposted with permission from the Enterprise Session Monitor team.   This will be about fundamentals, as "real world" as possible. A big part of that will involve explaining how these fundamentals fit into the modern cloud landscape and affect performance and cost, including SAS Viya. But to get to that, and to truly appreciate it, it's worth starting from the basics. The Basics: How Processors Process Data Consider a program that looks like this. data work.inventory; infile '/some/data/inventory.csv'; input someColumns : $someFormats. /* more columns probably */ run; This simple snippet of SAS code uses a SAS DATA step - one of the oldest and most widely used data load mechanisms in SAS - to read a text-based CSV dataset (/some/data/inventory.csv), and store it as a SAS dataset in the temporary WORK library (work.inventory). The WORK library is a bit like a temporary SAS scratch disk, used to store intermediate datasets between processing steps.   So, what is it that happens as this program runs? Consider:   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   For this code to run, and for the machine to do the work we're asking it to do, the following things must happen: The input data, that CSV file, needs to be read from storage - a disk somewhere. That data then needs to be transported from that storage device to the CPU on which the code is running. The CPU will receive the data in chunks and must process it. It needs to work out which segments of those chunks are the lines of the CSV file, which segments of each line are the actual columns of data, and how each of those column segments should be interpreted - whether the text between those two commas is a phone number, a timestamp, or someone's address. Once it has worked all that out, its task is to write the resulting data out. In the case of this code, it will be writing the data out in a format it will find easier to understand next time round - the SAS binary data format (sas7bdat). That resulting data must then be transported from that CPU to a storage device somewhere where it can be written, to be read back again later. These fundamentals are important. If you've ever run out of storage space on your laptop, you'll know that the amount of data a disk can hold is limited by its capacity. If you then tried to free up some space by moving some of the larger files to an external disk, you'll have discovered that the rate at which disks are able to read and write data is also very limited - by either the speed of the disk, or the speed of its connection to your laptop.   It's a common assumption that the speed and age of a machine's processor dictate its performance. While it's true that a faster, newer CPU can execute more instructions than an older one, if those instructions consist of 'process this data', that CPU can't do anything useful if the data it's meant to be processing hasn't reached it yet. It will spend a lot of its time waiting for the rest of the system to catch up and supply it with more data; the faster the processor, the more time it'll likely spend waiting.   The question when processing data is 'how much data is enough data to keep my processor busy?'. Time for an example from the real world. An example from the real world For this example, a large CSV file was loaded using code very similar to the snippet above, and the process was monitored with SAS Enterprise Session Monitor. So information about the data load step’s performance, and other information like the rate at which the data was reaching the processor, and how busy it was as a result.   The following graph is a straight export from ESM. From this graph, we can see that this code started executing at 17:00:35 and ran for just under two minutes, until 17:02:30. Here are a few other things the graph shows. CPU Capacity Throughout the time it ran, the code used close to 100% CPU. While the 'percent CPU' process metric used to describe CPU utilization in this context is commonly used, it can be quite confusing. Technically it means that 'a program which is running at 100% CPU is using the amount of CPU cycles equivalent to the capacity of a single processor core, or utilizing all processing time available to a single thread of execution'.   A word on threads. Some programming languages, and many SAS Foundation procedures, can split their work into multiple independent streams of instruction, enabling them to distribute their work to multiple workers - multiple threads. This is how they're able to 'go above 100% CPU', by utilizing more than one CPU core at a time. CAS, the core compute component of SAS Viya, is a good example of a compute engine that does almost all its processing in this way. CAS not only distributes its work across multiple CPU cores but is able to also distribute it across multiple nodes in a cluster, achieving 'massively parallel processing' and huge gains in performance and processing speed.   However, older procedures like the classic SAS DATA step, or the Python runtime (with its Global Interpreter Lock), are strictly single-threaded. They are only capable of processing one linear stream of instructions at a time, and a single piece of code can never use more than 100% of CPU - regardless of how many available CPU cores it has as its disposal.   You can think about it like this: when a step flatlines at 100% like the one in this chart, it is a single-threaded step. As such, because it's flatlining at its peak possible utilization, it is running at optimal performance, as efficiently as it possibly can. In other words, here the processor *is* being supplied with enough data to keep it as busy as it could be, and the limiting factor in the completion time of this code is indeed the processor. Data Throughput But at what rate does the data need to flow to and from the processor for it to maintain this optimal 100% CPU utilization? How much of that CSV data is it capable of converting into that easier-to-read-back-in-later SAS binary format in each time period, and how much of the resulting binary data does it produce as output in that time?     Hovering over the chart above shows that for this code, this rate was around 100 MB per second for the incoming CSV file being read from disk, and a little over 200MB per second for the binary dataset being produced and written to disk.   The SAS binary dataset this step produces will be 'easier to read next time round'. One reason that future steps will find it easier to read is precisely because they won't have to figure out where each bit of data starts and stops - the work the data step does when it first reads the CSV file from disk. In the binary dataset, that data is stored in fixed width variables, where each record and each column are aligned and occupy the same amount of space on disk. The downside to this is that, because all those trailing blank characters are also written to disk, the output.sas7bdat binary dataset takes up more disk space. This also explains why, in this graph, the resulting data is written at twice the rate that the source data is read in.   With that in mind, if we were in a situation where we didn't have enough disk space to store that padded out file, or our target disk device wasn't fast enough to cope with a 200MB/s write speed, we could tell SAS to compress that output before writing it to disk. This would change the way those blanks are stored and drop the size of the output file by around 60%, and therefore also reduce the rate at which it needed to be written. However, from the analysis earlier, we know that the CPU already has no time spare (it was running at 100%) and asking it to also apply compression to the output data would give it even more work to do. Doing that in this instance would increase the overall completion time for our load step by around 30%, and as we are not currently sharing this instance with anyone else, that's not something we care about.   You could say that here, we are choosing to keep the computational complexity of our program lower, because it lets our CPU process more data faster. We are doing that because, as our metrics suggest, our instance can cope with the higher disk throughput and extra storage space required to execute the data load in the shortest possible time. Computational Complexity Loosely speaking, computational complexity is this idea that sometimes, when performing a complex operation on a given volume of data, the processor can take a long time to finish processing that volume of data. Other times, when performing a different simpler operation on the same volume of data, it can finish that work much, much faster. It all depends on how complex, or computationally expensive, the thing it's been asked to do with that data is.   Here's another example program execution. I started with the same code as above, but this time I added an extra step to describe the dataset (a PROC Means), and another to sort it (a PROC Sort).     Here, the load of the CSV data again took around 2 minutes. Then, describing the data then took another 2 minutes, and sorting it took a little over 4 minutes.   Let's focus on the first two steps for now - the Data Step and the Proc Means.   We know what the first step does - it reads data from a CSV file and writes it to a SAS dataset. What the new, second step then does is read the dataset created by the first step and calculate some summary statistics for it. For this to happen, the same data written by the first DATA step is read back by the second - the PROC Means step. This is why, if you squint a little, you can make out that the volume of teal bars covering the second step is the same as the volume of soft red bars covering the first.   However, while the two steps do work on the exact same volume of data, there's an obvious difference between them in terms of the amount of CPU resource they require.   You'd be forgiven for assuming that the second step, the one that performs actual calculations on the data, would require more processing power than the first, which just loads it. But, because in this case the data contains few numeric variables, and the input dataset is already in that SAS binary format, the Means procedure doesn't have to work very hard there. The entire second step barely needs any CPU time to do its thing.   So, if that's the case, if it requires barely any work to process that data, why does the second step take almost as long to complete as the first?   The answer to this may seem obvious by now. It's because all of that data still needs to be read back from disk for the Means procedure to process it. Unlike the first step, this instance of the Means procedure is a typical example of a step where the storage system isn't able to supply the processor with data at a sufficient rate to make use of all the available CPU time. To approach 100% CPU, it would need to read it in at about 1.1GB/sec.   A chart showing this in SAS Enterprise Session Monitor showing a job only using a fraction of a CPU thread while there are still plenty of CPU resources available on that node, almost always indicates that the job is either waiting on more data to be read in or waiting for the data it's producing to be written before more can be processed. The hold-up can be either a network connection, a database query, or like in this case, the artificial data throughput limits our cloud provider is applying. The limit is reducing the read speed from our virtual disk device to 200MB/second because of the instance type we have selected for this particular node in our cluster.     Finally, in the chart above there is a final step, which Sorts the dataset produced by the first step. It is included here as an example of what other, 'less simple' procedures can look like in SAS Enterprise Session Monitor. PROC SORT is an example of a step that uses memory and depends on memory limits, and only uses the disk if it realizes that it can't do what it needs to within those memory limits. It's also an example of a multi-threaded operation. Why is any of this Important for me? The huge range in the computational complexity of operations that data scientists commonly throw at their data is a key characteristic of analytic workloads. Understanding the underlying fundamentals of how your code uses system resources will help you build better, more efficient data jobs, particularly when building data engineering & data management pipelines.   Why should you care if your code is better or more efficient?   If you are using SAS professionally, there is a very good chance that you share your server with multiple other users, maybe even multiple teams of users. The resources on that server will be limited, and writing code that make best use of what you have available will result in faster completion times and an increase in system responsiveness and overall capacity.   The ability to look at a job profile and recognize what resource intensive code looks like can be extremely powerful in quickly identifying the root causes of poor system performance. The global observability that SAS Enterprise Session Monitor offers let experienced developers keep an eye on the workloads their teams are running and perform on-the-fly root cause analysis, identifying bottlenecks and even mentoring colleagues who might not know that the code they are generating could run faster or be more efficient. We've seen this 'collaborative analysis' approach result in collective increases in performance that more than double the effective capacity of a platform. The Cloud If you are considering moving your analytic workload to the Cloud, a precise understanding of your resource requirements can play a huge role in both your projected cloud infrastructure costs and your confidence in those projections.   All the major Cloud providers offer a range of instance types that are skewed towards CPU, disk or memory intensive workloads. All the providers allow you to scale each of those instances up, with the same resource skew, almost indefinitely. Deciding which instance type to choose, or which batches are best suited to which instance type and size, is difficult to do without properly profiling your workloads. Your IT department may be able to derive some of this data from the physical hardware you're running your workloads on now, but often, this kind of sizing results cloud infrastructure cost projections that are astronomical.   An exact knowledge of the size and skew of your workloads' resource requirements, and consistency with which they use those resources - broken down by individual flow, job or group of users - lets you identify and select quick win candidates that can easily be 'lifted and shifted' to the Cloud in their current state. It can highlight candidates for optimization that can be augmented and migrated in a second wave and drive the decision around which parts of your pipeline need to be re-engineered before they are redeployed. It enables a precise, data-first approach to cloud architecture that can eliminate guesswork and make the move much simpler. Digital transformation with confidence. Conclusion In this post we looked at how the CPU is “fed” with data, we learned about the resource requirements of different steps and operations, and talked about how those requirements can impact overall system performance.   Find more articles from SAS Global Enablement and Learning here. ... View more

SAS Enterprise Session Monitor Usage Series Part 1 I’ve frequently referred people who were interested in learning practical information about Enterprise Session Monitor usage to two blog posts on the old Boemska website. But as all good things on the internet do over time, the old host disappeared. So, I’ve reached out to the Enterprise Session Monitor team and received permission to bring the information back, in the hopes that you too will find it useful. - Erik Pearsall Links Obsessing over Observability SAS Enterprise Session Monitor - Obsessing over Observability - SAS Users   ESM Deployment Using Ansible SAS Enterprise Session Monitor – Deployment Using Ansible   Custom Process Monitoring with filterSpec in Enterprise Session Monitor Taking Control: Custom Process Monitoring with filterSpec in Enterprise Session Monitor Tag! You're it! Mastering ESM Tags. Tag, You’re It!: Mastering ESM Tags SAS Enterprise Session Monitor Usage Series Part 2 How Analytic Workloads Work Using Enterprise Session Monitor Heatmaps A version of this article originally appeared on boemskats.com in July of 2020 as “Unlocking SAS Performance Secrets” by Chris Blake. This has been modified and reposted with permission from the Enterprise Session Monitor team.   One of the most useful features of ESM is the "Top 50 Heatmap". It provides a neat, very visual way of quickly seeing what's happening (live or historically) on a server at any point in time. It's a bit like a spectrogram, but for workloads. It can instantly show you what your users and jobs are doing and even highlight if they are affecting each other. In cases like the one in this post, it can also show fundamental issues with the underlying system configuration (spoiler alert!).   What we're looking at above is a picture of the top 50 most active user sessions on a SAS 9.4 server, showing their effective CPU utilization over the selected 5 minute timespan. Looking at the heatmap, you can quite clearly see a pattern where most of those 50 user sessions - I count around 30 - are trying to do sustained work but are really struggling. Every 30 seconds or so there is a burst of activity, where CPU utilization increases in perfect sync - but only for a few seconds at a time. Not long after, those sessions go back to barely using any CPU at all. The scale is not in the image, but a hover tooltip shows us that the darkest of those red tiles represents around 90% of a single CPU, which is good performance for typical workloads.   On a day-to-day basis, our customers typically use the Heatmap feature to identify when a process is using a disproportionate amount of a resource and causing problems for everyone else. In that scenario, you would see a pattern like this one, but there would be some lines that get darker just as the others get lighter, thus highlighting the greedy process(es) causing the problem. These situations also tend to be a lot more obvious when using ESM's disk I/O heatmaps, which highlight when disk-hungry or inefficient code is affecting the performance and developer experience for everyone else.   However, in this case, you can see that there aren't any processes that stand out in that way. This suggests that there may be a real environment-wide resource availability issue at play. Looking at overall CPU usage on that node showed that there was plenty of CPU time available during those periods of starvation (there's a node-level graph further down this post), so this suggests that this may be a problem with getting enough data to the CPU. In other words, the environment either has insufficient or misconfigured storage infrastructure, or it's just not getting the provisioned disk bandwidth.   Time to pick up the phone and speak to the storage team. Talking to IT and Infrastructure teams In most organizations - but particularly large enterprises - this is where the fun and games start. We're confident that the issue is caused by insufficient disk bandwidth. However, especially as this server is virtualized, the storage team will quite likely say something like: "This is not a storage problem. You need to speak to the VMWare team. Your machine is probably starved of CPU because you are running on an over provisioned VM host. We know those guys; they're always doing that".   At this point, you may be in for a lengthy game of ping-pong.   However, because the SAS Administrator has SAS Enterprise Session Monitor, they can confidently say that over provisioning and CPU starvation is not the issue here. Here's why:   Let's look at that heatmap again. You will notice that among the sea of simultaneously suffering processes, there are some that don't seem to be affected by this problem at all: the 5th one from top, the one 12 sessions below that, then two below that one, then the second one from bottom. Here is the above image, but highlighting the sessions I'm talking about: These outliers prove that this is not a CPU starvation problem. If it was, they too would display the exact same CPU starvation phasing pattern, which they do not. These outliers appear to be getting almost optimal performance on an otherwise extremely slow host. Whatever they're doing, it probably doesn't depend on those slow disks.   Drilling in to look at the performance graphs for each of these processes showed us that in each case the amount of disk throughput they were using was very low and the size of their individual SAS WORK and UTIL directories was close to zero, meaning the data they were getting was probably coming from another disk.   In other words, it was clear that these exceptions - these optimally running processes - were either doing in-memory work or running computational tasks that didn't seem to be interacting with the SAS WORK disk.   Now is probably a good time to remind ourselves of the rule of thumb for architecting SAS "mixed analytic workloads". The rule says to have a SASWORK storage volume capable of at least 100MB/second sustained write, and 100MB/second sustained read throughput for each CPU core. (Important Performance Considerations When Moving SAS to a Public Cloud).   With that in mind, here is the overall node-level chart I mentioned earlier. It shows CPU utilization in blue, with light green and soft red bars representing device-level reads and writes to/from the SASWORK disk: Without the support of our session-level heatmap, anyone could assume from these host-level metrics that the server is underutilized, except for those occasional bursts of activity. But, because of our heatmap, we can prove that this is not the case.   What we can observe from this graph is that the sustained disk bandwidth the customer is getting from their SASWORK storage device is clearly not enough for an 8 vCPU machine (note the scale on the right-hand y-axis). However, when provided with sufficient bandwidth during those ~800MB/sec bursts, the SAS workload successfully utilizes ~80% of available CPU, which is exactly the optimal performance we would expect to see. The peaks in this graph line up perfectly with the dark phases in our heatmap, and the bursting pattern seen here is typical of storage that depends on a small fast cache capable of providing the required bandwidth.   We're only getting our required throughput while the cache is available, for a few seconds at a time - enough to get past a simple post-install infrastructure validation test, for example. At all other times, while that cache is getting flushed to disk, we get the underlying disk's true sustained rate of throughput. That real, sustained rate is clearly not enough to feed the CPU the data it needs. That is why we see the phasing patterns in our heatmaps, and that is why the end users are suffering and complaining about unusually poor performance. Conclusion It's often tough talking to your IT department and trying to explain why you believe there is a problem with the underlying infrastructure they've provisioned for you. They'll look at the graphs they have, see CPU headroom, and assume that there's nothing wrong and tell you the problem probably lies within your application. And to be fair to them, with most other applications they manage, and with most business users they interact with, that would be the correct answer. But SAS works a little differently, and SAS users aren't typical business users. Our clients very often find that the granular level of observability SAS Enterprise Session Monitor provides helps them either solve their own problems or enables them to work together with IT to almost instantly resolve issues that they've been working on for months - sometimes, years.   Find more articles from SAS Global Enablement and Learning here. ... View more

In SAS Viya, there are two authorization systems, one for the Cloud Analytics Services server and other for the General Authorization components. The CAS authorization layer deals with access to the in-memory analytics engine and its data, while the General Authorization layer controls access to content and functionality in other parts of SAS Viya. Understanding both authorization systems will help ensure the security for the entire SAS Viya system. This series of posts will cover the essentials of securing content and controlling access to functionality so that you will be equipped to perform one of the common SAS Viya administration tasks: securing the environment. Links General Authorization Essentials Part 1   General Authorization Essentials Part 1: Content | SAS Communities   Simple general authorization patterns https://go.documentation.sas.com/doc/en/sasadmincdc/default/calauthzcas/n00a7yg0yy97d4n1c1kgqvny3ofm.htm   CAS Authorization Documentation https://go.documentation.sas.com/doc/en/sasadmincdc/default/calauthzcas/n00a7yg0yy97d4n1c1kgqvny3ofm.htm   General Authorization Documentation https://go.documentation.sas.com/doc/en/sasadmincdc/default/calauthzgen/p0ro419uuj1cjqn1gw2jmofmg0bn.htm   The Rules Page https://go.documentation.sas.com/doc/en/sasadmincdc/default/evfun/n1uw3er96phzpfn1pxvnf01f6sw3.htm   Authorization Overview https://go.documentation.sas.com/doc/en/sasadmincdc/default/viyaov/p0i3vcgjpciz45n1of1v4vkffwbn.htm#p163fnzkr3l6m3n14wt1g2k6roi4 Functionality in SAS Viya When we discuss the concept of functionality in SAS Viya, we mean access to applications, selected features, and selected components. In the first part of this series on general authorization we learned about how to control access to objects like reports, SAS programs, or other content information stored within the SAS Viya folder structure. General Authorization Concept Reminders Several concepts carry over and apply when controlling access to functionality. Principals are the identities to which rules apply and allow or disallow access to endpoints or interface elements. General authorization makes sure that someone is allowed to perform a given action when they attempt it.   Principals refer to users, groups, or custom groups. A best practice is to use groups or custom groups and then adding or removing users from those groups, making maintenance of the system easier in the future.   Remember also that there is an Authenticated Users principal which refers to all the users who can successfully sign-in to SAS Viya. Rules can target this principal too. Basic Approach in Controlling Access to Functionality The general authorization security model inherently secures SAS Viya. By design, access that is not granted is implicitly disallowed. There are some predefined groups that exist that can grant access to components or features. See this link for more information on the predefined custom groups and the functionality to which they grant access: SAS Help Center: Identity Management: Reference   If these groups are appropriate for your needs, your security tasks will be to assign users to those custom groups. If you need additional customization, read on. Targeting URIs to Modify Access to Functionality Remember that authenticated users have a default level of access that is usually appropriate for default users of the software.  It is recommended to leave Authenticated Users as the principal in rules that control functional access. However, if after careful review you want your environment to modify that default access, you’ll need to modify the rules that control this access to functionality.   An important item of note: when you modify access to applications, this does not affect access to underlying services. For example, modifying a principal’s access to SAS Environment Manager might still allow them to access SAS Content folders through another interface.   Begin by identifying the access that you want to enforce. Do you want all users to be able to export PDF reports to their local machines? Do you want anyone to be able to upload data to CAS? Do you want all authenticated users to be able to edit reports? Plan out the access levels to give to various business units or teams in your organization.   Then you will need to identify the URI that targets the functionality that you want to modify or restrict for these access levels. There are documented URIs, which you can see here for a list of published Application URIs SAS Help Center: Access to Functionality: Application URIs. Some of those URIs support fine-grained functionality targeting, and you can find functional URI references for applications, components, or features.   Some examples of those URIs mentioned above:   Application, Component, or Feature URI Export Visual Analytics reports as PDF. /reportRenderer/reports/** Show the Import tab in SAS Data Explorer /casManagement_capabilities/importData Create and edit reports. /SASVisualAnalytics_capabilities/edit   See here for more information on working with URIs: Uniform Resource Identifiers (URI) in SAS Viya - SAS Support Communities   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page. Key Points about Prohibits and Authenticated Users Harkening back to the first part of this series, a reminder that SAS documentation recommends minimizing the use of prohibit rules. That includes never prohibiting Authenticated Users. Prohibit rules have absolute precedence and can even deny SAS Administrators from access.   Instead, use selective grants to provide access to other principals instead of Authenticated users. Simply change the principal from Authenticated Users to another group or custom group. This goes back to one of the primary tenants of authorization in SAS Viya: any access that is not granted is implicitly disallowed.   SAS Administrators already have a universal grant through an existing rule, and you don’t need to add additional rules to grant them access. Taking it Further There are many URIs in SAS Viya, and you can make more extensive changes by looking over the rules page in SAS Environment Manager. You can see the list of existing rules that target various URIs. Not only can you replace principals on these rules, but also edit them in other ways such as making copies, changing permissions, or even deleting rules. Rules can also have conditions on them, only granting access based off the originating IP address, or removing access on the weekends. More information here: SAS Help Center: General Authorization: Concepts   An advanced user can also attempt to work with URIs that are not documented. You can use developer.sas.com to understand the structure of a service before attempting to build a Target URI for a bit of functionality dealing with that service.   There is also a page named the Capabilities page in SAS Environment Manager, which shows content if you license a product that uses capabilities. Currently, capabilities can only be viewed at this time. In general, access to functionality is controlled by the Read permission. Some predefined rules grant additional permissions. You can view a rule’s description for more information or details about a rule.   Don’t forget to perform backups before and after you make significant changes to your system. Managing access to functionality can be a complex task. Test your changes to make sure you do not introduce unintended effects.   Find more articles from SAS Global Enablement and Learning here. ... View more

In SAS Viya, there are two authorization systems, one for the Cloud Analytics Services server and other for the General Authorization components. The CAS authorization layer deals with access to the in-memory analytics engine and its data, while the General Authorization layer controls access to content and functionality in other parts of SAS Viya. Understanding both authorization systems will help ensure the security for the entire SAS Viya system. This series of posts will cover the essentials of securing content and controlling access to functionality so that you will be equipped to perform one of the common SAS Viya administration tasks: securing the environment. Links General Authorization Essentials Part 2 General Authorization Essentials Part 2: Functionality   Simple general authorization patterns https://blogs.sas.com/content/sgf/2019/10/18/simple-general-authorization-patterns/   CAS Authorization Documentation https://documentation.sas.com/doc/en/sasadmincdc/default/calauthzcas/n00a7yg0yy97d4n1c1kgqvny3ofm.htm   General Authorization Documentation https://documentation.sas.com/doc/en/sasadmincdc/default/calauthzgen/p0ro419uuj1cjqn1gw2jmofmg0bn.htm   The Rules Page https://documentation.sas.com/doc/en/sasadmincdc/default/evfun/n1uw3er96phzpfn1pxvnf01f6sw3.htm   Authorization Overview https://documentation.sas.com/doc/en/sasadmincdc/default/viyaov/p0i3vcgjpciz45n1of1v4vkffwbn.htm#p163fnzkr3l6m3n14wt1g2k6roi4 Content in SAS Viya SAS Viya content objects are information that you or other users save in SAS Viya. These objects can be SAS programs, reports, models, or anything else that SAS Viya users create and save within the SAS Viya folder structure. These objects are accessed through the SAS content interface in SAS web applications. The folders are virtual containers rather than an actual file system, the data being stored in the SAS Infrastructure Data server. Managing who can view or modify this content is important for maintaining data integrity, confidentiality, and compliance with regulations when using SAS Viya. Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   Because these objects are stored in a database, there is no physical file path to locate these objects on disk. This is by design, as SAS Viya is designed to run on cloud-based architecture where the file system might not be available for user access. (Note: Users can still access SAS content programmatically using the FILESRVC file access method. More information and examples here: https://go.documentation.sas.com/doc/en/pgmsascdc/default/lestmtsglobal/p0qapul7pyz9hmn0zfoefj0c278a.htm#n0jkbbdvak6hann1og91a9o280rh).   Objects in SAS content can be secured with permissions. But before diving into how to apply permissions, let’s go over some basics. Principals Principals are the identities to which access controls apply on an item of content. When someone tries to read (or change/delete/modify/move, et. cetera) a piece of content, general authorization makes sure that someone is allowed to perform that given action.   Principals refer to users, groups, or custom groups. A best practice is to use groups or custom groups and then adding or removing users from those groups, making maintenance of the system easier in the future.   There’s also an Authenticated Users principal which refers to all the users who can successfully sign-in to SAS Viya. Access controls on objects can be applied to this principal too.   More information here: SAS Help Center: General Authorization: Concepts Removing principals from SAS Content: How to remove a principal from a SAS Content folder permission Secure by Default The general authorization security model inherently secures SAS Viya content. By design, access that is not granted is implicitly disallowed. When a new top-level folder is created within SAS Content, only the user who created those folders has access. This means that Authenticated Users (remember, this is everyone who can log in to SAS Viya) have no access to those folders by default, and the person who created the top-level folder would need to edit its authorization and grant other principals access to the folder.   Authenticated Users have access to their own personal folder and the Public folder, even if they have no other group memberships. Permissions on Objects and Containers The permissions that can be assigned to objects in SAS Content are listed below:   Permission Affected Activity Create Create a new object Read Read an object Update Update or edit an object Delete Delete an object Secure Set permissions on an object (also required to share) Add Add a member to a container Remove Move a member out of a container   When permissions are viewed in SAS Environment Manager, icons are used to explain effective access, and where a permission is applied. A green check for authorized, a red no symbol for not authorized, a half-filled circle for a conditional or row-level permission, and a grey circle for an unknown permission (usually seen when a permission is removed but before the new effective access is previewed). There is also a black diamond symbol used when the access control is placed on the specific object that is being viewed. This means the permission is explicitly placed on this object, rather than inherited from somewhere above it. And when I mean, “inherited from somewhere above it”, I mean from a folder that contains this object.   To understand access controls on folders, there are two different sets of permissions. There are permissions on the object itself, and a set of conveyed permissions on the container. The object permissions control whether a given principal can affect the folder itself. The container permissions become object permissions on any content stored within the folder. It is through this second set of permissions that the permissions cascade down and become inherited by reports, SAS programs, other folders stored within this one. These permissions lend themselves to being applied in common patterns of access. You can learn more about these common patterns at this post: https://blogs.sas.com/content/sgf/2019/10/18/simple-general-authorization-patterns/   These patterns can be applied so that users may access a top-level folder and have different access for subfolders for different business units. These patterns can be combined to form a simple security model that can be effective in SAS Viya’s content tree. Sharing Content Not limited to administrators, SAS Viya users can collaborate on reports, SAS programs, and other content by sharing them with other users and groups. Prior to 2024.12, SAS Drive was where users shared content items, more info at this link: https://communities.sas.com/t5/SAS-Communities-Library/Sharing-in-SAS-Drive-part-2/ta-p/500442   With SAS Drive being deprecated, this functionality has moved to the Content Page in SAS Environment Manager. Users can right-click on a folder or item that they want to share and select Share. They can pick another principal to share with, and then grant which permissions to share with that principal. You can also edit or unshare these permissions on an item already shared.   A user needs the Secure permission to share content with other users and groups. Administrators can modify sharing settings, more information here: SAS Help Center: General Authorization: Concepts   Interfaces for Securing General Authorization There are several interfaces that users and administrators can use to work with general authorization within SAS Viya. You can use the graphical user interface of SAS Environment Manager or the sas-viya command-line interface to secure SAS Content. Users can make use of the Sharing Window on the Content Page of SAS Environment manager for simple sharing with coworkers. Advanced methods would be to use either the Rules page in SAS Environment Manager or the SAS Viya REST APIs to work with authorization in SAS Viya.   More information here on using REST APIs for authorization: Authorization | SAS for Developers   Using the sas-viya command-line interface for general authorization: Updating SAS Viya general authorization permissions using the sas-viya command line Rules Page and Content URIs When access controls are applied to content in general authorization, a rule is created in the SAS Infrastructure Data Server to enforce those access controls. You can find a list of these rules on the Rules Page in SAS Environment Manager, or by using the authorization plug-in on the sas-viya command-line interface. The rules page is an advanced interface, and you can filter on existing rules, duplicate and modify them, or replace principals in rules. Administrators can also toggle the enforcement of a given rule on or off, which might be useful in troubleshooting access issues. See SAS Help Center: General Authorization: Troubleshooting for more information about troubleshooting using the rules page.   Uniform Resource Identifiers are the unique identifiers that allow rules to target a specific content object. When dealing with content, we call these Target URIs. Target URIs can be either an objectURI or a containerURI identifier. See here for more information on targeting content using URIs: SAS Help Center: General Authorization: Concepts   General Authorization Reminders As a best practice, SAS documentation recommends minimizing the use of prohibit rules. Prohibit rules have absolute precedence and can even deny SAS Administrators from accessing content. Try to apply your access controls broadly and let them filter down using conveyed container permissions. Try to apply your permissions broadly to groups as well and add/remove members from those groups for easier maintenance in the future. Lastly, consider this your friendly reminder to perform backups before and after you make significant changes to your system.   General authorization manages content such as folders, reports, and stored SAS programs. It also governs functionality such as access to applications, features, and services. Head to part two of this series to learn about controlling access to functionality.   Find more articles from SAS Global Enablement and Learning here. ... View more

I’ve taught at a few sites now where the administrators work with sensitive data, and a common question that inevitably gets asked is, “My enterprise works with X data, and I don’t want the liability of the SAS Viya administrators looking inside that data. How can I prevent this?” The purpose of this post is to explore just that.   Substitute X for any of the following: Healthcare data that might contain personally-identifying-information, financial systems that might contain highly sensitive data, or government data that might rely on specific clearances to access. That’s fine—SAS Viya has access controls, right? We’ll just use the SAS Viya Cloud Analytics Services Authorization layer and set proper permissions so that our administrators cannot access the data.    Except that there’s an issue--Any SAS Viya administrator can promote themselves to become the CAS Superuser. The CAS Superuser is exempt from any access controls placed on CAS libraries (caslibs) or the tables stored within those caslibs. (Note: I've since updated this article with information about the Select permission at the bottom. Scroll down to learn more. The CAS Superuser does obey the Select permission.)   Now, I’m not implying anything about the trustworthiness or character of administrators. Administrators are crucial individuals responsible for maintaining these platforms. Rather, it’s a fundamental practice in data security to mitigate risks to a company and comply with privacy regulations. By implementing security measures that limit direct access to sensitive data, organizations are proactively safeguarding against internal and external threats. This approach is based on the principle of least privilege, where individuals are granted the minimum level of access necessary to perform their job duties. Meet Susan. She’s the person cleared to access the sensitive data in our organization.   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   We’ll use Host Access Controls to Secure the sensitive data at the source, limiting access to specific users. By default, the CAS Server in SAS Viya launches its sessions as a shared user. We’re going to deny the shared service account access. And grant access only to the users who are allowed to access that sensitive data. In this example, we’ll grant Susan the required access and deny others at the host level.   Then we configure SAS Viya to launch its CAS Sessions as each individual user, using a technique called Host launch. Host launch is configured by including an overlay in the environment’s kustomization.yaml file, and then placing Susan in a special custom group named CASHostAccountRequired. Further details can be found in the readme file located in the directory: $deploy/sas-bases/examples/cas/configure/readme.md   Next, the administrator defines a caslib, or a pointer to the data, for Susan to use. We’ll name it susandata. Even at this step, when the administrator attempts to test their access to the data source, they will get an error. Anyone except for Susan will be denied access. However, we have defined the connection to the data source, so let’s move on and grant Susan access in SAS Viya.   Granting Susan's access and ensuring that Administrators have none. Administrators are already denied access at the data source, but we can never be too careful.   We define CAS Authorization access controls so that Susan can access the susandata caslib and any tables that exist in the data location. You’ll notice that I denied access for SAS Administrators—this is redundant due to the lack of access at the host level, but still a good practice.   If the administrator tests their access, even using the CAS Superuser role they are denied access.   I have assumed the CAS Superuser role and am still being denied access. This is exactly what we want!    Susan gets to work. She tests her access and loads the table into memory to perform her analyses. In the SAS Data Explorer, the Available tab shows all tables that are loaded in memory on the server. I assume the CAS Superuser role and can see the table name and column headings that exist on the table that contains sensitive data. Uh oh, is all this post going to be for nothing? Luckily no. The CAS Superuser is prohibited from viewing the contents of that secure CUSTOMERS table.   The CAS Superuser can see the column headings and the title of the table if it is loaded into memory but is denied access to any of its contents.    SAS Administrators, including the CAS Superuser, do not have access to this sensitive data table. The risk and information security teams can sleep soundly. Thanks for exploring this with me. The walkthrough above only uses a single user example, but you could modify this to secure data for multiple users or groups as well. (Update) One additional item of note: even without implementing Host Launch and changing the permissions on-disk to source files, the CAS Superuser is still subject to the Select permission in the CAS Authorization layer, even when they have assumed the superuser role. CAS Superuser Role   If the administrator’s account is not already granted Select on a table, they would still need to explicitly give themselves this permission before being allowed to read rows from the table. As a superuser, they can of course give themselves a grant of Select on any caslib or table within a caslib, however this action will be logged and can be audited for review later. Some supplemental reading:    @StuartRogers' post on SAS Viya 2021.1.3 CAS Security Changes @GerryNelson's post on Securing OS Resources in SAS Viya SAS Documentation for more information on The CASHostAccountRequired Custom Group     Find more articles from SAS Global Enablement and Learning here. ... View more

Enterprise Session Monitor Tags are annotations that appear on the Enterprise Session Monitor time-series graph when they occur. This seemingly simple concept can be quite powerful for the developer or the administrator in understanding what’s happening in each session. The purpose of this post is to explore these tags in SAS Enterprise Session Monitor. New to Enterprise Session Monitor? Catch my earlier posts on the topic here: Obsessing Over Observability and Taking Control: Custom Process Monitoring with filterSpec in Enterprise Session Monitor.     Program tagging is performed with the %esmtag() macro, which is defined by SAS Enterprise Session Monitor. Tags are available in all systems monitored by SAS Enterprise Session Monitor: SAS 9, SAS Viya 3.5, and SAS Viya 4. In order to enable program tagging in SAS Viya, here are the steps that need to be taken: SAS Enterprise Session Monitor: Enable Program Tagging.   These user-generated annotations function similarly to the %PUT statement, allowing for a developer to leave comments or notes for themselves in the SAS program log. With tagging, this is taken a step further -- instead of only printing output to the SAS log, a labeled flag appears on the time-series graph in the Live View and Timespans View of SAS Enterprise Session Monitor. So, when they are observing the session, they can see these markers. They can be used for marking the start or end of code submission, the beginning of various sections of program code, and much more.   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   Tags are used to provide extra information to people who are administrating, monitoring, or developing for SAS platforms. Tagging adds additional, meaningful context to understand what is occurring inside these sessions without needing to go and read the log of every single program execution. They can also be inserted into autoexec files or supplied with custom pre- or post- code.   I’ve gotten a few paragraphs in and haven’t mentioned syntax yet, so I probably should. Once program tagging has been enabled, tags can be written to SAS Enterprise session monitor by calling the %esmtag() macro. The macro accepts up to three parameters: the flag label, text to appear on mouse hover, and the Hex color code for the flag.   %esmtag(flag text, text on hover, #BBBBBB);   Tags also accept some basic HTML markup for line breaks and text formatting:     %esmtag(flag text, <b>This text is bold</b><br> <i>This text is italic</i><br> <small>This is some smaller text.</small><br> This is _subscripted text.<br> This is ^{superscripted} text. );   This is where things begin to get interesting, as tags can also accept macro variables. With macro variables, tags can display dynamic information to the time series graph, not only static text strings.   Some ideas for tag use include displaying a user’s name when they submit a program for execution. A string like this can be added to custom code within SAS Studio: This is probably pretty cool on reusable compute servers.   %esmtag(start,<b>&sysuserid</b> is starting code submission.);   You can also use macro variables to display the number of observations in a table after a DATA step completes.    Phew, that's a lot of rows.    proc sql noprint; select count(*) into :obs_count from eriktable; quit; %esmtag(&syslast has <b>&obs_count</b> obs.)   What about that last parameter on the &esmtag() macro? The third parameter sets the color of the flag. Let’s add some interesting colors based on the output. This is a spin on one of the more complex tagging examples from documentation.   Pretend we’re working with some variable data that might fall below a specific threshold. We can use an IF-THEN-ELSE-IF statement to build several different tags, one for low, medium, or expected numbers of observations. Within this code, each tag dynamically adjusts its color—red, yellow, or green—based on the number of observations generated in the table. This dynamic approach enhances both readability and visual appeal. While this example deals with rows of table data, this same concept is transferable for other uses. I’d love to see what kinds of interesting tags administrators or developers create.    These tags change their color based on the number of observations generated in the preceding table.   proc sql; create table eriktable as select * from sashelp.cars; quit; data _null_; set eriktable end=final; if final then do; if &sqlobs. >= 300 then call execute('%esmtag(Good Number of Rows (&sqlobs.),, #AAFFAA);'); /* Green */ else if 99 <= &sqlobs. <= 299 then call execute('%esmtag(Medium Number of Rows (&sqlobs.),, #FFD324);'); /* Yellow */ else call execute('%esmtag(Low Number of Rows (&sqlobs.),, #DDAAAA);'); /* Red */ end; run;   Once these tags have passed by in the time series graph, they are not gone. You can search for them in the Events and Logs page of SAS Enterprise Session Monitor. This way, these meaningful events are valuable information and is retained for future reference.     Thank you for reading. Ready to learn more about SAS Enterprise Session Manager? Here are some additional resources to explore:   Observability Solutions for the SAS Viya Platform SAS Enterprise Session Monitor – Deployment Using Ansible Course: System Tuning Using SAS® Enterprise Session Monitor     Find more articles from SAS Global Enablement and Learning here. ... View more

Team Name PS2G Track Public Sector Use Case As part of the SAS Hackathon, our focus will be creating an OSTP Program Finder Dashboard. The goal of this project is to create an interactive and user-friendly tool that allows students, parents, and school administrators to easily find out which Out-of-School Time Programs (OSTP) are available at different schools and grade levels. Technology SAS Visual Analytics Region Americas Team lead Kevin Collins Team members Sidorela Kollcaku, Khine Chin Chin Soe Technology expertise Python & R Programming, Full Stack Web Development Overview The dashboard will allow users to: Select a Grade: Users can choose their specific grade level using a drop-down menu. Explore Available Programs: Once a grade is selected, the dashboard will display the available programs (e.g., STEM, Arts, Sports) across all participating schools for that grade. School and Program Insights: A map or table will show the schools offering the selected programs. Users can filter by program type to see which schools provide specific options like STEM or tutoring. Program Information: Detailed descriptions of each program (e.g., STEM, Arts, Sports) will be provided to help parents and students understand the nature of each offering. Why This Project Matters: Parents and Students: This dashboard will simplify the process of finding extracurricular activities based on the student’s grade level, making it easier to engage in programs that align with their interests. School Administrators: The tool can also be used by school administrators to track program availability and ensure equitable access across different schools and grade levels. Accessibility: By making the dashboard mobile-friendly, parents and students can easily check program availability on the go. Key Features: Interactive Filters: Users can filter by grade and program to instantly see which schools offer those programs. Program Details: Each program will have a description so that parents and students know what to expect. Visual Map: Schools offering selected programs will be visualized on a map, making it easy to find nearby schools. Best Practices: Simplicity and Usability: The dashboard will focus on providing an intuitive, easy-to-navigate interface. It’s important that users, especially parents and students, can quickly find the information they need without confusion. Mobile Accessibility: Ensuring that the dashboard is mobile-friendly is crucial for accessibility, allowing users to browse program options from their smartphones or tablets. Scalability: The dashboard will be designed in a way that it can be easily updated as new programs, schools, or grades are added in the future. This project has the potential to improve program awareness and engagement by making extracurricular opportunities more accessible to everyone in the school community. I'm excited to continue refining the dashboard and explore the impact it can have on increasing participation in after-school programs! Jury video   Pitch Video ... View more

Good news! We're staffing an Administration Booth in the Innovation Hub at SAS Innovate in April. Make sure you swing by and say hello! We'll have some experts on hand to talk about a variety of topics. Come say hello! ... View more

By default, information that your developers and users create and store in SAS Studio are organized in SAS Content folders. This SAS Content folder structure is not connected to the underlying server file system, but rather is stored in the SAS Infrastructure Data Server. This is a PostgreSQL database that persists content like SAS Programs, reports, authorization rules, models, attachments, and other uploaded files, sometimes even data. This content is shared internally across SAS Viya web applications, so you can find the same content in SAS Studio, SAS Environment Manager, SAS Model Studio, SAS Visual Analytics, and the list goes on.  In this blog, I'll show you how to enable the local file system for SAS Studio.  For those of you administering SAS Studio, here’s a link to the SAS Studio Administrator’s Guide. Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   You can view SAS Content by opening SAS Environment Manager and viewing the content page. You’ll see a list of SAS Viya folders. Content within the SAS Infrastructure Data Server is backed up by the included SAS Viya backup processes. When users log in to any SAS application for the first time, a folder is created for them called My Folder.  This folder contains items that they do not want to share with other users. The contents of this folder are visible only to individual users and administrative users. There’s also a My Favorites folder, where users can add frequently used reports and data, as well as a My History folder with a reference to the 40 most recent content items that the user accessed. They also have access to other folders such as Public and any other folders that the General Authorization system allows the user to access, based on their group memberships.    The idea behind this allows users to easily store SAS Content in a single location, seamlessly accessible with other SAS Viya applications, and backed up by SAS Viya.  However, as they are stored in a database, they are not easily referenced by other non-SAS scripts or programs. What if you were building a SAS program and then wanted to have a 3 rd party scheduler or script use it later.  Some tools only currently work well when they are stored in SAS Content. But also, items in SAS Content do not have Git support or some other version control that a company might use.   There is a way for administrators to grant access back to the underlying file system on the SAS Server. SAS Studio configuration properties allow for the showServerFiles property to be set to true. When the property is set to true, users can access the file system from SAS Studio.  The administrator has some flexibility in defining the location for SAS Studio users to access. Depending on the configuration, this could expose sensitive information about the environment and allow any user to read files in the shared directory. There are additional configuration properties to define where the file system access begins, in the configuration property named fileNavigationRoot. The configuration property allows the administrator to choose between the following options: User – which allows home directory access. System – which enables full server access. Custom – which uses the configuration value: fileNavigationCustomRootPath The configuration property named fileNavigationCustomRootPath specifies a custom directory that the administrator wants users to be able to use, like external persistent storage.     Be aware that some customers might lock down and prevent access to the underlying file system. The underlying file system is also not backed up by the SAS Viya default backup, and a system will need to be in place to back up any custom locations where data is being stored.   Additionally, the latest releases of SAS Viya are configured by default to automatically generate UIDs for users when interacting with the mounted file server. To ensure access with the automatically generated ID, see this additional resource SAS Viya 2022.10 UID and GID Changes.   This is all about flexibility—granting your users the ability to choose between saving their programs to SAS Content or on the file system, where they can easily be used and shared outside of the SAS Infrastructure Data Server. ... View more

SAS Enterprise Session Monitor is an observability tool used to keep watch on server operations and performance. It monitors specific sessions and processes and displays that information. In short, it gives you analytics about your analytics. Enterprise Session Monitor is built for SAS, bringing insight to SAS session metrics such as PROC and DATA Steps, and CAS Actions. A developer or administrator can perceive real-time feedback on their company’s SAS workloads and see the effects of a session on the server resources.   SAS Enterprise Session Monitor Server can observe multiple environments simultaneously, on premises or in the cloud. Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   It's been available for a while now, and Enterprise Session Monitor agents run on the same environments where SAS 9, SAS Viya 3.X, and the latest SAS Viya environments run: Windows, Linux/Unix, and Kubernetes.  The default configuration tracks those SAS 9, SAS Viya 3.X, and SAS Viya processes and sessions in your environment. This is not an exhaustive list by any means, but sessions such as: SAS Workspace servers, Compute servers, CAS controller and worker sessions, and distributed SAS Grid Workspace servers are all monitored and displayed by default. Depending on the operating system and platform, administrators can see which data tables have been accessed within a period of time, accurately count concurrent users on environments, and see which SAS statements are being used in programmer sessions across the organization. All this information is kept in a database, so you have a historical record and can generate analysis or reports on workloads that ran in your environments.   I write environments because the Enterprise Session Monitor agents run on multiple nodes, and can glean session information from multiple SAS deployments, even if they are different types. They all send their information to the same SAS Enterprise Session Monitor Server to view and store.   How does Enterprise Session Monitor discover sessions?   By default, SAS Enterprise Session monitor has two methods of finding sessions on these remote hosts. The agents are configured with either a passive method or an active method. The passive discovery method is the default. In this passive discovery method, SAS Enterprise Session Monitor Agents watch in a specific directory or folder for trigger files that contain session information. These files are generated by SAS sessions because of the Enterprise Session Monitor configuration which appends some code to SAS servers when they run. That code generates these trigger files when each session executes.   Passive Process Discovery and Active Process Discovery may work together to provide a complete record of sessions and jobs.   The active process discovery method causes the agent to query the task list or process table of a node at regular intervals, and then searches that table for sessions (defined in another file) that it is interested in tracking. The result of that search is sent to the Enterprise Session Monitor Server.   Both active and passive process discovery can be active on a node simultaneously. By default, the active process scan runs every ten seconds, and the passive discovery method’s trigger files ensure that processes are discovered even if they started and finished in the nine second window between process table scans.   All of this is configurable, and you are the administrator who gets to configure this.   Why Track Custom Sessions?    Enterprise Session Monitor can track SAS sessions out of the box by virtue of the initial configuration of SAS Enterprise Session Monitor. Enabling active monitoring and configuring the filterSpec.yaml file allows us as administrators to track other processes and sessions. And it's completely customizable.   Let’s say you want to track a version control or DevOps application that your company uses. Perhaps you want to track severs which host third-party databases. Perhaps the R&D department wants to track their custom Java application to ensure the tool they are building isn’t using too many system resources. Maybe you just want to benchmark how much slower open-source analytic tools run. You get the picture—the possibilities for node monitoring with an Enterprise Session Monitor agent is up to you.   Okay! I’m convinced. How to begin?   In the Enterprise Session Monitor Agent install location, there is a bin directory (or folder for those of you on Windows) which holds the agent configuration in a file called config.properties . Two lines in that file are of interest to you right now: monitorAllSessions and monitoringInterval . Uncommenting these lines will allow the Enterprise Session Monitor Agent to use active process discovery. The agent then references a file called filterSpec.yaml to configure the processes that you are interested in sending to the Enterprise Session Monitor Server. You'll also need to restart the Enterprise Session Monitor agent once your configuration is complete.     Uncommenting monitorAllSessions and monitoringInterval (and configuring the filterSpec, and restarting the Enterprise Session Monitor Agent) engages Active Monitoring.   The filterSpec.yaml file contains process filter blocks, and each block contains a different set of properties to identify sessions for the agent to track.   The filterSpec.yaml file contains filter blocks that specify which processes you are interested in adding to SAS Enterprise Session Monitor.   Starting each block is the processFilterRegex property, which defines a regular expression to identify the session from the node's process table. When building these, it is best to run the process table on a server, such as with ps -ef . Locate a process that you want to target and copy several lines of the process table to the clipboard, including the sample process. Use a tool like regex101.com to build your regular expressions (disclaimer: this links to a non-SAS site, but the regex101 tool is a web site that's useful in building, testing, and debugging regular expressions).   The expressions you build can be as simple as just finding the name of the process somewhere in the string, or as complicated as accounting for each delimiter, character, word, or number. You can also use capture groups to specify a string to use in the name or type of session in Enterprise Session Monitor.   Using the Regex101 interface to build a regular expression to find Java processes, and store the process name in a variable called ${appname}   Each block of properties is processed in sequential order, so if there are sessions that match multiple rules, the first one is the one that is applied. So, if your custom application also appeared in a list of all programs, the custom application one would apply if it were higher in the filterSpec.yaml file.   The process filter block can specify a userFilterRegex , which can disregard processes which don’t match a specific user. This is useful if you are tracking a service account, or it can be left alone to capture a matching process by any user.   The process filter block can specify the location of the session’s log file on Unix systems in the processLogfile  property. Using this, Enterprise Session Monitor can open the log file from the graphical user interface.   The ignoreMatching property, if set to true, will ignore matching processes. If set to false, the agent will send the session information to the Enterprise Session Monitor Server.   There are three other sub properties under the esmPattern property which determine information for the Enterprise Session Monitor server to display: processName , processType , and processQueue . These three properties can be hardcoded or accept variables from the processFilterRegex, like ${appname} , ${port} , ${msg} , or others. These capture groups and variables are defined by you when writing your regular expressions.  A sample process filter block might look similar to the yaml code below:   #track java processes - processFilterRegex: (?<user>\w+).*\/usr\/bin\/java\s-jar\s(?<appname>\w+).* userFilterRegex: '' ignoreMatching: False esmPattern: processName: ${appname} processType: java processQueue:   These configurations and filterSpec.yaml files must be modified on each agent, and work on both Windows and Linux/Unix systems. On Kubernetes systems, all processes that run on a node are automatically discovered by default.   The result of all your hard work is being able to see your specified processes in the Enterprise Session Monitor process list and time series graphs.   I hope this helps in your understanding and helps you create some interesting uses for extending SAS Enterprise Session Monitor. Perhaps you'll be tracking additional processes across your analytical ecosystems soon!     Find more articles from SAS Global Enablement and Learning here. ... View more