BookmarkSubscribeRSS Feed

Introducing the SAS 9.4 Environment Manager Smart Agent

Started ‎02-14-2019 by
Modified ‎02-14-2019 by
Views 3,820

SAS Environment Manager monitors and administers many aspects of a SAS 9.4 deployment. EV (as we refer to it internally) is comprised of two different software components: a Server and its Agents. An agent is deployed to each host of a SAS deployment where it gathers information about ongoing operations. The agent relays that information back to the EV server where it can be analyzed and shared with the SAS software's administration personnel.


In this way, SAS Environment Manager is flexible to the many possible topologies of SAS software deployments and enables administration, alerts, auditing, logging, and many other features to help keep the SAS deployment running smoothly.


However, there are some SAS deployment topologies for which EV needs configuration beyond what the standard install process can provide. In particular, SAS installations which require additional configuration of the EV agents are those where the SAS Compute Tier runs across multiple hosts machines which all share a single deployment of the SAS configuration directory. In the past we relied on a separately downloadable utility to help configure the EV agents for this scenario. But with SAS 9.4 M6, we have a new smart agent technology to make it even easier.


Tiers of software functionality

This post dives right into some of the deeper aspects of multi-tiered, multi-machine deployments of SAS. Where possible, I'll try to provide relevant background to keep us all on the same happy path.


A multi-tiered SAS deployment is a reference to distributing SAS software components to different machines based on their primary function. We typically see tiers referenced like:

  • SAS Metadata Tier: one, three, or more machines dedicated to hosting the SAS Metadata Server
  • SAS Middle Tier: one or more machines that host web technologies, such as the SAS Web Server, SAS Web Application Server, SAS Cache Locator, and more
  • SAS Compute Tier: one or more (sometimes dozens or possibly hundreds) machines which provide analytic and data processing functionality with Foundation SAS
  • SAS Client Tier: laptops, desktops, mobile devices, etc. which access the power of the SAS deployment over the network

- When talking about the new EV Smart Agent functionality, we're focused exclusively on the SAS Compute Tier.


Why is the SAS Compute Tier special?

It comes down to how the SAS software is deployed initially and then accessed for ongoing operations. To understand why the compute tier is different, let's first look at how other SAS software tiers are deployed. Consider this clustered SAS Metadata Server on three host machines:



Select any image to see a larger version.


To deploy the SAS Metadata Server, we must run the SAS Deployment Wizard (SDW) on each host. The SDW lays down physical files for sashome and sasconfig on to each machine. Once the files are in place, then the SDW starts up the SAS software services on that host automatically, including the EV Agents.


The process is pretty similar for a clustered SAS Middle Tier as well:




In each case, every host machine has its own set of sashome and sasconfig files - and each one has its own SAS Environment Manager Agent as well. Also, one (or more) of these hosts is also running the SAS Environment Manager Server component, too.


So if you're keeping count, you'll notice that this approach requires running the SDW five times so far - once on each machine shown in these two illustrations.


The SAS Compute Tier can be treated differently. It's possible that the SAS Compute Tier might not just have one, two, or three hosts - it may have dozens, scores, potentially hundreds of hosts. Instead of running the SDW on every single one, wouldn't it be great if we could just run the SDW once and share that one set of files across all of the compute tier hosts?


TAKE NOTE: The SAS Compute Tier for this post is in reference to the software components which provide Foundation SAS through connectivity provided by either SAS/CONNECT or SAS Integrations Technology (IOM) software. This includes the SAS/CONNECT Server or SAS processes spawned by the SAS/CONNECT Spawner as well as IOM application servers like the SAS Workspace Server, SAS Stored Process Server, SAS Pooled Workspace Server, and SAS OLAP Server. And so, of course, SAS Grid Manager is included in this, too. For this post, the compute tier does not include SAS LASR Analytic Server, SAS Event Stream Processing, anything from SAS Viya, etc.


Share the what now?

For the hosts in the SAS Compute Tier, we can choose to share one instance of the SAS configuration directory. And if our SAS deployment is running on Linux (or UNIX) hosts, then we can also share the sashome directory (with all of SAS' binary executables) as well:




In this illustration, notice that the SDW is only run on one host machine of the compute tier. All of the other compute tier hosts will be configured to access the files for sashome and sasconfig through the use of shared filesystem technology. This simplifies the deployment significantly since we only need run the SDW for the compute tier just one time. It also simplifies ongoing administration and maintenance because we can make one change (or apply a hotfix just once) and all of the machines in the compute tier will see it automatically.


Isn't the EV Smart Agent supposed to be in this post somewhere?

Yes, indeed. I'm just getting to it. But first you need to know that the standard EV Agent is already installed, configured, and running on every host where the SDW was executed. In the illustrations above, the SDW has already provided a perfectly good EV agent on:

  • SAS Metadata Tier: meta01, meta02, meta03
  • SAS Middle Tier: mid01, mid02
  • SAS Compute Tier: comp01

We don’t change those. Leave them alone. They're fine.


However, notice that we don't have any EV agents running on the compute tier hosts where the SDW was not executed: comp02, comp03, comp04, …, compn.


This is where the new EV Smart Agent comes into play.


Deploying the EV Smart Agent to the SAS Compute Tier hosts

This is actually super simple. There's nothing to deploy. As of 9.4 M6, the EV Smart Agent software files are already in place in the sasconfig directory. You just need to use them.


On every SAS Compute Tier host which relies on a shared configuration *and* has not run the SDW:

  1. Navigate to the SAS configuration directory, similar to:   /[SASCONFIGDIR]/Lev1/Web/SASEnvironmentManager


  2. Display a directory listing with results similar to:


    drwxr-xr-x.  8 sasinst sas  195 Dec 11 11:06 agent-5.8.0-EE
    drwxr-xr-x. 11 sasinst sas  168 Dec 11 11:05 emi-client
    drwxr-xr-x. 18 sasinst sas 4096 Dec 11 11:04 emi-framework
    drwxr-xr-x.  2 sasinst sas   91 Dec 11 11:05 smart-agent


    Notice the standard EV agent directory is still there: agent-5.8.0-EE. And remember it's actively being used on the compute tier host machine where the SDW did run.


  3. Now navigate into the smart-agent directory and display the listing:


    -rw-r--r--. 1 sasinst sas 2213 Dec 11 11:07
    -rw-r--r--. 1 sasinst sas 7461 Dec 11 11:05 hq-agent.bat
    -rwxr-x--x. 1 sasinst sas 6638 Aug 30 00:33
    -rw-r--r--. 1 sasinst sas  264 Dec 11 11:05


  4. Run the script and specify the start parameter. For it's first-ever execution, the smart agent will automatically create a new directory structure just for this host's files:


    [running on comp02] $ ./ start
    Cloning a new HQ Agent for host comp02..
    Cloned a new HQ Agent for host comp02 under the directory /[SASCONFIGDIR]/Lev1/Web/SASEnvironmentManager/cloned_agents/comp02/agent-5.8.0-EE.
    Starting HQ Agent...... running (19889).
    [ Running agent setup ]
    Should Agent communications to HQ be unidirectional [default=no]: yes
    What is the HQ server IP address:
    Should Agent communications to HQ always be secure [default=yes]: Yes
    What is the HQ server SSL port [default=7443]: 7443
    - Testing secure connection ... Success
    What is your HQ login [default=hqadmin]: sasevs@saspw
    What is your HQ password: **Not echoing value**
    What is the agent IP address [default=]: comp02
    - Received temporary auth token from agent
    - Registering agent with HQ
    - HQ gave us the following agent token
    - Informing agent of new HQ server
    - Validating
    - Successfully setup agent
    - Agent using new transport, unidirectional=true, port=7443


  5. The new directory structure created in the shared SAS compute tier configuration looks similar to:


    +-- cloned_agents
        +-- [HOSTNAME]
            +-- agent-5.8.0-EE 
                +-- bin
                +-- bundles
                +-- conf
                +-- data
                +-- log
                +-- wrapper
                 -- open_source_licenses-SpringSource_tc_Server_Plug-in.txt
                 -- open_source_licenses.txt
                 -- README.txt


  6. For ongoing operations of the EV Smart Agent on the compute tier hosts, use /[SASCONFIGDIR]/Lev1/Web/SASEnvironmentManager/smart-agent/ [ start | stop | restart | status ]


Keep it all straight

Let's be clear. For every machine where the SDW ran, we already have the standard EV Agent and it works great. Each of those hosts where the SDW ran also has its own sas.servers utility to start (and stop and status) all of the SAS services, including the default EV Agent. So continue working with those as you always have.


The EV Smart Agent is used on the compute tier only - and only those hosts which rely on the shared SAS configuration directory - and only those which did not run the SDW. So your SAS deployment when its complete will run with the standard EV Agent running on some hosts and the EV Smart Agent on the compute tier hosts which need it.




This is important to remember. Here's why…


On the first-deployed host of the SAS Compute Tier, you'll typically use the sas.servers utility to operate all of the SAS services together. But if you want to operate just the EV Agent, you can do so by calling the correct script:




$ /[SASCONFIGDIR]/Lev1/Web/SASEnvironmentManager/agent-5.8.0-EE/bin/ status

HQ Agent is running (PID:4812).
Current agent bundle: agent-5.8.0
Server IP address:
Server (SSL) port: 7443
Using new transport; unidirectional=true
Agent listen port: 2144


However, on the other hosts of the SAS Compute Tier which are running the EV Smart Agent, the command path is different:



$ /[SASCONFIGDIR]/Lev1/Web/SASEnvironmentManager/smart-agent/ status

HQ Agent is running (PID:1485).
Current agent bundle: agent-5.8.0
Server IP address:
Server (SSL) port: 7443
Using new transport; unidirectional=true
Agent listen port: 2144


Why can't I…?

I know. You want to run the EV Smart Agent on the first-deployed compute tier host (where the SDW ran). And technically, you can. It'll work (assuming you stop the standard EV agent first) and report statistics to the EV Server just the same. But it's a bad idea.


It's a bad idea because the sas.servers utility - which you want to continue using - is not equipped to work with the EV Smart Agent. So you'd end up wrangling with competing executions of the EV agent to no real benefit.


Okay, now I bet you want to run the standard EV Agent on the other hosts of the compute tier, right? And well, you can. It's just that's the old way of doing things. And the EV Smart Agent is the new way.


Red pill or blue pill?

Unfortunately, no one can be told what the Matrix is. You have to see it for yourself.




Here's an interesting twist to consider: there is no EV Smart Agent.


As I said above, whether you're running the standard EV Agent or the new EV Smart Agent, you're getting the exact same agent-5.8.0 process. It's just that the parameters fed to that agent which tell it how to run are different. The "smarts" are only in the scripting which invokes the agent - specifically to create that dedicated directory structure per host and direct the agent to use it. In terms of what the EV agent reports on and monitors, the standard and smart agents are exactly the same.


Go and find out more

With so much power and flexibility in how we deploy SAS software, there's still a lot of ground to cover when discussing the concepts here, not just for SAS Environment Manager, but the various topologies, operational considerations, and so on.


Learn more about saving time and effort with a shared configuration directory.


And when working with a multi-tiered (or multi-machine) deployment of SAS software, coordination of SAS software services as they run across hosts is important. SAS Technical Support provides the SAS_lsm utility to help manage the operations of SAS software on multiple machines.


@RobCollum , thank you very much for sharing. HUGE improvement. I will definetely play with it and test it. Specially usefull for SAS Grid deployments.

Thanks for this!  As mentioned, this is very nice for a Grid environment, and was very timely information for completing our new 9.4M6 Grid just today 🙂

Thank you for this Rob... had I not been reading up on the new SAS Workload Orchestrator I would have missed setting this up.  


Makes perfect sense!


Renée La Forest

Hi Rob


Great post. But would it be possible to have the cloned agent run on all compute nodes on a grid and then alter the sas.server.mid and sas.server.mid.template script to point to the cloned_agent for stop/start/status




if [ "$OSTYPE" = "aix" ];


I have had this setup run at a customer and haven't run into any problems yet.


Best Regards


@HåvardPehrson , 


this is an interesting approach, however, I would like to ask you: 

Have you got the chance to check with top, htop or any resource monitoring tool, how many resources (specially CPU) are in use by the cloned agents? 


I would like to hear your experiences in this regard, @HåvardPehrson  and @RobCollum 


Best regards,


Hi @JuanS_OCS 


No. I haven't used any resource monitoring tools to check cpu usage. However, I would expect any significant difference compared to the old setup.


Best Regards


Thank you for the thought.


In my experience, the cloned agent is using much more resources. While the normal EVM agent uses 1%-2% as much, I perceive the cloned agent often uses ~20% cpu, which I think is something to pay attention to, specially in grid environments.

Hi @JuanS_OCS 


I checked on my server and I don't see any increased cpu usage for the smart agent pid

[smart-agent]$ ./ status
HQ Agent is running (PID:21125).


top - 11:48:29 up 17 days, 22:44, 2 users, load average: 0,18, 0,36, 0,58
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
%Cpu(s): 1,1 us, 0,9 sy, 0,0 ni, 97,9 id, 0,0 wa, 0,0 hi, 0,0 si, 0,0 st
KiB Mem : 65361320 total, 34909836 free, 24110000 used, 6341484 buff/cache
KiB Swap: 2097148 total, 48 free, 2097100 used. 39747240 avail Mem

21125 dsas 20 0 136368 1200 640 S 0,0 0,0 0:26.28 wrapper-linux-x


But thanks, that is something to keep in mind 🙂


Best Regards


RE: alter the sas.server.mid and sas.server.mid.template script to point to the cloned_agent for stop/start/status

I'm glad that works for you! Personally, I don't recommend manually changing those files. The next time you install a hotfix or maintenance update, those files are fair game to be replaced. I'm sure the dev teams would enjoy hearing your request for that functionality. I'll share with them, but best would be if you made a feature request. SAS Technical Support should be able to help with that.


RE: 20% CPU utilization:

As I wrote above:

...whether you're running the standard EV Agent or the new EV Smart Agent, you're getting the exact same agent-5.8.0 process. It's just that the parameters fed to that agent which tell it how to run are different. The "smarts" are only in the scripting which invokes the agent - specifically to create that dedicated directory structure per host and direct the agent to use it. In terms of what the EV agent reports on and monitors, the standard and smart agents are exactly the same.

If you're experiencing unusual resource consumption, then I suggest opening a track with SAS Technical Support to run down the cause. I wouldn't want a simple monitoring agent to consume so much processing power. 


Version history
Last update:
‎02-14-2019 11:20 AM
Updated by:



Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.

If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website. 

Register now!

Free course: Data Literacy Essentials

Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning  and boost your career prospects.

Get Started

Article Tags