Architecting, installing and maintaining your SAS environment

Alerts in SAS Environment Manager

Reply
New Contributor
Posts: 4

Alerts in SAS Environment Manager

I am currently setting up EVM 2.5 on a Win 2008R2 platform running SAS 9.4 M3 in a four server cluster. After partially enable APM the SAS EVM dashboard is reporting alerts about "SASWork Disk Use %>70". It seems APM can not compleatey be enabled since it gives me an error on the kit server, see below.

 

D:\SAS\Config\Lev1\Web\SASEnvironmentManager\emi-framework\bin>validate.bat --le
vel 2
Service Architecture Validation - Mon Nov 16 11:07:37 EST 2015
Starting level 1 validation...
Level 1 validation completed with 0 error(s)
Starting extended level 2 validation (this could take a while)...

Service Architecture validation completed with 1 error(s)
Errors:
        1) The SAS Environment Manager Server is unable to process additional ta
sks at this time. This may be due to high change request volume. If the server h
as recently been (re)started, it may be synchronizing with its Agents. This proc
ess could take up to an hour.
Please try again at a later time to validate your environment.

 

This error accoring to EVM is on the metadata server but if one opens MS resource manger and look on disk acitivity disk que is very low .05 and disk activity is low as well.

 

Has anyone else come across "SASWork Disk Use %>70" alert in EVM and if so how did you reslove it?

All four servers in this cluser are virtual servers on a VMware platform.

Thanks for any input on the matter, Ulf

SAS Super FREQ
Posts: 683

Re: Alerts in SAS Environment Manager

Hi

 

I would say this metric, "SASWork Disk Use %>70", is how full the file system is. So this means the disk where SASWORK is, is more than 70% full. 

 

Also have a look at this blog entry http://blogs.sas.com/content/sgf/2015/08/19/monitor-your-saswork-directory-from-sas-environment-mana...

 

bruno

Occasional Contributor
Posts: 6

Re: Alerts in SAS Environment Manager

Right now I am receiving the same "SASWork > 70%" alert and the 2.0TB disk has only 18GB (<1%) used on it (according to df command) and the alert is telling me that the dis is 82% full.

 

That's not useful.

Trusted Advisor
Posts: 1,141

Re: Alerts in SAS Environment Manager

[ Edited ]

Hello @DeanHatch,

 

if you have any doubts, you can always go to the Inventory tab of the monitored resource by the Alert, and check its configuration. '

Probably the SASWORK defined on SAS foundation it is a different one than the one configured on your SASApp level. Therefore, the disk volumen might differ as well.

 

It is often the case that the "blame" is not on a "bug" but in our own configurations or our own misunderstanding of the configuration.

 

Anyway, if you have still questions after that check, you can always come back to us, or ask the question in your SAS Technical Support team.

Occasional Contributor
Posts: 6

Re: Alerts in SAS Environment Manager

 

Juan,

 

I definitely agree that "It is often the case that the "blame" is not on a "bug" but in our own configurations or our own misunderstanding of the configuration."  However, the Alert is pretty specific about the location of the SAS Work directory and the location of the Alert agrees with the location of the Indicators, which indicates 0% used.

 

Also, the page for the Resource tells me that no Alerts have been created, which is odd.

 

Also, this behavior started yesterday when I enabled the SAS Environment Manager Service Architecture for Extended Monitoring.  I have several other nearly identical systems on which I have not enabled the Service Architecture for Extended Monitoring and non of those report such an Alert.  I think in this case it might be called Over-Extended Monitoring, Smiley Wink

 

 

Occasional Contributor
Posts: 6

Re: Alerts in SAS Environment Manager

Recent Alerts
   
Date / Time Alert Name Resource Name Fixed Ack
07/06/2017 09:45 AM SASWork Disk Use % > 70 steps2-grid9-2.dev.econ.census… No
07/06/2017 09:40 AM SASWork Disk Use % > 70 steps2-grid9-2.dev.econ.census… No
07/06/2017 09:35 AM SASWork Disk Use % > 70 steps2-grid9-2.dev.econ.census… No

 

 

 

  • Total KBytes Free Move Metric Up Move Metric Down Remove Metric
    (SAS Home Directory 9.4) LOW:1.9 TB   AVG:1.9 TB   PEAK:1.9 TB
  • Chart Use Percent Move Metric Up Move Metric Down Remove Metric
    (SAS Home Directory 9.4) LOW:1.0%   AVG:1.0%   PEAK:1.0%
Trusted Advisor
Posts: 1,141

Re: Alerts in SAS Environment Manager

Hello again @DeanHatch,

 

no, this is not the definition, either the inventory. What I mean:

 

- Go to your Resources menu

- Select Services

- Search for MACHINE SAS Home Directory 9.4 SAS work directory

- Select Inventory

- Scroll down and click on the last Edit button

 

On the directory text box, you will see the directory/logical volume it is actually being monitored by this alert.

Occasional Contributor
Posts: 6

Re: Alerts in SAS Environment Manager

You are right, Juan.

 

The Alert and the Indicator ARE referring to two different locations.

 

Thanks for the clarification.  I will configure the Alert to Monitor the correct location.

Trusted Advisor
Posts: 1,141

Re: Alerts in SAS Environment Manager

[ Edited ]

No problem, you are welcome.

 

I usually try to over-understand anything. When I believe to understand something, I try to dive a bit deeper.

 

FYI, what happened here is:

 

When you enabled EMI (the extended monitoring), is when all of those alerts where created, and meny things more happened on the background, depending on the components (APM,ACM, Kits, VALoad) you enabled and activated. So this part is expected, and that is why in your other systems you don;t receive the alerts, because they don't exist by default, unless EMI is enabled.

 

When you enable EMI, a lot of SAS code runs, and feeds the data to the EM agent. This SAS code takes the _SASWORK directory that is defined on your SASFoundation directory. But what people normally calls SASWORK is the same parameter that is normally overwritten at the SASApp level. And here is where the confussion comes from sometimes.

 

By default, SASWORK on SAS foundation is pointing to /tmp (or the user folder on windows). Therefore SAS recommends to change this configuration to a separated drive/vlolume, with high I/O. But,as said, many people change this config on the SASApp level, and they forget about SAS Foundation, which is a mistake from my point of view, because many calls to SAS maight not be using the configuration of a SASApp server.

 

As conclussion, my advise is to change this configuration (SASWORK path in the SAS Foundation. And just to skip you from the "pain" of deleting the PWT8155 SAS Home Directory 9.4 SAS work directory resource and to re-define the alerts, it is better if you change the "directory" where we mentioned on the previous post.

Occasional Contributor
Posts: 6

Re: Alerts in SAS Environment Manager

"So this part is expected"

You nailed THAT one on the head. I have been using SAS for almost 31 years now and it never ceases to amaze me how counter-intuitive or random some of the designers/developers can be there. The Post-Configuration-Customization-Modifications drive me crazy.
Trusted Advisor
Posts: 1,141

Re: Alerts in SAS Environment Manager

Smiley Happy Well, SAS is not easy, historically speaking. However, I feel positive, I see a great improvement since SAS 9.1.3 on post-configuration activities (at that point, every single action was a challenging journey), specially now on SAS 9.4. The more mature the product is, the easier it becomes.

 

And I hope to see even greater progress with Viya, the expectations are elevated. I think Ansible will help a lot, as explained by @ErwanGranger here https://communities.sas.com/t5/SAS-Communities-Library/Desired-State-is-a-Destination-not-a-Journey/...

 

I am curious!

Occasional Contributor
Posts: 6

Re: Alerts in SAS Environment Manager

Also, couldn't the Monitor and the Indicator use the same location? For me, at least, that would be less confusing than a green check mark being accompanied by an Alert.
Trusted Advisor
Posts: 1,141

Re: Alerts in SAS Environment Manager

Yes, they should. But perhaps I am not understanding well here what is a Monitor, and what is an Indicator.

 

For Hyperic there are Resources (I guess what you call Monitor), and over Resources, Alerts are created. And I have no clue what might be an Indicator here, sorry.

Ask a Question
Discussion stats
  • 12 replies
  • 495 views
  • 2 likes
  • 4 in conversation