BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
Nigel_Pain
Lapis Lazuli | Level 10

9.4M7 on Windows 2012R2

I have an alert which has been triggered in EVM for one of our servers, "SASConfig Disk Use % > 95". This is generating an alert every five minutes or so. I'm aware of the issue, and am working to rectify it so I don't need the alerts to keep firing. I tried modifying the alert definition so that it disables until it's been marked as fixed:

Nigel_Pain_0-1618239869814.png

but I get an error:

Nigel_Pain_1-1618239923603.png

Thanks for any suggestions.

1 ACCEPTED SOLUTION

Accepted Solutions
Nigel_Pain
Lapis Lazuli | Level 10

We finally got this sorted after a lot of investigation by Tech. Support. The analyst was convinced that it was a performance issue but there appeared to be plenty of spare CPU and memory. Then it just started working again (coincidentally, during an online session with Tech. Support). It turned out that our Systems Management had moved the mid-tier VM to a different host, so it was actually host resources that were the problem. Now there's a moral to learn from!

 

Thanks for your time and effort @gwootton 

View solution in original post

8 REPLIES 8
gwootton
SAS Super FREQ
Do you see a corresponding error in your Environment Manager's server.log file?
--
Greg Wootton | Principal Systems Technical Support Engineer
JuanS_OCS
Azurite | Level 17

Hello @Nigel_Pain ,

 

first of all, yes, indeed, as @gwootton indicates, when we get an errors, first thought is to give a look to the logs and the database.

 

On the other hand, most monitoring tools work though alert vs acknowledgement, and Hyperic, on which SAS EVM is based, is not an exception. You might want to look a the the guide, especially:

  • Preventing Multiple Alerts for the Same Problem
  • Disable all Alert Notifications (not recommended)
  • Setting a Notification Throttle (the best idea)

https://docs.vmware.com/en/vRealize-Hyperic/5.8.4/vrealize-hyperic-584-admin-guide.pdf

 

This being said, for yous specific problem, I am wondering if you are having a permissions issue either in your file system, in hyperic configuration (and SAS Metadata) or, what could be harder to investigate, in your PSQL database (WIP) for EVM. Hopefully the log should be specific on the matter.

 

Let us know!


Best regards,

Juan

Nigel_Pain
Lapis Lazuli | Level 10

Thanks @JuanS_OCS and @gwootton for your suggestions.

 

I would expect to have permissions at all levels: I have an unrestricted SAS account, member of the Superuser role in EVM, and local admins on the server.

The Hyperic guide is just telling me to do what I was trying to!

 

Presumably the server.log file to check is the one in config\Lev1\Web\SASEnvironmentManager\server-5.8.0-EE\logs? It'll take a bit of searching. A job for tomorrow! Or it maybe easier to tail the log and reproduce the problem.

 

gwootton
SAS Super FREQ
The action you are taking to stop getting multiple alerts is correct (checking the box).
I think the problem is that when you go to edit the alert, Environment Manager is automatically populating the if condition with a unit (%) which the field won't accept. Changing it to a decimal value (i.e. 0.90 instead of 90.0%) allowed me to get past the error.
--
Greg Wootton | Principal Systems Technical Support Engineer
gwootton
SAS Super FREQ
I tested changing this in my environment and encountered a similar failure. To resolve: In addition to checking the box, modify the "greater than" section from "90.0%" to "0.90" and then click Ok.
--
Greg Wootton | Principal Systems Technical Support Engineer
Nigel_Pain
Lapis Lazuli | Level 10

Thanks Greg. I was quite hopeful when I saw your posting, but unfortunately when I tried this there was no change. I can see some errors in the server.log but they don't mean an awful lot to me:

13-04-2021 09:25:28,580 BST ERROR [tomcat-http--34] [org.hyperic.hq.web.spring.exception.GlobalExceptionController@35] org.hyperic.hq.ui.exception.ParameterNotFoundException: entity ids not found
org.hyperic.hq.ui.exception.ParameterNotFoundException: entity ids not found
 at org.hyperic.hq.ui.spring.util.RequestUtils.getEntityIds(RequestUtils.java:360) ~[sas.evmanager.web.jar:904700.0.0.20200715190000_v940m7]
 at org.hyperic.hq.ui.spring.util.RequestUtils.getEntityId(RequestUtils.java:314) ~[sas.evmanager.web.jar:904700.0.0.20200715190000_v940m7]
 at org.hyperic.hq.ui.spring.action.resource.common.monitor.alerts.config.EditDefinitionConditionsController.eidtConfig(EditDefinitionConditionsController.java:93) ~[sas.evmanager.web.jar:904700.0.0.20200715190000_v940m7]
 at sun.reflect.GeneratedMethodAccessor2403.invoke(Unknown Source) ~[?:?]
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_252]
 at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_252]
 at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221) ~[spring-web-3.2.18.RELEASE.jar:3.2.18]
 at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136) ~[spring-web-3.2.18.RELEASE.jar:3.2.18]
 at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104) ~[spring-webmvc-3.2.18.RELEASE.jar:3.2.18.RELEASE]
 at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:743) ~[spring-webmvc-3.2.18.RELEASE.jar:3.2.18.RELEASE]
 at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:672) ~[spring-webmvc-3.2.18.RELEASE.jar:3.2.18.RELEASE]
 at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:82) ~[spring-webmvc-3.2.18.RELEASE.jar:3.2.18.RELEASE]
 at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:933) [spring-webmvc-3.2.18.RELEASE.jar:3.2.18.RELEASE]
 at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:867) [spring-webmvc-3.2.18.RELEASE.jar:3.2.18.RELEASE]
 at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:951) [spring-webmvc-3.2.18.RELEASE.jar:3.2.18.RELEASE]
 at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:853) [spring-webmvc-3.2.18.RELEASE.jar:3.2.18.RELEASE]
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:660) [servlet-api.jar:?]
 at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:827) [spring-webmvc-3.2.18.RELEASE.jar:3.2.18.RELEASE]
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) [servlet-api.jar:?]
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [catalina.jar:?]
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:?]
 at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77) [spring-web-3.2.18.RELEASE.jar:3.2.18]
 at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:106) [spring-web-3.2.18.RELEASE.jar:3.2.18]
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:?]
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:?]
 at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) [tomcat-websocket.jar:9.0.31.A.RELEASE]
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:?]
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:?]
 at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126) [catalina.jar:?]
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:?]
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:?]
 at com.sas.hyperic.security.servlet.SecurityServletFilter.doFilter(SecurityServletFilter.java:27) [sas.hyperic.security.sso.jar:904700.0.0.20200715190000_v940m7]
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:?]
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:?]
 at com.sas.hyperic.security.servlet.LocaleServletFilter.doFilter(LocaleServletFilter.java:54) [sas.hyperic.security.sso.jar:904700.0.0.20200715190000_v940m7]
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:?]
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:?]
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:154) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:199) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:150) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at com.sas.framework.webapp.servlet.SanitizingRequestFilter.onDoFilter(SanitizingRequestFilter.java:101) [sas.framework.webapp.jar:904700.1.0.20200722190000_v940m7]
 at com.sas.framework.webapp.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:82) [sas.framework.webapp.jar:904700.1.0.20200722190000_v940m7]
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:110) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160) [spring-security-web-3.2.9.RELEASE.jar:3.2.9.RELEASE]
 at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) [spring-web-3.2.18.RELEASE.jar:3.2.18]
 at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) [spring-web-3.2.18.RELEASE.jar:3.2.18]
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:?]
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:?]
 at org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:230) [spring-orm-3.2.18.RELEASE.jar:3.2.18.RELEASE]
 at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:106) [spring-web-3.2.18.RELEASE.jar:3.2.18]
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:?]
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:?]
 at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:164) [urlrewritefilter-3.1.0.jar:3.1.0]
 at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:141) [urlrewritefilter-3.1.0.jar:3.1.0]
 at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:90) [urlrewritefilter-3.1.0.jar:3.1.0]
 at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:417) [urlrewritefilter-3.1.0.jar:3.1.0]
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:?]
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:?]
 at org.hyperic.hq.web.filters.CsrfIgnoreAllGetsFilter.doFilter(CsrfIgnoreAllGetsFilter.java:106) [hq-web-5.8.0-classes.jar:?]
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:?]
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:?]
 at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88) [spring-web-3.2.18.RELEASE.jar:3.2.18]
 at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:106) [spring-web-3.2.18.RELEASE.jar:3.2.18]
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:?]
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:?]
 at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) [catalina.jar:?]
 at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [catalina.jar:?]
 at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) [catalina.jar:?]
 at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) [catalina.jar:?]
 at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [catalina.jar:?]
 at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) [catalina.jar:?]
 at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) [catalina.jar:?]
 at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) [tomcat-coyote.jar:9.0.31.A.RELEASE]
 at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) [tomcat-coyote.jar:9.0.31.A.RELEASE]
 at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) [tomcat-coyote.jar:9.0.31.A.RELEASE]
 at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639) [tomcat-coyote.jar:9.0.31.A.RELEASE]
 at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-coyote.jar:9.0.31.A.RELEASE]
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_252]
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_252]
 at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-util.jar:9.0.31.A.RELEASE]
 at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
13-04-2021 09:25:28,679 BST INFO [tomcat-http--34] [com.sas.framework.webapp.servlet.SanitizingRequestFilter@209] Could not decode the value for POST name: [jsContentBlocks]. Consider adding jsContentBlocks to web.xml allowedPostNames init-param for the SanitizingRequestFilter
gwootton
SAS Super FREQ
I think a technical support track may be your best bet here. I'm not sure the "entity IDs not found" error is related to this or not.

One other thing I thought of you might try is making the adjustment to the alert under Manage -> Monitoring Defaults and find the "Edit Alerts" link next to "SAS Config Level Directory 9.4", and try making the same change there.
--
Greg Wootton | Principal Systems Technical Support Engineer
Nigel_Pain
Lapis Lazuli | Level 10

We finally got this sorted after a lot of investigation by Tech. Support. The analyst was convinced that it was a performance issue but there appeared to be plenty of spare CPU and memory. Then it just started working again (coincidentally, during an online session with Tech. Support). It turned out that our Systems Management had moved the mid-tier VM to a different host, so it was actually host resources that were the problem. Now there's a moral to learn from!

 

Thanks for your time and effort @gwootton 

suga badge.PNGThe SAS Users Group for Administrators (SUGA) is open to all SAS administrators and architects who install, update, manage or maintain a SAS deployment. 

Join SUGA 

Get Started with SAS Information Catalog in SAS Viya

SAS technical trainer Erin Winters shows you how to explore assets, create new data discovery agents, schedule data discovery agents, and much more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 8 replies
  • 2282 views
  • 4 likes
  • 3 in conversation