SAS EVM Agent service is not running on two servers (Metadata and MidTier)

[Asif4862]

Hello All,

SAS EVM Agent is not running on two servers Metadata and MidTier but it is running on ComputeTier server.

The following are the errors in log files of test env. servers:

Test env. Metadata server (D:\SAS\Config\Lev1\Web\SASEnvironmentManager\agent-5.8.0-EE\log)

1. agent log

11-09-2020 11:54:52,124 EDT ERROR [AgentDaemonMain] [AgentDaemon@958] Critical error running agent

java.lang.NoClassDefFoundError: org/codehaus/jackson/map/ObjectMapper

               at com.sas.grid.hyperic.sgmg.SASGridServerMeasurementPlugin.<clinit>(SASGridServerMeasurementPlugin.java:43)

               at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

               at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

               at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

               at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

               at java.lang.Class.newInstance(Class.java:442)

               at org.hyperic.hq.product.ProductPlugin.getPlugin(ProductPlugin.java:267)

               at org.hyperic.hq.product.ProductPlugin.getPlugin(ProductPlugin.java:287)

               at org.hyperic.hq.product.ProductPluginManager.addPluginTypes(ProductPluginManager.java:1024)

               at org.hyperic.hq.product.ProductPluginManager.registerPluginJar(ProductPluginManager.java:991)

               at org.hyperic.hq.product.ProductPluginManager.registerPluginJar(ProductPluginManager.java:711)

               at org.hyperic.hq.product.ProductPluginManager.register(ProductPluginManager.java:811)

               at org.hyperic.hq.product.ProductPluginManager.registerPlugins(ProductPluginManager.java:791)

               at org.hyperic.hq.agent.server.AgentDaemon.startPluginManagers(AgentDaemon.java:693)

               at org.hyperic.hq.agent.server.AgentDaemon.start(AgentDaemon.java:925)

               at org.hyperic.hq.agent.server.AgentDaemon$RunnableAgent.run(AgentDaemon.java:1043)

               at java.lang.Thread.run(Thread.java:748)

Caused by: java.lang.ClassNotFoundException: org.codehaus.jackson.map.ObjectMapper

               at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

               at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

               at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

               ... 17 more

11-09-2020 11:54:52,126 EDT INFO  [AgentDaemonMain] [FileMonitor@216] Entered FileMonitor.stop()

11-09-2020 11:54:52,126 EDT INFO  [AgentDaemonMain] [AgentDaemon@989] Agent shut down

11-09-2020 11:54:52,129 EDT INFO  [Watcher] [Watcher@253] watch service closed, terminating.

1. wrapper log

INFO   | jvm 5    | 2020/09/11 11:55:10 | Success -- agent is stopped!

ERROR  | wrapper  | 2020/09/11 11:55:26 | Shutdown failed: Timed out waiting for signal from JVM.

ERROR  | wrapper  | 2020/09/11 11:55:26 | JVM did not exit on request, terminated

STATUS | wrapper  | 2020/09/11 11:55:27 | on_exit trigger matched.  Restarting the JVM.  (Exit code: 1)

FATAL  | wrapper  | 2020/09/11 11:55:27 | There were 5 failed launches in a row, each lasting less than 60 seconds.  Giving up.

FATAL  | wrapper  | 2020/09/11 11:55:27 |   There may be a configuration problem: please check the logs.

STATUS | wrapper  | 2020/09/11 11:55:27 | <-- Wrapper Stopped

Test env. MidTier server (D:\SAS\Config\Lev1\Web\SASEnvironmentManager\agent-5.8.0-EE\log)

1. agent log

11-09-2020 12:03:28,713 EDT ERROR [AgentDaemonMain] [AgentDaemon@958] Critical error running agent

java.lang.NoClassDefFoundError: org/codehaus/jackson/map/ObjectMapper

               at com.sas.grid.hyperic.sgmg.SASGridServerMeasurementPlugin.<clinit>(SASGridServerMeasurementPlugin.java:43)

               at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

               at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

               at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

               at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

               at java.lang.Class.newInstance(Class.java:442)

               at org.hyperic.hq.product.ProductPlugin.getPlugin(ProductPlugin.java:267)

               at org.hyperic.hq.product.ProductPlugin.getPlugin(ProductPlugin.java:287)

               at org.hyperic.hq.product.ProductPluginManager.addPluginTypes(ProductPluginManager.java:1024)

               at org.hyperic.hq.product.ProductPluginManager.registerPluginJar(ProductPluginManager.java:991)

               at org.hyperic.hq.product.ProductPluginManager.registerPluginJar(ProductPluginManager.java:711)

               at org.hyperic.hq.product.ProductPluginManager.register(ProductPluginManager.java:811)

               at org.hyperic.hq.product.ProductPluginManager.registerPlugins(ProductPluginManager.java:791)

               at org.hyperic.hq.agent.server.AgentDaemon.startPluginManagers(AgentDaemon.java:693)

               at org.hyperic.hq.agent.server.AgentDaemon.start(AgentDaemon.java:925)

               at org.hyperic.hq.agent.server.AgentDaemon$RunnableAgent.run(AgentDaemon.java:1043)

               at java.lang.Thread.run(Thread.java:748)

Caused by: java.lang.ClassNotFoundException: org.codehaus.jackson.map.ObjectMapper

               at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

               at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

               at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

               ... 17 more

11-09-2020 12:03:28,721 EDT INFO  [AgentDaemonMain] [FileMonitor@216] Entered FileMonitor.stop()

11-09-2020 12:03:28,722 EDT INFO  [AgentDaemonMain] [AgentDaemon@989] Agent shut down

11-09-2020 12:03:30,570 EDT INFO  [Watcher] [Watcher@253] watch service closed, terminating.

1. wrapper log

INFO   | jvm 5    | 2020/09/11 12:03:51 | Success -- agent is stopped!

ERROR  | wrapper  | 2020/09/11 12:04:05 | Shutdown failed: Timed out waiting for signal from JVM.

ERROR  | wrapper  | 2020/09/11 12:04:05 | JVM did not exit on request, terminated

STATUS | wrapper  | 2020/09/11 12:04:06 | on_exit trigger matched.  Restarting the JVM.  (Exit code: 1)

FATAL  | wrapper  | 2020/09/11 12:04:06 | There were 5 failed launches in a row, each lasting less than 60 seconds.  Giving up.

FATAL  | wrapper  | 2020/09/11 12:04:06 |   There may be a configuration problem: please check the logs.

STATUS | wrapper  | 2020/09/11 12:04:06 | <-- Wrapper Stopped

As a resolution step I followed this SAS note: https://support.sas.com/kb/52/668.html

But there is an error when I started SAS EVM agent from CMD dos command on both servers

Error verbiage:

wrapper  | SAS [Config-Lev1] SAS Environment Manager Agent started.

Error contacting agent: Error sending argument: Unable to connect to 127.0.0.1:2

144: Connection refused: connect, retried 5 times, cmd=agent:ping

I researched and I found this documentation link: https://documentation.sas.com/?docsetId=evug&docsetTarget=n07iigw9mfj4eln1nyktpc3p1x3a.htm&docsetVer...

The above similar error resolution step are

To resolve this issue, follow these steps:

1.       Ensure that no other process is running on port 2144 (or on the port on which the SAS Environment Manager Agent is configured).
2.       Enter this command:

sas-config-dir/Lev1/Web/SASEnvironmentManager/agent-5.8.0-EE/bin/hq-agent.bat start

I checked for this specific port using this command: netstat -an |find /i "2144" but there is no process is running on this port 2144

I also checked telnet 127.0.0.1 2144
C:\Users\abc>telnet 127.0.0.1 2144
Connecting To 127.0.0.1...Could not open connection to the host, on port 2144: Connect failed

Please advise

Thank you in advance

Thanks,
Asif

[shoin · Accepted solution]

You may be missing some libraries (UIP issue sometimes observed with problematic depot, tip: always run depot checker utility).

Copy
jackson-core-asl-1.8.6.jar
jackson-jaxrs-1.8.6.jar
jackson-mapper-asl-1.8.6.jar

from
<../some/path/>Config/Lev1/Web/SASEnvironmentManager/server-5.8.0-EE/hq-engine/hq-server/webapps/ROOT/WEB-INF/lib/
to
<../some/path/Config>/Lev1/Web/SASEnvironmentManager/agent-5.8.0-EE/bundles/agent-5.8.0/pdk/lib/

Start agents