topic Re: SAS WebApp installation issue- SAS 94 M5 in Administration and Deployment

SAS WebApp installation issue- SAS 94 M5

Keerthip — Fri, 09 Mar 2018 13:56:42 GMT

Hi All,

I am facing the below error while installing Web application server . Could you please help with this issue.

The following problem was encountered while configuring SAS Web Application Server Configuration: Application Server has started, however web applications with the following contexts failed to start. Please refer to Application Server logs for more details.
PlatformWebServices.

For more information, see the log file located at

/SASConfig/wap1/Lev1/Logs/Configure/vfabrictcsvr_startMidtierServers_2018-03-09-13.26.56.log

Vfabric log :

[appserverTarget] 2018-03-09 13:32:08,235 [main] ERROR com.sas.appserver.vfabrictcsvr.Server - Webapp with PlatformWebServices context failed to start
[appserverTarget] 2018-03-09 13:32:08,235 [main] INFO com.sas.appserver.vfabrictcsvr.Server - getFailedApps() >>>
[appserverTarget] 2018-03-09 13:32:08,272 [main] INFO com.sas.appserver.utils.AppServerException - /en_US/AppServerMessages.properties does not exist.
[appserverTarget] 2018-03-09 13:32:08,273 [main] INFO com.sas.appserver.vfabrictcsvr.Server - testServer() - Throwing AppServerException
[appserverTarget] 2018-03-09 13:32:08,273 [main] INFO com.sas.appserver.vfabrictcsvr.Server - testServer() <<<
[appserverTarget] 2018-03-09 13:32:08,279 [main] INFO com.sas.sdw.SDWExceptionHandler - handle() >>>
[appserverTarget] 2018-03-09 13:32:08,279 [main] INFO com.sas.sdw.SDWExceptionHandler - handleSDWException() >>>
[appserverTarget] 2018-03-09 13:32:08,279 [main] INFO com.sas.sdw.SDWExceptionHandler - target=startMidtierServers, task=com.sas.sdw.SDWExceptionHandler
[appserverTarget] 2018-03-09 13:32:08,279 [main] INFO com.sas.sdw.SDWExceptionHandler - message=Application Server has started, however web applications with the following contexts failed to start. Please refer to Application Server logs for more details.</br><ul><li>PlatformWebServices</li></ul>
[propertyfile] Updating property file: /sascommon/SAS/SASConfig/wap1/Lev1/Logs/Configure/vfabrictcsvr_config_status.properties
[propertyfile] Updating property file: /sascommon/SAS/SASConfig/wap1/Lev1/Logs/Configure/vfabrictcsvr_config_status.properties
2018-03-09 13:32:08,287 [main] INFO com.sas.sdw.SDWExceptionHandler - configStatusFile set.
2018-03-09 13:32:08,293 [main] ERROR com.sas.sdw.SDWExceptionHandler - com.sas.appserver.utils.AppServerException: Application Server has started, however web applications with the following contexts failed to start. Please refer to Application Server logs for more details.</br><ul><li>PlatformWebServices

Re: SAS WebApp installation issue- SAS 94 M5

JuanS_OCS — Fri, 09 Mar 2018 13:58:51 GMT

Hello @Keerthip,

any change that this note contains your problem, and your solution?

Problem Note 56135: The web application might not deploy when you add a middle-tier cluster that uses a third-party database for SAS® Shared Services http://support.sas.com/kb/56/135.html

Solution is to fix the jdbc URL for your SharedServices database.

Re: SAS WebApp installation issue- SAS 94 M5

Keerthip — Fri, 09 Mar 2018 14:03:13 GMT

I have checked that kb article. jdbc postgres is configured correctly in server.xml file. Issue is still there..

Re: SAS WebApp installation issue- SAS 94 M5

JuanS_OCS — Fri, 09 Mar 2018 14:06:24 GMT

Do you have clustered web application servers or not?

Something else: could you check the logs for that application? In Lev1/Web/Logs/SASServer1_1

Re: SAS WebApp installation issue- SAS 94 M5

Keerthip — Fri, 09 Mar 2018 15:08:52 GMT

Hi Juan

I found the below error in server.xml & catalina.out under SASServer1_1. Could you infer anything from these logs?

Server.xml :

... 6 more
2018-03-09 13:27:27,456 ERROR (Thread-129) [org.apache.catalina.startup.Catalina] Catalina.stop
org.apache.catalina.LifecycleException: Failed to stop component [StandardServer[-1]]
at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:238)
at org.apache.catalina.startup.Catalina.stop(Catalina.java:728)
at org.apache.catalina.startup.Catalina$CatalinaShutdownHook.run(Catalina.java:829)
Caused by: org.apache.catalina.LifecycleException: Failed to stop component [StandardService[Catalina]]
at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:238)
at org.apache.catalina.core.StandardServer.stopInternal(StandardServer.java:814)
at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:226)
... 2 more
Caused by: org.apache.catalina.LifecycleException: Failed to stop component [StandardEngine[Catalina]]
at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:238)
at org.apache.catalina.core.StandardService.stopInternal(StandardService.java:483)
at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:226)
... 4 more
Caused by: org.apache.catalina.LifecycleException: A child container failed during stop
at org.apache.catalina.core.ContainerBase.stopInternal(ContainerBase.java:1002)
at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:226)
... 6 more
2018-03-09 13:28:12,347 INFO (ResponseThread-0aac8edb-0a36-b43e-3346-958111edba11) [org.apache.catalina.loader.WebappClassLoaderBase] Illegal access: this web application instance has been stopped already. Could not load [org.springframework.jms.core.JmsTemplate$10]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [org.springframework.jms.core.JmsTemplate$10]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1305)
at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1293)

catalina.out :

INFO [RMI TCP Connection(3)-10.54.180.62] - Metadata connection unavailable. Creating...
2018-03-09 13:27:11,264 [listenerContainer-1] ERROR [unknown] org.springframework.jms.listener.DefaultMessageListenerContainer - Could not refresh JMS Connection for destination 'topic://sas.jms.SASAOPCacheTopic' - retrying in 5000 ms. Cause: The JMS connection has failed: java.io.EOFException
2018-03-09 13:27:11,283 [DefaultMessageListenerContainer-1] ERROR [unknown] org.springframework.jms.listener.DefaultMessageListenerContainer - Could not refresh JMS Connection for destination 'topic://sas.jms.scheduler.pip.broadcastTopic' - retrying in 5000 ms. Cause: The JMS connection has failed: java.io.EOFException
2018-03-09 13:27:11,283 [schedulingListenerContainer-1] WARN [unknown] com.atomikos.jms.AtomikosJMSException - Error in proxy
javax.jms.JMSException: java.io.EOFException

2018-03-09 13:27:14,895 ERROR (localhost-startStop-9) [org.apache.catalina.loader.WebappClassLoaderBase] Found RMI Target with stub class class [com.sas.metadata.remote.MdObjectListImpl_Stub] and value [MdObjectListImpl_Stub[UnicastRef [liveRef: [endpoint:xxxxxx.net:44130](local),objID:[1fdf52af:1620aaa0d8f:-7aa0, -3850858118489109859]]]]]. This RMI Target has been forcibly removed to prevent a memory leak.

Re: SAS WebApp installation issue- SAS 94 M5

JuanS_OCS — Fri, 09 Mar 2018 15:15:14 GMT

It does not seem to me as real errors.

Those errors are generated when you are stopping services (probably in the wrong order) and the services cannot connect to other services any more.

I think you should look or before those errors, or start them again.

Important to mention: please pay attention to the path I have sent. You sent logs from other locations.

Very different Lev1/Web/Logs/SASServer1_1/

than /Lev1/Web/SASServer1_1/logs

Re: SAS WebApp installation issue- SAS 94 M5

Keerthip — Fri, 09 Mar 2018 16:34:44 GMT

Hi Juan,

i have checked logs in the mentioned path SASConfig/wap1/Lev1/Web/Logs/SASServer1_1. I am seeing the below errors:

... 12 more
2018-03-09 13:27:21,265 [MasterServiceThread-0aac8edb-0a36-b43e-3346-958111edba11] WARN [unknown] com.sas.svcs.scheduling.server.impl.ScheduleManager - SAS Distributed In-Process Scheduling exception occurred.
org.springframework.jms.UncategorizedJmsException: Uncategorized exception occurred during JMS processing; nested exception is com.atomikos.jms.AtomikosJMSException: Failed to grow the connection pool; nested exception is com.atomikos.datasource.pool.CreateConnectionException: error creating JMS connection
at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316)
at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:469)
at org.springframework.jms.core.JmsTemplate.receiveSelected(JmsTemplate.java:695)
at org.springframework.jms.core.JmsTemplate.receiveSelected(JmsTemplate.java:687)
at com.sas.scheduler.api.servers.ip.engine.mq.JMSClusterSupport$6.run(JMSClusterSupport.java:846)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.atomikos.jms.AtomikosJMSException: Failed to grow the connection pool
at com.atomikos.jms.AtomikosJMSException.throwAtomikosJMSException(AtomikosJMSException.java:55)
at com.atomikos.jms.AtomikosConnectionFactoryBean.throwAtomikosJMSException(AtomikosConnectionFactoryBean.java:174)
at com.atomikos.jms.AtomikosConnectionFactoryBean.createConnection(AtomikosConnectionFactoryBean.java:593)
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:184)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:456)
... 4 more
Caused by: com.atomikos.datasource.pool.CreateConnectionException: error creating JMS connection
at com.atomikos.jms.AtomikosJmsXAConnectionFactory.createPooledConnection(AtomikosJmsXAConnectionFactory.java:61)
at com.atomikos.datasource.pool.ConnectionPool.growPool(ConnectionPool.java:250)
at com.atomikos.datasource.pool.ConnectionPool.borrowConnection(ConnectionPool.java:201)
at com.atomikos.jms.AtomikosConnectionFactoryBean.createConnection(AtomikosConnectionFactoryBean.java:591)
... 6 more
Caused by: javax.jms.JMSException: Could not connect to broker URL: tcp://xxxxxx.net:61616. Reason: java.net.ConnectException: Connection refused (Connection refused)
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:36)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:357)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:302)
at org.apache.activemq.ActiveMQXAConnectionFactory.createXAConnection(ActiveMQXAConnectionFactory.java:60)
at com.atomikos.jms.AtomikosJmsXAConnectionFactory.createPooledConnection(AtomikosJmsXAConnectionFactory.java:58)
... 9 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:344)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.activemq.transport.tcp.TcpTransport.connect(TcpTransport.java:501)
at org.apache.activemq.transport.tcp.TcpTransport.doStart(TcpTransport.java:464)
at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)
at org.apache.activemq.transport.AbstractInactivityMonitor.start(AbstractInactivityMonitor.java:168)
at org.apache.activemq.transport.InactivityMonitor.start(InactivityMonitor.java:52)
at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
at org.apache.activemq.transport.WireFormatNegotiator.start(WireFormatNegotiator.java:72)
at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:337)
... 12 more
2018-03-09 13:27:21,266 [MasterServiceThread-0aac8edb-0a36-b43e-3346-958111edba11] WARN [unknown] com.sas.svcs.scheduling.server.impl.ScheduleManager - SAS Distributed In-Process Scheduling exception occurred.
org.springframework.jms.UncategorizedJmsException: Uncategorized exception occurred during JMS processing; nested exception is com.atomikos.jms.AtomikosJMSException: Failed to grow the connection pool; nested exception is com.atomikos.datasource.pool.CreateConnectionException: error creating JMS connection
at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316)
at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:469)
at org.springframework.jms.core.JmsTemplate.receiveSelected(JmsTemplate.java:695)
at org.springframework.jms.core.JmsTemplate.receiveSelected(JmsTemplate.java:687)
at com.sas.scheduler.api.servers.ip.engine.mq.JMSClusterSupport$6.run(JMSClusterSupport.java:846)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.atomikos.jms.AtomikosJMSException: Failed to grow the connection pool
at com.atomikos.jms.AtomikosJMSException.throwAtomikosJMSException(AtomikosJMSException.java:55)
at com.atomikos.jms.AtomikosConnectionFactoryBean.throwAtomikosJMSException(AtomikosConnectionFactoryBean.java:174)
at com.atomikos.jms.AtomikosConnectionFactoryBean.createConnection(AtomikosConnectionFactoryBean.java:593)
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:184)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:456)
... 4 more
Caused by: com.atomikos.datasource.pool.CreateConnectionException: error creating JMS connection
at com.atomikos.jms.AtomikosJmsXAConnectionFactory.createPooledConnection(AtomikosJmsXAConnectionFactory.java:61)
at com.atomikos.datasource.pool.ConnectionPool.growPool(ConnectionPool.java:250)
at com.atomikos.datasource.pool.ConnectionPool.borrowConnection(ConnectionPool.java:201)
at com.atomikos.jms.AtomikosConnectionFactoryBean.createConnection(AtomikosConnectionFactoryBean.java:591)
... 6 more
Caused by: javax.jms.JMSException: Could not connect to broker URL: tcp://xxxxxx.net:61616. Reason: java.net.ConnectException: Connection refused (Connection refused)
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:36)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:357)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:302)
at org.apache.activemq.ActiveMQXAConnectionFactory.createXAConnection(ActiveMQXAConnectionFactory.java:60)
at com.atomikos.jms.AtomikosJmsXAConnectionFactory.createPooledConnection(AtomikosJmsXAConnectionFactory.java:58)
... 9 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:344)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)

Re: SAS WebApp installation issue- SAS 94 M5

SimonDawson — Mon, 12 Mar 2018 09:22:51 GMT

The root of the exception is -

Caused by: javax.jms.JMSException: Could not connect to broker URL: tcp://xxxxxx.net:61616. Reason: java.net.ConnectException: Connection refused (Connection refused)

This means either you have a firewall preventing your web application server contacting the host where the SAS JMS Broker is running or your JMS broker isn't running.

Check the logs (path from the top of my head similar to below) of your JMS broker and try start it.

Config/Lev1/Web/activemq/data/activemq.log

edit: typos

Re: SAS WebApp installation issue- SAS 94 M5

NunoTGrunho — Mon, 12 Mar 2018 10:11:17 GMT

Hi,

Take a look at this sas note http://support.sas.com/kb/54/066.html.