SAS WebApp installation issue- SAS 94 M5

[Keerthip]

Hi All,

I am facing the below error while installing Web application server . Could you please help with this issue.

The following problem was encountered while configuring SAS Web Application Server Configuration: Application Server has started, however web applications with the following contexts failed to start. Please refer to Application Server logs for more details.
PlatformWebServices.

For more information, see the log file located at 

/SASConfig/wap1/Lev1/Logs/Configure/vfabrictcsvr_startMidtierServers_2018-03-09-13.26.56.log

Vfabric log : 

[appserverTarget] 2018-03-09 13:32:08,235 [main] ERROR com.sas.appserver.vfabrictcsvr.Server - Webapp with PlatformWebServices context failed to start
[appserverTarget] 2018-03-09 13:32:08,235 [main] INFO com.sas.appserver.vfabrictcsvr.Server - getFailedApps() >>>
[appserverTarget] 2018-03-09 13:32:08,272 [main] INFO com.sas.appserver.utils.AppServerException - /en_US/AppServerMessages.properties does not exist.
[appserverTarget] 2018-03-09 13:32:08,273 [main] INFO com.sas.appserver.vfabrictcsvr.Server - testServer() - Throwing AppServerException
[appserverTarget] 2018-03-09 13:32:08,273 [main] INFO com.sas.appserver.vfabrictcsvr.Server - testServer() <<<
[appserverTarget] 2018-03-09 13:32:08,279 [main] INFO com.sas.sdw.SDWExceptionHandler - handle() >>>
[appserverTarget] 2018-03-09 13:32:08,279 [main] INFO com.sas.sdw.SDWExceptionHandler - handleSDWException() >>>
[appserverTarget] 2018-03-09 13:32:08,279 [main] INFO com.sas.sdw.SDWExceptionHandler - target=startMidtierServers, task=com.sas.sdw.SDWExceptionHandler
[appserverTarget] 2018-03-09 13:32:08,279 [main] INFO com.sas.sdw.SDWExceptionHandler - message=Application Server has started, however web applications with the following contexts failed to start. Please refer to Application Server logs for more details.</br><ul><li>PlatformWebServices</li></ul>
[propertyfile] Updating property file: /sascommon/SAS/SASConfig/wap1/Lev1/Logs/Configure/vfabrictcsvr_config_status.properties
[propertyfile] Updating property file: /sascommon/SAS/SASConfig/wap1/Lev1/Logs/Configure/vfabrictcsvr_config_status.properties
2018-03-09 13:32:08,287 [main] INFO com.sas.sdw.SDWExceptionHandler - configStatusFile set.
2018-03-09 13:32:08,293 [main] ERROR com.sas.sdw.SDWExceptionHandler - com.sas.appserver.utils.AppServerException: Application Server has started, however web applications with the following contexts failed to start. Please refer to Application Server logs for more details.</br><ul><li>PlatformWebServices

[JuanS_OCS]

Hello @Keerthip,

any change that this note contains your problem, and your solution?

Problem Note 56135: The web application might not deploy when you add a middle-tier cluster that uses a third-party database for SAS® Shared Services http://support.sas.com/kb/56/135.html 

Solution is to fix the jdbc URL for your SharedServices database.

[Keerthip]

I have checked that kb article. jdbc postgres is configured correctly in server.xml file. Issue is still there..

[JuanS_OCS]

Do you have  clustered web application servers or not?

Something else: could you check the logs for that application? In Lev1/Web/Logs/SASServer1_1

[Keerthip]

Hi Juan

I found the below error in server.xml & catalina.out  under SASServer1_1. Could you infer anything from these logs?

Server.xml :

... 6 more
2018-03-09 13:27:27,456 ERROR (Thread-129) [org.apache.catalina.startup.Catalina] Catalina.stop
org.apache.catalina.LifecycleException: Failed to stop component [StandardServer[-1]]
at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:238)
at org.apache.catalina.startup.Catalina.stop(Catalina.java:728)
at org.apache.catalina.startup.Catalina$CatalinaShutdownHook.run(Catalina.java:829)
Caused by: org.apache.catalina.LifecycleException: Failed to stop component [StandardService[Catalina]]
at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:238)
at org.apache.catalina.core.StandardServer.stopInternal(StandardServer.java:814)
at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:226)
... 2 more
Caused by: org.apache.catalina.LifecycleException: Failed to stop component [StandardEngine[Catalina]]
at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:238)
at org.apache.catalina.core.StandardService.stopInternal(StandardService.java:483)
at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:226)
... 4 more
Caused by: org.apache.catalina.LifecycleException: A child container failed during stop
at org.apache.catalina.core.ContainerBase.stopInternal(ContainerBase.java:1002)
at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:226)
... 6 more
2018-03-09 13:28:12,347 INFO (ResponseThread-0aac8edb-0a36-b43e-3346-958111edba11) [org.apache.catalina.loader.WebappClassLoaderBase] Illegal access: this web application instance has been stopped already. Could not load [org.springframework.jms.core.JmsTemplate$10]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [org.springframework.jms.core.JmsTemplate$10]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1305)
at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1293)

catalina.out : 

INFO [RMI TCP Connection(3)-10.54.180.62] - Metadata connection unavailable. Creating...
2018-03-09 13:27:11,264 [listenerContainer-1] ERROR [unknown] org.springframework.jms.listener.DefaultMessageListenerContainer - Could not refresh JMS Connection for destination 'topic://sas.jms.SASAOPCacheTopic' - retrying in 5000 ms. Cause: The JMS connection has failed: java.io.EOFException
2018-03-09 13:27:11,283 [DefaultMessageListenerContainer-1] ERROR [unknown] org.springframework.jms.listener.DefaultMessageListenerContainer - Could not refresh JMS Connection for destination 'topic://sas.jms.scheduler.pip.broadcastTopic' - retrying in 5000 ms. Cause: The JMS connection has failed: java.io.EOFException
2018-03-09 13:27:11,283 [schedulingListenerContainer-1] WARN [unknown] com.atomikos.jms.AtomikosJMSException - Error in proxy
javax.jms.JMSException: java.io.EOFException

2018-03-09 13:27:14,895 ERROR (localhost-startStop-9) [org.apache.catalina.loader.WebappClassLoaderBase] Found RMI Target with stub class class [com.sas.metadata.remote.MdObjectListImpl_Stub] and value [MdObjectListImpl_Stub[UnicastRef [liveRef: [endpoint:xxxxxx.net:44130](local),objID:[1fdf52af:1620aaa0d8f:-7aa0, -3850858118489109859]]]]]. This RMI Target has been forcibly removed to prevent a memory leak.

[JuanS_OCS]

It does not seem to me as real errors.

Those errors are generated when you are stopping services (probably in the wrong order) and the services cannot connect to other services any more.

I think you should look or before those errors, or start them again.

Important to mention: please pay attention to the path I have sent. You sent logs from other locations.

Very different Lev1/Web/Logs/SASServer1_1/

than /Lev1/Web/SASServer1_1/logs

[Keerthip]

Hi Juan,

i have checked logs in the mentioned path SASConfig/wap1/Lev1/Web/Logs/SASServer1_1. I am seeing the below errors:

... 12 more
2018-03-09 13:27:21,265 [MasterServiceThread-0aac8edb-0a36-b43e-3346-958111edba11] WARN [unknown] com.sas.svcs.scheduling.server.impl.ScheduleManager - SAS Distributed In-Process Scheduling exception occurred.
org.springframework.jms.UncategorizedJmsException: Uncategorized exception occurred during JMS processing; nested exception is com.atomikos.jms.AtomikosJMSException: Failed to grow the connection pool; nested exception is com.atomikos.datasource.pool.CreateConnectionException: error creating JMS connection
at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316)
at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:469)
at org.springframework.jms.core.JmsTemplate.receiveSelected(JmsTemplate.java:695)
at org.springframework.jms.core.JmsTemplate.receiveSelected(JmsTemplate.java:687)
at com.sas.scheduler.api.servers.ip.engine.mq.JMSClusterSupport$6.run(JMSClusterSupport.java:846)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.atomikos.jms.AtomikosJMSException: Failed to grow the connection pool
at com.atomikos.jms.AtomikosJMSException.throwAtomikosJMSException(AtomikosJMSException.java:55)
at com.atomikos.jms.AtomikosConnectionFactoryBean.throwAtomikosJMSException(AtomikosConnectionFactoryBean.java:174)
at com.atomikos.jms.AtomikosConnectionFactoryBean.createConnection(AtomikosConnectionFactoryBean.java:593)
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:184)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:456)
... 4 more
Caused by: com.atomikos.datasource.pool.CreateConnectionException: error creating JMS connection
at com.atomikos.jms.AtomikosJmsXAConnectionFactory.createPooledConnection(AtomikosJmsXAConnectionFactory.java:61)
at com.atomikos.datasource.pool.ConnectionPool.growPool(ConnectionPool.java:250)
at com.atomikos.datasource.pool.ConnectionPool.borrowConnection(ConnectionPool.java:201)
at com.atomikos.jms.AtomikosConnectionFactoryBean.createConnection(AtomikosConnectionFactoryBean.java:591)
... 6 more
Caused by: javax.jms.JMSException: Could not connect to broker URL: tcp://xxxxxx.net:61616. Reason: java.net.ConnectException: Connection refused (Connection refused)
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:36)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:357)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:302)
at org.apache.activemq.ActiveMQXAConnectionFactory.createXAConnection(ActiveMQXAConnectionFactory.java:60)
at com.atomikos.jms.AtomikosJmsXAConnectionFactory.createPooledConnection(AtomikosJmsXAConnectionFactory.java:58)
... 9 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:344)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.activemq.transport.tcp.TcpTransport.connect(TcpTransport.java:501)
at org.apache.activemq.transport.tcp.TcpTransport.doStart(TcpTransport.java:464)
at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)
at org.apache.activemq.transport.AbstractInactivityMonitor.start(AbstractInactivityMonitor.java:168)
at org.apache.activemq.transport.InactivityMonitor.start(InactivityMonitor.java:52)
at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
at org.apache.activemq.transport.WireFormatNegotiator.start(WireFormatNegotiator.java:72)
at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:337)
... 12 more
2018-03-09 13:27:21,266 [MasterServiceThread-0aac8edb-0a36-b43e-3346-958111edba11] WARN [unknown] com.sas.svcs.scheduling.server.impl.ScheduleManager - SAS Distributed In-Process Scheduling exception occurred.
org.springframework.jms.UncategorizedJmsException: Uncategorized exception occurred during JMS processing; nested exception is com.atomikos.jms.AtomikosJMSException: Failed to grow the connection pool; nested exception is com.atomikos.datasource.pool.CreateConnectionException: error creating JMS connection
at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316)
at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:469)
at org.springframework.jms.core.JmsTemplate.receiveSelected(JmsTemplate.java:695)
at org.springframework.jms.core.JmsTemplate.receiveSelected(JmsTemplate.java:687)
at com.sas.scheduler.api.servers.ip.engine.mq.JMSClusterSupport$6.run(JMSClusterSupport.java:846)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.atomikos.jms.AtomikosJMSException: Failed to grow the connection pool
at com.atomikos.jms.AtomikosJMSException.throwAtomikosJMSException(AtomikosJMSException.java:55)
at com.atomikos.jms.AtomikosConnectionFactoryBean.throwAtomikosJMSException(AtomikosConnectionFactoryBean.java:174)
at com.atomikos.jms.AtomikosConnectionFactoryBean.createConnection(AtomikosConnectionFactoryBean.java:593)
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:184)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:456)
... 4 more
Caused by: com.atomikos.datasource.pool.CreateConnectionException: error creating JMS connection
at com.atomikos.jms.AtomikosJmsXAConnectionFactory.createPooledConnection(AtomikosJmsXAConnectionFactory.java:61)
at com.atomikos.datasource.pool.ConnectionPool.growPool(ConnectionPool.java:250)
at com.atomikos.datasource.pool.ConnectionPool.borrowConnection(ConnectionPool.java:201)
at com.atomikos.jms.AtomikosConnectionFactoryBean.createConnection(AtomikosConnectionFactoryBean.java:591)
... 6 more
Caused by: javax.jms.JMSException: Could not connect to broker URL: tcp://xxxxxx.net:61616. Reason: java.net.ConnectException: Connection refused (Connection refused)
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:36)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:357)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:302)
at org.apache.activemq.ActiveMQXAConnectionFactory.createXAConnection(ActiveMQXAConnectionFactory.java:60)
at com.atomikos.jms.AtomikosJmsXAConnectionFactory.createPooledConnection(AtomikosJmsXAConnectionFactory.java:58)
... 9 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:344)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)

[SimonDawson · Accepted solution]

The root of the exception is -

Caused by: javax.jms.JMSException: Could not connect to broker URL: tcp://xxxxxx.net:61616. Reason: java.net.ConnectException: Connection refused (Connection refused)

This means either you have a firewall preventing your web application server contacting the host where the SAS JMS Broker is running or your JMS broker isn't running.

Check the logs (path from the top of my head similar to below) of your JMS broker and try start it.

Config/Lev1/Web/activemq/data/activemq.log 

edit: typos

[NunoTGrunho]

Hi,

Take a look at this sas note http://support.sas.com/kb/54/066.html.