topic Re: SASServer1_1 not starting in Administration and Deployment

SASServer1_1 not starting

woo — Fri, 11 Jan 2019 16:25:44 GMT

Hello, we are running sas 9.4 on Linux machine. there was no environment changes happened and we started receiving web application error. please help to troubleshoot. Thank you.

http:/hostname:7980 is fine so don't see any issue in web server or either jms broker or cache locator. when try -> http:/hostname:7980/any-sas-web-application is throws error saying web page cannot be found.

this is the server.log for server1_1 while starting up mid-tier server.

09:34:43,904 WARN (main) [com.springsource.tcserver.serviceability.rmi.JmxSocketListener] Detected LDAP configuration '/sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/jaas.config', but there is no ldap configuration entry specified.
09:34:43,968 INFO (main) [org.apache.catalina.startup.Catalina] Initialization processed in 1099 ms
09:34:46,083 INFO (main) [org.apache.catalina.core.StandardService] Starting service Catalina
09:34:46,083 INFO (main) [org.apache.catalina.core.StandardEngine] Starting Servlet Engine: Pivotal tc Runtime 3.0.0.RELEASE/7.0.55.A.RELEASE
09:34:46,093 INFO (localhost-startStop-1) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASThemeDesignerForFlex.xml
09:34:46,093 INFO (localhost-startStop-4) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASPermissionManager.xml
09:34:46,093 INFO (localhost-startStop-2) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASWIPSoapServices.xml
09:34:46,093 INFO (localhost-startStop-3) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASIdentityServices.xml
09:34:56,606 INFO (localhost-startStop-1) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASThemeDesignerForFlex]] No Spring WebApplicationInitializer types detected on classpath
09:34:56,648 INFO (localhost-startStop-1) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASThemeDesignerForFlex]] Config path: file:////sasconfig/Lev1/Web/Common/LogConfig/
09:34:56,648 INFO (localhost-startStop-1) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASThemeDesignerForFlex]] Context path: SASThemeDesignerForFlex
09:34:56,649 INFO (localhost-startStop-1) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASThemeDesignerForFlex]] Configuring Log4j from URL at path: /file:////sasconfig/Lev1/Web/Common/LogConfig/SASThemeDesignerForFlex-log4j.xml
09:34:56,649 INFO (localhost-startStop-1) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASThemeDesignerForFlex]] isXMLConfigFile: true
09:34:56,659 INFO (localhost-startStop-3) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASIdentityServices]] No Spring WebApplicationInitializer types detected on classpath
09:34:56,675 INFO (localhost-startStop-2) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASWIPSoapServices]] No Spring WebApplicationInitializer types detected on classpath
09:34:56,678 INFO (localhost-startStop-3) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASIdentityServices]] Config path: file:////sasconfig/Lev1/Web/Common/LogConfig/
09:34:56,691 INFO (localhost-startStop-3) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASIdentityServices]] Context path: SASIdentityServices
09:34:56,692 INFO (localhost-startStop-3) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASIdentityServices]] Configuring Log4j from URL at path: /file:////sasconfig/Lev1/Web/Common/LogConfig/SASIdentityServices-log4j.xml
09:34:56,692 INFO (localhost-startStop-3) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASIdentityServices]] isXMLConfigFile: true
09:34:56,712 INFO (localhost-startStop-1) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASThemeDesignerForFlex]] Initializing Spring root WebApplicationContext
09:34:56,725 INFO (localhost-startStop-3) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASIdentityServices]] Initializing Spring root WebApplicationContext
09:34:56,730 INFO (localhost-startStop-2) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASWIPSoapServices]] Config path: file:////sasconfig/Lev1/Web/Common/LogConfig/
09:34:56,730 INFO (localhost-startStop-2) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASWIPSoapServices]] Context path: SASWIPSoapServices
09:34:56,730 INFO (localhost-startStop-2) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASWIPSoapServices]] Configuring Log4j from URL at path: /file:////sasconfig/Lev1/Web/Common/LogConfig/SASWIPSoapServices-log4j.xml
09:34:56,731 INFO (localhost-startStop-2) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASWIPSoapServices]] isXMLConfigFile: true
09:34:56,786 INFO (localhost-startStop-2) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASWIPSoapServices]] Initializing Spring root WebApplicationContext
09:34:56,995 INFO (localhost-startStop-4) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASPermissionManager]] No Spring WebApplicationInitializer types detected on classpath
09:34:57,053 INFO (localhost-startStop-4) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASPermissionManager]] Config path: file:////sasconfig/Lev1/Web/Common/LogConfig/
09:34:57,054 INFO (localhost-startStop-4) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASPermissionManager]] Context path: SASPermissionManager
09:34:57,054 INFO (localhost-startStop-4) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASPermissionManager]] Configuring Log4j from URL at path: /file:////sasconfig/Lev1/Web/Common/LogConfig/SASPermissionManager-log4j.xml
09:34:57,054 INFO (localhost-startStop-4) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASPermissionManager]] isXMLConfigFile: true
09:34:57,080 INFO (localhost-startStop-4) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASPermissionManager]] Initializing Spring root WebApplicationContext
09:35:01,863 ERROR (localhost-startStop-4) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASPermissionManager]] Exception sending context initialized event to listener instance of class org.springframework.web.context.ContextLoaderListener
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cachedConfigurer' defined in ServletContext resource [/WEB-INF/spring-config/infrastructure-config.xml]: Cannot resolve reference to bean 'environmentProperties' while setting bean property 'properties'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'environmentProperties': FactoryBean threw exception on object creation; nested exception is org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is com.atomikos.jdbc.AtomikosSQLException: Failed to grow the connection pool
        at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:334)
        at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1419)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1160)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:195)
        at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:659)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:461)
        at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:410)
        at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:306)
        at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:112)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4992)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5490)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:649)
        at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:672)
        at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1862)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'environmentProperties': FactoryBean threw exception on object creation; nested exception is org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is com.atomikos.jdbc.AtomikosSQLException: Failed to grow the connection pool
        at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:173)
        at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:126)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1465)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:304)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:191)
        at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328)
        ... 27 more
Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is com.atomikos.jdbc.AtomikosSQLException: Failed to grow the connection pool
        at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80)
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:575)
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:639)
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:668)
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:676)
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:716)
        at com.sas.framework.services.config.dao.PropertySetDatabaseDAO.getProperties(PropertySetDatabaseDAO.java:180)
        at com.sas.framework.services.config.dao.PropertySetDatabaseDAO.getPropertyMap(PropertySetDatabaseDAO.java:223)
        at com.sas.framework.services.config.dao.PropertySetFactoryBean.getObject(PropertySetFactoryBean.java:56)
        at com.sas.framework.services.config.dao.PropertySetFactoryBean.getObject(PropertySetFactoryBean.java:21)
        at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:166)
        ... 32 more
Caused by: com.atomikos.jdbc.AtomikosSQLException: Failed to grow the connection pool
        at com.atomikos.jdbc.AtomikosSQLException.throwAtomikosSQLException(AtomikosSQLException.java:46)
        at com.atomikos.jdbc.AbstractDataSourceBean.throwAtomikosSQLException(AbstractDataSourceBean.java:90)
        at com.atomikos.jdbc.AbstractDataSourceBean.getConnection(AbstractDataSourceBean.java:328)
        at com.atomikos.jdbc.nonxa.AtomikosNonXADataSourceBean.getConnection(AtomikosNonXADataSourceBean.java:215)
        at com.atomikos.jdbc.AbstractDataSourceBean.getConnection(AbstractDataSourceBean.java:377)
        at com.sas.framework.services.config.dao.PropertySetDataSource.getConnection(PropertySetDataSource.java:241)
        at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111)
        at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77)
        ... 42 more
Caused by: com.atomikos.datasource.pool.CreateConnectionException: Could not create JDBC connection
        at com.atomikos.jdbc.nonxa.AtomikosNonXAConnectionFactory.createPooledConnection(AtomikosNonXAConnectionFactory.java:130)
        at com.atomikos.datasource.pool.ConnectionPool.growPool(ConnectionPool.java:250)
        at com.atomikos.datasource.pool.ConnectionPool.borrowConnection(ConnectionPool.java:201)
        at com.atomikos.jdbc.AbstractDataSourceBean.getConnection(AbstractDataSourceBean.java:325)
        ... 47 more
Caused by: java.sql.SQLException: ORA-28001: the password has expired

        at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:445)
        at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:389)
        at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:382)
        at oracle.jdbc.driver.T4CTTIfun.processError(T4CTTIfun.java:600)
        at oracle.jdbc.driver.T4CTTIoauthenticate.processError(T4CTTIoauthenticate.java:445)
        at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:450)
        at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
        at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:380)
        at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:760)
        at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:401)
        at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:546)
        at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:236)
        at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
        at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:521)
        at com.atomikos.jdbc.nonxa.AtomikosNonXAConnectionFactory.getConnection(AtomikosNonXAConnectionFactory.java:119)
        at com.atomikos.jdbc.nonxa.AtomikosNonXAConnectionFactory.createPooledConnection(AtomikosNonXAConnectionFactory.java:127)
        ... 51 more
09:35:02,624 ERROR (localhost-startStop-2) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:02,625 ERROR (localhost-startStop-2) [org.apache.catalina.core.StandardContext] Context [/SASWIPSoapServices] startup failed due to previous errors
        at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
        at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:380)
        at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:760)
        at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:401)
        at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:546)
        at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:236)
        at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
        at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:521)
        at com.atomikos.jdbc.nonxa.AtomikosNonXAConnectionFactory.getConnection(AtomikosNonXAConnectionFactory.java:119)
        at com.atomikos.jdbc.nonxa.AtomikosNonXAConnectionFactory.createPooledConnection(AtomikosNonXAConnectionFactory.java:127)
        ... 51 more
09:35:02,624 ERROR (localhost-startStop-2) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:02,625 ERROR (localhost-startStop-2) [org.apache.catalina.core.StandardContext] Context [/SASWIPSoapServices] startup failed due to previous errors
09:35:02,883 INFO (localhost-startStop-4) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASPermissionManager]] Cleaning up Log4j resources for context: null...
09:35:02,883 INFO (localhost-startStop-4) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASPermissionManager]] Shutting down all loggers and appenders...
09:35:02,884 INFO (localhost-startStop-4) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASPermissionManager]] Log4j cleaned up.
09:35:02,890 INFO (localhost-startStop-4) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASPermissionManager.xml has finished in 16,797 ms
09:35:02,890 INFO (localhost-startStop-4) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASTheme_default.xml
09:35:03,153 INFO (localhost-startStop-3) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASIdentityServices]] Cleaning up Log4j resources for context: null...
09:35:03,153 INFO (localhost-startStop-3) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASIdentityServices]] Shutting down all loggers and appenders...
09:35:03,153 INFO (localhost-startStop-3) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASIdentityServices]] Shutting down all loggers and appenders...
09:35:03,154 INFO (localhost-startStop-3) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASIdentityServices]] Log4j cleaned up.
09:35:03,160 INFO (localhost-startStop-3) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASIdentityServices.xml has finished in 17,067 ms
09:35:03,160 INFO (localhost-startStop-3) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASWIPClientAccess.xml
09:35:03,627 INFO (localhost-startStop-2) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASWIPSoapServices]] Closing Spring root WebApplicationContext
09:35:03,635 INFO (localhost-startStop-2) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASWIPSoapServices]] Cleaning up Log4j resources for context: null...
09:35:03,636 INFO (localhost-startStop-2) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASWIPSoapServices]] Shutting down all loggers and appenders...
09:35:03,636 INFO (localhost-startStop-2) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASWIPSoapServices]] Log4j cleaned up.
09:35:03,644 INFO (localhost-startStop-2) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASWIPSoapServices.xml has finished in 17,551 ms
09:35:03,645 INFO (localhost-startStop-2) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASPrincipalServices.xml
09:35:03,819 INFO (localhost-startStop-4) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASTheme_default.xml has finished in 929 ms
09:35:03,822 INFO (localhost-startStop-4) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASFlexThemes.xml
09:35:04,964 INFO (localhost-startStop-4) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASFlexThemes.xml has finished in 1,142 ms
09:35:04,965 INFO (localhost-startStop-4) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASBIWS.xml
09:35:08,859 ERROR (localhost-startStop-1) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:08,860 ERROR (localhost-startStop-1) [org.apache.catalina.core.StandardContext] Context [/SASContentServer] startup failed due to previous errors
09:35:08,867 INFO (localhost-startStop-1) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASContentServer.xml has finished in 6,505 ms
09:35:08,867 INFO (localhost-startStop-1) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASSharedApps.xml
09:35:10,468 ERROR (localhost-startStop-2) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:10,469 ERROR (localhost-startStop-2) [org.apache.catalina.core.StandardContext] Context [/SASPrincipalServices] startup failed due to previous errors
09:35:11,161 ERROR (localhost-startStop-3) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:11,162 ERROR (localhost-startStop-3) [org.apache.catalina.core.StandardContext] Context [/SASWIPClientAccess] startup failed due to previous errors
09:35:11,469 ERROR (localhost-startStop-4) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:11,470 ERROR (localhost-startStop-4) [org.apache.catalina.core.StandardContext] Context [/SASBIWS] startup failed due to previous errors
09:35:11,480 INFO (localhost-startStop-4) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASBIWS.xml has finished in 6,515 ms
09:35:11,480 INFO (localhost-startStop-4) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASWorkflowServices.xml
09:35:11,516 INFO (localhost-startStop-2) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASPrincipalServices.xml has finished in 7,871 ms
09:35:11,516 INFO (localhost-startStop-2) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/sasweb.xml
09:35:12,224 INFO (localhost-startStop-3) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASWIPClientAccess.xml has finished in 9,064 ms
09:35:12,224 INFO (localhost-startStop-3) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASDeploymentBackup.xml
09:35:12,428 INFO (localhost-startStop-2) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/sasweb.xml has finished in 912 ms
09:35:12,428 INFO (localhost-startStop-2) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASPreferences.xml
09:35:15,432 ERROR (localhost-startStop-1) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:15,433 ERROR (localhost-startStop-1) [org.apache.catalina.core.StandardContext] Context [/SASSharedApps] startup failed due to previous errors
09:35:16,452 INFO (localhost-startStop-1) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASSharedApps.xml has finished in 7,585 ms
09:35:16,453 INFO (localhost-startStop-1) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASTheme_company.xml
09:35:17,349 INFO (localhost-startStop-1) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASTheme_company.xml has finished in 896 ms
09:35:17,350 INFO (localhost-startStop-1) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASAuthorizationServices.xml
09:35:18,885 ERROR (localhost-startStop-2) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:18,919 ERROR (localhost-startStop-2) [org.apache.catalina.core.StandardContext] Context [/SASPreferences] startup failed due to previous errors
09:35:19,935 INFO (localhost-startStop-2) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASPreferences.xml has finished in 7,507 ms
09:35:19,935 INFO (localhost-startStop-2) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASTemplateEditor.xml
09:35:19,935 INFO (localhost-startStop-2) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASTemplateEditor.xml
09:35:20,029 ERROR (localhost-startStop-4) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:20,031 ERROR (localhost-startStop-4) [org.apache.catalina.core.StandardContext] Context [/SASWorkflowServices] startup failed due to previous errors
09:35:21,051 INFO (localhost-startStop-4) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASWorkflowServices.xml has finished in 9,570 ms
09:35:21,051 INFO (localhost-startStop-4) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASAdmin.xml
09:35:21,231 ERROR (localhost-startStop-3) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:21,231 ERROR (localhost-startStop-3) [org.apache.catalina.core.StandardContext] Context [/SASDeploymentBackup] startup failed due to previous errors
09:35:22,148 ERROR (localhost-startStop-1) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:22,149 ERROR (localhost-startStop-1) [org.apache.catalina.core.StandardContext] Context [/SASAuthorizationServices] startup failed due to previous errors
09:35:22,242 INFO (localhost-startStop-3) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASDeploymentBackup.xml has finished in 10,018 ms
09:35:22,242 INFO (localhost-startStop-3) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASWorkflowWebServices.xml
09:35:23,166 INFO (localhost-startStop-1) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASAuthorizationServices.xml has finished in 5,816 ms
09:35:23,167 INFO (localhost-startStop-1) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASLogon.xml
09:35:24,748 ERROR (localhost-startStop-2) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:24,749 ERROR (localhost-startStop-2) [org.apache.catalina.core.StandardContext] Context [/SASTemplateEditor] startup failed due to previous errors
09:35:25,766 INFO (localhost-startStop-2) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASTemplateEditor.xml has finished in 5,831 ms
09:35:25,767 INFO (localhost-startStop-2) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASWIPServices.xml
09:35:26,540 ERROR (localhost-startStop-4) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:26,541 ERROR (localhost-startStop-4) [org.apache.catalina.core.StandardContext] Context [/SASAdmin] startup failed due to previous errors
09:35:27,552 INFO (localhost-startStop-4) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASAdmin.xml has finished in 6,501 ms
09:35:27,552 INFO (localhost-startStop-4) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASBackupManager.xml
09:35:28,373 ERROR (localhost-startStop-3) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:28,374 ERROR (localhost-startStop-3) [org.apache.catalina.core.StandardContext] Context [/SASWorkflowWebServices] startup failed due to previous errors
09:35:29,395 INFO (localhost-startStop-3) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASWorkflowWebServices.xml has finished in 7,153 ms
09:35:29,397 INFO (localhost-startStop-3) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASStoredProcess.xml
09:35:29,397 INFO (localhost-startStop-3) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASStoredProcess.xml
09:35:29,406 ERROR (localhost-startStop-1) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:29,407 ERROR (localhost-startStop-1) [org.apache.catalina.core.StandardContext] Context [/SASLogon] startup failed due to previous errors
09:35:30,438 ERROR (localhost-startStop-1) [org.apache.catalina.loader.WebappClassLoader] The web application [/SASLogon] appears to have started a thread named [perf4j-async-stats-appender-sink-CoalescingStatistics] but has failed to stop it. This is very likely to create a memory leak.
09:35:30,442 INFO (localhost-startStop-1) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASLogon.xml has finished in 7,275 ms
09:35:34,109 ERROR (localhost-startStop-4) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:34,110 ERROR (localhost-startStop-4) [org.apache.catalina.core.StandardContext] Context [/SASBackupManager] startup failed due to previous errors
09:35:34,153 INFO (localhost-startStop-4) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASBackupManager.xml has finished in 6,601 ms
09:35:34,811 ERROR (localhost-startStop-3) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:34,812 ERROR (localhost-startStop-3) [org.apache.catalina.core.StandardContext] Context [/SASStoredProcess] startup failed due to previous errors
09:35:34,838 INFO (localhost-startStop-3) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASStoredProcess.xml has finished in 5,440 ms
09:35:38,875 ERROR (localhost-startStop-2) [org.apache.catalina.core.StandardContext] Error listenerStart
09:35:38,876 ERROR (localhost-startStop-2) [org.apache.catalina.core.StandardContext] Context [/SASWIPServices] startup failed due to previous errors
09:35:39,893 INFO (localhost-startStop-2) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASWIPServices.xml has finished in 14,126 ms
09:35:39,894 INFO (localhost-startStop-1) [org.apache.catalina.startup.HostConfig] Deploying web application directory /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/webapps/ROOT
09:35:40,477 INFO (localhost-startStop-1) [org.apache.catalina.startup.HostConfig] Deployment of web application directory /sasconfig/Lev1/Web/WebAppServer/SASServer1_1/webapps/ROOT has finished in 582 ms
09:35:40,494 INFO (main) [org.apache.catalina.startup.Catalina] Server startup in 56525 ms
^C
-sh-4.1$

Re: SASServer1_1 not starting

dpage — Fri, 11 Jan 2019 16:33:53 GMT

"Caused by: java.sql.SQLException: ORA-28001: the password has expired"

Oracle isn't letting the JDBC connection through due to an expired password. If you can unexpire and keep the same password, do that. If you need to update it on the Oracle side to something new, follow the doc here to update SAS's references to it. https://go.documentation.sas.com/?docsetId=bisecag&docsetTarget=p0kb2gtnuyjnrnn1hduu5by88u4f.htm&docsetVersion=9.4&locale=en

Re: SASServer1_1 not starting

woo — Fri, 11 Jan 2019 16:43:34 GMT

we have unexpired password in place. do we still need to update using sdm? we have sas metadata and other compute servers are up and seems fine. services came up fine for those tiers. issue is only mid-tier...

Re: SASServer1_1 not starting

dpage — Fri, 11 Jan 2019 18:12:45 GMT

I'm not sure what that means - but if the username/password combination for SharedServices is invalid, sasserver1 will fail to start with the errors you're seeing. There's nothing in the compute or metadata tier that would care about those credentials, so they'd work fine. Some solutions have database credentials that are used in multiple places, but typically the compute tier won't fail until you run a job / stored process / application action that triggers a database access - it's not usually a startup issue.

Re: SASServer1_1 not starting

woo — Fri, 11 Jan 2019 19:34:53 GMT

so if I use internal sas account, example: sas installer account, below libname statement runs fine. if password needs to update, would I able to still define libname statement?

however, at the moment, I, as a user, cannot use EG and connect to metadata server (same with other users). Profile error says userid or password is incorrect. only can connect using internal account and define libname statement.

libname myora oracle path="" schema="" user="" password="";

Re: SASServer1_1 not starting

dpage — Fri, 11 Jan 2019 21:10:43 GMT

It depends on how your Oracle authentication is setup - if you're using something like Kerberos auth it may be delegating that properly, but not the JDBC user/password login. You can see which userid it's attempting to use in SASServer1_1/conf/server.xml - look for the jdbc line(s) - then attempt to use a libname statement to that same host/port/SID with that user/password.

I wouldn't expect that libname to do much of anything if you're not filling those fields out.

For your EG error, that seems unrelated but likely has something to do with the metadata server -> OS authentication layer - look for more details in the metadata server logs (is it supposed to be configured to validate external accounts in a system like LDAP?)

Re: SASServer1_1 not starting

woo — Fri, 11 Jan 2019 22:50:20 GMT

Thanks for your details. Also, I followed sas documentation and was able to update password on compute tier and mid-tier. all server tiers came up fine except server8 which is throwing below error.

I also modified "db.AdminTools.properties" /sasconfig/Lev1/Applications/SASModelRiskManagementAdministrativeTools/properties with proc pwencode process.

server.log for server8

17:36:12,185 INFO (main) [org.apache.catalina.core.StandardService] Starting service Catalina
17:36:12,185 INFO (main) [org.apache.catalina.core.StandardEngine] Starting Servlet Engine: Pivotal tc Runtime 3.0.0.RELEASE/7.0.55.A.RELEASE
17:36:12,210 INFO (localhost-startStop-1) [org.apache.catalina.startup.HostConfig] Deploying configuration descriptor ../sasconfig/Lev1/Web/WebAppServer/SASServer8_1/conf/Catalina/localhost/SASModelRisk.xml
17:36:17,068 INFO (localhost-startStop-1) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASModelRisk]] No Spring WebApplicationInitializer types detected on classpath
17:36:17,096 INFO (localhost-startStop-1) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASModelRisk]] Config path: file:///../sasconfig/Lev1/Web/Common/LogConfig/
17:36:17,097 INFO (localhost-startStop-1) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASModelRisk]] Context path: SASModelRiskManagementMidTier
17:36:17,097 INFO (localhost-startStop-1) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASModelRisk]] Configuring Log4j from URL at path: /file:///../sasconfig/Lev1/Web/Common/LogConfig/SASModelRiskManagementMidTier-log4j.xml
17:36:17,097 INFO (localhost-startStop-1) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASModelRisk]] isXMLConfigFile: true
17:36:17,119 INFO (localhost-startStop-1) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASModelRisk]] Initializing Spring root WebApplicationContext
17:37:19,199 ERROR (localhost-startStop-1) [org.apache.tomcat.jdbc.pool.ConnectionPool] Unable to create initial connections of pool.
java.sql.SQLException: Listener refused the connection with the following error:
ORA-12505, TNS:listener does not currently know of SID given in connect descriptor

        at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:458)
        at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:546)
        at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:236)
        at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
        at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:521)
        at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:278)
        at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:182)
        at org.apache.tomcat.jdbc.pool.ConnectionPool.createConnection(ConnectionPool.java:701)
        at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:635)
        at org.apache.tomcat.jdbc.pool.ConnectionPool.init(ConnectionPool.java:486)
        at org.apache.tomcat.jdbc.pool.ConnectionPool.<init>(ConnectionPool.java:144)
        at org.apache.tomcat.jdbc.pool.DataSourceProxy.pCreatePool(DataSourceProxy.java:116)
        at org.apache.tomcat.jdbc.pool.DataSourceProxy.createPool(DataSourceProxy.java:103)
        at org.apache.tomcat.jdbc.pool.DataSourceFactory.createDataSource(DataSourceFactory.java:554)
        at org.apache.tomcat.jdbc.pool.DataSourceFactory.getObjectInstance(DataSourceFactory.java:242)
        at org.apache.naming.factory.ResourceFactory.getObjectInstance(ResourceFactory.java:141)
        at javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:321)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:842)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:153)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:830)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:153)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:830)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:167)
        at org.apache.naming.factory.ResourceLinkFactory.getObjectInstance(ResourceLinkFactory.java:92)
        at javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:321)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:842)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:153)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:830)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:153)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:830)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:153)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:830)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:167)
        at org.apache.naming.SelectorContext.lookup(SelectorContext.java:156)
        at javax.naming.InitialContext.lookup(InitialContext.java:411)
        at com.sas.oprisk.framework.server.persistence.ContainerManagedConnectionProvider.<init>(ContainerManagedConnectionProvider.java:38)
        at com.sas.oprisk.monitor.web.MonitorContextListener.contextInitialized(MonitorContextListener.java:55)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4992)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5490)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:649)
        at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:672)
        at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1862)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: oracle.net.ns.NetException: Listener refused the connection with the following error:
ORA-12505, TNS:listener does not currently know of SID given in connect descriptor

        at oracle.net.ns.NSProtocol.connect(NSProtocol.java:395)
        at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1102)
        at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:320)
        ... 49 more
17:37:19,203 WARN (localhost-startStop-1) [org.apache.naming.NamingContext] Unexpected exception resolving reference
java.sql.SQLException: Listener refused the connection with the following error:
ORA-12505, TNS:listener does not currently know of SID given in connect descriptor

        at oracle.net.ns.NSProtocol.connect(NSProtocol.java:395)
        at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1102)
        at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:320)
        ... 49 more
17:37:19,205 ERROR (localhost-startStop-1) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASModelRisk]] Exception sending context initialized event to listener instance of class com.sas.oprisk.monitor.web.MonitorContextListener
com.sas.oprisk.framework.server.OpRiskConfigurationError: Listener refused the connection with the following error:
ORA-12505, TNS:listener does not currently know of SID given in connect descriptor

        at com.sas.oprisk.framework.server.persistence.ContainerManagedConnectionProvider.<init>(ContainerManagedConnectionProvider.java:47)
        at com.sas.oprisk.monitor.web.MonitorContextListener.contextInitialized(MonitorContextListener.java:55)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4992)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5490)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:649)
        at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:672)
        at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1862)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: javax.naming.NamingException: Listener refused the connection with the following error:
ORA-12505, TNS:listener does not currently know of SID given in connect descriptor

Caused By:

javax.naming.NamingException: Listener refused the connection with the following error:
ORA-12505, TNS:listener does not currently know of SID given in connect descriptor

        at org.apache.naming.NamingContext.lookup(NamingContext.java:859)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:153)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:830)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:153)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:830)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:167)
        at org.apache.naming.factory.ResourceLinkFactory.getObjectInstance(ResourceLinkFactory.java:92)
        at javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:321)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:842)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:153)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:830)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:153)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:830)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:153)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:830)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:167)
        at org.apache.naming.SelectorContext.lookup(SelectorContext.java:156)
        at javax.naming.InitialContext.lookup(InitialContext.java:411)
        at com.sas.oprisk.framework.server.persistence.ContainerManagedConnectionProvider.<init>(ContainerManagedConnectionProvider.java:38)
        at com.sas.oprisk.monitor.web.MonitorContextListener.contextInitialized(MonitorContextListener.java:55)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4992)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5490)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:649)
        at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:672)
        at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1862)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
17:37:19,207 ERROR (localhost-startStop-1) [org.apache.catalina.core.StandardContext] Error listenerStart
17:37:19,209 ERROR (localhost-startStop-1) [org.apache.catalina.core.StandardContext] Context [/SASModelRisk] startup failed due to previous errors
17:37:19,294 WARN (localhost-startStop-1) [com.sas.oprisk.monitor.web.MonitorContextListener] Cannot unlock secured user session context. Lock object was null.
17:37:19,294 INFO (localhost-startStop-1) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASModelRisk]] Closing Spring root WebApplicationContext
17:37:19,830 INFO (localhost-startStop-1) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASModelRisk]] Cleaning up Log4j resources for context: Model Risk Management Mid-Tier...
17:37:19,830 INFO (localhost-startStop-1) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASModelRisk]] Shutting down all loggers and appenders...
17:37:19,832 INFO (localhost-startStop-1) [org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/SASModelRisk]] Log4j cleaned up.
17:37:19,841 ERROR (localhost-startStop-1) [org.apache.catalina.loader.WebappClassLoader] The web application [/SASModelRisk] appears to have started a thread named [disconnect thread for jabdl3908(27471)<v9>:47666/35271] but has failed to stop it. This is very likely to create a memory leak.
17:37:19,841 ERROR (localhost-startStop-1) [org.apache.catalina.loader.WebappClassLoader] The web application [/SASModelRisk] appears to have started a thread named [SystemFailure WatchDog] but has failed to stop it. This is very likely to create a memory leak.
17:37:19,842 ERROR (localhost-startStop-1) [org.apache.catalina.loader.WebappClassLoader] The web application [/SASModelRisk] appears to have started a thread named [SystemFailure Proctor] but has failed to stop it. This is very likely to create a memory leak.
17:37:19,842 ERROR (localhost-startStop-1) [org.apache.catalina.loader.WebappClassLoader] The web application [/SASModelRisk] appears to have started a thread named [State Logger Consumer Thread] but has failed to stop it. This is very likely to create a memory leak.
17:37:19,857 INFO (localhost-startStop-1) [org.apache.catalina.startup.HostConfig] Deployment of configuration descriptor ../sasconfig/Lev1/Web/WebAppServer/SASServer8_1/conf/Catalina/localhost/SASModelRisk.xml has finished in 67,647 ms
17:37:19,858 INFO (localhost-startStop-2) [org.apache.catalina.startup.HostConfig] Deploying web application directory ../sasconfig/Lev1/Web/WebAppServer/SASServer8_1/webapps/ROOT
17:37:20,611 INFO (localhost-startStop-2) [org.apache.catalina.startup.HostConfig] Deployment of web application directory ../sasconfig/Lev1/Web/WebAppServer/SASServer8_1/webapps/ROOT has finished in 752 ms
17:37:20,626 INFO (main) [org.apache.catalina.startup.Catalina] Server startup in 69325 ms
^C

Re: SASServer1_1 not starting

woo — Mon, 14 Jan 2019 14:06:24 GMT

Also, I am able to define a libname statement successfully from EG (connect as installer id) and from base sas (./sas -nodms - from sas compute server) as installer id.

LIBNAME MYORA Oracle PATH="XXX" SCHEMA="XXX" USER=XXX PASSWORD="{SAS002}XXX";

for EG error, we are having only internal connection happening. user exist on metadata server but somehow profile is not being accepting...at same time, profile with sas installer id connecting fine...

Re: SASServer1_1 not starting

dpage — Mon, 14 Jan 2019 14:42:53 GMT

that error sounds like the Oracle server has changed since the SAS stack was deployed - so the URL we're using isn't valid. We're accessing a SID the server doesn't know about. This is JDBC so it doesn't consume the tnsnames.ora file local to the webappserver, but talks directly to the database server so it must know about the SID. That's different from what you're doing in EG and SAS where it will honor your local tnsnames.ora.

At this point engaging tech support will be the best path forward.

Re: SASServer1_1 not starting

woo — Mon, 14 Jan 2019 15:22:23 GMT

Thank you so much. Really appreciate your input.

Just last check. for EG error, we have only internal authentication happening. however, we don't have users defined under /etc/passwd file, only internal users define there. when I, as user, try to connect metadata server with my eg profile, eg profile throwing "user id or password is not valid" error. does this means my id is not defined the way it should be at OS level?

New client connection (12345) rejected from server port 8561 for user xyz. Peer IP address and port are [::ffff:10.24.30.136]:54581 for APPNAME=SAS Enterprise Guide.

Client connection 12345 closed.

Access denied.

Re: SASServer1_1 not starting

Kurt_Bremser — Mon, 14 Jan 2019 15:36:21 GMT

Purely internal users will have the structure xyz@saspw (similar to sasadm@saspw), because the use the non-default authentication domain "saspw".

The default for DefaultAuth is authenticating against the OS, so for every user-id that does not have an authentication domain attached, SAS will go looking in the OS for the credentials. Also note that internal users are mostly useless in EG, as they can't start a workspace server.

Re: SASServer1_1 not starting

woo — Mon, 14 Jan 2019 15:55:14 GMT

Thank you Kurt. in this case, I am sorry its my bad. actually my installer id is not internal id then as it is defined as "DefaultAuth" for Authentication Domain in smc (like me). And since installer id is clearly defined under /etc/passwd (but not me) it is being authenticated fine.

So looks like, OS/Linux team will have to define my id same way or any other possible way where I can get authenticated as well.

Re: SASServer1_1 not starting

Debi_SAS — Tue, 15 Jan 2019 06:17:51 GMT

For your ID ; Are you able to login (direct SSH) to Linux server using your ID ?? If yes, then can you login to Unix once and try to create a small file (touch abc) in your home path and check if you have permission there. If everything above goes well, then you should be able to connect through SAS EG without any issue.

Regarding your original error, I found "Caused by: java.sql.SQLException: ORA-28001: the password has expired" message from your log . Along with kurt's reply, you can also check if the password defined for the connection ID is correct and not expired.

Regards,
Debi

Re: SASServer1_1 not starting

woo — Tue, 15 Jan 2019 18:08:02 GMT

for EG - I can login as me (as user) on Linux machines and can create directory or files. however, eg connection profile still showing error (userid or password invalid). however, can connect using sas installer id (which is defined at /etc/passwd) but users are not (they are not defined at /etc/passwd)

for oracle issue - we don't have expired password. however, there was error and it was pointing to updating password where we perform updating password (same password through) for compute server and mid-tier server - then everything else came up but not server81 - where we receive oracle error. however, I can connect using sqlplus + oracle libname statement define fine with sas installer id from EG.

Re: SASServer1_1 not starting

Debi_SAS — Tue, 15 Jan 2019 18:42:30 GMT

Thanks Woo.

For EG - If you are able to login to the linux server with your user ID and that ID is not mentioned in /etc/passwd file; then it must be an AD or LDAP ID / or any authentication provider ID . If it is AD/LDAP ID (PAM Authentication), then you need to enable PAM authentication to resolve this issue .

Please find below link for the same.

http://support.sas.com/kb/49/432.html

Re: SASServer1_1 not starting

woo — Tue, 15 Jan 2019 20:42:13 GMT

Thanks Debi. I will take a look.

Re: SASServer1_1 not starting

woo — Thu, 17 Jan 2019 03:05:20 GMT

it worked fine Debi. Thanks a lot.