topic Re: SAS Web App Server unable to start in Administration and Deployment

SAS Web App Server unable to start

MariaD — Thu, 16 Mar 2023 16:59:48 GMT

Hi Folks,

We have a Linux server running SAS 9.4M7. We applied the new license renewal 10 days ago on our server. On Monday, when the user tried to open SAS Visual Analytics appears 404 error. After that error, we restarted all SAS services. We checked that no were pending processes before starting the services. All services started ok except for SAS Web App Server (Server1_1, Server11_1, Server12_1, Server2_1, Server7_1, Server8_1).

We even restarted the server physically, but the same situation occurred. We can't find any error in the logs.

We have the same server, on Test environment. On that environment, we didn't apply the license yet. So, we applied the license on it and restarted the services. To our surprise, the same error occurs. The SAS Web App Server was unable to start. All other services, like JMS, Cache Locator, Web Server, and Environment Manager appear to be fine.

Any ideas or suggestions? We already contact Tech Support, but until now we can't solve it and we don't have the SAS available.

Regards,

Re: SAS Web App Server unable to start

Nigel_Pain — Fri, 17 Mar 2023 09:25:24 GMT

Hi Maria

Just to ask the obvious question, have you updated the SID in the metadata? This is something you need to do as well as renewing the licence, and covers the middle-tier products. It's an option in SAS Deployment Manager:

Re: SAS Web App Server unable to start

MariaD — Fri, 17 Mar 2023 12:33:49 GMT

Hi @Nigel_Pain ,

Yes, we updated the SID file in the metadata too.

Regards,

Re: SAS Web App Server unable to start

411752 — Fri, 17 Mar 2023 13:10:51 GMT

What does your Webserver log and WebApp server(SAServer1_1) log say, You should get a good clue looking at these both logs. Please attach them here ,if possible.

Re: SAS Web App Server unable to start

MariaD — Fri, 17 Mar 2023 13:53:02 GMT

Hi @411752 ,

I've just looked again on .../Lev1/Web/WebAppServer/SASServer1_1/logs and there isn't any new log generated after restarting the services.

On ../Lev1/Web/WebServer/logs, the log called error_2023-03-17-12.13.log only have the lines below:

[Fri Mar 17 12:13:46.962498 2023] [mpm_event:notice] [pid 21601:tid 139623033902912] AH00489: Apache/2.4.43 (Unix) mod_bmx/0.9.6 OpenSSL/1.0.2u-fips configured -- resuming normal operations
[Fri Mar 17 12:13:46.962619 2023] [core:notice] [pid 21601:tid 139623033902912] AH00094: Command line: '/sas/software/SASWebServer/9.4/httpd-2.4/bin/httpd -d /sas/config/Lev1/Web/WebServer'

If you need any other log, please let me know.

Re: SAS Web App Server unable to start

StefanM — Fri, 17 Mar 2023 14:23:14 GMT

Did you try it directly?
/sso/biconfig/940/Lev1/Web/WebServer/bin/httpdctl status|start

Re: SAS Web App Server unable to start

MariaD — Fri, 17 Mar 2023 14:27:41 GMT

Hi @StefanM ,

Yes, the WebServer is running.

Regards,

Re: SAS Web App Server unable to start

StefanM — Fri, 17 Mar 2023 14:35:01 GMT

And the same with tomcat (Server1)?

/sso/biconfig/940/Lev1/Web/WebAppServer/SASServer1_1/bin/tcruntime-ctl.sh status|start

Re: SAS Web App Server unable to start

411752 — Fri, 17 Mar 2023 14:42:13 GMT

@MariaD If all shows running fine and no errors on logs, I doubt some issues with Cache locator (on your primary compute and primary midtier) . You need to stop Cache locator on primary grid and stop all services on midtier nodes and restart.

Stop Cache locator on both nodes and delete below file and directory /opt/sas/config/Lev1/Web/gemfire/instances/ins_41415/ConfigDiskDir_

/opt/sas/config/Lev1/Web/gemfire/instances/ins_41415/locator41415state.dat

Re: SAS Web App Server unable to start

MariaD — Fri, 17 Mar 2023 14:45:03 GMT

Hi @StefanM

thanks, I'll tried but the following error appear:

Am I doing something wrong?

Re: SAS Web App Server unable to start

StefanM — Fri, 17 Mar 2023 14:53:34 GMT

It should work like this, independant of the other idea with gemfire/cache locator.

I would use the trace:
sh -xv ./tcruntime-ctl.sh status

Re: SAS Web App Server unable to start

MariaD — Fri, 17 Mar 2023 15:12:36 GMT

Hi @StefanM ,

Follows the results:

[sas@sas bin]$ sh -xv ./tcruntime-ctl.sh status
#!/bin/sh

# ---------------------------------------------------------------------------
# Copyright (c) 2010-2020 VMware, Inc. or its affiliates. All rights reserved.
# ---------------------------------------------------------------------------

##
# This version calls the top-level tcruntime-ctl.sh
# script
##

INSTALL_BASE="/sas/software/SASWebApplicationServer/9.4"
+ INSTALL_BASE=/sas/software/SASWebApplicationServer/9.4

os400=false
+ os400=false
darwin=false
+ darwin=false
case "`uname`" in
CYGWIN*) cygwin=true;;
OS400*) os400=true;;
Darwin*) darwin=true;;
esac
+ case "`uname`" in
uname
++ uname

# resolve links - $0 may be a softlink
PRG="$0"
+ PRG=./tcruntime-ctl.sh

while [ -h "$PRG" ] ; do
ls=`ls -ld "$PRG"`
link=`expr "$ls" : '.*-> $.*$$'`
if expr "$link" : '/.*' > /dev/null; then
PRG="$link"
else
PRG=`dirname "$PRG"`/"$link"
fi
done
+ '[' -h ./tcruntime-ctl.sh ']'

PRGDIR=`dirname "$PRG"`
dirname "$PRG"
++ dirname ./tcruntime-ctl.sh
+ PRGDIR=.
PRGDIR=`cd "$PRGDIR"; pwd -P`
cd "$PRGDIR"; pwd -P
++ cd .
++ pwd -P
+ PRGDIR=/sas/config/Lev1/Web/WebAppServer/SASServer1_1/bin

#Guess instance name
BASEDIR=`cd "$PRGDIR/../." ; pwd -P`
cd "$PRGDIR/../." ; pwd -P
++ cd /sas/config/Lev1/Web/WebAppServer/SASServer1_1/bin/../.
++ pwd -P
+ BASEDIR=/sas/config/Lev1/Web/WebAppServer/SASServer1_1
INSTANCE_NAME=`basename "$BASEDIR"`
basename "$BASEDIR"
++ basename /sas/config/Lev1/Web/WebAppServer/SASServer1_1
+ INSTANCE_NAME=SASServer1_1

#Absolute path
PRGDIR=`cd "$PRGDIR/../../." ; pwd -P`
cd "$PRGDIR/../../." ; pwd -P
++ cd /sas/config/Lev1/Web/WebAppServer/SASServer1_1/bin/../../.
++ pwd -P
+ PRGDIR=/sas/config/Lev1/Web/WebAppServer

#Set instance base if it's not already set
if [ -z "$INSTANCE_BASE" ]; then
INSTANCE_BASE="$PRGDIR"
export INSTANCE_BASE
fi
+ '[' -z '' ']'
+ INSTANCE_BASE=/sas/config/Lev1/Web/WebAppServer
+ export INSTANCE_BASE

EXECUTABLE=tcruntime-ctl.sh
+ EXECUTABLE=tcruntime-ctl.sh

PROGRAM=""
+ PROGRAM=

#first check install base

# Check that target executable exists
if [ -d "$INSTALL_BASE" ]; then
if [ -x "$INSTALL_BASE"/"$EXECUTABLE" ]; then
PROGRAM="$INSTALL_BASE"/"$EXECUTABLE"
elif $os400; then
# -x will Only work on the os400 if the files are:
# 1. owned by the user
# 2. owned by the PRIMARY group of the user
# this will not work if the user belongs in secondary groups
eval
PROGRAM="$INSTALL_BASE"/"$EXECUTABLE"
else
echo "ERROR Cannot find $INSTALL_BASE/$EXECUTABLE or it's not executable."
fi
fi
+ '[' -d /sas/software/SASWebApplicationServer/9.4 ']'
+ '[' -x /sas/software/SASWebApplicationServer/9.4/tcruntime-ctl.sh ']'
+ PROGRAM=/sas/software/SASWebApplicationServer/9.4/tcruntime-ctl.sh

if [ -z "$PROGRAM" ]; then
if [ ! -x "$PRGDIR"/"$EXECUTABLE" ]; then
echo "ERROR Cannot find $PRGDIR/$EXECUTABLE or it's not executable."
echo "ERROR One of these files is needed to run this program"
exit 1
else
PROGRAM="$PRGDIR"/"$EXECUTABLE"
fi
fi
+ '[' -z /sas/software/SASWebApplicationServer/9.4/tcruntime-ctl.sh ']'

if [ "verbose-status" = "$1" ]; then
echo "Derived instance name: $INSTANCE_NAME"
echo "Executing script: $PROGRAM"
fi
+ '[' verbose-status = status ']'
exec "$PROGRAM" "$INSTANCE_NAME" "$1" "$2"
+ exec /sas/software/SASWebApplicationServer/9.4/tcruntime-ctl.sh SASServer1_1 status ''
Invalid sytax.

Thanks,

Re: SAS Web App Server unable to start

MariaD — Fri, 17 Mar 2023 15:13:44 GMT

Hi @411752 ,

I also tried cleaning the file and directory of the Cache Locator but nothing changes.

Regards,

Re: SAS Web App Server unable to start

StefanM — Fri, 17 Mar 2023 15:21:27 GMT

And this script (will not work even if all fine this way):

sh -xv /sas/software/SASWebApplicationServer/9.4/tcruntime-ctl.sh SASServer1_1 status ''

Re: SAS Web App Server unable to start

411752 — Fri, 17 Mar 2023 15:22:30 GMT

@MariaD : Sure, those are the basic troubleshooting steps and I would wait for Tech Support for any other methods.

Please update us here once you find the solution.

Re: SAS Web App Server unable to start

MariaD — Fri, 17 Mar 2023 15:25:59 GMT

Hi @StefanM ,

Nope, same thing:

[sas@sas bin]$ sh -xv /sas/software/SASWebApplicationServer/9.4/tcruntime-ctl.sh SASServer1_1 status
#!/bin/sh

# ------------------------------------------------------------------------------
# Copyright (c) 2021 by SAS Institute Inc., Cary, NC USA 27513
# ------------------------------------------------------------------------------
#
# Invoke Tomcat's catalina shell script to control the web application server.
#
# ------------------------------------------------------------------------------

# Read the instance name
if [ "$#" -gt 0 ]; then
name=$1
shift
else
echo "Invalid sytax."
exit 1
fi
+ '[' 2 -gt 0 ']'
+ name=SASServer1_1
+ shift

# Read the command
if [ "$#" -gt 0 ]; then
command=$1
shift
else
echo "Invalid sytax."
exit 1
fi
+ '[' 1 -gt 0 ']'
+ command=status
+ shift

# Read the path to the instance
if [ "$#" -gt 1 ]; then
if [ "$1" = "-n" ]; then
# Shift off the recognized command switch
shift
# Remember and shift off the template name
instancedir=$1
shift
else
echo "Invalid syntax."
exit 1
fi
else
echo "Invalid sytax."
exit 1
fi
+ '[' 0 -gt 1 ']'
+ echo 'Invalid sytax.'
Invalid sytax.
+ exit 1

Re: SAS Web App Server unable to start

StefanM — Fri, 17 Mar 2023 15:26:13 GMT

Something is really strange, stepping one to another might show at some point the root cause.

Re: SAS Web App Server unable to start

StefanM — Fri, 17 Mar 2023 15:37:19 GMT

export INSTANCE_BASE=/sas/config/Lev1/Web/WebAppServer

/sso/sfw/sas/940/SASWebApplicationServer/9.4/tcruntime-ctl.sh SASServer1_1 status ''

should work. '' is for empty params
Our tcruntime-ctl.sh looks different

Re: SAS Web App Server unable to start

StefanM — Fri, 17 Mar 2023 15:39:41 GMT

Error, your path (sso is ours)

/sas/software/SASWebApplicationServer/9.4/tcruntime-ctl.sh SASServer1_1 status ''

Re: SAS Web App Server unable to start

StefanM — Fri, 17 Mar 2023 15:54:29 GMT

I am out as the installation is different than ours.

Looks like to start the tomcat server you need a 2nd parameter:
"Read the path to the instance"