SAS Server unresponsive

[mmmAol]

Hi,

We have a sas machine(unix machine) that gets rebooted every saturday. Just after reboot, the sas server is not accesible. That is client can not connect to the sever.

Looking through error logs, I found following:

The TCP/IP tcpSockConnect() support routine failed with error 111 (The connection was refused.).

ERROR [00000008] sas - The Bridge Protocol Engine Socket Access Method was unable to connect to host myserver1, port 1234.

ERROR [00000008] sas - A connection refused usually indicates the target server is not active.  Or, if active, it may be listening on a different port or service.

ERROR [00000008] sas - Unable to connect to the metadata server.

ERROR [00000008] sas - Unable to connect to the metadata server.

ERROR [00000004] sas - Unable to initiate the IOM run-time.

We run the sas startup script when servers/services become unresponsive. Once we the run the start up script, users are able to connect to the sas servers. 

I was wondering if there is any settings or installation issue that we may need to address. I was wondering if someone can shed some light.

[PaulHomes]

Hi,

I wonder if there's an issue with the start up ordering of the various SAS services.  Can you provide a bit more info? Are your SAS services all on the 1 machine or spread over multiple machines? How have the SAS services been configured by the UNIX sysdmin to start during server boot - a call to the sas.servers script or calls to individual service scripts (metadata server, object spawner, olap server etc)? When/how is your Java app server started in relation to the SAS services? If spread over multiple machines, how are they being coordinated to start in the correct order? In which particular SAS service log file are you looking where you see this error? Is your metadata server really on host myserver1 and port 1234 or was that changed to anonymize the log?

Cheers

Paul

[mmmAol]

Thank you Paul for quick follow up. Please find answers below.
Q) Are your SAS services all on the 1 machine or spread over multiple machines?

A) 1 machine

Q) How have the SAS services been configured by the UNIX sysdmin to start during server boot - a call to the sas.servers script or calls to individual service scripts

A) Call to sas.servers script

Q) When/how is your Java app server started in relation to the SAS services?

A) I don't know. if you can tell me where to look, i will be able to give you answer.

We have one machine that does the actual computation and the other one is metadata server.  You are right I have removed host and port# to anonymize the message post here.

I did a grep on "cat /sas92/config/Lev1/Logs/*.log. | grep '2012-01-28' | grep -ni error"

Looking for errors on all the log files under that directory.

[flredsterling]

I suggest you look at the UNIX/Linux start up script you are using. Keep in mind that when this script runs it should be running as the SAS installer ID and also realize that no environment has been created; check that full paths are being used for all commands. Your UNIX sysadmin should have it right, but it wouldn't hurt it check. It would be interesting to run a "status" before running a "start", though I would guess that it would show none of the servers to be running.

-Mark-

[PaulHomes]

Hi,

Although you mention you only have 1 machine you later mention that compute work is done on one and metadata is available on another.  That sounds like a 2 machine config with dedicated app and metadata servers.  With that type of config I would confirm with your sysadmin that when they reboot the servers the app server doesn't get started until the metadata server machine has started and the metadata server has fully initialized.

You may or may not have a Java app server depending on the SAS products you have.  Do you use any of SAS Web Report Studio, SAS Information Delivery Portal, SAS BI Dashboard?

It would be handy to know which log file contains the error so you know which service is failing.  This is a command I can use on my Linux server to grep all the logs for a specific day looking for the word error but also printing out the name of the file it occurs in and the line number.  Not sure if your UNIX flavor supports all these options but, if not, the man pages might have alternatives.

find /opt/sas92/ebiedieg/Lev1 -name '*2012-01-28*.log' -exec grep -Hni error {} \;

Cheers

Paul