Unable to connect to workspace server

[vaishnavpranam]

Hi,

I am unable to connect to a workspace server.

I was created a month ago and i was able to connect to it till yesterday.

There have been no configuration changes in system.

When i try to validate the server the screen just hangs an i could only see a small dailog box saying Validating and after that nothing happens.

Please suggest what steps should i take to check this issue.

Thanks in advance !!

[Kurt_Bremser]

Look at the spawner logs (Lev1/ObjectSpawner/Logs).

[vaishnavpranam]

Hi Kurt,

This is a Grid system with 6 nodes.
Do i need to check object spawner logs for each node?

[Kurt_Bremser]

You need to check the spawner log on the physical server(s) that is/are defined for this workspace server.

[vaishnavpranam]

Nothing in the Logs

[Kurt_Bremser]

See if those logs are current. If the latest log file has not been written to recently, it's a sign for a hanging process.

Try to connect to the spawner from SMC and restart it from there. If that doesn't work, you may have to use the commandline to kill and restart the spawner process. Which may reveal a configuration problem that keeps the spawner from starting up.

All the while take care that you are following down the right path (looking on the correct physical server, looking for the correct userid - upcase/lowcase! - and so on), and use the correct userid from the commandline. The spawner runs under the userid of the SAS install user.

[JuanS_OCS]

Hello @vaishnavpranam,

when it stoped working for you, have you changed your password, recently? Is it your account locked, somehow?

If that is the case, your admin should grant you access to change the password on lsf (lspasswd command line). 

Anyway, if it is a workspace you can always debug on several ways:

- If you can, connecting to the OS and launching a WorkspaceServer.sh/.bat session and checking what is there.

- Enabling WorkspaceServer logs, temporarily, and extending info to INFO or even DEBUG

- You can also extend log info on Object Spawner, it will require you to refresh or restart the Object Spawner

- Your system admin can check the PAM logs

- You can check logs on LSF.

I would start just with the question regarding the password than perhaps the 2-3 first questions, it should be enough.

[vaishnavpranam]

Hi JuanS_OCS,

The issue is not with a particular id. None of our users is able to connect that WSServer.

I am considering the ways you specified for debugging.
I just tried to test connect from SMC and i was able to make a connection to node1,2,3,4, & 6, But my node 5 was not responding.

I am attaching the screenshot for your reference.

Thanks !!

[JuanS_OCS]

Good idea.

It seems to me that this node has lost a configuration, or perhaps you just need to restart the Object Spawner on that machine. You will need to, anyway, because of the change on logging config. 

[vaishnavpranam · Accepted solution]

Did not change the logging.
Just restarted the Object Spawner and the issue is resolved.

Thanks !!

[JuanS_OCS]

Very glad to read! 🙂

[eirigoyen]

HI,
I know this topic is a couple of months old, but we have been experiencing the same issue since December 6, intermittently, and we are getting around by restarting the ObjectSpawner in our compute machines, 4 in a grid environment. Is there a known root cause for this behavior to occur? We contacted SAS tech support and we were told that we need to install 2 hot fixes (N22016 and N27004) we are running SAS 9.4 M1. We are scheduled to apply the hot fixes this weekend, and we hope this will correct the issue with the spawner.

Thank you.

[JuanS_OCS]

Hello @eirigoyen,

Please let us know how it works after the hotfixes application. Every case might be different. I have seen different kind of scenarios that this might happen, on GIR and no GRID environments.

Is your environment windows or linux based? How many connections are in the pool for your SAS Stored Process servers on each SASApp?

On my experience, there is a small bug in SAS 9.4M1 (and further if I am correct) and it should be more stable on later versions. But I have also seen that if you increment the number of connections on the pool for the STP servers it helps to make Object Spawners more stable. The reason is that long or too often STP sessions can make the STP server to crash, eventually, and this can cause, in some scenarios, a domino effect that makes crush the Object Spawner itself. Hence, extending the pool and some other settings in the STP servers (with no hotfixes)  can reduce, in some scenarios, availability from 75% to 99%. 

[eirigoyen]

Hi @JuanS_OCS,

Thank you very much for your reply, I am a fairly new SAS Admin and I am learning as I go :-). We are running on Linux  RHEL 6.7. I will post the results after our hot fix patch.

Cheers.