11-22-2017 01:36 PM
Our users are encountering this error message when attempting to connect to SAS VA Hub.
HTTP Status 500 - GemFire on <HostName>(7959)<v18>:36179/45490 started at <datetime>: Message distribution has terminated.
Any help would he greatly appreciated.
04-12-2018 04:50 PM
Hi @Alok_Pal, did you found a resolution for this issue? My users are having the same exact issue.
04-13-2018 06:08 AM
ensure the health of your cache locator and JMS services and that they start in the right order and they won;t go down automatically after a few minutes started.
Reasons might be many: log too large (solution: clean it), in proper start up order (solution: start them in the right order), memory problems (solution: analyze and solve the Jvm problem), several network addresses/cards (solution: bind the service to a single IP address), etc
04-13-2018 10:47 AM
Hi @JuanS_OCS thanks for your reply. In regards to services I do run a batch file that does loads the services in order and will not load a services if the previous one is not loaded. Also in my situation server was running for past month and nothing has been changed (network, large files...) it is my Dev environment and hardly being used at this time just for testing. I am digging into log files to see if I can discover something. To fix the problem I just stopped and restarted the services and it solved the issue but I want to understand why! Don't want this to happen in our prod environment.
thanks again for your input
04-13-2018 10:50 AM
it seems to me as you do not check dependencies, or not properly, in your script. I would say that some services are starting before some previous services do start. Is your server a unix one? Then better use sas.servers script.
04-13-2018 11:15 AM
Hi @JuanS_OCS - I am in Windows environment and I do check for dependencies. When I was creating the scripts I noticed that. Services indicated they are started but actually they were not running in full yet. Also, for reasons (security patches...) I have restarted servers and each time scripts stopped and started the services in orders and for past 7 months never had any issues. This just came out of blue and being new in SAS I really want to understand and find out Why. Thank you again.
04-15-2018 04:35 AM
as you might be aware, the gemfire, as any other web app server, is a JVM (a Java Virtual Machine), which has some known memory boundaries, as Heap sizes and Garbage Collection loads. If you would analyze the service with SAS Environment Manager or a JMX Console, you could have some clues on what goes wrong.
I can think on several options: handled log too large (solution: change configuration to rollover smaller logs), GC is too high/reaching heapsize limit (solution: further analysis of logs to know more about it)... etc.
In the end and in short, the best way to understand your problem is to analyze your logs (increasing the info level to debug level if required) and the status of your memory. The causes could be many.