SAS Mid-Tier WebApplication Server Dependencies - Restarts and Web App Issues

[mrtball]

Hello,

Since moving to 9.4 and the SAS Web Application Server architecture we have an issue where if we restart services/servers via BAT scripts (in order as per http://support.sas.com/documentation/cdl/en/bisag/68240/HTML/default/viewer.htm#p0d9d5nzmd8i4yn1usv2...

And SAS web app servers in the order

1. SAS [Config-Lev5] SASServer1_1 - WebAppServer
2. SAS [Config-Lev5] SASServer2_1 - WebAppServer
3. SAS [Config-Lev5] SASServer3_1 - WebAppServer
4. SAS [Config-Lev5] SASServer4_1 - WebAppServer
5. SAS [Config-Lev5] SASServer12_1 - WebAppServer

We run into an issue where CI studio does not start - this is only fixed by then manually restarting SAS [Config-Lev5] SASServer3_1 - WebAppServer.

Just wondering if anybody know of what a potential cause of this might be? And also as these are Web App Server services could it be that we don't have the correct dependencies in place? And whether during our restart's could i be a case that the SAS [Config-Lev5] SASServer1_1 - WebAppServer is not started correctly in TomCat?

This seemingly only affects SAS [Config-Lev5] SASServer3_1 - WebAppServer (CI Studio Web App Server). And doesn't affect the VA server SAS [Config-Lev5] SASServer12_1 - WebAppServer.

This is on Windows Server 2008.

Thanks,

[jag_SAS]

You mention CI Studio in your post, but SASServer3 manages web applications in the Finanacial Management solution suite. Customer Intelligence web applications are managed under SASServer6, which I don't see in your list. The two solutions do not depend on each other in any way that I am aware so perhaps SASServer3 failing to start correctly is preventing SASServer6 from starting. However, you don't seem to have a SASServer6 so we need to get to the bottom of what solutions you have deployed on this server. To that end please post back the contents of the status.xml file you have under Lev5/ConfigData on your middle tier. That will give us some idea of exactly what components are deployed on your middle tier. 

SASServer12 starting correctly makes sense to me only because I know the SAS web application server startup script has this wierd thing where it starts 1 then 12, 13 14 ... before it moves on to 2, 3, 4. I'm not sure if it still does that in SAS 9.4M4 (started shipping in November), but it did in earlier 9.4 maintenance releases. 

There can be many reasons why SASServer3 is not starting correctly so I'm going to give you many things you can investigate.

You inidicate you see this when you use the .bat file to restart the web application servers. Are you certain this is the only time you see this and that the issue does not occur when the server/machine is rebooted?

When this occurs what errors is the SASServer3 log reporting? The log I'm referring to should be under Lev5/Web/WebAppServer/SASServer3_1/logs. SASServer1 could be reporting issues as well. 

Financial Management stores some of its content in a database on your compute tier. I don't have the name of that service offhand, but you might want to see if restarting all of your SAS processes produces the same experience or is it only after some time has passed and then you restart only the SAS web application servers?

Is Financial Management still licensed on the system? Could it be that the license for this solution has expired or was terminated, but the solution was not unconfigured? That could cause startup problems for SASServer3 an SASServer4, but I would expect you to see the same problem when the machine reboots and not just when running the script. 

As you can see there are a lot of things to investigate. SAS Technical Support is going to be your best option to reach out to for the type of indepth investigation you need. I'm happy to continue providing tips and things I would investigated, but that approach could take us a lot longer than just reaching out to them. 

[mrtball · Accepted solution]

Hello Jag,

Thank you for your response, in the time that passed since I logged this - I've identified some changes that have rectified this behaviour for us. These were.

1. In our Windows Batch scripts SAS service stop / start 's commands were changed from 'net stop'/'net start' to 'sc stop'/'sc start'  - Investigations identified that SC is more reliable than NET.

2. Our SAS [Config-Lev4] Web Infrastructure Platform Data Server resides on a different server to the Web or Mid-tier so we were not necessarily stopping / starting as per SAS' advised order (immediately before we stopped the metadata server). The SC provides the functionality to stop services on other servers over the network which enables us to stop all the service exactly how SAS recommends.

In the 2-3months we have been using batch scripts with these changes we have not run into any issues.

Thanks,
Tom