Object Spawner instance in 9.4

[BStone]

Guys,

Just a confirmation required for the following please:

In a Grid 9.4 deployment and having SAS Stored process, Workspace, Logical Workspace servers set up as Grid Launched, where will the Object Spawner sit?

1. Will the Ojbect Spawner sit on the Grid Control Server

2. Will the Object Spawner sit on each Grid node

To me it almost makes sense to have the Object Spawner only on the Grid Control Server, since the Grid Manager will start a ... server session.

If you only have the Workspace server cofigured as Grid Launched with the exception of the Stored Process Server and Logical WS, only then I could see having the Object Spawner on each Grid node.

Any thoughts?

Thanks in advance...

[jakarman]

You are referring to Installing and Configuring a SAS Grid Environment. As you asking for technical backgrounds, let us go for the technical backgrounds.

The grid-environment is really build on LSF, That us a scheduler with Load balancing.

It should run sas-processes (sas.exe), so for sure on each node sas code (Foundation) must be there.

The architecture of Grid is showing more Scalability Community: SAS Grid Architecture   

You will need a SAS/connect service running on each machine. Starting with 9.3 the configuration of that has got more approaches. The whole set of parameters can be done in the configuration part of the SAS installation ( .../Lev-/SASApp/... ) in the same way as the batch-server. (scripting/files no service).All kind of parameters being stored in the SAS metadata. Before 9.3 all of that could be done with only OS-scripting starting the service indepently of the SAS metadata server.

The workspace server is only shown at the control server. As that one is needing the objectsspawner that one must also running on the same server.

Remember the process is that the object-spawner will fork new processes and than change the user-context of the new process. That is why sasauth needs to be owned/running by root and having the suid bit set.   LSF has the same kind of technical approach, a masterprocess will fork new ones and change the usercontext.     

[BStone]

:, thanks for your response.

However, it's still not clear to me unfortunately. In 9.4 you can configure the Workspace server to be grid enabled - meaning the service will be started by the Grid Manager and not the Object Spawner. This is why I want to know if the Object Spawner and Workspace Server would exist:

1. Only on the Grid Control Server OR

2. Across all nodes OR

3. ... is there is a Master instance of the Object Spawner and WS on Grid Control Server and a slave/light version of the Object Spawner and WS on the other nodes?

Point 1 or 3 for me would make sense, but I'm not too sure and therefore welcome any comments.

Cheers.

[jakarman]

Think on the forked process by the object spawner using the sasauth.

You need that one having started. The only process capable doing that is the objectspawner.   (see figure link  architecture in control-server WS)

Running the grid

That is needing a process, master process, running that will communicate to the grid. (WS visible at control server) 

LSF And SAS/connect processes will communicate with that to execute the request. Both are not requiring a SAS objectspawner process to be run .

The sas/connect will require a Connect-spawner.      I would go for 1/ with that argumentation.

Planning the work, it really does not make much impact with your question.

As you are needing a batch-server and connectserver you must set up the configuration directories.  That is the most of the work to do.   

[BStone]

Dank je Jaap

.. makes sense if the OS and WS only exist on the GCS.

[jchaionn]

Hi BStone,

I have a similar concern as we are building our environment as well and want to take advantage of Grid Launched Workspace Servers, but there is no clear instructions on how to set it up.  Did you ever clarify how many Object Spawners are required?  I was thinking that one is all this needed because the Grid should start the workspace server on the remote nodes if they are to be manageable by LSF.  At least that is what makes sense to me.

[BStone]

, unfortunately I never got a full confirmation of where it sits, I therefore have to go with my assumption that the Object Spawner only exist on the GCS. Technically, I do not see why it will sit on each node if LSF is chosing the best host and starting a WS on that host.

[jakarman]

Bstone, remember those figures of the grid concepts.

Scalability Community: SAS Grid Architecture
There is not WS (SAS workspace service) shown on the other nodes. A WS is started by the objected spawner, that combination is mandatory by design.

LSF is not able to do anything on those specials of SAS, LSF is a IBM product. IT can make use of OS scripts and that is what a batch server, connect and base can do. SAS can name them servers (as seen in SMC), but it are just OS scripts. 

[BStone]

Agree, this is why I'm stating that the Object Spawner sits on the Grid Control Server where the WS Server sits.

Take "Defining Multiple Application Servers" in the Intelligence Platform Admin Guide as an example. It states the following:

"If you are adding logical servers that require an object spawner (one of the workspace servers and the stored process server), then when you deploy your SAS Application Server, the wizard also deploys an object spawner to start the servers. If the machine on which you are adding the SAS Application Server already contains an object spawner, then the wizard updates the pre-existing spawner definition for you to include the new servers that you want your spawner to manage"

Remember we are talking about Grid-Launched WS Servers, in other words an application request a WS session from the Object Spawner, the Object Spawner pass the request to LSF, LSF (IBM) executes the /WorkspaceServer/WorkspaceServer.sh script. This is why the jobs appears as LSF jobs.

[jakarman]

I would guess LSF is not using the workspaceserver.sh script but instead that batchserver.sh script on the other machines.
Not a big difference as that workpspaceserver script will work with a X11  server (validating/debugging) as will that other one.
Just a setup difference with naming and possible different settings in all those config files.   

[jchaionn]

I have only been playing with this configuration so I don't know for sure if this is correct yet or not.  But what I did was this:

1. Setup clustered filesystems between all of my workspace server (5 of them)
   1. /opt/sas/sashome
   2. /opt/sas/config
   3. /opt/sas/grid/lsf
2. Installed Platform LSF according to SAS docs
3. Installed Metadata Server Cluster according to SAS docs
4. Installed and configured SAS on WKS1 with SASApp context, Workspace Server, Stored Process Server and Object Spawner
   1. Didn't do anything with the others (2-5)
5. In SAS Management Console, I modified the SASApp - Workspace Server Properties to include 4 additional servers
   1. Right-clicked on SASApp - Logical Workspace Server and converted to Load Balancing
      1. Balancing Algorithm: Grid
      2. Checked "Launch servers via Grid"
6. In SAS Management Console, I modified the SASApp - Stored Process Server Properties to include 4 additional servers
   1. Go to SASApp - Logical Stored Process Server Properties -> Load Balancing tab
      1. Balancing Algorithm: Grid
      2. Checked "Launch servers via Grid"
7. Restart SAS processes on WKS1

Now when I use EG to connect to my environment, I get a new workspace server session on any of the 5 WKS servers.  I can see it balancing the load and my Management Console settings of SASApp visually look very similar to what I found in this article: http://support.sas.com/resources/papers/proceedings14/SAS375-2014.pdf

So, I actually only have one object spawner running on WKS1 but the Metadata server is aware of 5 possible servers that are tied to SASApp and my Grid Control Server is also aware of these 5 servers.

What I believe is happening is EG is communicating with the Metadata server, authenticating, then sending me to the object spawner on my WKS1 server.  The object spawner tells the GCS that it needs a Workspace server and then Grid launches one on the least busy server and passes control of that back to EG.

Jaap, I think is describing how grid worked prior to the introduction of grid-launched workspace servers.

BStone, does my scenario match what you have?

[BStone]

yes that's seems close enough.

Our's are designed slightly different/complexed. Using the same scenario as you have with additonal workspace servers based on additional App Servers that is required for the project. However, it still comes down to X1 Object Spawner per App Server and users will only ever see one Workspace Server, unless they belong to another group that is linked to a different App Server.

Hope my explanantion is not too confusing

[jakarman]

jchaionn, Your descritption is exactly matching my previous description. As you have used the instructions there is al lot done without telling you exactly what is done.

At some point My assumptions is wrong. it not the batchserver but there is a nice dedicated griderserver folder with the sasgrid scritps (page 4) in your doc.

All LSF queue/job handling is visible although it might be confusing the made an interface to see all of LSF in the SMC.

The naming convention SAS is using in the SMC is confusing as what the are calling a server is mostly a logical service sometimes just being only some script.

Bstone may be your text is confusing but I understand your set up is having an central metdataserver and segregated computer severs each having a objectspawner and as workspacserver. That is a nice setup for splitting and isolating work over a lot of small sized machines.    

[doug_sas]

The object spawner can sit on any machine in the grid. The object spawner can grid-launch servers to any other machine on the grid provided

1. The machine has SAS installed (kinda obvious)
2. The installed SAS has Integration Technologies licensed (to be able to run as a workspace/stored process server)
3. The machine has been configured to run workspace servers
4. The machine is listed as a server component under the logical workspace/stored process server

You can have one object spawner or, if you want high availability, multiple load-balanced object spawners.

[jakarman]

Doug as lsf (that ibm product) is doing the load balancing in the grid, why should there by sas ws servers directly started and disturbing what lsf is managing?