BookmarkSubscribeRSS Feed
Manoj07k
Obsidian | Level 7

Guys , We are approaching physical server end of life. so we have a thought of expanding the cluster by add new node and shift the services from existing server.

 

We have 9.4 M6 Grid with metadata clustered on 3 nodes , Web on 2 nodes (load balanced) and compute nodes . I just wanted to understand is the idea is feasible and if any supporting documents we have in SAS.

 

Thanks in advance.

5 REPLIES 5
Sajid01
Meteorite | Level 14


As a general rule, if your license permits you should be able to add another node to the cluster.

Alternatively in a three node cluster, two nodes complete a quorum. Shut down one node and replace with a new machine. 
Your specification of your grid architecture is not very clear. If you can mention it clearly we can give a better idea, (Metadata, midtier, grid control and grid nodes - these need to be specified)

 

Manoj07k
Obsidian | Level 7

Appreciate  your response.

 

We have below arch now with 11 nodes ( 1 to 11 ) , we want to add nodes 12 to 22 to the cluster and move the binaries and config, since node 1 to 11 is approaching End of life. Once we moved to 12 to 22 we will drop 1 to 11 nodes.

 

Node NamePurpose of Node
Node1Primary Grid Control server Compute
Node2Secondary Grid Control server Compute
Node3Master   Metadata
Node4Failover   Metadata  + Compute
Node5Failover   Metadata  + Compute
Node6Web Tier node 1
Node7Web Tier node 2 ( Load Balanced)  + Compute
Node8Compute node
Node9Compute node
Node10Compute node
Node11Compute node

 

 

 
Node NamePurpose of Node
Node12Primary Grid Control server + Compute
Node13Secondary Grid Control server + Compute
Node14Master   Metadata node 1
Node15Failover   Metadata node 2 + Compute
Node16Failover   Metadata nod3 + Compute
Node17Web Tier node 1
Node18Web Tier node 2 ( Load Balanced) + Compute
Node19Compute node
Node20Compute node
Node21Compute node
Node22Compute node

 

 

We use SAS 9.4 M6 with Linux 7.1. 

 

Also , we have LSF and SPDS installed in the cluster.

 

Let me know your thoughts. 

 

Thanks.

huntdoug
SAS Employee

As a general rule do not put any other SAS server or share the server but with one metadata server.  The metadata server should have it's own system.  If using a metadata server I would recommend anyone server be as big as all 3.  When the cluster is down 1 node it's really down half - 1 slave and 1 master.  The slaves handle all the work.  The master updates metadata (rw) and the slaves(ro) handle the users connections and read only queries.  So running 1 metadata slave means it does the work of 2 and the master still does the read-write work.  Consider the CPU/Memory/IO that the 3 do.  I'd triple it.  I'd give lots of cpu cores so that the Metadata Server has the most threads to work with as this determines by algorithm how wide that is.  There still can be a bottleneck if you have a user running a long read that may have to finish before the update can be done so that there are no dirty reads.   Honestly a single metadata server node flattens the hierarchy and latency in my mind over a cluster of metadata servers.  A cluster complicates administration.  A cluster is something one must be robust with.

Sajid01
Meteorite | Level 14

Thanks @Manoj07k 

1. Basically you are replacing your existing setup with a new environment. It will be simpler for you to leave the existing arrangement as it is and build new server setup. Once the new one's have been tested and operational the older ones can be decommissioned. Building a new system will help you rationalize your architecture and may help bring down the costs. You have the opportunity to remove any bottlenecks and trouble spots.

2. I see that your metadata and middle tier (Web) are also serving as compute nodes. I would not do this.
3. If I   had the choice, I would prefer to separate the entire setup  into three logical groups each with a distinct role.( Make your own judgement to see if this works for you.)

  • Group 1: The Metadata tier :Existing nodes 3,4 and 5. Will function as Metadata Servers only. Remove the compute functionality from these server. They will be clustered.
  • Group 2:The middle tier : The existing 6 and 7. I will add one more and remove the compute functionality. Clustered. My understanding is that pure middle tier with no compute activity would not add to licensing cost.(SAS employees please enlighten on this. And Manoj you need to verify this.)
  • Group 3: The Compute tier : The Grid Control Servers and the grid nodes. My thinking in a true Grid would not need a secondary grid control server.( May be good to have). If the Grid Control server fails, a grid node should take over.  If I am the decision maker, would do away with the secondary Grid Control server.

4. If you do not want to follow the approach of building a new system, You can do the replacement in phase wise manner . In the first phase have one compute tier, one metadata server and the secondary grid control server. As for the Web tier there are only two , you have to make the judgement..
However this approach has the issue of integrating the newly built servers with existing one's. I would like to avoid this. scenario.

5.My choice would be a new rational system  with just the right number of servers .However  I do not know your requirements and constraints. Make you own judgement.

 

Manoj07k
Obsidian | Level 7

Thanks @Sajid01 for expressing your thoughts clearly .

 

I would also love to do a fresh deployment. But we're just thinking of different ideas of how to reduce the timelines and cost of deployment.

 

As you said, the best approach is to keep meta and mid tier dedicated but that's the project requirements 🙂 

 

suga badge.PNGThe SAS Users Group for Administrators (SUGA) is open to all SAS administrators and architects who install, update, manage or maintain a SAS deployment. 

Join SUGA 

Get Started with SAS Information Catalog in SAS Viya

SAS technical trainer Erin Winters shows you how to explore assets, create new data discovery agents, schedule data discovery agents, and much more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 5 replies
  • 1272 views
  • 5 likes
  • 3 in conversation