BookmarkSubscribeRSS Feed
☑ This topic is solved. Need further help from the community? Please sign in and ask a new question.
sas_chinmay
Fluorite | Level 6

With the kubernetes command, we have nearly finished all of the deployment steps. SAS Help Center: Deploy the Software However, when we were executing step number seven, we noticed that sas-rabbitmq-server and sas-workload-orchestrator are in PENDING state.
So, what will be the correct way to fix these issues and any monitoring tool that we can refer?

PFB:

Events:
  Type     Reason             Age                    From                Message
  ----     ------             ----                   ----                -------
  Warning  FailedScheduling   27m                    default-scheduler   0/10 nodes are available: 1 Insufficient memory, 1 node(s) had untolerated taint {workload.sas.com/class: cas}, 1 node(s) had untolerated taint {workload.sas.com/class: compute}, 2 node(s) didn't match Pod's node affinity/selector, 6 Insufficient cpu. preemption: 0/10 nodes are available: 4 Preemption is not helpful for scheduling, 6 No preemption victims found for incoming pod.
  Warning  FailedScheduling   6m39s (x7 over 22m)    default-scheduler   0/10 nodes are available: 1 Insufficient memory, 1 node(s) had untolerated taint {workload.sas.com/class: cas}, 1 node(s) had untolerated taint {workload.sas.com/class: compute}, 2 node(s) didn't match Pod's node affinity/selector, 6 Insufficient cpu. preemption: 0/10 nodes are available: 4 Preemption is not helpful for scheduling, 6 No preemption victims found for incoming pod.
  Normal   NotTriggerScaleUp  17m (x6 over 26m)      cluster-autoscaler  pod didn't trigger scale-up: 2 max node group size reached, 1 node(s) didn't match Pod's node affinity/selector
  Normal   NotTriggerScaleUp  2m34s (x130 over 27m)  cluster-autoscaler  pod didn't trigger scale-up: 1 node(s) didn't match Pod's node affinity/selector, 2 max node group size reached
[azureuser@sasviya ~]$

 

sas_chinmay_0-1728892915561.png

kubectl top nodes snap.png

 

 

1 ACCEPTED SOLUTION

Accepted Solutions
sasefj
SAS Employee

Which version of Viya are you deploying?  There were Changes for SAS Workload Orchestrator Services in 2024.08 that might affect a potential solution.  It looks like your services in Pending can't find a node to run on.  I note you are on Azure.  You might try increasing your max nodes setting for your stateful and/or stateless node pools.  It's easy to adjust max nodes in the Azure Console.  

 

View solution in original post

1 REPLY 1
sasefj
SAS Employee

Which version of Viya are you deploying?  There were Changes for SAS Workload Orchestrator Services in 2024.08 that might affect a potential solution.  It looks like your services in Pending can't find a node to run on.  I note you are on Azure.  You might try increasing your max nodes setting for your stateful and/or stateless node pools.  It's easy to adjust max nodes in the Azure Console.  

 

suga badge.PNGThe SAS Users Group for Administrators (SUGA) is open to all SAS administrators and architects who install, update, manage or maintain a SAS deployment. 

Join SUGA 

Get Started with SAS Information Catalog in SAS Viya

SAS technical trainer Erin Winters shows you how to explore assets, create new data discovery agents, schedule data discovery agents, and much more.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 1 reply
  • 1941 views
  • 1 like
  • 2 in conversation