A Practical Guide to Argo CD Part 2: Granular SAS Viya Deployment using Argo CD

[#1]

Introduction

Welcome back to the second installment of our blog series about ArgoCD. Here are the other parts in case you missed them: Part 1 Introduction and Core Concepts and Part 3 Customizing Argo CD using Custom Tools. In this blog, we’ll take a look at how to align ArgoCD with the responsibility boundaries in a SAS Viya deployment, so cluster administrators and SAS Viya administrators can each manage the parts that belong to them. We will then demonstrate how ArgoCD can be used to deploy Viya granularly following these respective boundaries.

SAS Viya can be deployed in several ways, including using the SAS Deployment Operator, the sas-orchestration tool, or by manually applying Kubernetes manifests with targeted label selectors via CLI. Each deployment method relies on Kustomize to generate SAS Viya manifests from the deployment assets located in the sas-bases directory.

The first two approaches are well-suited for automated and streamlined deployments, but they typically require persistent cluster administrator privileges to be granted to a SAS administrator. In many organizations, there is a strict separation between cluster administration and application administration roles. As a result, these methods may not be suitable in environments where cluster admin involvement must be minimized during the SAS Viya deployment process.

Just as it is important to understand how Kubernetes manifests are deployed to a cluster, it is equally important to understand what is being deployed and who is responsible for deploying it.

A granular deployment approach helps enforce a secure, reliable, and consistent deployment strategy. The third option—manually applying Kubernetes manifests—can support these requirements when carefully controlled. Luckily, by leveraging ArgoCD, these manual apply operations can be replaced with an automated, declarative workflow that clearly defines what is deployed, as well as who deploys it and when.

Deployment Personas

As we begin, we first define two key personas responsible for the SAS Viya deployment.

Cluster Admin

• The cluster admin is responsible for cluster-wide operations and must have elevated permissions to create and manage SAS Viya custom resource definitions (CRDs), cluster role bindings (CRBs), and any custom security context constraints (SCCs) required by certain SAS Viya components specific to only OpenShift.

SAS Viya Admin

• The SAS Viya admin is responsible for managing all resources within the namespace that hosts Viya workloads. However, a standard namespace administrator role is not sufficient on its own. In addition to namespace-level access, the SAS Viya admin must also manage additional Viya created objects not included by the namespace admin role.
• To support this, a custom role should be assigned alongside namespace permissions, granting the SAS Viya admin the ability to manage Viya specific CRs, PodTemplates, and any other objects not covered by the default namespace admin role.
• This custom role aggregates additional capabilities to the namespace admin role and is referred to as the sas-extended-admin-rbac role within this article.

These personas can be implemented as two separate ArgoCD instances, with one instance assigned to each persona.

The cluster admin ArgoCD instance is intended for exclusive use by the cluster admin, while the SAS Viya admin ArgoCD instance is used by the SAS Viya administrator.

When both instances are deployed, a dedicated service account for each instance is automatically created. These service accounts are used by the ArgoCD Application Controller to deploy workloads from connected Git repositories.

The cluster-admin instance is backed by a service account with cluster-wide privileges, whereas the SAS Viya admin instance uses a service account with namespace-level access and the additional sas-extended-admin role to manage SAS Viya specific resources.

sas-extended-admin-rbac role

Below is an example of what the sas-extended-admin-rbac role should contain:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: sas-extended-admin-rbac
  namespace: sasviyadev
  labels:
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
    rbac.authorization.k8s.io/aggregate-to-nsadmin: "true"
rules:
- apiGroups: ["viya.sas.com"]
  resources: ["casdeployments", "casdeployments/finalizers", "casdeployments/status"]
  verbs: ["create", "get", "list", "update", "patch", "watch", "delete", "deletecollection"]
...

The full example of the sas-extended-admin-rbac role is attached to the article.

Once the sas-extended-admin-rbac role is defined, it can be bound to the viya-openshift-gitops-argocd-application-controller service account used by the ArgoCD Application Controller within the SAS Viya admin ArgoCD instance. In addition, the same service account can be granted the namespace admin role within the viyagitops namespace:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: argocd-sas-extended-admin
subjects:
  - kind: ServiceAccount
    name: viya-openshift-gitops-argocd-application-controller
    namespace: viyagitops
roleRef:
  kind: ClusterRole
  name: sas-extended-admin-rbac
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: argocd-namespace-admin
  namespace: viyagitops
subjects:
  - kind: ServiceAccount
    name: viya-openshift-gitops-argocd-application-controller
    namespace: viyagitops
roleRef:
  kind: ClusterRole
  name: admin
  apiGroup: rbac.authorization.k8s.io

The personas and their dedicated service accounts are illustrated in the figure below:

Now that we have our personas defined and their respective ArgoCD instances created with appropriate permissions assigned to each instance’s service account, we can proceed to customizing the SAS Viya deployment directory to support our granular deployment approach.

Viya Manifest Labels

A granular SAS Viya deployment requires more than simply separating responsibilities between cluster administrators and SAS Viya administrators. The deployment content itself must also be separated so that each persona only deploys the Kubernetes resources they are responsible for managing.

SAS Viya deployment assets already provide a strong foundation for this model through the manifests generated by Kustomize. When Kustomize builds the final site.yaml manifest from the sas-bases and site-config directories, the generated resources include predefined labels that categorize components by function and scope.

These labels make it possible to selectively deploy subsets of the overall Viya platform. Instead of applying the entire site.yaml manifest at once, kubectl apply commands can target specific label selectors to deploy only the resources required for a particular administrative boundary.

For example, the following commands demonstrate how the site.yaml manifest can be applied selectively by targeting specific label selectors:

• kubectl apply --selector="sas.com/admin=cluster-api" --server-side --force-conflicts -f site.yaml
• kubectl wait --for condition=established --timeout=60s -l "sas.com/admin=cluster-api" crd
• kubectl apply --selector="sas.com/admin=cluster-wide" -f site.yaml
• kubectl apply --selector="sas.com/admin=cluster-local" -f site.yaml --prune
• kubectl apply --selector="sas.com/admin=namespace" -f site.yaml --prune

These commands explicitly target specific labels defined within the site.yaml manifest:

sas.com/admin=cluster-api

Resources labeled with sas.com/admin=cluster-api include the custom resource definitions (CRDs) required for the SAS Viya deployment. For example, this label includes CRDs such as casdeployments.viya.sas.com for the CAS deployment CRs, along with others like opendistroclusters.opendistro.sas.com used by the OpenDistro components installed with Viya.

Applying these resources requires cluster-admin permissions and should only be executed by a cluster administrator.

sas.com/admin=cluster-wide

Resources labeled with sas.com/admin=cluster-wide include Viya specific roles and service accounts deployed within the target Viya namespace.

Most of these resources are intended to be applied by the SAS Viya administrator using the sas-extended-admin-rbac role in combination with namespace-admin privileges. However, it is important to note that some of these roles may require permissions beyond what sas-extended-admin-rbac provides.

Since Kubernetes RBAC does not allow a user to create roles with permissions, they do not already hold, the sas-extended-admin-rbac role must be carefully designed to include all necessary privileges required by the roles defined under the sas.com/admin=cluster-wide label.

In addition, some of these resources include ClusterRoleBindings (CRBs), which should not be applied by the SAS Viya administrator. These should instead be handled by the cluster administrator. To support this separation, CRB manifests can be patched to change their label from sas.com/admin=cluster-wide to a value such as sas.com/admin=crb-admin, allowing them to be filtered out of the SAS Viya admin workflow and explicitly provided to the cluster admin for deployment.

The following example shows a patch transformer used to modify the label on the SAS Logon CRB, changing it from cluster-wide to crb-admin:

apiVersion: builtin
kind: PatchTransformer
metadata:
  name: replace-cluster-wide-admin-label-sas-logon
patch: |-
  - op: replace
    path: /metadata/labels/sas.com~1admin
    value: crb-admin
target:
  group: rbac.authorization.k8s.io
  version: v1
  kind: ClusterRoleBinding

The attached sas-extended-admin-rbac role YAML defines a comprehensive set of aggregated permissions required for the SAS Viya administrator. It is important to note that future SAS Viya releases may introduce new roles or modify existing ones, so the sas-extended-admin-rbac role should be reviewed and updated prior to each Viya upgrade to ensure it remains aligned with these changes.

sas.com/admin=cluster-local

Resources labeled with sas.com/admin=cluster-local include Viya specific RoleBindings, ConfigMaps, Secrets, PersistentVolumeClaims, and a few PodTemplates deployed within the target Viya namespace.

These resources will be created and managed by the SAS Viya administrator.

sas.com/admin=namespace

Resources labeled with sas.com/admin=namespace include Viya specific workloads such as Deployments, StatefulSets, Jobs, and CronJobs, along with additional ConfigMaps and PodTemplates.

This category also includes namespace-scoped custom resources such as CASDeployment, OpenDistroCluster, and DataServer, among others depending on your customizations.

These resources will be created and managed by the SAS Viya administrator.

In summary, the resources below are applied and managed by the two personas discussed:

• Cluster Admin
  • CRDs
  • CRBs
  • Custom SCCs if deploying Viya on OpenShift
• SAS Viya Admin
  • Deployments
  • StatefulSets
  • Roles
  • RoleBindings
  • ServiceAccounts
  • ConfigMaps
  • PodTemplates
  • Jobs
  • CronJobs
  • DaemonSets
  • CRs

Splitting the Viya Deployment Based on Personas

Now that we have identified the deployment personas and how built-in label selectors can be used to scope resources by administrative responsibility, we can move on to generating custom site.yaml manifests for each persona. These manifests can then be deployed through ArgoCD using targeted label selectors aligned to each deployment role.

To begin, we should first define how the deployment directory and Git repository will be organized.

Viya Deployment Directory

The following example illustrates one way the SAS Viya deployment directory can be organized to support a granular deployment model:

After Kustomize generates the full site.yaml manifest, the deployment can be split into smaller manifests based on the built-in SAS Viya labels applied to resources.

These segmented manifests allow resources to be deployed independently according to the responsibilities of each deployment persona.

These manifests can be carved from the original site.yaml using yq and label selectors. For example:

yq 'select(.metadata.labels["sas.com/admin"] == "cluster-api")' "site.yaml"   > cluster-api/site-cluster-api.yaml

The yq command above will gather every resource with the sas.com/admin=cluster-api label and store these resources into cluster-api/site-cluster-api.yaml. 

The yq commands can also be incorporated into scripts and combined with the Kustomize build process shown below to automatically generate the segmented deployment manifests.

#!/usr/bin/env bash
 
ORIGINAL_SITE="site.yaml"
 
echo "Running kustomize build..."
kustomize build -o "${ORIGINAL_SITE}"

echo “Create scoped directories”
mkdir -p namespace cluster-local cluster-wide cluster-api cluster-crb
 
echo "Filtering cluster-api resources..."
yq 'select(.metadata.labels["sas.com/admin"] == "cluster-api")' "${ORIGINAL_SITE}" \
  > cluster-api/site-cluster-api.yaml
 
echo "Filtering cluster-crb resources..."
yq 'select(.metadata.labels["sas.com/admin"] == "crb-admin")' "${ORIGINAL_SITE}" \
  > cluster-crb/site-cluster-crb.yaml
 
echo "Filtering cluster-wide resources..."
yq 'select(.metadata.labels["sas.com/admin"] == "cluster-wide")' "${ORIGINAL_SITE}" \
  > cluster-wide/site-cluster-wide.yaml
 
echo "Filtering cluster-local resources..."
yq 'select(.metadata.labels["sas.com/admin"] == "cluster-local")' "${ORIGINAL_SITE}" \
  > cluster-local/site-cluster-local.yaml
 
echo "Filtering namespace-scoped resources..."
yq 'select(.metadata.labels["sas.com/admin"] == "namespace")' "${ORIGINAL_SITE}" \
  > namespace/site-namespace.yaml
 
echo "Done."

The site-cluster-api.yaml and site-cluster-crb.yaml manifests can be provided directly to the cluster administrator for deployment and ongoing management.

Connecting our Git Repository

Once the SAS Viya deployment directory has been organized with the segmented site.yaml manifests, it must be pushed to a Git repository so that the ArgoCD instance can connect to it and use it as the source of truth for deployments.

There are several ways to connect a Git repository to Argo CD. The simplest approach is to configure the repository directly through the ArgoCD instance UI: ArgoCD Console -> Settings -> Repositories -> Connect Repo.

A Git repository can also be connected using the ArgoCD CLI. The CLI can be installed from here: https://argo-cd.readthedocs.io/en/latest/cli_installation/

Below are a few sample commands demonstrating the use of the ArgoCD CLI to connect a Git repository:

argocd login <argoCD Server>
argocd repo add https://git.example.com/org/repo.git \
 --username myuser \
 --password "$GIT_PASSWORD"
argocd repo list

Creating ArgoCD Applications

To keep the configuration simple, and because we are only managing a single SAS Viya deployment through ArgoCD, we will create our ArgoCD Applications explicitly rather than generating them through ApplicationSets.

Before deploying the Viya workloads, ArgoCD Applications must be configured for the specific workloads they are responsible for managing.

For example, the cluster administrator’s ArgoCD instance should contain an Application dedicated to deploying only the cluster-level administrative components defined in:

• cluster-api/site-cluster-api.yaml
• cluster-crb/site-cluster-crb.yaml

In contrast, the SAS Viya administrator’s Argo CD instance will contain Applications dedicated to deploying SAS Viya namespace-scoped components, such as:

• cluster-wide/site-cluster-wide.yaml
• cluster-local/site-cluster-local.yaml
• namespace/site-namespace.yaml

This approach establishes a level of granularity that enables tighter control over what is deployed through Git and who is authorized to manage those deployments.

Control over which components an ArgoCD Application is permitted to manage is defined through the repository URL of the connected Git repository and the specific path within that repository where the manifests reside.

For example, we can configure an Application called viya-cluster-wide to point to a specific path in the Git repository that contains only the cluster-wide/site-cluster-wide.yaml manifest by defining path: accordingly.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: viya-namespace
  namespace: viyagitops
spec:
...
  source:
    path: deploy/stable-202603/cluster-wide
    repoURL: 'https://github.com/stic-lab/viyagitops.git'
    targetRevision: HEAD
...

This enables us to know exactly which resources our ArgoCD Application is responsible for.

Cluster-Admin ArgoCD Application

The cluster administrator Argo CD Application below resides within the cluster-admin ArgoCD instance and is configured with a path pointing to the deploy/viyagitops-cluster-admin directory in the Git repository. This directory contains cluster administrator-level manifests extracted from the original site.yaml via labels:

• site-cluster-api.yaml
• site-cluster-crb.yaml

As well as some other cluster-scoped resources.

Expanding the cluster-admin Application, we can see all the resources the Application manages:

SAS Viya Admin ArgoCD Applications

The SAS Viya administrator will be responsible for managing non-cluster-admin resources and, as a result, will manage three separate ArgoCD Applications within their ArgoCD instance.

These Applications correspond to the three non-cluster-admin components separated from the original site.yaml manifest:

• cluster-wide:
  • Path: cluster-wide/site-cluster-wide.yaml
• cluster-local:
  • Path: cluster-local/site-cluster-local.yaml
• namespace:
  • Path: namespace/site-namespace.yaml

Synching Applications for Deployment

Once the ArgoCD Applications are created within their respective Argo CD instances, the SAS Viya resources can be deployed by initiating a sync operation. After synchronization, the resources managed by each Application are deployed immediately, depending on the configured sync policy.

To ensure obsolete resources are removed from the cluster as the manifests evolve in Git, especially for viya-cluster-local and viya-namespace resources, the ArgoCD sync policy should be configured accordingly. For these Applications, the sync policy should be set to automatic with resource pruning enabled.

This can be configured either through the Argo CD UI or directly within the Application CR YAML. Below is an example of configuring the viya-namespace Application CR to enable automated synching and resource pruning:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: viya-namespace
  namespace: viyagitops
spec:
...
  syncPolicy:
    automated:
      enabled: true
      prune: true

Automatic synchronization ensures that Argo CD continuously reconciles Applications whenever changes are made to the manifests in Git. This guarantees that the Git repository remains the single source of truth and that the cluster state is always aligned with the desired state defined in Git with minimal user involvement.

Once an ArgoCD Application has been synchronized, you can expand it to inspect and view the resources it manages. For example, the viya-namespace Application is shown expanded below:

Summary Architecture

To summarize, we have implemented a granular SAS Viya deployment with strict RBAC controls, using ArgoCD to automate and enforce deployment boundaries.

We began by defining two personas: the cluster administrator and the SAS Viya administrator. We then demonstrated why separate ArgoCD instances are required to maintain clear responsibility boundaries, including controlled access based on the service accounts and roles assigned to the ArgoCD Applications managed within each instance.

Next, we showed how SAS Viya manifests can be split according to these personas and how the Git repository structure reflects this separation. We created ArgoCD Applications within their respective instances, each targeting specific subsets of manifests stored in Git. Finally, we enabled automatic synchronization to ensure that any changes in Git are immediately reflected in the cluster state, keeping deployments continuously aligned with the source of truth.

The diagrams below summarize this granular SAS Viya deployment approach using ArgoCD.

SAS Viya Admin

Cluster Admin

Conclusion

The approach we described in this blog builds on the labels SAS Viya already adds to its resources, using them to split the deployment along responsibility boundaries so each part can be managed by the right ArgoCD instance with the right level of privilege.

In Part 3, we’ll build on this and look at a few practical details that make the setup easier to run in the real world.