Welcome back! This second post serves as a follow-up to our previous post, where we initiated discussions on DevOps processes with SAS Viya CLI container images, available here, Introducing New SAS Viya CLI container image in Azure DevOps (March 2024 Update). In this two-part blog, we delve deeper into security customizations, addressing aspects such as managing refresh tokens, handling certificates not chained to public Certificate Authorities (CAs), and enhancing security measures within Azure Pipelines and Kubernetes clusters. These enhancements build upon our previous discussions, aiming to fortify your DevOps pipelines against security vulnerabilities while ensuring seamless execution. Let's explore these advancements in detail.   Here's an illustrative diagram presenting the functional flow involved.   Managing the Use / Revocation of the Refresh-Token   Why does it matter?   Access tokens play a pivotal role in facilitating secure communication between various components. However, the limitations of access tokens, such as their short lifespan of 1 hour (default for the SAS Viya) and irreversibility, often pose challenges. . Here is a link from my colleague Stuart Rogers  that explain the SAS Viya CLI Token Expiry (additional note, starting with 2023.11 the default refresh token expiration is 14 days). Unlike access tokens, refresh tokens have a longer validity period of 14 days and are revocable, providing better control and flexibility in managing authentication within DevOps pipelines. In this example, we will show you how to consume the SAS Refresh Token (retrieved from your Azure Key Vault) while running your Azure pipeline.   Using the Refresh-Token while executing your DevOps pipeline   Step 1: Define the pipeline parameter   This Azure Pipeline parameter allows you to specify whether you want to use the refresh token during pipeline execution, providing flexibility in authentication. - name: USE_REFRESH_TOKEN displayName: Do you want to use the refresh token (valid 14 days)? type: boolean default: true   Step 2: Modify pipeline task   Enhance the step titled "Get JSON Web Token from Azure Key Vault" to extract the refresh token and its expiry date: - task: AzureCLI@2 name: TgetJWTfromAKV displayName: Get JSON Web Token from Azure Key Vault inputs: azureSubscription: '$(KV_SERVICE_CONNECTION)' scriptType: 'bash' scriptLocation: 'inlineScript' inlineScript: | az keyvault secret download \ --name ${{ parameters.AZURE_KEYVAULT_KEY }} \ --vault-name ${{ parameters.AZURE_KEYVAULT }} \ --file $PWD/credentials.json export JWT_TOKEN=$(cat $PWD/credentials.json | jq -r '.Default."access-token"') export REFRESH_TOKEN=$(cat $PWD/credentials.json | jq -r '.Default."refresh-token"') export EXPIRY=$(cat $PWD/credentials.json | jq -r '.Default."expiry"') echo "##vso[task.setvariable variable=JWT_TOKEN;isOutput=true]$JWT_TOKEN" echo "##vso[task.setvariable variable=REFRESH_TOKEN;isOutput=true]$REFRESH_TOKEN" echo "##vso[task.setvariable variable=EXPIRY;isOutput=true]$EXPIRY"   Step 3: Add Context Variables   Add the variable to the context of the container job: variables: - name: JWT_TOKEN value: $[ dependencies.getJWTfromAKV.outputs['TgetJWTfromAKV.JWT_TOKEN'] ] - name: REFRESH_TOKEN value: $[ dependencies.getJWTfromAKV.outputs['TgetJWTfromAKV.REFRESH_TOKEN'] ] - name: EXPIRY value: $[ dependencies.getJWTfromAKV.outputs['TgetJWTfromAKV.EXPIRY'] ]   Step 4: Add pipeline task   This step creates a credentials file containing the refresh token, access token, and expiry date, allowing the use of the refresh token mechanism. - task: Bash@3 displayName: Generate credentials.json enabled: ${{ eq(parameters.USE_REFRESH_TOKEN, true) }} inputs: targetType: 'inline' script: | jq -n \ --arg access-token $JWT_TOKEN \ --arg expiry $EXPIRY \ --arg refresh-token $REFRESH_TOKEN \ '{Default: $ARGS.named}'>~/.sas/credentials.json Reminder: In my previous post, I was leveraging the JWT_TOKEN environment variable as the authentication mechanism. Now, I will be using the credentials.json file. With these improvements in place, you should be able to use the refresh token mechanism.   Revoke the refresh token after execution   Revoking refresh tokens after use is a security practice aimed at mitigating the risk of unauthorized access to resources. Refresh tokens are long-lived credentials used to obtain new access tokens without requiring the user to re-enter their credentials. However, if a refresh token is compromised or falls into the wrong hands, it can be abused to gain ongoing access to sensitive data or perform unauthorized actions. Here are the steps needed to implement the revocation of the refresh token when you have completed all your DevOps tasks with your SAS Viya CLI.   Step 1: Define the parameter   Define an Azure Pipeline parameter to determine whether you want to revoke the refresh token after use: - name: REVOKE_REFRESH_TOKEN displayName: Do you want to revoke the refresh token after use ? type: boolean   Step 2: Add pipeline task   Include a step to log out using the SAS Viya CLI and, subsequently, revoke the refresh token: - task: Bash@3 displayName: Log out (revoke the Refresh Token) enabled: ${{ eq(parameters.REVOKE_REFRESH_TOKEN, true) }} inputs: targetType: 'inline' script: | /opt/sas/viya/home/bin/sas-viya auth logout The log out will sign you out and revoke your refresh token. Note: If your Access token is still valid it will remain valid until its expiration. By incorporating these practices, you can effectively manage the use and revocation of refresh tokens within your DevOps processes, enhancing security and control over authentication mechanisms.   Handling certificates not chained to a public Certificate Authority   By default, the SAS Viya platform is configured to trust the CA certificates that are distributed by mozilla.org. By including this bundle of trusted CA certificates, you will be able to connect to your platform (signed by one of these CA) without any additional configuration. If you are using a self/site-signed certificates, you will need to add your CA to your trust store in order to validate the certificate that will be send by your SAS Deployment. When your SAS Viya platform's certificate is not chained to a public CA, it poses a challenge to secure communication. Without proper validation, your system may reject connections, leading to potential disruptions in your pipeline. But fear not, as we've outlined steps to navigate this challenge seamlessly. The issue of untrusted authority can rear its head at two critical junctures in your pipeline process: During Credential Generation: Using the sas-viya-cli, you might encounter verification failures when generating credentials. This can halt your DevOps process before it even begins. In Azure DevOps Pipelines: Similar challenges can arise when running pipelines, necessitating additional steps to ensure smooth execution.   During credential generation   During the initial stages of setting up your DevOps pipeline, you may encounter verification failures when generating credentials. Let's break down the steps to address this issue:   Step 1: Identify verification failure   Using the check_certificatescript, you can identify verification failures. Here's a snippet of code illustrating this step (full code is presented in Annex 1): # Set some variables SAS_SERVICES_DNS=yoursasviyaurl.company.com # Can be retrieved automatically if you have configured kubectl # NS=gelenv; SAS_SERVICES_DNS=$(kubectl -n ${NS} get configmap -o=jsonpath='{.items[*].data.SAS_SERVICES_URL}') # Example usage: check_certificate <hostname> <port> check_certificate "$SAS_SERVICES_DNS" 443 # Example usage: check_certificate <hostname> <port> <truststore> check_certificate "$SAS_SERVICES_DNS" 443 $HOME/trustedcerts.pem log output: fradae@cldlgn01:~$ check_certificate "$SAS_SERVICES_DNS" 443 Certificate verification: Failed (Unable to verify the first certificate) fradae@cldlgn01:~$ check_certificate "$SAS_SERVICES_DNS" 443 $HOME/trustedcerts.pem Certificate verification: OK   Step 2: Download the trust store   Downloading the SAS Viya Platform trust store is crucial for establishing secure connections. Use the provided command to retrieve it: # Set some variables NS=gelenv # Extract the trust store file and store it in the $HOME directory kubectl -n $NS cp $(kubectl get pod -n $NS | grep "sas-logon-app" | head -1 | awk -F" " '{print $1}'):security/trustedcerts.pem ~/trustedcerts.pem   Step 3: Utilize the trust store   Ensure seamless authentication by mounting the trust store as a volume in your Docker container: docker run --rm -it -v /$HOME:/security -v $PWD:/cli-home/.sas -e SAS_SERVICES_ENDPOINT $DOCKER_REGISTRY/viya-4-x64_oci_linux_2-docker/sas-viya-cli:latest auth loginCode Note: Adding /$HOME:/security as a mounted volume ensures that trustedcerts.pem is readily accessible. Without the trust store, login attempts may fail due to unrecognized authorities. Your trust store must be named exactly trustedcerts.pem. If you try without the trust store, you'll receive the following ERROR message: Login failed due to an error with the security certificate. The certificate is signed by an unknown authority. Run with the ‘–verbose' global option to see additional details.   In Azure DevOps pipelines   In your Azure DevOps pipelines, additional steps are necessary to ensure smooth execution when certificates are not chained to public CAs.   Step 1: Define pipeline parameter   Define a parameter to determine whether importing the trust store from your SAS Viya platform is necessary: - name: PUB_SIGN_CERT displayName: TLS certificate is chained to a public CA? type: boolean default: true   Step 2: Incorporate Bash task   Add a Bash task to retrieve the trusted certificates just before executing the SAS Viya CLI command: - task: Bash@3 displayName: Get the Trusted Certs (if not publicly available) enabled: ${{ eq(parameters.PUB_SIGN_CERT, false) }} inputs: targetType: 'inline' script: | kubectl -n ${K8S_NAMESPACE} cp $(kubectl get pod -n ${K8S_NAMESPACE} | grep "sas-logon-app" | head -1 | awk -F" " '{print $1}'):security/trustedcerts.pem /tmp/trustedcerts.pem   Step 3: Modify Bash task   Modify the pipeline step titled "Retrieve all configuration definition list" to utilize the downloaded trust store: - task: Bash@3 displayName: Retrieve all configuration definition list inputs: targetType: 'inline' script: | if [ -f /tmp/trustedcerts.pem ]; then export SSL_CERT_FILE=/tmp/trustedcerts.pem fi cd /opt/sas/viya/home/bin /opt/sas/viya/home/bin/sas-viya-wrapper --output text configuration configurations list By completing these steps, you ensure secure communication with your SAS Viya platform, even when using certificates not chained to a public CA.   Conclusion   By effectively managing refresh tokens, handling certificates and enhancing security measures within Azure pipelines and Kubernetes clusters, organizations can fortify their DevOps pipelines against potential security threats. These practices not only ensure security but also ensure the reliability and integrity of your applications and data in cloud environments.   Annex 1 #!/bin/bash   # Function to check server certificate verification status check_certificate() { hostname="$1" port="$2" trusted_ca_file="${3:-}"   # Prepare CAfile option if trusted_ca_file is provided cafile_option="" if [ -n "$trusted_ca_file" ]; then cafile_option="-CAfile $trusted_ca_file" fi   # Run openssl s_client command and capture output output=$(echo X | openssl s_client -connect "$hostname":"$port" -servername "$hostname" $cafile_option 2>&1)   # Look for the "Verify return code" line and extract the return code verify_return_code=$(echo "$output" | grep -oP "Verify return code: \K\d+")   # Check the verification status and print detailed information case "$verify_return_code" in 0) echo "Certificate verification: OK" ;; 10) echo "Certificate verification: Failed (Certificate has expired)" ;; 20) echo "Certificate verification: Failed (Unable to get local issuer certificate)" ;; 21) echo "Certificate verification: Failed (Unable to verify the first certificate)" ;; 18) echo "Certificate verification: Failed (Self-signed certificate)" ;; *) echo "Certificate verification: Failed (Unknown reason, Verify return code: $verify_return_code)" ;; esac }   Additional Azure DevOps Resources with SAS Viya   David Estreich (myself) related posts with Azure DevOps: SAS Viya CLI container image in Azure DevOps (March 2024 Update) SAS Viya CLI container image in Azure DevOps, security topics – Part 1 -> IP Restrictions  Unlocking Efficiency with Container Jobs in Azure DevOps Streamlining SAS Viya Authentication in Azure DevOps Bogdan Teleuca's other posts about SAS Viya with Azure DevOps: SAS Viya with Azure DevOps Data Pipeline. SAS Viya CI/CD Pipelines with Azure DevOps. SAS Viya Decision Pipeline with Azure DevOps SAS Viya and Azure Pipelines Self-Hosted Agents SAS Viya and Azure Pipelines Microsoft-Hosted Agents ... View more

Welcome back! This first post serves as a follow-up to our previous post, where we initiated discussions on DevOps processes with SAS Viya CLI container images, available here, Introducing New SAS Viya CLI container image in Azure DevOps (March 2024 Update). In this two-part blog, we delve deeper into security customizations, addressing aspects such as managing refresh tokens, handling certificates not chained to public Certificate Authorities (CAs), and enhancing security measures within Azure Pipelines and Kubernetes clusters. These enhancements build upon our previous discussions, aiming to fortify your DevOps pipelines against security vulnerabilities while ensuring seamless execution. Let's explore these advancements in detail.   Here's an illustrative diagram presenting the functional flow involved.   Azure Key Vault Security with IP Restrictions in Azure Pipelines   Ensuring that only authorized entities can access these secrets is crucial for maintaining the integrity of your applications and data. One effective way to enhance security is by enforcing IP restrictions, thereby limiting access to Azure Key Vault only from specified IP addresses. In this blog example, we'll explore how to integrate IP restrictions into your Azure Pipelines to bolster the security of your Azure Key Vault.   Step 1: Define Parameter for IP Restrictions   The first step is to define a parameter in your Azure Pipeline configuration to determine whether IP restrictions should be enforced. This parameter will provide flexibility in enabling or disabling IP restrictions as per your organization's security policies. Below is an example of how to define this parameter: - name: KV_IP_RESTRICTIONS displayName: Do you have IP restrictions for your Azure Key Vault? type: boolean default: true   Step 2: Allow Microsoft Agent IP in Azure Key Vault   Before retrieving credentials from Azure Key Vault, it's essential to add the IP address of the Microsoft Agent to the allowed IP list for the Key Vault. This ensures that the pipeline can access the Key Vault securely. To achieve this, include the following Azure CLI task in your pipeline: - task: AzureCLI@2 displayName: Allow MSFT Agent IP to Azure Key Vault condition: ${{ eq(parameters.KV_IP_RESTRICTIONS, true) }} inputs: azureSubscription: '$(KV_SERVICE_CONNECTION)' scriptType: 'bash' scriptLocation: 'inlineScript' inlineScript: | # Retrieve your IP address CURRENT_IP=$(curl -s https://ipinfo.io/ip) az keyvault network-rule add --name ${{ parameters.AZURE_KEYVAULT }} --ip-address $CURRENT_IP > /dev/null   Step 3: Remove Microsoft Agent IP from Azure Key Vault   After retrieving credentials from Azure Key Vault, it's crucial to remove the IP address of the Microsoft Agent from the allowed IP list to maintain security posture. This step ensures that unauthorized access attempts from the Microsoft Agent IP are thwarted. Add the following Azure CLI task to your pipeline: - task: AzureCLI@2 displayName: DisAllow MSFT Agent IP to Azure Key Vault condition: ${{ eq(parameters.KV_IP_RESTRICTIONS, true) }} inputs: azureSubscription: '$(azureSubscription)' scriptType: 'bash' scriptLocation: 'inlineScript' inlineScript: | # Retrieve your IP address CURRENT_IP=$(curl -s https://ipinfo.io/ip) az keyvault network-rule remove --name ${{ parameters.AZURE_KEYVAULT }} --ip-address $CURRENT_IP > /dev/null These steps ensure that only designated IP addresses, including the Microsoft Agent IP, are permitted to access sensitive information stored in Azure Key Vault. Implementing such security measures aligns with best practices for safeguarding data in the cloud and helps protect your applications from potential security threats.   Managing Azure Kubernetes Cluster API IP Restrictions   In this blog example, we'll delve into how to handle IP restrictions for AKS APIs, particularly when utilizing the sas-viya-cli in a pipeline executed on a Microsoft-hosted agent.   Understanding the Challenge   When the AKS API is restricted based on IP addresses, it poses a challenge for pipeline execution using public IPs. We need to ensure that the Microsoft-hosted agent's IP is authorized to access the AKS API. This involves installing the Azure CLI, modifying AKS API IP restrictions, and ensuring proper permissions within the Docker image.   Step1: Enhancements to Docker Image   To accommodate these requirements, enhancements to the Docker image are necessary. Below is a snippet showcasing the modifications: RUN dnf install -y --nodocs nodejs sudo && dnf clean all NOTE: sudo command is mandatory to be able to install the az cli (that will be done while executing the DevOps pipeline).   Step2: Define Parameter for AKS APIs IP Restrictions   This parameter will provide flexibility in enabling or disabling IP restrictions management as per your organization's security policies. Below is an example of how to define this parameter: - name: AKS_APIS_IP_RESTRICTIONS displayName: Do you have IP restrictions for your AKS APIs? type: boolean default: false   Step 3: Installing Azure CLI (while executing your DevOps pipeline)   Installing the Azure CLI within the pipeline environment is mandatory to accommodate the current requirement. Here's how to do it: - task: Bash@3 displayName: Install Azure CLI for ubi8 (RHEL8) condition: enabled: ${{ eq(parameters.AKS_APIS_IP_RESTRICTIONS, true) }} inputs: targetType: 'inline' script: | sudo rpm --import https://packages.microsoft.com/keys/microsoft.asc sudo dnf install -y https://packages.microsoft.com/config/rhel/8/packages-microsoft-prod.rpm sudo dnf install -y azure-cli   Step 4: Modifying AKS API IP Restrictions   To add the Microsoft agent's IP to the authorized list for AKS API access, follow these steps: Retrieve the original AKS API authorized IP ranges for later use. Add the Microsoft agent's IP to the allowed list. Ensure proper propagation of the rules by adding a delay (of 10 seconds in this case) - task: AzureCLI@2 displayName: Add Agent IP to the AKS API authorized IPs condition: ${{ eq(parameters.IP_RESTRICTIONS, true) }} inputs: azureSubscription: 'Service for PSGEL262' scriptType: 'bash' scriptLocation: 'inlineScript' inlineScript: | # set -x AZURE_RG=${RACE_PREFIX}-rg AZURE_AKS=${RACE_PREFIX}-aks AZURE_RANGE=$(az aks show \ --resource-group $AZURE_RG \ --name $AZURE_AKS \ --query apiServerAccessProfile.authorizedIpRanges | jq -r '. | join(",")') # Store the value to reset it later echo "##vso[task.setvariable variable=AZURE_RANGE]$AZURE_RANGE" # Retrieve your IP address CURRENT_IP=$(curl -s api.ipify.org) # Add to AKS approved list az aks update \ --resource-group  $AZURE_RG \ --name $AZURE_AKS \ --api-server-authorized-ip-ranges ${CURRENT_IP}/32,${AZURE_RANGE} --output none sleep 10   Step 5: Resetting AKS API IP Restrictions   After pipeline execution, it's essential to reset the AKS API IP restrictions to their original values to maintain the integrity of your environment: - task: AzureCLI@2 displayName: Remove Agent IP to the AKS API authorized IPs enabled: ${{ eq(parameters.AKS_APIS_IP_RESTRICTIONS, true) }} inputs: azureSubscription: '$(RG_SERVICE_CONNECTION)' scriptType: 'bash' scriptLocation: 'inlineScript' inlineScript: | # Add to AKS approved list az aks update \ --resource-group $AZURE_RG \ --name $AZURE_AKS \ --api-server-authorized-ip-ranges $(AZURE_RANGE) --output none By following these steps, you ensure that your pipeline can seamlessly interact with your AKS cluster while maintaining the necessary security measures.   Managing Kubernetes Ingress IP Restrictions   One common method of enforcing security is by implementing IP restrictions for Kubernetes Ingress. In this blog example, we'll explore how to manage IP restrictions for Kubernetes Ingress effectively.   Step 1: Define the parameter for Ingress IP restrictions   - name: INGRESS_IP_RESTRICTIONS displayName: Do you have IP restrictions for your K8s Ingress? type: boolean default: false   Step 2: Installing kubectl Command Line   Before proceeding, ensure that the kubectl command-line tool is installed while executing your pipeline. Here's how to do it: - task: KubectlInstaller@0 displayName: Install Kubectl client condition: ${{ or(eq(parameters.IP_RESTRICTIONS, true), eq(parameters.PUB_SIGN_CERT, false)) }} inputs: kubectlVersion: 'latest'   Step 3: Adding Microsoft Agent IP to the Ingress   To allow the Microsoft agent's IP access to the Kubernetes Ingress, follow these steps: Retrieve AKS Cluster Credentials: Firstly, we need to authenticate with the AKS cluster to execute commands. This involves retrieving the cluster credentials. Check Load Balancer Source Ranges: We check if the loadBalancerSourceRanges property exists in the Ingress service specification. This property specifies the allowed IP ranges for accessing the Ingress. Apply Patch: If the loadBalancerSourceRanges property exists, we apply a patch to add the Microsoft agent's IP address to the list of allowed ranges. If it doesn't exist, we create the property and add the IP address. Finalize Configuration: Once the patch is applied, the Ingress configuration is updated to allow access from the Microsoft agent's IP address. - task: AzureCLI@2 displayName: Add Agent IP to the AKS Ingress authorized IPs enabled: ${{ eq(parameters.INGRESS_IP_RESTRICTIONS, true) }} inputs: azureSubscription: '$(RG_SERVICE_CONNECTION)' scriptType: 'bash' scriptLocation: 'inlineScript' inlineScript: | az aks get-credentials --resource-group $AZURE_RG \ --name $AZURE_AKS \ --overwrite-existing # kubectl config view # Check if the property exists load_balancer_ranges=$(kubectl get services ingress-nginx-controller -n ingress-nginx -o=jsonpath='{.spec.loadBalancerSourceRanges}' 2>/dev/null) # Check if the output is empty or not if [ -n "$load_balancer_ranges" ]; then echo "Property exists, applying patch..." kubectl patch services ingress-nginx-controller -n ingress-nginx --type='json' -p='[{"op": "add", "path": "/spec/loadBalancerSourceRanges/-", "value": "'${CURRENT_IP}/32'" }]' else echo "Property does not exist, creating it..." kubectl patch services ingress-nginx-controller -n ingress-nginx --type=json '-p=[{"op": "add", "path": "/spec/loadBalancerSourceRanges", "value": ["'${CURRENT_IP}/32'"] }]' fi   Step 4: Removing the MSFT Agent IP from the Ingress Allowed List   After pipeline execution, it's essential to remove the Microsoft agent's IP from the Kubernetes Ingress allowed list: Find Index of IP Address: We retrieve the index of the Microsoft agent's IP address within the list of allowed IP ranges specified in the Ingress configuration. Patch the Ingress Service: Using kubectl, we patch the Ingress service to remove the IP address from the loadBalancerSourceRanges. Ensure Configuration Update: We ensure that the Ingress configuration is successfully updated by checking for the removal of the IP address from the allowed list. - task: Bash@3 displayName: Remove Agent IP from the AKS Ingress authorized IPs enabled: ${{ eq(parameters.INGRESS_IP_RESTRICTIONS, true) }} inputs: targetType: 'inline' script: | # Find the index of the IP address within the loadBalancerSourceRanges INDEX=$(kubectl get services ingress-nginx-controller -n ingress-nginx -o json  | jq '.spec.loadBalancerSourceRanges | map(. == "'${CURRENT_IP}/32'") | index(true)') # Patch the Ingress service to remove the IP address kubectl patch services ingress-nginx-controller -n ingress-nginx \ --type=json -p='[{"op": "remove", "path": "/spec/loadBalancerSourceRanges/'$INDEX'"}]' # Ensure the configuration is updated kubectl patch services ingress-nginx-controller -n ingress-nginx \ --type=json -p='[{"op": "remove", "path": "/spec/loadBalancerSourceRanges/'$INDEX'"}]' By following the steps outlined in this blog example, you can effectively control access to your applications while ensuring the necessary level of security.   Conclusion   By effectively managing network IP restrictions and enhancing security measures within Azure pipelines and Kubernetes clusters, organizations can fortify their DevOps pipelines against potential security threats. These practices not only ensure security but also ensure the reliability and integrity of your applications and data in cloud environments.   Additional Azure DevOps Resources with SAS Viya   David Estreich (myself) related posts with Azure DevOps: SAS Viya CLI container image in Azure DevOps (March 2024 Update) SAS Viya CLI container image in Azure DevOps, security topics – part 2 ->Certificates & Tokens Unlocking Efficiency with Container Jobs in Azure DevOps Streamlining SAS Viya Authentication in Azure DevOps Bogdan Teleuca's other posts about SAS Viya with Azure DevOps: SAS Viya with Azure DevOps Data Pipeline. SAS Viya CI/CD Pipelines with Azure DevOps. SAS Viya Decision Pipeline with Azure DevOps SAS Viya and Azure Pipelines Self-Hosted Agents SAS Viya and Azure Pipelines Microsoft-Hosted Agents ... View more

In the latest development in the realm of containerization, the SAS Viya command-line interface (CLI) has made its debut in a container image. This milestone brings with it a host of advantages, offering streamlined batch operations within SAS Viya Platform deployments or functioning autonomously as a standalone container. For a comprehensive overview, delve into the intricacies outlined in the SAS Viya Platform: Using the Command-Line Interface documentation available here: SAS® Help Center - What’s New in Administration - 2024.03 (March 2024) - Command-Line Interfaces On the same subject, Gerry Nelson will be releasing soon an article describing an "Overview of installing and using the SAS Viya containerized CLI". This article might be a good introduction of this new component prior jumping into customizing it for DevOps purposes. This post marks a significant update from my previous entries, "Unlocking Efficiency with Container Jobs in Azure DevOps" and "Streamlining SAS Viya Authentication in Azure DevOps". Noteworthy changes include a shift towards employing a Linux-based client, and, most notably, leveraging the newly available sas-viya-cli image included to any SAS Viya 2024.03 deployment.  Source: Streamlining SAS Viya Authentication in Azure DevOps   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   For those familiar with my prior writings, the procedural steps to integrate the SAS Viya CLI into an Azure Container Job may ring familiar: Retrieve Your Credentials: Authenticate to cr.sas.com to gain access to and utilize the new sas-viya-cli container image. (Optional) Consider pushing the sas-viya-cli container image into your Azure Container Registry for added convenience. Authenticate to your SAS Viya platform and procure your access and refresh tokens. Push Your Credentials: Upload your credentials.json file into your Microsoft Azure Key Vault. Utilize Your Credentials: Build an updated container image tailored for deployment within the container job via the Azure DevOps pipeline. Push the resultant image to your designated Azure Container Registry. Reflect the necessary adjustments in your Azure DevOps pipeline. Execute your Azure DevOps pipeline. In the interest of clarity and brevity, this post refrains from delving into additional configurations, reserving them for a forthcoming follow-up post. Topics slated for discussion include links to security enhancements, encompassing considerations such as: Azure KeyVault IP restrictions Azure Kubernetes Cluster API IP restrictions Kubernetes Ingress IP restrictions Use / Revocation of the Refresh-Token Certificates not chained to a public Certificate Authority Stay tuned for the subsequent post, where I will elaborate on these critical aspects. Now, let's delve into the intricacies of each step involved in integrating the SAS Viya CLI into Azure DevOps container jobs.   Authenticate to cr.sas.com be able to pull & use the new sas-viya-cli container image   In the pursuit of integrating the sas-viya-cli container image into your workflows, a crucial initial step entails authenticating to cr.sas.com to allow pulling the new image. Detailed instructions for these actions can be found in the SAS Help Center: Command-Line Interface SAS Help Center: Command-Line Interface documentation, specifically under the section titled "Instructions When Running the CLI from a Container".   Below, I outline the requisite commands, assuming you have already downloaded the SAS Mirror Manager and obtained your order certificates, ensuring a smooth progression through this authentication process:): # Set some variables MIRRORMGR_PATH=$HOME # Run the SAS Mirror Manager command to get cr.sas.com credentials CR_COMMAND=$($MIRRORMGR_PATH/mirrormgr list remote docker login --deployment-data $MIRRORMGR_PATH/SASViyaV4_XXXXXX_certs.zip) # Extracting username using grep and awk DOCKER_CR_USERNAME=$(echo $CR_COMMAND | grep -oP -- '-u \K\S+') # Extracting password using grep and awk DOCKER_CR_PASSWORD=$(echo $CR_COMMAND | grep -oP -- "-p '\K[^']+") docker login -u "$DOCKER_CR_USERNAME" -p "$DOCKER_CR_PASSWORD" cr.sas.com   (Optional) Push the sas-viya-cli container image into your Azure Container Registry   In scenarios where authentication via the cr.sas.com container registry is not feasible or permitted due to network restrictions, an alternative option involves pushing the sas-viya-cli container image into your Azure Container Registry. This ensures accessibility and control over the image within your Azure environment. Before proceeding, ensure you've logged in to Azure using the az login command and authenticated to cr.sas.com using the previous step commands.   Here are the step-by-step instructions, tailored for execution within a Linux Command prompt (bash): # az login SOURCE_DOCKER_REGISTRY=cr.sas.com TARGET_DOCKER_REGISTRY=myownacr TARGET_DOCKER_ACCESS_TOKEN=$(az acr login --name $TARGET_DOCKER_REGISTRY --expose-token | jq -r '.accessToken') docker login $TARGET_DOCKER_REGISTRY.azurecr.io -u 00000000-0000-0000-0000-000000000000 -p $TARGET_DOCKER_ACCESS_TOKEN az acr import --force -n $TARGET_DOCKER_REGISTRY \ --username $DOCKER_CR_USERNAME --password $DOCKER_CR_PASSWORD \ --source $SOURCE_DOCKER_REGISTRY/viya-4-x64_oci_linux_2-docker/sas-viya-cli:latest   Authenticating to Your SAS Viya Platform and Retrieving Access & Refresh Tokens   In preparation for running SAS Viya CLI from an Azure DevOps environment, the initial step involves retrieving your SAS Viya credentials locally before uploading them to an Azure Key Vault.   Below are the step-by-step instructions, tailored for execution within a Linux Command prompt (bash): # Set some variables DOCKER_REGISTRY=cr.sas.com export SAS_SERVICES_ENDPOINT=https://azureuse011070.my-trials.sas.com # docker run --rm -it -e JWT_TOKEN -e SAS_SERVICES_ENDPOINT $DOCKER_REGISTRY/viya-4-x64_oci_linux_2-docker/sas-viya-cli:latest auth loginCode docker run --rm -it -v $PWD:/cli-home/.sas -e SAS_SERVICES_ENDPOINT $DOCKER_REGISTRY/viya-4-x64_oci_linux_2-docker/sas-viya-cli:latest auth loginCode Upon executing these commands, you'll obtain your access and refresh tokens. Subsequently, you'll have two authentication options within your Azure DevOps pipeline: Option 1: Pass the Access token as the JWT_TOKEN variable. This approach is valid until the access token expires, typically within a short timeframe defined by the sas.logon.jwt.policy.accessTokenValiditySeconds parameter (default: 1 hour). While efficient, this method may not be suitable for lengthy DevOps pipeline executions. Option 2: Rebuild the credentials.json file using both the access and refresh tokens. This method remains valid until the refresh token expires, determined by the sas.logon.jwt.policy.refreshTokenValiditySeconds parameter (default: 14 days). This approach offers greater longevity, making it potentially more suitable for prolonged DevOps pipeline executions. Option 1 will be detailed in this post, option 2 will be detailed the follow-up post.   Uploading Your Credentials to Azure Microsoft KeyVault   Having obtained a valid JSON Web Token (JWT) and a refresh token, the next step is to securely upload these credentials to your Azure Key Vault. This ensures centralized management and enhanced security of your sensitive authentication data. Even if option 1 only require uploading the JWT_TOKEN, for flexibility and in anticipation to use option 2, I opted to upload the entire credentials.json file, incorporating both the access and refresh tokens along with their respective expiry dates, as a single object into your Azure Key Vault. This consolidated approach eliminates the need to manage these tokens separately, thereby streamlining the authentication process within your Azure environment while ensuring comprehensive security and compliance.   Below are the instructions for executing this task within a Linux Command prompt (bash): # az login (prerequisite) # Set some variables AZURE_KEYVAULT_NAME=SAS-COMMUNITIES-KV AZURE_SECRET_NAME=SAS-COMMUNITIES-CREDENTIALS # Create a secret (if one doesn't exist) or update a secret in a KeyVault. az keyvault secret set --vault-name $KV --name $CR --file $PWD/credentials.json --output none   Upon execution, your credentials will be securely stored within your Azure Key Vault, safeguarding them against unauthorized access and ensuring seamless integration with your Azure DevOps pipeline.   Enhancing Container Image Creation for Azure DevOps Pipeline   In my previous post, we manually installed the SAS Viya CLI during the Docker image build process. However, in this iteration, we will optimize our workflow by utilizing the new sas-viya-cli image as the foundation for our updated container job image. The only additional component we incorporate is nodejs, a prerequisite for images used in Azure Container Jobs.   Below is the Dockerfile showcasing the enhancements: FROM cr.sas.com/viya-4-x64_oci_linux_2-docker/sas-viya-cli:latest USER 0 RUN mv /etc/yum.repos.d/ubi.repo /etc/yum.repos.d/ubi.repo.sas \ mv /etc/yum.repos.d/ubi.repo.orig /etc/yum.repos.d/ubi.repo \ rm /etc/yum.repos.d/sas* RUN dnf install -y --nodocs nodejs && dnf clean all ENTRYPOINT [ "" ] CMD [ "node" ] Please note that this example employs cr.sas.com as the container registry. However, if you followed the optional step, you could utilize your own registry during the build process.   Now, let's proceed with the creation of the updated container image. Execute the following commands in your Linux Command prompt (bash): # Set some variables TARGET_DOCKER_REGISTRY=myownacr.azurecr.io VERSION=1.0 # Build locally the new Container Job docker image and tag it to be pushed to your own Azure Container Registry docker build . -t $TARGET_DOCKER_REGISTRY/viya-4-x64_oci_linux_custom/cj-sas-viya-cli:$VERSION-$(date +%Y%m%d.%s) -t $TARGET_DOCKER_REGISTRY/viya-4-x64_oci_linux_custom/cj-sas-viya-cli:latest The resulting image will then be tagged appropriately for utilization within your Azure DevOps pipeline and subsequently pushed to your designated Azure Container Registry.   Publishing Your Local Image to Azure DevOps Pipeline via Azure Container Registry   Now that you've successfully built your Docker image locally, the next step is to make it accessible to your Azure DevOps pipeline by hosting it on your own Azure Container Registry (ACR). This ensures seamless integration and deployment within your pipeline environment.   Below are the instructions to execute within a Linux Command prompt (bash): # Set some variables TARGET_DOCKER_REGISTRY=myownacr TARGET_DOCKER_ACCESS_TOKEN=$(az acr login --name $TARGET_DOCKER_REGISTRY --expose-token | jq -r '.accessToken') # Log in to the Azure Container Registry to be able to push the image docker login $TARGET_DOCKER_REGISTRY.azurecr.io -u 00000000-0000-0000-0000-000000000000 -p $TARGET_DOCKER_ACCESS_TOKEN docker push -a $TARGET_DOCKER_REGISTRY.azurecr.io/viya-4-x64_oci_linux_custom/cj-sas-viya-cli   Updating Your Azure DevOps Pipeline for Enhanced Integration   Now, we'll walk through the necessary steps to reflect the changes introduced, ensuring smooth deployment of your SAS Viya CLI commands in your Azure DevOps pipeline.   Defining Parameters for Enhanced Flexibility   To start, let's define the parameters required for your pipeline. These parameters provide flexibility and customization options tailored to your specific environment: parameters: - name: SAS_VIYA_URL displayName: Specify the URL for the SAS Viya platform environment type: string default: https://sasviyarg.gelenable.sas.com - name: AZURE_KEYVAULT displayName: Specify the Azure KeyVault you want to use type: string default: SAS-COMMUNITIES-KV - name: AZURE_KEYVAULT_KEY displayName: Specify the Secret you want to use type: string default: SAS-COMMUNITIES-CREDENTIALS These parameters allow you to customize the pipeline according to your SAS Viya platform URL and Azure KeyVault configuration, providing greater control and adaptability.   Updating the Pipeline Structure   With the parameters defined, let's update the pipeline structure to reflect the latest changes. We'll introduce two jobs: one to retrieve the credentials from KeyVault and another to execute your SAS Viya CLI commands.   Retrieving Your Credentials from Azure KeyVault   Now, we'll guide you through the process of retrieving your credentials.json file from Azure KeyVault.   Here is the yaml extract for the job to execute this step: - job: getJWTfromAKV displayName: Get SAS Viya Access Token steps: - task: AzureCLI@2 name: TgetJWTfromAKV displayName: Get JSON Web Token from Azure Key Vault inputs: azureSubscription: '$(KV_SERVICE_CONNECTION)' scriptType: 'bash' scriptLocation: 'inlineScript' inlineScript: | az keyvault secret download --name ${{ parameters.AZURE_KEYVAULT_KEY }} --vault-name ${{ parameters.AZURE_KEYVAULT }} --file $PWD/credentials.json export JWT_TOKEN=$(cat $PWD/credentials.json | jq -r '.Default."access-token"') echo "##vso[task.setvariable variable=JWT_TOKEN;isOutput=true]$JWT_TOKEN" This job, titled "Get SAS Viya Access Token," employs the AzureCLI task to extract the JSON Web Token (JWT) from Azure KeyVault. Here's a breakdown of the steps: Download Credentials: Utilizing the Azure CLI, the script downloads the credentials.json file from Azure KeyVault and saves it to the pipeline workspace. Extract JWT Token: The script extracts the access token from the downloaded credentials.json file using jq. Set Output Variable: Finally, the access token is stored as an output variable, enabling its use in subsequent pipeline steps. Notice that this step is NOT using a container job but uses a “standard” job where the Azure CLI is available by default.   Executing Your Azure DevOps Pipeline   Now, we'll guide you through the process of running your Azure DevOps pipeline, leveraging the SAS Viya CLI commands for streamlined execution.   Here is the yaml extract for the job to execute this step: - job: runViyaClilogin dependsOn: getJWTfromAKV displayName: Run the SAS Viya CLI commands variables: - name: JWT_TOKEN value: $[ dependencies.getJWTfromAKV.outputs['TgetJWTfromAKV.JWT_TOKEN'] ] container: image: $(AZURE_ACR).azurecr.io/viya-4-x64_oci_linux_custom/cj-sas-viya-cli:1.0-20240329.1711733600 endpoint: 'ACR Connection' env: SAS_SERVICES_ENDPOINT: ${{ parameters.SAS_VIYA_URL }} JWT_TOKEN: $(JWT_TOKEN) steps: - task: Bash@3 displayName: Retrieve all configuration definition list inputs: targetType: 'inline' script: | cd /opt/sas/viya/home/bin /opt/sas/viya/home/bin/sas-viya-wrapper --output text configuration configurations list This job, titled "Run the SAS Viya CLI commands," orchestrates the execution of SAS Viya CLI commands within your Azure DevOps pipeline. Here's an overview of the steps involved: Dependencies: This job depends on the successful execution of the previous job, ensuring that the SAS Viya access token is retrieved from Azure KeyVault beforehand. Define Variables: The access token retrieved from KeyVault is assigned to the JWT_TOKEN variable, facilitating secure authentication for SAS Viya CLI commands. Container Configuration: The job runs within a specified container image hosted in your Azure Container Registry (ACR), ensuring consistency and reproducibility of the execution environment. Environment Configuration: Environment variables are configured to specify the SAS Viya platform URL and pass the JWT token for authentication. Execution Steps: The SAS Viya CLI commands are executed within the pipeline, enabling tasks such as retrieving configuration definition lists for seamless management of your software solutions.   Conclusion   In this post, we've introduced the new SAS Viya Command-Line Interface (CLI) container image for Azure DevOps pipelines. This update streamlines SAS Viya Platform deployments, offering efficiency and flexibility. We began by outlining the transition to a Linux-based client and leveraging the sas-viya-cli image. Then, we integrated the CLI into Azure DevOps pipelines, covering credential retrieval from Azure KeyVault, Docker image optimization, and pipeline updates. Executing the pipeline, we seamlessly ran SAS Viya CLI commands within our containerized environment. Stay tuned for deeper dives into configurations and security enhancements in future posts. Our containerization journey continues with more insights and innovations.   Additional Azure DevOps Resources with SAS Viya   David Estreich (myself) related posts with Azure DevOps: Unlocking Efficiency with Container Jobs in Azure DevOps Streamlining SAS Viya Authentication in Azure DevOps Bogdan Teleuca's other posts about SAS Viya with Azure DevOps: SAS Viya with Azure DevOps Data Pipeline. SAS Viya CI/CD Pipelines with Azure DevOps. SAS Viya Decision Pipeline with Azure DevOps SAS Viya and Azure Pipelines Self-Hosted Agents SAS Viya and Azure Pipelines Microsoft-Hosted Agents Find more articles from SAS Global Enablement and Learning here. ... View more

This article is a follow-up post on my previous one about “Unlocking Efficiency with Container Jobs in Azure DevOps” available here. That article demonstrated the use of “Container Jobs” but more specifically a job containing the SAS Viya CLI (Command Line Interface). Now what you will need is a way to authenticate during the run of your Azure pipeline.   Our odyssey begins with the SAS Viya CLI, choreographing the Auth Code method for authentication, and seamlessly segues into the Azure Key Vault, where credentials.json finds a secure sanctuary. But that is not all; we are threading this narrative into the broader tapestry of configuring an environment through the Azure DevOps pipeline using this credential.json file.   Overall process   The main authentication steps are illustrated in the following diagram:   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   If you have not already installed your SAS Viya CLI, my colleague David Stern did a great post documenting all the steps here. There is also a YouTube video from the SAS Users channel available here. If you are looking for ideas or use-cases for the the SAS Viya CLI, here is a list of SAS Communities posts referencing the SAS Viya CLI.   Once this is done, you should be able to authenticate using the “Auth Code” method. I am recommending this method as it should be valid for any of your deployed authentication process (Direct LDAP / OpenID Connect or SCIM).   In the following steps, I have assumed you are on Windows and have deployed the SAS Viya CLI under the Download/sas-viya-cli folder. This should be easily adapted if you are on a Mac or Linux.   1. Authenticate on Azure & on SAS Viya using their CLIs   Copy & Paste the following instructions in a Windows Command prompt (cmd):   :: Variables section set PATH=%PATH%;%userprofile%\Downloads\sas-viya-cli set EP :: sign in to Azure az login --output none :: sign in to SAS Viya sas-viya profile init --sas-endpoint %EP% --output text --colors-enabled yes sas-viya authenticate loginCode   NOTE: Replace sas-communities.demo.sas.com with your SAS Viya URL. Once prompted to do so, open the provided URL and authenticate to your SAS Viya Environment. This step will generate a credentials.json file that will contain your access and refresh tokens that you will need to share with your pipeline for later execution on your behalf. This file will be stored here : %userprofile%/.sas/credentials.json.   2. Create a resource group on Azure to hold the Azure Key Vault (if you do not have one you want to use)   Copy & Paste the following instructions in in a Windows Command prompt (cmd):   :: Variables section set RG=SAS-COMMUNITIES-RG az group create --location eastus2 --resource-group %RG% --output table   3. Create your Azure Key Kault (if you do not have one you want to use)   Copy & Paste the following instructions in a Windows Command prompt (cmd):   :: Variables section set RG=SAS-COMMUNITIES-RG set KV=SAS-COMMUNITIES-KV az keyvault create --name %KV% --resource-group %RG% --output table   4. Upload your credentials.json into a secret into Azure KeyVault   Copy & Paste the following instructions in a Windows Command prompt (cmd):   :: Variables section set SC=Service for Pipeline Execution set KV=SAS-COMMUNITIES-KV set CR=SAS-COMMUNITIES-CREDS az keyvault secret set --vault-name SC% --name %CR% --file %userprofile%/.sas/credentials.json   5. Check the existence in the Microsoft Azure Portal   Go to: Home >> Key Vaults >> SAS-COMMUNITIES-KV >> Secrets >> SAS-COMMUNITIES-CREDS >> Select the “Current Version” id     Modify your pipeline to consume the credentials.json from Azure Key Vault   Now that you have uploaded your credentials into a Microsoft Key Vault secret, the next step is to modify your pipeline in order to consume them.   1. Identify your Service connection in your Azure DevOps pipeline   On my previous blog post you should spot the name after the AzureScription, in my case I used   variables: azureSubscription: Service for Pipeline Execution   On Azure DevOps, you can also confirm the name by clicking on “Project Settings” >> “Service Connections”     Here is some more information on Service Connection from the Azure documentation: Manage service connections   2. Give the appropriate rights on your Azure KeyVault   Your pipeline will need to be able to read your Azure Key Vault entry. To enable this, you will need to set two policies (get and list) on your Azure KeyVault for your Service Connection (aka. Service principals).   :: Variables section set SC=Service for Pipeline Execution set KV=SAS-COMMUNITIES-KV for /f %i in ('az ad sp list --filter "displayname eq '%SC%'" --query "[].appId" --output tsv') do set SP_ID=%i az keyvault set-policy --name  %KV% --object-id %SP_ID% --secret-permissions get list --output table   3. Update your Azure DevOps pipeline definition   This extract is taken from my previous post that you will need to update :   - task: Bash@3 # Second task referenced in the explanation bellow inputs: targetType: 'inline' script: | # set -x VIYA_URL=https://myownviya.communities.sas.com KV=SAS-COMMUNITIES-KV CR=SAS-COMMUNITIES-CREDS sas-viya profile init --sas-endpoint https://$VIYA_URL --output json  --colors-enabled yes rm -Rf /home/vsts_azpcontainer/.sas/credentials.json 2>/dev/null az keyvault secret download \ --file /home/vsts_azpcontainer/.sas/credentials.json \ --name $CR --vault-name $KV sas-viya --verbose configuration configurations list   If all went as expected, this task will display a list of configurations for your SAS Viya environment. Now, it is up to you to script whatever you want to automate using the SAS Viya CLI (i.e. User creation, folder creations, rules assignments, configurations deployment, …).   Conclusion   In conclusion, this post has walked you through the process of streamlining SAS Viya authentication in Azure DevOps. Starting with the SAS Viya CLI and the Auth Code method for authentication, we seamlessly integrated the Azure Key Vault to securely store credentials. The journey continued with configuring your environment through the Azure DevOps pipeline, using the credentials.json file generated through the authentication process.   By following the step-by-step instructions, you have learned how to:   Generate credentials using the local SAS Viya CLI. Authenticate on Azure and SAS Viya using their CLIs. Create a resource group and Azure Key Vault to securely store your credentials. Identify and set up your Service Connection in Azure DevOps. Grant appropriate rights on your Azure Key Vault for your Service Connection. Upload the credentials.json into the Azure Key Vault as a secret. Modify your Azure DevOps pipeline to consume the credentials from the Azure Key Vault.   This guide aims to empower you to enhance the security and efficiency of your Azure DevOps pipeline when working with SAS Viya. By leveraging the capabilities of Azure Key Vault and following proven practices, you can ensure a robust and secure authentication process for your SAS Viya environment. As you embark on your journey of orchestrating SAS Viya tasks within your pipelines, this post can serve as a valuable resource, enabling you to navigate the intricacies of authentication seamlessly. ... View more

In the fast-paced world of software development, streamlining workflows and ensuring consistency across environments are critical challenges. Azure DevOps, Microsoft's comprehensive set of development tools, provides a solution to these challenges through the use of container jobs. In this post, we'll explore why container jobs are essential, how to build and store containers, and the best practices for designing YAML files to consume them effectively.   In this post, I will illustrate the concept through the following SAS common use case:   How can we integrate SAS Viya CLI features into your Azure DevOps pipeline?   Below are a few supplementary constraints:   I opted against using a jump box, a typical scenario for installing SAS Viya CLI. I prefer steering clear of an Azure DevOps custom agent since it would require running either on the jump box or the Kubernetes Cluster. Thus, my sole viable choice is to utilize a Microsoft-Hosted Agent in conjunction with container jobs.   Why Use Container Jobs?   Containerization is a game-changer in software development, offering a lightweight and portable solution for packaging, distributing, and running applications. Azure DevOps leverages containerization through container jobs, providing a consistent environment for your builds and releases.   Overall Execution Workflow     Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   Get the execution workflow (YAML) from the Azure Repos Start the execution of the Azure Pipeline using a Microsoft-Hosted Agent Pull the container image containing the SAS Viya CLI Add the agent IP to the k8s API and the loadBalancerRange to allow the agent to communicate with the SAS Viya AKS Cluster Run the SAS Viya CLI task on the agent (optional) When completed, you can remove the agent IP to the loadBalancerRange and the k8s API allowed list.   Building and Storing Containers   To use container jobs effectively, you first need to build your container. Azure DevOps supports various container technologies such as Docker. In your project repository, create a Dockerfile specifying the dependencies and configurations required for your application.   The Dockerfile is set to install the SAS Viya CLI, along with all accessible plugins and the pyviyatools project from GitHub, ensuring the installation of all necessary prerequisites for these tools. Additionally, it will address context prerequisites, enabling container job execution by configuring the azpcontainer user with all required permissions and binaries and ensuring the execution of the node command, a prerequisite for Azure DevOps.   See Non glibc-based containers for guidelines.   Dockerfile : # Step 0: RedHat UBI8 base image FROM registry.access.redhat.com/ubi8/ubi USER 0 # Step 1: Install the container job prerequisites RUN dnf install -y --nodocs bind-utils sudo git python3 nodejs tar iputils jq && dnf clean all RUN groupadd -g 123 docker_azpcontainer \ && useradd -m -g 123 -u 1001 vsts_azpcontainer # Step 2: Copy and install your certificates (if you need) # COPY SAS_SHA2_Issuing_CA02_full_chain.pem /etc/pki/ca-trust/source/anchors # RUN update-ca-trust # Step 3: Install the Azure CLI RUN rpm --import https://packages.microsoft.com/keys/microsoft.asc \ && dnf install -y https://packages.microsoft.com/config/rhel/8/packages-microsoft-prod.rpm \ && dnf install -y azure-cli # Step 4: Copy and install the SAS Viya CLI COPY sas-*.tgz /tmp/ RUN mkdir -p /tmp/utilities \ && tar -xvf /tmp/sas-viya-cli-*.tgz -C /tmp/utilities \ && mv /tmp/utilities/sas-viya /usr/local/bin \ && rm -Rf /tmp/utilities \ && chmod +x /usr/local/bin/sas* \ && rm -f /tmp/sas-* \ && ln -s /usr/bin/python3 /usr/bin/python # Step 5 (optional): Install pyviyatools prerequisites RUN pip3 install requests argparse configobj USER 1001 WORKDIR /home/vsts_azpcontainer # Step 6: Install All plugins of the sas-viya CLI RUN sas-viya plugins install --repo SAS all # Step 7: Install pyviyatools from GitHub RUN git clone https://github.com/sassoftware/pyviyatools.git #LABEL "com.azure.dev.pipelines.agent.handler.node.path"="/usr/bin/node" USER 0 CMD [ "node" ]   Some clarifications: Step 0: I decided to use the ubiI8 image as this is a base image mostly used by SAS in SAS Viya. Step 1: While the pipeline is in progress, Azure DevOps will attempt to execute the container under the user called vsts_azpcontainer. To ensure that all SAS Viya plugins are installed in the user's directory, I opted to establish the user during the build phase and supply the plugins to the user's home directory.   Build your Docker image (first locally) using the following command:   docker build -t sas-viya-cli:latest .   Storing Containers: When designing a container, a frequently used strategy is to first build it locally on a Linux-based system with Docker installed (I am using WSL) or using Docker for Desktop. Afterward, validation tests are carried out prior to pushing the updated image version to a central repository (in our case an Azure Container Registry). Note: The most frequent issues often come from communication problems, network issues, or SSL certificate-related issues.   Azure Container Registry (ACR) is a fully managed Docker registry service that makes it easy to manage container images. Here is a snippet to be able to build & push your image to your ACR :   az login # or az logion --use-device-code az account set --subscription "myownsubscription" az acr build --registry myownacr --image viya-tool/sas-viya-cli:latest .   Designing YAML for Container Jobs   Now, let's design the YAML file to use the container in your Azure DevOps pipeline.   variables: azureSubscription: Service for Pipeline Execution trigger: - none jobs: - job: runViyaCliLogin container: image: myownacracr.azurecr.io/viya-tool/sas-viya-cli:latest endpoint: 'ACR Connection' steps: - task: KubectlInstaller@0 inputs: kubectlVersion: 'latest' - task: AzureCLI@2 # First task referenced in the explanation bellow inputs: azureSubscription: '$(azureSubscription)' scriptType: 'bash' scriptLocation: 'inlineScript' inlineScript: | # set -x AZURE_RG=sasviyarg AZURE_AKS=sasviyaaks AZURE_RANGE=$(az aks show \ --resource-group $AZURE_RG \ --name $AZURE_AKS \ --query apiServerAccessProfile.authorizedIpRanges \ | jq -r '. | join(",")') # Store the value to reset it later echo "##vso[task.setvariable variable=AZURE_RANGE]$AZURE_RANGE" # Retrieve your IP address CURRENT_IP=$(dig +short "myip.opendns.com" "@resolver1.opendns.com") # Add to AKS approved list az aks update -g $AZURE_RG -n $AZURE_AKS --api-server-authorized-ip-ranges ${CURRENT_IP}/32,${AZURE_RANGE} az aks get-credentials --resource-group $AZURE_RG \ --name $AZURE_AKS \ --overwrite-existing kubectl patch services ingress-nginx-controller -n ingress-nginx \ --type=json -p='[{"op": "add", "path": "/spec/loadBalancerSourceRanges/-", "value": "'${CURRENT_IP}/32'" }]' enabled: false displayName: Add Agent IP to the AKS Cluster (IPs) - task: Bash@3 # Second task referenced in the explanation bellow inputs: targetType: 'inline' script: | sas-viya --help - task: AzureCLI@2 # Third task referenced in the explanation bellow inputs: azureSubscription: '$(azureSubscription)' scriptType: 'bash' scriptLocation: 'inlineScript' inlineScript: | # set -x AZURE_RG=sasviyarg AZURE_AKS=sasviyaaks # Retrieve your IP address CURRENT_IP=$(dig +short "myip.opendns.com" "@resolver1.opendns.com") INDEX=$(kubectl get services ingress-nginx-controller -n ingress-nginx -o json | jq '.spec.loadBalancerSourceRanges | map(. == "'${CURRENT_IP}/32'") | index(true)') kubectl patch services ingress-nginx-controller -n ingress-nginx \ --type=json -p='[{"op": "remove", "path": "/spec/loadBalancerSourceRanges/'$INDEX'"}]' # Add to AKS approved list az aks update \ --resource-group $AZURE_RG \ --name $AZURE_AKS \ --api-server-authorized-ip-ranges $(AZURE_RANGE) enabled: false displayName: Remove Agent IP to the AKS Cluster (IPs)   The container section specifies the container image to use for this job. The first task is designed to add the agent IP to the api-server-authorized-ip-ranges to allow access to the Kubernetes APIs. Then, add the agent IP to the to the loadBalancerSourceRanges in the NGINX controller. This will automatically update the “internal” Network Security Group to allow the agent IP to connect to the AKS Cluster. The second task illustrate the “simple” usage of the sas-viya CLI. The third task removes the agent address to the loadBalancerSourceRanges and the api-server-authorized-ip-ranges.   Conclusion   By leveraging container jobs in Azure DevOps, you're not just building and testing your code; you're ensuring consistency, improving security, and optimizing resource utilization throughout your development pipeline.   This instance demonstrates that we can leverage Azure DevOps' standard features, such as installing the kubectl tool, while also incorporating additional functionalities not inherently provided, like integrating the sas-viya CLI into the same pipeline.   In an upcoming post, I will detail the process of integrating with Azure KeyVault to authenticate and fully utilize the features of the sas-viya CLI.   If you have an interest in Azure DevOps and exploring various ways to integrate it with SAS Viya, I recommend checking out Bogdan Teleuca posts, including topics such as:   Microsoft-hosted agents vs. self-hosted agents How to Create a SAS Viya Docker Agent for Azure DevOps     Find more articles from SAS Global Enablement and Learning here. ... View more

Some time ago, my colleague Raphael Poumarede shared how to Reduce the Azure bill when you are not using the SAS Viya environment.   To be even more cost effective in cloud resources, a common approach is to schedule the stop and the start of your Kubernetes cluster at fixed hours. At the same time and to not limit the SAS users, the decision is usually taken to delay the stop “as late as possible” in the evening and to anticipate the restart “as soon as possible” in the morning. What if the SAS Users can play a role to optimize the cloud costs? What if the SAS Users can decide to postpone a stop or anticipate a start by themself without anyone to be involved? In this post I will share an example on how this can be achieved. This example will mainly be using Azure resources as they simplify the integration with MS Teams, but the concept can be adapted to any cloud provider.   Microsoft Tools used   Microsoft Teams Microsoft Teams is a collaboration platform for communication, meetings, and teamwork within organizations. It integrates with Microsoft 365 apps and offers features like chat, video conferencing, and document sharing. We are using Teams as a “simple” way to trigger an action.   Azure Logic Apps Azure Logic App automates tasks and integrates apps and services in the cloud. It offers a visual interface for creating workflows without extensive coding. This simplifies and accelerates business processes.  We are using Logic App as a sequencer for all tasks.   Azure DevOps  Azure DevOps is Microsoft's integrated suite of development tools and services. It covers the software development lifecycle, enabling teams to collaborate, automate workflows, and deliver high-quality software efficiently. We are using Azure DevOps to get access to external commands (kubectl) not available in Logic App.   Prerequisites   SAS Viya platform is configured and available Microsoft - Create a standard or private channel in Microsoft Teams Microsoft - Create Incoming Webhooks The ability to create a Microsoft Logic App in the Azure Portal   Process overview in Microsoft Teams   The following screenshot shows an example of implementing this approach.    Select any image to see a larger version. Mobile users: If you do not see this image, scroll to the bottom of the page and select the "Full" version of this post.   Start Process   Step1: The user (here David) sends a message to start the SAS Viya platform. Step 2: The system will respond that the received has been received. This is only to confirm that the request has been received until the SAS Viya platform will be up-and-running. Step 3: When the SAS Viya platform is completely up-and running the system sends the message in the channel with the URL of the environment.   Stop Process   Step1: The user (here David) sends a message to stop the SAS Viya platform. Step2: The system sends the message when the SAS Viya platform is completely stopped.   Logic App Flow Design (for the Start Process)   The general idea is to use the Logic App as a sequencer for the following steps:   Trigger the sequence when a keyword is used in a specific channel in MS Teams Send a “request received” to acknowledge the request List the associated Azure Resources For each Flexible Server (if you are using an external PostGreSQL instances) List them (if more than one, i.e for CDS instances) Start them using the Azure Microsoft.DBforPostgreSQL API For each AKS Cluster List them Start them using the Azure Microsoft.ContainerService API This will start the AKS Cluster Therefore, will start the SAS Viya Platform Trigger an Azure DevOps pipeline to Run a kubectl command the check the status of the SAS Viya platform (and wait for a complete start) Send a message when the platform is fully available   Here is the graphical overview of the Logic App I designed in Microsoft Portal.     Now let’s describe each node of this Logic App.       When keywords are mentioned Step Type: Microsoft Teams: When keywords are mentioned This is where you will define your “trigger”. I used the keyword StartSASViya in Team “SAS Viya Environment” in the channel “Personal -Latest”. Hints: Don’t use # in the keywords     HTTP Post message in a chat or channel Step Type: HTTP This step is a call to your predefined WebHooks. The content of the message is in JSON format, click on the image to see the content     List resources by resource group Step Type: Azure Resource Manager This step will retrieve all Azure resources in the “fradae-stable-202308-rg” resource group.     Get Flexible Servers Step Type: Variables Built-in This step will filter the previous output only with object with type: “Microsoft.DBforPostgreSQL/flexibleServers”     For each Flexible Server Step Type: For each (Control) This step will loop on each “Flexible Server” in the resource group. This step will first get the full ID of the Flexible Server in this format: /subscriptions/c1b1cbfa-699a-47f5-95c5-aaaaaaaaaaaa/resourceGroups/fradae-stable- 202308/rg/providers/Microsoft.DBforPostgreSQL/ flexibleServers/fradae-stable-202308-default-flexpsql Then we will extract the variable needed to be able to start the Flexible Server through an API :   Subscription ID Resource Group Flexible Server Name   The method used is to “Split” the ID, (i.e for the Subscription ID : split(outputs('Extract_PG_ID'),'/')[2]). This will extract c1b1cbfa-699a-47f5-95c5-aaaaaaaaaaaa. The same method is used for the other needed variables. The last step is the actual start of the “Flexible Server” using the API described here       Get AKS Clusters (same logic as for Flexible Server) Step Type: Variables Built-in This step will filter the previous output only with object with type: “Microsoft.ContainerService/managedClusters”     For each AKS Cluster (same logic as for Flexible Server) Step Type: For each (Control) The last step is the actual start of the “AKS Cluster” using the API described here     Send an HTTP request to Azure DevOps Step Type: Send an HTTP requests to Azure DevOps This will trigger a DevOps pipeline (mainly to be able to use a kubectl command) to wait until the SAS Viya Platform will be fully functional. Then the pipeline will send the completion message in the same predefined WebHooks. The content of the request Body is in JSON format, click on the image to see the content Hints: The JSON object resources.repositories.self.refName is mandatory and should reflect your Azure DevOps branch name of the pipeline you want to execute   Conclusion   This example shows how SAS Users can be an active part of the cloud cost optimization. Some other examples could be implemented using the same approach:   Same logic could be used to stop the SAS Viya platform A more "controlled” flow could be designed to add some approval steps between a “SAS User” request for a restart and the request to the “SAS Admin” Team that will consume the logic to start the SAS Viya platform If you have scheduled stops (based on tags), you can define Logic Apps to avoid the stop Time-based Logic App to define when critical batch will run   I hope this brief example of the integration between Azure Logic App / Microsoft Teams and the SAS Viya platform will be useful to you. Special thanks to Thomas and Pooja as this example was inspired by the combination of this post from Thomas Stringer, Schedule AKS Clusters to Start and Stop Automatically and a post from my colleague Pooja Dalal and customized it to add the MS Teams interaction.     Find more articles from SAS Global Enablement and Learning here. ... View more