BookmarkSubscribeRSS Feed

SAS Viya CLI container image in Azure DevOps, security topics – Part 1 -> IP Restrictions

Started ‎07-23-2024 by
Modified ‎07-23-2024 by
Views 985

Welcome back! This first post serves as a follow-up to our previous post, where we initiated discussions on DevOps processes with SAS Viya CLI container images, available here, Introducing New SAS Viya CLI container image in Azure DevOps (March 2024 Update). In this two-part blog, we delve deeper into security customizations, addressing aspects such as managing refresh tokens, handling certificates not chained to public Certificate Authorities (CAs), and enhancing security measures within Azure Pipelines and Kubernetes clusters. These enhancements build upon our previous discussions, aiming to fortify your DevOps pipelines against security vulnerabilities while ensuring seamless execution. Let's explore these advancements in detail.

 

Here's an illustrative diagram presenting the functional flow involved.

de_1_SAS-Viya-CLI-container-image.png

 

Azure Key Vault Security with IP Restrictions in Azure Pipelines

 

Ensuring that only authorized entities can access these secrets is crucial for maintaining the integrity of your applications and data. One effective way to enhance security is by enforcing IP restrictions, thereby limiting access to Azure Key Vault only from specified IP addresses. In this blog example, we'll explore how to integrate IP restrictions into your Azure Pipelines to bolster the security of your Azure Key Vault.

 

Step 1: Define Parameter for IP Restrictions

 

The first step is to define a parameter in your Azure Pipeline configuration to determine whether IP restrictions should be enforced. This parameter will provide flexibility in enabling or disabling IP restrictions as per your organization's security policies. Below is an example of how to define this parameter:

- name: KV_IP_RESTRICTIONS
  displayName: Do you have IP restrictions for your Azure Key Vault?
  type: boolean
  default: true

 

Step 2: Allow Microsoft Agent IP in Azure Key Vault

 

Before retrieving credentials from Azure Key Vault, it's essential to add the IP address of the Microsoft Agent to the allowed IP list for the Key Vault. This ensures that the pipeline can access the Key Vault securely. To achieve this, include the following Azure CLI task in your pipeline:

- task: AzureCLI@2
  displayName: Allow MSFT Agent IP to Azure Key Vault
  condition: ${{ eq(parameters.KV_IP_RESTRICTIONS, true) }}
  inputs:
    azureSubscription: '$(KV_SERVICE_CONNECTION)'
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      # Retrieve your IP address
      CURRENT_IP=$(curl -s https://ipinfo.io/ip)
      az keyvault network-rule add --name ${{ parameters.AZURE_KEYVAULT }} --ip-address $CURRENT_IP > /dev/null

 

Step 3: Remove Microsoft Agent IP from Azure Key Vault

 

After retrieving credentials from Azure Key Vault, it's crucial to remove the IP address of the Microsoft Agent from the allowed IP list to maintain security posture. This step ensures that unauthorized access attempts from the Microsoft Agent IP are thwarted. Add the following Azure CLI task to your pipeline:

- task: AzureCLI@2
  displayName: DisAllow MSFT Agent IP to Azure Key Vault
  condition: ${{ eq(parameters.KV_IP_RESTRICTIONS, true) }}
  inputs:
    azureSubscription: '$(azureSubscription)'
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      # Retrieve your IP address
      CURRENT_IP=$(curl -s https://ipinfo.io/ip)
      az keyvault network-rule remove --name ${{ parameters.AZURE_KEYVAULT }} --ip-address $CURRENT_IP > /dev/null

These steps ensure that only designated IP addresses, including the Microsoft Agent IP, are permitted to access sensitive information stored in Azure Key Vault. Implementing such security measures aligns with best practices for safeguarding data in the cloud and helps protect your applications from potential security threats.

 

Managing Azure Kubernetes Cluster API IP Restrictions

 

In this blog example, we'll delve into how to handle IP restrictions for AKS APIs, particularly when utilizing the sas-viya-cli in a pipeline executed on a Microsoft-hosted agent.

 

Understanding the Challenge

 

When the AKS API is restricted based on IP addresses, it poses a challenge for pipeline execution using public IPs. We need to ensure that the Microsoft-hosted agent's IP is authorized to access the AKS API. This involves installing the Azure CLI, modifying AKS API IP restrictions, and ensuring proper permissions within the Docker image.

 

Step1: Enhancements to Docker Image

 

To accommodate these requirements, enhancements to the Docker image are necessary. Below is a snippet showcasing the modifications:

RUN dnf install -y --nodocs nodejs sudo && dnf clean all

NOTE: sudo command is mandatory to be able to install the az cli (that will be done while executing the DevOps pipeline).

 

Step2: Define Parameter for AKS APIs IP Restrictions

 

This parameter will provide flexibility in enabling or disabling IP restrictions management as per your organization's security policies. Below is an example of how to define this parameter:

- name: AKS_APIS_IP_RESTRICTIONS
  displayName: Do you have IP restrictions for your AKS APIs?
  type: boolean
  default: false

 

Step 3: Installing Azure CLI (while executing your DevOps pipeline)

 

Installing the Azure CLI within the pipeline environment is mandatory to accommodate the current requirement. Here's how to do it:

- task: Bash@3
  displayName: Install Azure CLI for ubi8 (RHEL8)
  condition: enabled: ${{ eq(parameters.AKS_APIS_IP_RESTRICTIONS, true) }}
  inputs:
    targetType: 'inline'
    script: |
      sudo rpm --import https://packages.microsoft.com/keys/microsoft.asc
      sudo dnf install -y https://packages.microsoft.com/config/rhel/8/packages-microsoft-prod.rpm
      sudo dnf install -y azure-cli

 

Step 4: Modifying AKS API IP Restrictions

 

To add the Microsoft agent's IP to the authorized list for AKS API access, follow these steps:

  1. Retrieve the original AKS API authorized IP ranges for later use.
  2. Add the Microsoft agent's IP to the allowed list.
  3. Ensure proper propagation of the rules by adding a delay (of 10 seconds in this case)
- task: AzureCLI@2
  displayName: Add Agent IP to the AKS API authorized IPs
  condition: ${{ eq(parameters.IP_RESTRICTIONS, true) }}
  inputs:
    azureSubscription: 'Service for PSGEL262'
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      # set -x
      AZURE_RG=${RACE_PREFIX}-rg
      AZURE_AKS=${RACE_PREFIX}-aks
      AZURE_RANGE=$(az aks show \
        --resource-group $AZURE_RG \
        --name $AZURE_AKS \
        --query apiServerAccessProfile.authorizedIpRanges | jq -r '. | join(",")')
      # Store the value to reset it later
      echo "##vso[task.setvariable variable=AZURE_RANGE]$AZURE_RANGE"
      # Retrieve your IP address
      CURRENT_IP=$(curl -s api.ipify.org)
      # Add to AKS approved list
      az aks update \
        --resource-group  $AZURE_RG \
        --name $AZURE_AKS \
        --api-server-authorized-ip-ranges ${CURRENT_IP}/32,${AZURE_RANGE} --output none
      sleep 10

 

Step 5: Resetting AKS API IP Restrictions

 

After pipeline execution, it's essential to reset the AKS API IP restrictions to their original values to maintain the integrity of your environment:

- task: AzureCLI@2
  displayName: Remove Agent IP to the AKS API authorized IPs
  enabled: ${{ eq(parameters.AKS_APIS_IP_RESTRICTIONS, true) }}
  inputs:
    azureSubscription: '$(RG_SERVICE_CONNECTION)'
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      # Add to AKS approved list
      az aks update \
        --resource-group $AZURE_RG \
        --name $AZURE_AKS \
        --api-server-authorized-ip-ranges $(AZURE_RANGE) --output none

By following these steps, you ensure that your pipeline can seamlessly interact with your AKS cluster while maintaining the necessary security measures.

 

Managing Kubernetes Ingress IP Restrictions

 

One common method of enforcing security is by implementing IP restrictions for Kubernetes Ingress. In this blog example, we'll explore how to manage IP restrictions for Kubernetes Ingress effectively.

 

Step 1: Define the parameter for Ingress IP restrictions

 

- name: INGRESS_IP_RESTRICTIONS
  displayName: Do you have IP restrictions for your K8s Ingress?
  type: boolean
  default: false

 

Step 2: Installing kubectl Command Line

 

Before proceeding, ensure that the kubectl command-line tool is installed while executing your pipeline. Here's how to do it:

- task: KubectlInstaller@0
  displayName: Install Kubectl client
  condition: ${{ or(eq(parameters.IP_RESTRICTIONS, true), eq(parameters.PUB_SIGN_CERT, false)) }}
  inputs:
    kubectlVersion: 'latest'

 

Step 3: Adding Microsoft Agent IP to the Ingress

 

To allow the Microsoft agent's IP access to the Kubernetes Ingress, follow these steps:

  1. Retrieve AKS Cluster Credentials: Firstly, we need to authenticate with the AKS cluster to execute commands. This involves retrieving the cluster credentials.
  2. Check Load Balancer Source Ranges: We check if the loadBalancerSourceRanges property exists in the Ingress service specification. This property specifies the allowed IP ranges for accessing the Ingress.
  3. Apply Patch: If the loadBalancerSourceRanges property exists, we apply a patch to add the Microsoft agent's IP address to the list of allowed ranges. If it doesn't exist, we create the property and add the IP address.
  4. Finalize Configuration: Once the patch is applied, the Ingress configuration is updated to allow access from the Microsoft agent's IP address.
- task: AzureCLI@2
  displayName: Add Agent IP to the AKS Ingress authorized IPs
  enabled: ${{ eq(parameters.INGRESS_IP_RESTRICTIONS, true) }}
  inputs:
    azureSubscription: '$(RG_SERVICE_CONNECTION)'
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      az aks get-credentials --resource-group $AZURE_RG \
        --name $AZURE_AKS \
        --overwrite-existing
      # kubectl config view
      # Check if the property exists
      load_balancer_ranges=$(kubectl get services ingress-nginx-controller -n ingress-nginx -o=jsonpath='{.spec.loadBalancerSourceRanges}' 2>/dev/null)
      # Check if the output is empty or not
      if [ -n "$load_balancer_ranges" ]; then
        echo "Property exists, applying patch..."
        kubectl patch services ingress-nginx-controller -n ingress-nginx --type='json' -p='[{"op": "add", "path": "/spec/loadBalancerSourceRanges/-", "value": "'${CURRENT_IP}/32'" }]'
      else
        echo "Property does not exist, creating it..."
        kubectl patch services ingress-nginx-controller -n ingress-nginx --type=json '-p=[{"op": "add", "path": "/spec/loadBalancerSourceRanges", "value": ["'${CURRENT_IP}/32'"] }]'
      fi

 

Step 4: Removing the MSFT Agent IP from the Ingress Allowed List

 

After pipeline execution, it's essential to remove the Microsoft agent's IP from the Kubernetes Ingress allowed list:

  1. Find Index of IP Address: We retrieve the index of the Microsoft agent's IP address within the list of allowed IP ranges specified in the Ingress configuration.
  2. Patch the Ingress Service: Using kubectl, we patch the Ingress service to remove the IP address from the loadBalancerSourceRanges.
  3. Ensure Configuration Update: We ensure that the Ingress configuration is successfully updated by checking for the removal of the IP address from the allowed list.
- task: Bash@3
  displayName: Remove Agent IP from the AKS Ingress authorized IPs
  enabled: ${{ eq(parameters.INGRESS_IP_RESTRICTIONS, true) }}
  inputs:
    targetType: 'inline'
    script: |
      # Find the index of the IP address within the loadBalancerSourceRanges
      INDEX=$(kubectl get services ingress-nginx-controller -n ingress-nginx -o json  | jq '.spec.loadBalancerSourceRanges | map(. == "'${CURRENT_IP}/32'") | index(true)')
      # Patch the Ingress service to remove the IP address
      kubectl patch services ingress-nginx-controller -n ingress-nginx \
        --type=json -p='[{"op": "remove", "path": "/spec/loadBalancerSourceRanges/'$INDEX'"}]'
      # Ensure the configuration is updated
      kubectl patch services ingress-nginx-controller -n ingress-nginx \
        --type=json -p='[{"op": "remove", "path": "/spec/loadBalancerSourceRanges/'$INDEX'"}]'

By following the steps outlined in this blog example, you can effectively control access to your applications while ensuring the necessary level of security.

 

Conclusion

 

By effectively managing network IP restrictions and enhancing security measures within Azure pipelines and Kubernetes clusters, organizations can fortify their DevOps pipelines against potential security threats. These practices not only ensure security but also ensure the reliability and integrity of your applications and data in cloud environments.

 

Additional Azure DevOps Resources with SAS Viya

 

David Estreich (myself) related posts with Azure DevOps:

Bogdan Teleuca's other posts about SAS Viya with Azure DevOps:

Version history
Last update:
‎07-23-2024 02:07 PM
Updated by:
Contributors

hackathon24-white-horiz.png

The 2025 SAS Hackathon Kicks Off on June 11!

Watch the live Hackathon Kickoff to get all the essential information about the SAS Hackathon—including how to join, how to participate, and expert tips for success.

YouTube LinkedIn

SAS AI and Machine Learning Courses

The rapid growth of AI technologies is driving an AI skills gap and demand for AI talent. Ready to grow your AI literacy? SAS offers free ways to get started for beginners, business leaders, and analytics professionals of all skill levels. Your future self will thank you.

Get started

Article Tags