In parts one and two of this blog series we’ve discussed the “Why” and “How” to get started with deployment automation. This third installment focuses on some of the common pitfalls you may encounter as you embark on creating your own automations in deploying SAS environments.   Perfect is the enemy of Good Creating a pipeline that works in all circumstances and encompasses all steps in a deployment is a complicated task. Trying to create the perfect pipeline from the get-go is a plan destined to fail. Like the move from waterfall projects to more agile projects, creating a pipeline in a step-by-step approach is a better way to get started.   The previous blog post showed the starting points you can use when creating a pipeline. Using these resources will kickstart your pipeline creation. So start out with deploying your infrastructure in a minimal viable configuration.  You may have several internal guidelines or SAS Viya design implementation decisions that you have to follow, but these should be implemented in a step-wise approach. The SAS IaC repositories contain sample files you can use to deploy a minimal set of infrastructure. See for instance the sample file for the IaC scripts for Microsoft Azure.   The same principal should be applied to deploying software onto the infrastructure as well. Whether it is supporting software like the NGINX ingress controller or the SAS software itself, again start out with the default configuration. Once you’ve got the initial deployment of the software out of the way, iteratively applying changes to the software makes for a much more pleasant experience. Like the IaC repositories, the SAS documentation also contains an initial kustomization file you can use to deploy the default configuration of the SAS software.   In short, start simple and improve over time. Redeploying software or infrastructure does not have to be a time-consuming process anymore, you’ve got automation for that.   Dependencies So, you’ve followed all the advice dispensed so far and built a fantastic pipeline that can deploy your infrastructure and software automatically according to all the requirements and specifications you were given. Time to move on to the next project, right? Not quite. You see, all these fantastic automations have brought in a ton of inter-dependencies. When any of these technologies need to be updated, it may trigger an update to many other components. Even if you want to remain completely static, you may be forced by outside influences to perform updates. Whether these are security vulnerabilities that need to be addressed or deprecations instigated by your cloud provider, it is up to you to address these changes.   Your approach should be similar to your initial development of your automation. Try to avoid having to do many changes at the same time, as the risk that something will fail will be much higher.   Perform regular maintenance on all these components to stay on top of a changing world around you. It is much more relaxed to be able to update a component at your discretion and following your timelines than having to perform multiple updates with your back against the wall with a looming deprecation over your head.   Short but sweet  We would like to end this series of blog posts with a number of short and sweet pieces of advice: As already stated in this blog start small and grow your pipeline over time. When using Terraform to deploy your infrastructure, make backups of your state file. It is a nightmare to reconstruct. (Yes, we’ve been there) If you need to have differences in your environments, first develop a common pipeline definition and save its configuration in Git. Next, fork the repository for every environment you need. This way you can make generic changes in the common definition and make specific changes in the forked repositories. You can also use a fork to test out patches to your deployment pipeline before backporting them into your common pipeline definition. After changes have been backported, they can be brought into the other pipelines by rebasing them.   Conclusion In this blog series about the deployment automation process we have provided you with you with information about why and how to get started with deployment automation. In this last post of the series we provided guidelines to prevent you from running into some common pitfalls. The next step is for you: Get started with your automation journey.     Articles in this deployment automation series Part 1: Deployment Automation - Why?  Part 2: How do I start with SAS Viya deployment Automation? Part 3: What are common pitfalls when starting with deployment automation?     Access the 3-Part Series         ... View more

In the first part of this blog series we focused on the “Why”. Why would you want to automate your deployments and when is deployment automation a good fit for your deployments?   This second installment will give you a foundation on how to start creating your deployment pipeline. An important note here is that no deployment pipeline will be exactly the same. Any pipeline will have to interface with existing infrastructure and fulfill specific requirements that may be unique to your situation. This blog will therefore not contain code, but merely link to existing assets that you can use to build your pipeline. This blog post will focus on Viya 4 deployments.   Automation Tooling As mentioned in the previous blog post, the automation tooling that you use to build your deployment pipelines is usually a given. It will probably be standardized for the entire company or an entire division. We will therefore not spend too much time on this topic. One important consideration for choosing automation tooling should be the capability to have well versioned pipeline definitions. As we will see in the next part of this blog series, pipelines consist of many moving parts that need to work together to deliver the environments you require. Keeping all components in line with each other, both in terms of versioning and configuration is a challenge that requires frequent updates to your pipelines. Keeping a version and change history of your pipeline definitions is therefore very important. Make sure the automation tooling that you use offers or integrates with a versioning system like Git.   Platforms A choice of automation tooling might also be influenced by the platform on which you will eventually deploy your environments. When deploying your software to Azure, the natural choice will generally be to use Azure DevOps. It comes with its own Git repository for each project you create and provides out-of-the-box tasks for working with the Azure CLI and extensions for working with Terraform for example.   Although Azure DevOps can also be used to deploy infrastructure in AWS or GCP, a more platform agnostic choice might be more logical. Examples of these include Gitlab, which comes with extensive support for CI/CD pipelines and Jenkins with a very modular plugin-based approach.   Your platform might also come with predefined automation tooling. When deploying RedHat OpenShift for example it provides three CI/CD solutions: OpenShift Build, OpenShift Pipelines and OpenShift GitOps.   Pipeline Phases Although terminology differs between different automation tooling, pipelines usually consist of three levels of building blocks: phases, jobs and tasks. A task is the smallest element in a pipeline and executes a single instruction, a job is a sequence of tasks and a phase is a collection of jobs that ideally can be executed in parallel. By dividing your pipeline into phases you can parallelize where possible while maintaining a sequence where this is necessary.   Phase 1: Infrastructure Provisioning The first phase of your pipeline will usually provision the infrastructure needed to support your deployments. This phase uses Infrastructure as Code scripts to accomplish this. The sassoftware Github project provides four different IaC templates to help you get started with creating this phase: Azure AWS Google Cloud Open Source Kubernetes   The IaC templates for Azure, AWS and Google Cloud use Terraform to define and provision the required infrastructure. Terraform is also used when deploying Open Source Kubernetes to VMWare vSphere. When deploying Open Source Kubernetes to existing physical or virtual machines, the IaC template will use Ansible scripts to deploy Kubernetes on the existing machines.   For RedHat OpenShift and Google Anthos on VMWare there are no templates available. These Kubernetes environments have their own deployment mechanisms, which can be found here: RedHat OpenShift - Installing a cluster on vSphere Google Anthos - Clusters on VMware Phase 2: Installation Prerequisites Before SAS can be deployed, the prerequisites needed for the deployment need to be satisfied. These requirements include deploying an Ingress Controller in the Kubernetes cluster and/or creating a PostgreSQL database. Steps like obtaining the software license or mirroring the SAS container images are also part of this installation prerequisites phase.   Satisfying these installation requirements can be done by using native tasks available in the automation tooling. A number of these requirements can be fulfilled by leveraging an Ansible playbook provided on the sassoftware Github project.   SAS Viya 4 Deployment This project contains ansible code that creates a baseline cluster in an existing Kubernetes environment for use with the SAS Viya platform, generates the manifest for a SAS Viya platform software order, and then deploys that order into the specified Kubernetes environment. As such the SAS Viya 4 Deployment repository can do more than implement prerequisites and can also be used in to implement phase 3: Software deployment.   Phase 3: Software Deployment Once all prerequisites have been put into place, deploying the SAS software is pretty straightforward. There are a couple of options documented in the Operations Guide: SAS Help Center: Deploy the Software   For automated deployments, using the sas-orchestration docker image is probably the easiest approach. One of the sas-orchestration command parameters is the location of the deployment files. It is advisable to store these deployment files in a Git repository as well alongside the pipeline definition. This way any changes required to the deployment files will be versioned and deployments of specific SAS versions can reference the deployment assets corresponding to that version.   Alternatively, you can use the SAS Viya 4 Deployment project mentioned above.   Phase 4: Post-Installation Tasks There are a number of post-installation tasks that need to be performed depending on your software bundle. These are listed here: SAS Help Center: Post-Installation Tasks   A lot of these tasks include changes that can be incorporated into the SAS deployment assets (preferably stored in Git). For changes that cannot be incorporated into the SAS deployment assets, the sas-viya CLI is the best way to introduce changes to a system post-deployment.   SAS Help Center: Command-Line Interface: Overview   In the end you will end up with a pipeline that combines all these phases:         Articles in this deployment automation series Part 1: Deployment Automation - Why?  Part 2: Deployment Automation - How?  Part 3: Deployment Automation - Common Pitfalls    Access the 3-Part Series       ... View more

Deployment automation - Why?    The purpose of this blog series is to inspire you with general considerations of the pros and cons of automation and SAS Viya in particular. This series exists out of three blogs:     1: Why – Explains why when and where you may want to introduce deployment automation 2: How – Explains to start with deployment automation 3: Common Pitfalls – Will help you avoid some mistakes during the creation of deployment automation         You might be thinking, do I really need another blog post about deployment automation, DevOps, pipelines, Infrastructure as Code (IAC), Deployment as Code (DaC), desired state configuration or any other automation related topic? Good question, so when we set out to indeed make another blog post we thought it best to first take a step back and consider “Why do I want automation in the first place?” and “How does SAS Viya fit into the picture?”   This first blog post in the series will focus on these questions and tries to give some guidelines on why, when and where you may want to introduce automation. And although many of the concepts discussed here are general in nature, we will tie them to SAS related concepts where useful.   Although the need for automation can sometimes be simply mandated by a set of standards that an organization has adopted, it is important to consider the benefits and costs of automation to determine whether it is worth making the investment.   Benefits In requirements engineering, three concepts are usually used together to express the quality of a requirement: Completeness, Correctness and Consistency. In short, completeness refers to whether a requirement describes everything it needs to, correctness refers to whether a requirement accurately describes the need and consistency refers to whether there are no conflicting or contradicting requirements. These three concepts also neatly describe the main benefits of deployment automation, especially when combined with descriptive or desired state configuration.   Completeness When deployment automation is being strictly applied, meaning that any action taken to deploy an application is being captured in an automated step, it enforces completeness. That is to say that when there is only one option to make a change to an environment, it forces you to capture all changes in your automation scripts. This means that your scripts will by definition contain all necessary changes to deploy an application.   This does not mean that a deployment automation script necessarily contains all required steps from the start. On the contrary, an automation script is usually a living document that is constantly evolving, but it does mean that all changes that were required to get to a certain state of the application are written down and documented at some level.   Correctness We have seen that strictly applying automation enforces completeness. It also enforces correctness. When deployment steps or configuration changes are automated, there is no possibility of these steps or changes being applied incorrectly. At least as long as they are coded into the automation correctly of course. The removal of human error from deployments is often one of the main reasons behind employing automation.   Consistency Consistency in requirements engineering means that there are no contradicting or conflicting requirements. However, consistency between multiple deployments is a more widely seen advantage.   When deployment automation is used, one can be sure that the same steps are being applied, or the same configuration is being achieved, no matter where or how often the automation is run. It again avoids human error and ensures multiple environments are configured identically. This is especially valuable when deploying multiple environments in DTAP configurations where you want to minimalize differences between environments to consciously made ones.   Speed One benefit that was not mentioned in the introduction to this section is speed. Now this benefit comes with a big caveat in that deployment automation only speeds up deployments after you have developed your automation. The development of deployment automation takes time and effort, which we will talk about later in the “Costs” section. However, once you have created your automated deployment flow it should generally deploy an environment quicker than when this would be performed manually.   This increase in deployment speed is useful in lots of scenarios: When deploying multiple environments, like the above mentioned DTAP configurations. When deploying Just-In-Time or Ad-Hoc environments, where environments are provisioned when they are needed and decommissioned once they are no longer needed. This is only really feasible if environments can be deployed in a very short space of time. When performing disaster recovery activities. Where traditional disaster recovery strategies rely on having a cold, warm or hot standby in an alternate location available for when disaster strikes, having an automated and quick deployment automation means you can deploy a new environment within the time constraints usually associated with disaster recovery scenarios.   All of the benefits mentioned above should lead to increase business continuity and eventually a higher value of the solution.     Cost Now that we discussed the main benefits of deployment automation, it is time we look at the costs of setting up automation. We will consider three main areas: Automation tooling, Initial Development and Continuous Maintenance.   Automation Tooling You may already have automation tooling in place, but its worth factoring in the cost of creating and maintaining the automation tooling required to perform automated deployments. Depending on your choice of automation software, this could range from simply setting up an account and creating a project, to setting up dedicated hardware, network and identity and access management. Using a sequence of automation technologies allows you to create deployment pipelines.   The setup of automation tooling is an up-front cost that is independent of the software you are going to deploy. Whether it be SAS or another piece of software, the product or products you use to guide your automation is a choice you generally make for an entire company or division.   Initial Development The initial development of an automation pipeline takes time. There may be templates that you can start from and adjust to your needs, or you may have to start from scratch entirely.   The initial development cost will largely be determined by the easy of which the software and the infrastructure that supports it can be deployed automatically. Software that can only be deployed through a GUI and requires physical or specialized hardware is probably not a good fit for automated deployments. On the other hand, container-based software that can be deployed on cloud-native infrastructure will most likely require much less effort. The development cost will also be influenced by the amount of configuration or specialization that needs to be applied to the software to make it fit for purpose. Software that requires a lot of post-deployment configuration will be more costly to automate than software that does not.   Continuous Maintenance The popular practice of Continuous Integration and Continuous Delivery (CI/CD) is usually applied to delivering software as an end-product and not so much to software that is supporting the software development cycle itself such as deployment automation pipelines. Pipelines consists of software just as much as end-products do, which mean they come with updates and upgrades just as much as end-products do. Maintaining your deployment pipeline to incorporate and deal with changing requirements in both the supporting software as well as in the software delivery to be automated is a cost that will be continuous as well.   In part 3 of this series we will cover some of the challenges in maintaining deployment pipelines.     Automating the SAS deployment With the main benefits and costs clear, the question remains: “How does SAS fit into this?” Typical Use Cases We should first discuss the typical use cases you may consider when determining whether your SAS deployments should be automated. SAS deployments are often software development environments themselves, i.e. SAS is not the end-product, the code, model or decision you make with SAS is the end-product. This means SAS is often deployed in DTAP configurations, although blue/green deployment strategies gain in popularity. As discussed in the benefits section, deploying  software many times over with as little differences as possible is a great use-case for deployment automation.   Suitability for Automation The suitability of SAS software for automation differs greatly between SAS software versions and has generally improved from version to version. We can safely say that the latest version of SAS Viya is the easiest to deploy using automation tooling. That is not to say that SAS 9.4 or Viya 3.X cannot be deployed automatically; they can and many have, but the effort required to completely automate their deployment will be higher than it would be when using the latest version of SAS Viya. This means that the number of situations where a completely automated deployment of SAS 9.4 will be cost-effective will be less compared to SAS Viya.   Support from SAS Although creating an automated deployment pipeline is a task that integrates with an organization in specific ways a software vendor would not be involved in, SAS does supply useful content to help you jump-start your automated deployments. This content is provided to customers through the SAS Software GitHub project.   The next installment of this blog series will focus on getting you started with automating the deployment of SAS Viya.   Articles in this deployment automation series Part 1: Deployment Automation - Why?  Part 2: Deployment Automation - How?  Part 3: Deployment Automation - Common Pitfalls    Access the 3-Part Series ... View more