Most likely you've heard the term Jenkins, but maybe you haven't had the time to dig into the ways it may benefit you as a SAS Viya installer, architect, administrator, or user. In brief, Jenkins is an open-source automation tool used heavily in software development. However, its capabilities are wide-ranging and can certainly be applied to many facets of a SAS Viya 3.5 deployment.
One of the commonly-used features of Jenkins is its ability to trigger builds based on an event or a schedule. For example, when configured to interact with Gitlab, you can trigger a build to execute each time an update to a Gitlab project occurs. Assuming that example, let's take a look at what is required to trigger a Jenkins build and review some potential use cases.
Jenkins is an open-source automation server with many available plugins to extend functionality. It is used by R&D and testing to automate building, deploying, and testing throughout the software development life cycle.
But don't be fooled into thinking that it is strictly for software development. There are many capabilities that enable it to be an effective tool to automate many SAS Viya 3.5 processes, both deployment and operations related.
If you've ever seen a screenshot or have logged on to the application, don't let the "classic" web interface fool you. Behind that interface is a ton of functionality to automate your most-loved processes.
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
Let's review some key Jenkins terminology.
Plugins - Much of that functionality is the result of the plugins that interface with other technologies (e.g. Gitlab, Github, cloud providers, etc). And lest I forget, it is extensible. So you can enrich the functionality ever further if you wish by writing your own plugin.
Master - The central coordinating process/server which stores the configuration, provides and manages plugins, and renders the interface.
Project (Job) - At one point these terms were used interchangeably. However, the term "Job" has been deprecated. Essentially projects are the core runnable tasks that are controlled and monitored by Jenkins. Projects are created using the "New Item" menu and after selecting a type of project, you are presented with a series of configuration sections. The project can represent a single step or a complex combination of steps that run across multiple machines.
Build - A build is simply an execution or run of a project.
Artifact - Artifacts are files or objects created during a build. Artifacts can be tossed at the end of a run or saved for future use.
Node - Nodes are the hosts/machines that are set up to run the project builds when run from a master. In order for it to run tasks on a node, an agent must be defined on that machine. After a node is registered to the master, the master then distributes work to the nodes. Work can be directed using a variety of options. The Master is also a node.
Workspace - This is a disposable directory on the node that is used as a working directory for the build.
Pipeline - A user-defined model of a continuous delivery pipeline. Okay, that definition may not be too helpful. A pipeline is a definition of a series of stages to be executed either serially or in parallel on one or more hosts. Stages may contain one or more steps. For you mainframers, it reminds me of an uber-functional JCL with a web interface. But I digress…
Stage - a grouping in a Pipeline that defines a subset of the entire pipeline
Step - a single task within a pipeline; a group of steps can be used to define a stage. An example would be a single host command
For more terms, check out the Jenkins glossary.
Here is a simple architecture diagram of the Jenkins architecture (thanks Rob Collum). Notice we can have agents on a variety of platforms and each agent needs a workspace to store its repositories and project content (artifacts). If running a parallel pipeline, all three agents can be actively working at the same time. Logs for each stage are made available to and stored on the Jenkins server (Master). The Master can integrate with a variety of Git flavors as well as other software technologies.
In this example, we take advantage of Jenkins' Docker image. Using Docker to initiate a Jenkins environment provides a fast path to quickly getting access to a Jenkins server.
I'll keep this section short.
Most likely you've dealt with one or more of the Git flavors in recent years. Gitlab, and its competitors, are web-based DevOps lifecycle tools which provide a Git-repository manager for versioning, wiki capabilities, and issue-tracking along with various CI/CD pipeline features. Gitlab operates using an open-source license. A fair amount of overlap exists with the CI/CD pipeline features and Jenkins. In this example, we use Gitlab purely as a versioning tool.
A key feature in building automation in Jenkins is the ability to automatically kick-off (trigger) a project build as a result of an event, a request, or a predefined schedule. One manifestation of this ability is configuring a Jenkins project to perform a task when a push to a Gitlab project is issued and completed. For example, you have a YAML file that you are modifying in Gitlab and each time you modify it you want to perform syntax checker on the updated file (could be JSON, XML, etc). This is a simple example, but the point is to drive home the idea behind the example. So let's walk through the steps to enable this ability.
In order to create the trigger build we need to ensure we have the required software so that Jenkins can integrate with a Gitlab repository.
At this point you should have the key software pieces in place to begin configuration to perform a triggered build.
To activate a triggered build, we will break this up into two phases, configuration and operation.
The following diagram highlights the sequence of actions required to configure a Jenkins project to be triggered when an update is pushed/merged/committed to a Gitlab project. There are four major steps.
So there you have a simplistic view of the steps to configure Jenkins and Gitlab to initiate a trigger build job.
Earlier we mentioned a very simple use case. Assume we have the SAS Viya playbook stored in a Gitlab repository and we are planning a new deployment. Most likely you will be making changes to the vars.yml file. As you are making changes you could define a trigger build that will invoke a YAML syntax checker, such as YAMLlint, each time you update vars.yml to validate it. This could save you time down the road with debugging changes to your deployment.
One key phase of a new deployment, or changes to an existing deployment, is to validate that services in a new or updated environment are functioning as expected. A trigger build could be established to be initiated after all services are running to verify the health of services and subsequently perform basic functionality testing. Jenkins can be configured to retain the logs and output. It captures response times as well. As a result the Jenkins user interface can provide a historical view of past validation tests and you can quickly identify executions that do not complete within the expected execution time.
Beyond these, it is up to your imagination to identify the many possibilities of creating trigger builds for a SAS Viya 3.5 deployment.
In this article, we've reviewed the basic process flow of triggering a Jenkins project build. If you haven't been exposed to Jenkins before and your responsibilities include performing repetitive processes that would benefit from automation it may be worth your time to take a look at its capabilities. With Gitlab and Jenkins Docker images you can fairly quickly have an environment running and begin trigger testing.
Thanks for reading!
Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning and boost your career prospects.