Manage your SAS Viya Platform configuration in a Git Repository

2 Likes

When the SAS Deployment Operator was initially released, one of its key capabilities was to read the SAS Viya configuration from a remote source (such as a source code repository or a Web server).

Today, the latest deployment method, known as the sas-orchestration command also offers this option.

This article discusses the benefits to read the SAS Viya Configuration from a remote git repository (instead of using local files) and explains how to do it with the sas-orchestration command. It also shows an implementation example.

Why?

A SAS Viya deployment configuration is comprised of two main items:

a kustomization.yaml "main" file,
and the site-config folder files

which are typically stored under the same file system structure on your jump host.

Managing these configuration items can become a bit challenging overtime and make it difficult, to know what has changed and when…

For traceability purposes you could, for example, create a new copy of your SAS Viya configuration folder for each update (each month if you are on a stable cadence, and each time you modify the configuration).

After a little while that historical group of directories might look something like:

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

As you can see it is not very efficient; a lot of files are duplicated, and it could quickly become pretty messy…(and I’m not even showing the structure under site-config in the screenshot).

In addition, keeping the SAS Viya configuration on a jump machine’s file system means that you need to keep this machine around, make sure it is secured and backed up, etc… which is not very convenient or aligned with the current DevOps standards (where everything should be ephemeral and automatically built by a pipeline).

On the other hand, having your SAS Viya configuration files stored in a modern source code repository system (such as GitLab or GitHub) brings all the benefits of a modern code versioning system. You could organize your environments in different branches and keep track of all the changes, tag your deployed version and easily come back to them, etc…

It would also be much easier to integrate the SAS Viya configuration in a modern DevOps system.

Deploy SAS Viya from a remote location

The go-getter project provides a library, used behind the scene by the DaaS* tools, to pull the required SAS Viya configuration from a remote location.

As explained in the go-getter project’s README, “go-getter is a library for Go (golang) for downloading files or directories from various sources using a URL as the primary form of input”. Out of the box it supports the following protocols : local files, Git, Mercurial, HTTP, Amazon S3, Google GCS.

So, in theory, as long as you can setup the authentication to the remote repository and figure out the right “go-getter” URL (in the Deployment operator CRD or in the sas-orchestration command line), you could place and manage your SAS Viya configuration in any storage system (providing access through one of these go-getter protocols).

However, the current reality is a bit different 😊

While go-getter is used by the DaaS tools, it does not mean every possible system and authentication combination will automatically work when used from the sas-orchestration tool or Deployment operator.

For example, local files and "Git over SSH" access is currently not supported by the DaaS tools. So far, the only "go-getter" URLs tested and supported are the ones using the "Git over http(s)" protocol, like :

For a public repository : git::http://github.com/jsmith/public.git//folder
For a private repository: git::http://jsmith:<git_token>@github.com/jsmith/myviya.git//conf

Also, note that the "Git over http/s" go-getter URL only works for the SAS Viya configuration files (kustomization.yaml and site-config folder files), but not for the order’s certificate information or the license file.

The DaaS tools can read the order’s certificate information or the license file either from a local file system or from an HTTP server.

(*) DaaS = Deployment as a Service: DaaS tools are the Deployment operator and sas-orchestration command.

Using GitLab

In terms of infrastructure, we can install a containerized instance of the Open Source GitLab* platform inside our on-premise (Upstream Open source Kubernetes) or Cloud Managed Kubernetes Clusters (Google GKE, Amazon EKS or Microsoft AKS).

In such case, sometimes for demo or training puproses, it is possible to take some shortcuts, using the default "root" account in GitLab and making the project public, so we don’t have to perform any real authentication in the go-getter Git URL.

But a much better practice is to store and manage the SAS Viya configuration in a private GitLab repository.

So, in the next paragraph, we’ll show an example with the “SAS orchestration deploy command” using a git personal token to authenticate and to read the order’s asset and SAS Viya configuration from a private GitLab repository.

(*) GitLab is a very popular web-based Git repository that provides free open and private repositories, issue-following capabilities, and wikis.

An example with GitLab and personal access tokens

Create your GitLab token

GitLab personal access tokens allow you to authenticate with Git using HTTP Basic Authentication. This is what is expected to be used in the sas-orchestration tool.

Users can easily create their own personal access token from the GitLab web interface.

All you have to do, once you have signed in, is to right click on your GitLab avatar image, select "Edit Profile", then click on "Access Token".

Then you simply give a name to the token, consider changing the expiration date, select a scope (read_repository and write_repository scopes are sufficient) and click on the blue button "create personal access token".

Once it is done, you should see a message that confirms that "Your new personal access token has been created."

Now, it is important that you take note of the token content (you can reveal it or just copy it in your buffer), because we will need it later when we run the deployment command.

Store your SAS Viya configuration in GitLab

After that, you can create your "Viya" GitLab repository and store your SAS Viya configuration files there. It can be done either with the GitLab web UI or through Git commands.

If you use Git commands, you’ll first have to setup and configure the Git client on your machine, here is an example (on Linux):

git config --global credential.helper store
git config --global http.sslVerify false
git config --global user.name "GEL Student"
git config --global user.username jsmith
git config --global user.password "lnxsas123"
git config --global user.email "GELStudent@mycompany.com"

These commands update the Git configuration file located in the user’s home directory (~/.gitconfig). You can easily check your Git configuration just by running "git config --list".

In our scenario, we assume that we don't have the SAS Viya configuration in the remote repository yet.

So, we first need to create a new set of SAS Viya configuration files (kustomization.yaml and "site-config" directory) in a local folder.

Once they are in the local folder, we can initialize the repository, reference the remote location, perform our first commit (and finally push to our remote GitLab repository).

You can find below an example of the commands to do that (on a linux machine).

# move to the local repo folder (that contains our SAS Viya configuration files)
cd ~/project/deploy/viyaorch-repo
# Initialize the git repository
git init
# Reference the remote location (it also creates the project in GitLab).
git remote add origin http://gitlab.devops.company.com/jsmith/viyaorch-repo.git
# Now commit our changes in the local git repository.
git add . git commit -m "Initial commit of the viyaorch configuration."

You would see something like that:

At this point the Git repository has been initiated but it remains local.

Now it is time to "push" our changes toward the remote GitLab repository.

git push http://gitlab.devops.company.com/jsmith/viyaorch-repo.git

The first time, you are asked for your git username and password*, but then your credentials (that are in your .gitconfig file) are stored for the next time (remember the credentials.helper parameter was set to store).

Back to the GitLab web interface, you should then be able to see your SAS Viya configuration files in your GitLab project’s repository:

Congratulations, your SAS Viya Configuration is now stored in GitLab ! 🙂

(*) If you want to avoid this first time prompt, you can create a ~/.git-credentials file in advance with the credential in the following format : "http://<git username>:<git password@gitlab.devops.company.com"

Deploy SAS Viya

The final step is to set up the go-getter URL in the sas-orchestration command and run it to deploy SAS Viya!

In the user-content parameter for the sas-orchestration command, we specify the GitLab personal access token that we created earlier (instead of the user’s GitLab password).

Here is an example:


docker run --rm \
-v $(pwd):/workingdir/ \
-v ~/.kube/config:/kube/config \
-e "KUBECONFIG=/kube/config" \
--user $(id -u):$(id -g) \
sas-orch \
deploy \
--namespace viyaorch \
--deployment-data /workingdir/viyaorch/SASViyaV4_certs.zip \
--license /workingdir/viyaorch/SASViyaV4_license.jwt \
--user-content "git::http://jsmith:<gitlab personal access token>@gitlab.devops.company.com/jsmith/viyaorch-repo.git//config" \
--cadence-name stable \
--cadence-version 2023.06

Note that in the example we still read the order’s certificate (SASViyaV4_certs.zip) and license (SASViyaV4_license.jwt) files from the local filesystem in this case (but they could also be read from a remote HTTP server).

Finally, if you would like to experiment by yourself, note that the SAS Global Enablement and Learning team is developing specific SAS® Viya 4 deployment hands-on content within workshop style content. Contact your local SAS Education team for more details if this is of interest to you.

Cloud Git Repositories alternatives

While it makes sense in an "on-premise" environment to run a local GitLab instance, for deployment in the Public Cloud the use of the Cloud Provider code repository solution could be a better and more consistent option in the long run.

Your customer might even have this as a requirement if the Cloud Provider code repository is part of their standards and is already used for other projects.

Azure, AWS and Google Cloud Platform offer “git compatible” repository services, such as Azure Repos, AWS CodeCommit or Google Cloud Source Repositories.

As explained before, while go-getter can used by the DaaS tools (for the "git over http(s)" protocol), it does not mean every possible Cloud Provided Git Repository system and authentication combination will work automatically when used from the sas-orchestration tool or Deployment operator.

However, in the next article we will show how (with some tweaks and tricks) the Google Cloud Source Repositories service can be used as the SAS Viya configuration source for a SAS Viya deployment in the Google Cloud Platform.

Thanks for reading!