How fast can we get it built? An introduction to the SAS Viya QuickStart for GCP!

[#1]

Since early October a new SAS Viya QuickStart has been made available for Google Cloud Platform.

TL;DR

This tool is a great fit if you want to quickly deploy a "ready to use" SAS Viya 3.4 environment in the Google Cloud Platform without having to go through the process of Cloud provisioning (choosing machines, storage, load-balancers, configuring networks, firewall rules, etc...) and Viya pre-requisite and deployment.

Read carefully the  README file provided in the GitHub project and after some configuration, you can provision and deploy a full Viya environment in GCP by running a single command in the Google SDK command-line.

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

If you want to better understand how the QuickStart works and learn from real hands-on experience, keep on reading...:)

As it turned out to be a longer article than expected, here is a table of content below that you can use to jump directly to the topic of your interest:

• Google Deployment Manager
• The Architecture
• The Prerequisites
• Some tips to increase your chances of success
• Important things to note
• Conclusion

Google Deployment Manager

There are many ways to automate the provisioning and deployment of an application or a Software in the Google Cloud Platform (aka GCP).

One can use automation tools which have connectors to Cloud provider (such as ansible) or specialized IaaC (Infrastructure as a code) tools such as Terraform.

But what we provide in the QuickStart is a "Google Deployment Template" for the Google Deployment Manager which is the official way to go and is documented by Google.

This GitHub project is the main source of instructions and documentation for the Viya Quick Start: https://github.com/sassoftware/quickstart-sas-viya-gcp

Let's have a look at the architecture, pre-requisites  and see what it takes to use the QuickStart to deploy and run Viya in the Google Cloud Platform.

The architecture

The diagram below comes from the GitHub project README file.

The QuickStart creates the GCP infrastructure for you. The QuickStart focuses on a subset of the Viya offerings and available topologies available to customers who have licensed SAS:

• A single CAS SMP Server (in the diagram it's termed "CAS Controller", but to be clear it performs analytical processing)
• Single Viya services server
• Only a small set of supported products (VA/VS/VDMML 8.3 and higher)
• The SAS Data agent is currently not supported in this QuickStart.

The deployment also includes a local openLDAP server (Instructions are provided to create new users). Post-deployment instructions are provided to:

• replace Self-Signed TLS Certificate with custom certificate
• enable access to existing Data Sources (for SAS/ACCESS engines)

It is also important to note that currently, there is no single point and click license agreement with Google & SAS, for running SAS Viya in GCP. Therefore, there is no contractual support from Google. And whoever uses the QuickStart is responsible for the costs of the GCP resources that are consumed by its deployment.

The Prerequisites

• A GCP account with access to a GCP project
• A SAS software order Email
• The SAS Deployment Data zip file needs to be placed in a GCS bucket (if you have no idea of what a GCS bucket means, no worries. There is an explanation further down in the blog).

During our tests, we discovered that there was an additional level of requirement relating to the GCP account. This relates to the IAM role.

The service account XXXXXXXXXXXX@cloudservices.gserviceaccount.com, associated to the GCP project, needs to have the "Project IAM Admin" role.

This role is required to allow the necessary privileges to execute the "setIamPolicy" command that is used by the QuickStart template.

The user account you are using to do that (like john.smith@mycompany.com) needs to have the "Owner role" on the Google Cloud project. So, for a customer it is someone who likely plays the role of the administrator of the project.

How easy is the Deployment process?

The deployment process, itself is straight forward:

1) Grab a "jumphost" machine from where you can install git then the GCP SDK (gcloud commands) and configure it to work with a Google Cloud Project (the "jumphost" can be your own laptop, a VM or a container running on your laptop or a machine in Openstack)

2) Clone the "Viya for GCP QuickStart" project from GitHub

git clone https://github.com/sassoftware/quickstart-sas-viya-gc

3) Edit a few settings in a configuration file (instance size, GCP Zone ,etc...)

4) Run a single command to deploy the template.

Then wait 1 to 2 hours and BADABOUM! You have a brand new Viya deployment up and running in the Google Cloud!

Looks nice and easy, right?

But...as with other deployments, the need to pay attention to the detail is critical.

Why?

Well...behind these 4 simple steps, there are some instructions that might not appear so clear if you are new to the Google Cloud Platform. For example, you might have missed or misunderstood one of the values required in the configuration or maybe you did not know how to prepare the deployment data file in the GCS bucket, or your deployment was successful but you are not able to open the "SAS Logon" page, because you made a mistake in the Ingress rules definitions...hopefully the next section will be helpful.

Some tips to increase your chances of success

How to upload the order deployment data in the GCS bucket

To run a Viya platform in GCP, you need to make a license file available for it, right?

For that you must make the license available in the Google Storage services, called "Google Cloud Storage" (equivalent of S3 in AWS).

In the GCP console, find the Storage service and click on "Browser".

Just click on "Create Bucket", give it a name.

Then, in this new bucket, you can upload the "SAS_Data_Deployment.zip" file that is attached to your Software Order Email confirmation (which contains the license itself).

Finally, if you click on the uploaded file, you can see the URI.

Of course, you could automate all these steps by simply running a few Google SDK commands (like gsutil cp).

Copy the URI value. This is what you will need in the configuration file (for the DeploymentDataLocation variable).

Example of configuration settings

This is the most important step for the GCP quick start. If it is correctly set, then there should not be any issue with your deployment. Let's have a look as this file.

The README file has a nice table which provides some help to understand what we are supposed to provide for the configuration variables.

However, some parameters involve a little more work than others.

• GCP zone. Although it does not appear in the above Configuration table, you also need to provide the GCP deployment zone. For example:

  Zone: us-east1-b
  

  If you leave the default value (GCP_ZONE), the deployment will fail.
• SSH Public key.

  

  The QuickStart creates a Bastion machine in GCP from which you can orchestrate the other machines. If you don't have one already, create a key-pair with a command like this:

  ssh-keygen -q -t rsa -N '' -f ~/.ssh/frapovqs
  

  The command creates 2 files in your ".ssh" folder: frarpovqs and frarpovqs.pub which respectively contain the private and the public key. The public key needs to be configured in the GCP VM (and the QuickStart does it for you) so you can connect with SSH using the private key as your credential. We need to copy the content of the .pub file and set it as the value of the SSHPublicKey varaiable in the configuration file, it should look like in the example below:

  SSHPublicKey: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDY9wbxpRk19YwPtjYJOyyRlrUwDR9WiVLRD/obbY887JpPd5DDSsgdy4/t+KVUu8x8Ma8XiXG7aeP6iAr+kXmnag7/Gn1cFd4AbjUmjaoWNZIg3tsCWiWt2xo14nSzu980Py3VvXX8/mu681bLQfA7MdjlroAMDI4y2xWneldJhPELe02HrQHNtT/5feWiPDfmgdD+0AdjfyF/eFDna3TFXjkp5tVRrSnLf46IIzHwntTLawBiG/DSBs45G7r31AbEqT7Yaiz1pn48CbD5qzddNhQXFlmTvUnAJgzMJpvZuIXaRc6WDm45wPMSGphR8zpddPgrG8eMpQKOHxsO/7xH centos@frarpo-jumphost.dept-gbc.central4-r1.infraserv1.sas.com
  

  Make sure the value is on one single line.
• Deployment Data Location It corresponds to the "SAS_Data_Deployment.zip" file that we placed. We need to provide the URI of the file in GCS. For example:

  DeploymentDataLocation: gs://frarpo-viyaquickstart/SAS_Viya_deployment_data_9C7SRH.zip
  

  DeploymentDataLocation: gs://frarpo-viyaquickstart/SAS_Viya_deployment_data_9C7SRH.zip
• AdminIngressLocation and WebIngressLocation These parameters allow the opening or restriction of network access from the outside to the environment that gets deployed in GCP. As an example, here are the values I used:

  AdminIngressLocation: 149.173.8.44/32
  # Allow inbound HTTP(S) traffic to the Web Server from this CIDR block (IP address range). Must be a valid IP CIDR range of the form x.x.x.x/x.
  WebIngressLocation: 149.173.8.65/32
  

  As I used the range "32", it means that only the specified IP address can can connect to the Bastion host in GCP.
• For AdminIngressLocation it corresponds to the external IP of the jumphost machine in Openstack that I'm using to drive the QuickStart deployment. Be careful. The IP that is reported locally by the jump host does not necessarily correspond to the value that is required here. Remember that our deployment is in the Public Cloud on Internet. So we need to use the IP of the jump host as known from the Internet. For example:

  [centos@frarpo-jumphost ~]$ curl https://www.whatismyip.org/ | grep "Your IP"
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                   Dload  Upload   Total   Spent    Left  Speed
    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0<h3 class="text-center global-ip"><strong>Your IP:</strong> <a href="/my-ip-address">149.173.8.44</a></h3>
  100 36377    0 36377    0     0  74097      0 --:--:-- --:--:-- --:--:-- 74087 
  

• For WebIngressLocation it corresponds to the IP of my own workstation (also returned by a web site such as whatismyip.org).But a different CIDR range can also be used if one wants to open to multiple IP addresses. If you are not so comfortable with the CIDR IP range notation (with the "/") you can use this handy calculator: https://mxtoolbox.com/subnetcalculator.aspx# to understand how many IP addresses correspond to a specific CIDR IP range.

One ring command to rule them all...

Once everything is ready in the configuration file, just run the magic command:

gcloud deployment-manager deployments create frarpo-deployment --config ~/gcpquickstart/quickstart-sas-viya-gcp/templates/sas-viya-config.yaml --async

When you run this command, after 2 seconds you see:

The fingerprint of the deployment is iayJnj_Es9_eA-hP0D9raQ==
NAME                                                     TYPE    STATUS   TARGET             ERRORS  WARNINGS
operation-1572527013070-596347a03863e-7c1fc568-999c9dff  insert  RUNNING  frarpo-deployment  []      []
[centos@frarpo-jumphost ~]$

If it is the first time you do it, it's worth monitoring the deployment...because it may fail (see below for my own troubleshooting experience). If you are confident and did it before with success, you can go for a walk and come back 1 or 2 hours later.

Monitor the deployment progress

You can monitor what's going on with the GCP web console:

You can click on it to have more details on the deployment progress from the Google cloud perspective.

Now if you want to see deployment logs from a Viya perspective, follow the instructions in the README file to select the ansible controller VM and see the  "Serial port 1" log:

What if it does not work?

The README file provides some Troubleshooting guidelines.

Here is a little feedback from my own troubleshooting experience...As many people have said "learning from failures is the best way to learn"

Click on the link provided in the error message:

And enable it.

Then retry.

Another issue that I faced was related to missing privileges for the service account.

After adding the right role or privilege to the GCP project service account ("Project IAM Admin" role" as explained in the pre-requisite section), the issue was resolved.

A successful deployment will look like this:

Note: The warnings come from the fact that the OS image used by the QuickStart has been fixed to a specific version to avoid a known issue (http://support.sas.com/kb/64/703.html) of Viya 3.4 and specific versions of systemd in Linux.

How to access the Viya environment?

As explained in the GitHub README you can find the IP address to access your newly deployed environment by browsing in the GCP console (Network Services then Load-balancing, advanced view)

Assuming you have the correct information, you should be able to login with the sasadmin account and see something like:

Otherwise you will see something like:

This error likely means that the network security configuration (created by the Viya QuickStart deployment) does not allow you to access the load balancer.

In such case, go in the "Network Security", then "Cloud Armor" and check if the access rule match with your local IP address, like below:

Check out https://www.whatismyip.com/ if you are not sure what your IP address is).

Finally, you can also access though ssh, from your jumphost machine with the sasinstall account to the ansible controller machine, like:

ssh -i ~/.ssh/frapovqs sasinstall@35.196.188.241

Important things to note

• You can use a mirror There is a parameter for it in the configuration file (otherwise the ansible controller will instruct GCP VMs to download the SAS packages from the SAS Hosted repository machines on Internet)
• Where are my users? An openLDAP server is installed as part of the QuickStart deployment, the Viya service GCP VM will get the OpenLDAP playbook (thanks to Erwan Granger) from https://github.com/sassoftware/quickstart-sas-viya-common and run it to deploy and configure openLDAP and also SSSD. However, the environment could then be configured to point to a real LDAP server (such as the customer corporate Active Directory server) as long it is available from the GCP VPC network. The README file provides instructions on how to add new users.
• Use the right GCP project When you run the single command to perform the deployment in GCP, you don't need to specify explicitly state your Google account and Google project, because it is already configured in the Google SDK configuration. But if you are working with multiple Google projects, you have to be careful because you need to point to the project where you plan to deploy Viya. For example: Show current project:

  gcloud config get-value core/project
  sas-gelsandbox
  

  Switch project:

  gcloud config set project sas-gelsandbox
  Updated property [core/project].
  

  List VM instances in the current project:

  gcloud compute instances list
  

• Don't forget to remove/clean-up the environment when you don't need the it anymore The great thing with the Google Deployment Manager is that all the resources created by a deployment are considered as a single entity, the "deployment" object. So if we want to get rid of the deployed environment and all associated resources (network, load-balancer, accounts, etc...) it is very easy, you just need to run a command like:

  gcloud deployment-manager deployments delete frarpo-deployment
  

Conclusion

The QuickStart is great to quickly stand-up a pre-defined Viya environment in GCP with the proper cloud provisioning (service accounts, machines, networks, load-balancer).

Once you have used it, you can save some "cloud money" by removing it (and all associated resources) as easily as you created it and then whenever you need it again, recreate it with the same single command.

As it is provided as free source code in GitHub, the QuickStart can also be a great baseline for anyone who would like to extend it for more complex deployments or to build its own Viya IaaC framework.

Many thanks to Dante Bergamasco for his help and support in the write up of this article.

Thanks for reading!