With the recent updates of the Viya Quickstarts in the Cloud, I thought it would be a good time to come back on the Quickstart process and talk about some recent deployment experience in Azure.
SAS is a "Microsoft One" Commercial Partner which means that SAS and Microsoft teams closely collaborate on cloud technologies, solution areas and rapid deployment models, It makes Azure is a platform of choice for a SAS Viya deployment.
Also, with the move to Containers deployment in Kubernetes for the next Viya 4.0.1 release, we can expect the Cloud platforms such as Azure to become even more popular for Viya deployments.
So if you are an Architect or an Installation Engineer and completely new to Azure, now is probably a good time to get a first exposure to it 😊
The basics of the Viya Quickstart for Azure deployment are covered in the excellent post and 30 minute video from Michael Goddard. Note that the video is also available on YouTube, so any customer can use it to prepare a Quickstart deployment !
The purpose of this blog is to revisit the Quickstart deployment in Azure with a Viya 3.5 order and a CAS MPP environment but also to provide some extra guidance to secure your in Azure and make sure you don’t face some of the issues I encountered in my own deployment 😉
Before diving into the details, I'd like to thank Chris Lynch for his precious help in the write-up of this blog.
SAS's Quickstarts are getting better and better ! Not only have the three of them (AWS, GCP and Azure) have been updated to support the latest Viya version (3.5), but now, they also support CAS MPP configurations.
A new parameter has been added for that in each Quickstart and allows you to have up to 10 CAS workers in your Quickstart deployment. This makes Viya with CAS MPP deployments very easy.
As an example all you have to do is for the Azure Quickstart is to specify the number of CAS workers that you want, in the Quickstart template form (3 for example) :
Each Quickstart documentation (either README files on GitHub pages or PDF documentation for AWS), provides Architecture Diagrams for CAS SMP and CAS MPP cases.
See below the architecture diagram for the Quickstart architecture for SAS Viya on Azure with a CAS MPP configuration.
While the "Quick" word in "Quickstart" means "Quick deployment", it does not mean "Quick (and dirty) Architecture design".
To the contrary, one look a the architecture diagram is enough to see that many important Architecture principles and best practices for a Cloud deployment are implemented by the Quickstarts :
The Quickstarts are not simply provisioning a bunch of VMs to deploy our software. They also provision all the Cloud services or components required to create a robust and secured environment.
Of course, in a customized Cloud deployment, a more sophisticated topology and model could be implemented, but the Quickstart provides a solid starting point, that can be extended as the deployment templates are public.
So, with the addition of MPP CAS, it makes the Quickstart a valid option not only for demo or PoC/V (Proof of Concept/Value) but also for real project deployments for customers licensing a "Quickstart supported" solution (see below).
So let's recap on the changes for our Quick Start !
Regarding Azure, please note that there is also a new excellent post from Margaret Crevar (from the EEC team), with considerations on the Azure VM types, network and storage choices that impact the performances of our Software.
Ok, so now let's zoom on the Azure Quickstart and talk a little bit about the deployment.
As said previously, if you are fairly new to the Viya QuickStart on Azure, the best starting point is the Michael Goddard’s YouTube video : “SAS Viya Quickstart Template for Azure | Introduction and Demonstration”.
If you already know the basics and want to learn more advanced techniques like use Mirror repository or monitor the deployment progress, you will be interested by the following sections.
For many reasons, using a Mirror repository for the SAS packages is a best practice for any Viya deployment (see this post if if you need to remember why). Using a mirror also allows you to test the Quickstart with an Internal order.
According to the Quickstart README file, to use a Mirror repository, all you have to do is to:
It seem very simple, right ? However, the Azure portal interfaces can sometimes be a little bit confusing, especially if you are new to them. So let's detail a little bit more the steps with a real example.
Let's start with the first step: "upoad all the mirror content"
If you open the Storage explorer in the Azure portal, it should look like this:
You can copy the value of the Key field either for key1 or key2, that's what you will use for your "$STORAGEKEY" in the upload command that we need to run.
Now, you should be in position, to run the Azure CLI command to upload the mirror content into the blob container. The README file of the Quickstart gives the command line syntax :
Here is a real life example for the command using the previous example, values:
From there, your screen should start to scroll pretty quickly as it reports each file upload into the Azure blob container 😊. As a result, a folder named "09QSDS" is created in your "mirrors" blog container and contains all the files and folder of the mirrored repository.
Actually, if you use the Azure CLI on linux and want to increase the automation of the process, there is even a trick do this all as one line without needing to get the key out of the web interface (by having bash fetch the key for you in a sub var):
So far, so good. Now let's see the second part which is primarily to find the proper URL/URI to use to help our Quickstart deployment to find our brand new Mirror repository 😊
Here is the information that we have from the README file :
But don’t get confused ! in this context "SAS" means "Shared Access Signature" which is the Azure mechanism to provides secure delegated access to resources in your storage account (such as the "mirrors" blob containers) without compromising the security of your data. See this page for additional details.
Here are the steps from the portal to create the "SAS" key:
- In your storage group, select "Shared access signature" on the left side menu.
Now create a new one with:
- allowed services
- allowed resource types- All of them
- allowed permissions - Read List
And set the end date far enough in the future to cover your install window.
Then click "generate SAS and Connection String" and grab the value in the "SAS Token" string
If like me, you prefer to use the new “Storage explorer” feature, it would look like this:
Right click on "Mirrors"
Select "Get Shared Access Signature".
Set the expiry time (the "SAS" key used by the Quickstart to download the packages from the mirror repository will only be valid for this period of time), For the permissions, check "Read" and "List". Then click on the "Create" button, to get your URI, that contains the "SAS" key.
We are now in position to build the URL to set in the Viya Quickstart template form ! But before using it, we want to make sure the URL/URI actually works !
So copy the URI that is shown in the portal screen above, for example :
If we use this URL directly, it does not work:
We need to manipulate a little bit the url to get the form that is required for the Quickstart template.
Keep the first part of URI string (until the "?"), but then modify the path, as if you wanted to point on a specific file that is available in your mirror repository, for example : entitlement.json.
In our example that would be :
Then add the Query string (starting with the "?" that contains the validity dates and the signature/ token)
Now, it is working :
Now, it's time to set the value for the "Deployment Mirror" parameter in the QuickStart template.
So to get the final URL, we just need to "chop off" the entitlements.json, it gives something like that :
It won't work in a web browser, but ,trust me, this is what needs to be set in the Quickstart template.
As you have seen, the value for some of the template parameters can be a little bit tricky to find. So you surely want to monitor your deployment as closely as possible to make sure everything works as expected. Shortly after clicking on the "Purchase" button that launch the Quickstart deployment, the Ansible Controller Virtual machine is created and you can connect to it with the "vmuser" account. As soon as you see in the Azure portal that the "Ansible" Virtual Machine has been created, get the public IP address and connect to it from your ssh client.
You need to provide the SSH private key corresponding that corresponds to the SSH public key that was set in the Quick Start template. From there, you can monitor the deployment log exactly the same way you would to it from an on-premise deployment. For example:
At the end, you can also check the Viya services status using the Viya ARK MMSU scripts (embedded in the deployment):
On my first attempt, quite optimistically, I decided to deploy a Viya environment with 3 CAS workers and to keep the default Azure instance types, predefined in the Quickstart template:
But near the end of the deployment I had the unpleasant surprise to see messages below in the Azure Portal for my running deployment :
Humm...It does not look good, does it ? However, using the described technique above I was closely monitoring the deployment via the ansible deployment log and noticed that the deployment log shows no failure for the ansible deployment. Moreover, when I checked the status of my Viya services they all reported an "up and running" status. I even can logon with success to SAS Environment manager where everything seems perfect (with my 3 CAS workers as requested):
So...is my deployment valid or not ?
With a better look at the error message I can isolate an interesting message:
"After restart, services were still found to have not successfully started. This likely indicates that the boxes have been undersized. Try increasing the size of the services box."
Thanks to the Development team, I had an explanation:
"During the install process, the startup of the Viya services becomes very cpu bound. Once everything loads into ram, they settle down a lot, but starting all the services tends to be the point where a larger box is needed for a bit. But the services definitely can come online after the timeouts for the build".
Unfortunately some Azure time out settings (that can not be moved past) introduce a time limit for some tasks that the deployment process cannot meet because of limited CPU resources.
So, although my Viya deployment was successful, my Azure deployment was not.
Of course it would have been better to have a completed successful Azure deployment but as long as there is no failure in the ansible deployment results and all the Viya services are up and running, we can use Viya as is or increase the instances sizes and redeploy in case we want more reinsurance on how it performs.
Azure deployments are more records of what happened (we can redeploy or undeploy a deployment, but resource groups are really the management object that azure uses).
As the Quickstart has its own resource group and is expected to be the only thing in that resource group so it can also be cleaned up as a unit (also prevents name collisions).
Hopefully after reading this article you know a little bit more about the SAS's Quickstarts (and learned from my own experience and mistakes 😊 ). The Quickstarts are a great way to automate your deployments of Viya in the Public Cloud infrastructure while complying with the acknowledged rules of the art of Cloud deployments.
SAS is headed back to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team.
Interested in speaking? Content from our attendees is one of the reasons that makes SAS Innovate such a special event!
Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning and boost your career prospects.