As is often the case when I sit down to write about a topic, I thought I could write a short post about the ways in which you can update a License in a Viya Container. And as is often the case, a few thousand words later, I realize how wrong I was in the assumption that it would be short. 😞   Containers, like any other technology, require knowledge, experience, and skills. Although they can make life easier in some circumstances, this will be especially true for those who understand what makes containers tick. I hope that this post will help you acquire and cultivate some of that knowledge.   Help! My SAS container won't start anymore! As container adoption is maturing amongst SAS customers, I suspect that there is a slow wave headed towards SAS Tech Support. This consists of those customers who have been using a Programming-Only Viya Container for the better part of a year, and are about to start seeing some License Expiry warnings in their logs. Should they ignore these warnings for too long, they might even be facing the even more dire prospect of said containers no longer starting up.   Starting and dying is just as bad as not starting up I need a little parenthesis here to mention that yes, technically, the container will start, realize that the license has expired, the process inside will stop, and therefore the container will stop. Further to this, and because many might be the using --rm option on their docker run statement, the dead container will be cleaned up automatically. Which is why it really does look like it did not even start. Trust me on this: it started, it died within a second, and the body was removed immediately. If you are an avid fan of "CSI: Containers", you know it's hard to determine cause of death when there is no body.   What that looks like In order to illustrate what that looks like, compare these 2 results, edited for brevity:   First, an interactive and expressive way of running the container with a BAD location for the license:   docker container run \ -it \ --volume ${PWD}/BAD/sasinside:/sasinside \ prebuilt:v1   This results in something easy enough to parse for a human:   [INFO] : Logging files to stdout/stderr [INFO]: License location is '/opt/sas/viya/config/license.sas' Running host authentication script ERROR: No license file found at /opt/sas/viya/config/license.sas.   However, if instead of the -it parameters, you have used -d --rm , there is much less help.   docker container run \ -d --rm \ --volume ${PWD}/BAD/sasinside:/sasinside \ prebuilt:v1   will only output the container id:   72dfd4ed889624ccb2894b662390b8b8cfb0cb5240221f7c1cbddfa5c2869e70   However, this container will be nowhere to be found, as it will have died and been removed already. So, if you are in this situation already, or if you are looking down the barrel of a soon-to-expire Viya license in a container, here is a piece of advice for you. Don't panic! This is the howto you've been looking for.   Harder than it first looked I have to admit here that this has taken me a lot more time than I thought it would when my manager suggested it as a topic.   I had to do some digging, read a bit of the container recipe code, and even refer to the Viya 3.5 Documentation on applying licenses manually.   Getting your Viya 3.5 Programming-Only docker image If you know your Viya containers, you will know that there are 2 ways of obtaining a Viya 3.5 Programming-Only Single Image. And it's important to know which one you are using.   I like to refer to those 2 ways as follows: Pre-Built BYO (Build-Your-Own) In both cases, you are required to have a valid Viya 3.5 Order at your disposal. But the similarities stop there quickly.   Pre-Built If you decide that the Pre-Built image is right for you, at a high level, you will: Upload your SAS_Viya_deployment_data.zip file to a Linux Docker host Extract files from the zip archive Place certificates in some specific locations Install the mirrormgr utility on that host Use mirrormgr to determine the precise image tag to be used Download the pre-built image from SAS-Hosted Docker Registries, using the docker pull command   Build Your Own If instead you prefer to build your own image, then you will: Upload your SAS_Viya_deployment_data.zip file to a Linux Docker host Install the mirrormgr utility on that host Use mirrormgr to download the RPMs in that Order Serve the Yum Mirror over http/https Download the SAS Container Recipes from Github Execute the build.sh script to create the image   Naming conventions To make things simpler to follow, we are going to assume the following: We have a linux Docker host We have both images available on that docker host They are called: prebuilt:v1 byo:v1 We'll assume that both images are currently working, with their respective current licenses, but we need to extend the licenses nonetheless.   Pre-Built Image - Test container Let's start an interactive container from the Pre-Built image, on port 8888. cd ~/working03/ docker container run \ --interactive --tty --rm \ --volume ${PWD}/sasinside:/sasinside \ --volume ${PWD}/sasdemo:/data \ --volume ${PWD}/cas/data:/cas/data \ --volume ${PWD}/cas/cache:/cas/cache \ --volume ${PWD}/cas/permstore:/cas/permstore \ --env CASENV_CASDATADIR=/cas/data \ --env CASENV_CASPERMSTORE=/cas/permstore \ --publish 8888:80 \ prebuilt:v1   BYO Image - Test container And let's also start an interactive container from the BYO image on port 9999. docker container run \ --interactive --tty --rm \ --publish 9999:80 \ byo:v1 If you look at the above, you'll notice some differences in syntax between the byo:v1 image, and the prebuilt:v1 one. However, the important difference between the two is the lack of --volume ${PWD}/sasinside:/sasinside for the BYO image. This small difference is important. Keep it in mind for later.   Checking existing licenses Now that both test containers are started, let's execute the following code:   proc setinit; run; cas mysess; cas mysess listabout;   For the Base SAS part of the license we can see something like:       And for the CAS part of the license:       Updating the License for the Pre-Built image Now that we have established that both containers have a valid license that expires on 06JAN2021, we will go about updating them. For the purpose of this demonstration, I have another license file that will expire on 27JAN2021.   The quick way In the case of the Pre-Built image, the license is NOT part of the image. It is placed into a folder that is then mounted inside the running instance of the container. You can see that by browsing the content of the sasinside folder on the docker host:   [cloud-user@dockerbox sasinside]$ pwd /home/cloud-user/working03/sasinside [cloud-user@dockerbox sasinside]$ ls -al drwxrwxr-x 2 cloud-user cloud-user 142 Feb 3 06:25 . drwxrwxr-x 10 cloud-user cloud-user 322 Feb 21 09:38 .. -rw-rw-r-- 1 cloud-user cloud-user 0 Feb 3 06:25 cas_usermods.settings -rw-r--r-- 1 cloud-user cloud-user 3607 Feb 3 06:25 license.sas -rw-r--r-- 1 cloud-user cloud-user 9118 Feb 3 06:25 SASViyaV0300_00AAA0_12345678_Linux_x86-64.jwt -rw-rw-r-- 1 cloud-user cloud-user 0 Feb 3 06:25 workspaceserver_usermods.sh [cloud-user@dockerbox sasinside]$   So, to swap out the license, the operation could be as simple as the following (please don't do that quite yet, though!):   cd /home/cloud-user/working03/sasinside mkdir old_license_06JAN2021 mv license.sas old_license_06JAN2021 mv SASViya*.jwt old_license_06JAN2021 cp /tmp/newlicense/SASViya*.txt ./license.sas cp /tmp/newlicense/SASViya*.jwt .   Simple, right? Well, not so fast. First of all, you have made changes, but you have not yet confirmed that you got the intended result. Don't walk away yet, we still need to validate that this worked. However, and potentially more damaging, we have not really taken into account the fact that there may have been existing containers running already, and what this little manipulation might have done to them. The last thing you want is to kill someone's session just because you wanted to refresh the license.   The safer approach If one was extremely worried about uptime, and not impacting the end-users (as one might need to be sometimes), one should then proceed a lot more carefully than what I have shown above. A better alternative could be as follows:   cp -rp ./sasinside/ ./sasinside_new/ rm ./sasinside_new/license.sas ./sasinside_new/SASViya*.jwt cp /tmp/newlicense/SASViya*.txt ./sasinside_new/license.sas cp /tmp/newlicense/SASViya*.jwt ./sasinside_new/   What this allows us to do, for newly launched containers, is to mount ./sasinside_new/ instead of ./sasinside/ into them.   You are now fully ready and able to test your changes without fear of impacting the end users. You can toggle back and forth between the 2 versions of the license on each container that you launch.   So now for the validation:   Validation We stop the running container, and restart it, but pointing at sasinside_new rather than the original sasinside . Notice that we are using the same image:   docker container run \ --interactive --tty --rm \ --volume ${PWD}/sasinside_new:/sasinside \ --volume ${PWD}/sasdemo:/data \ --volume ${PWD}/cas/data:/cas/data \ --volume ${PWD}/cas/cache:/cas/cache \ --volume ${PWD}/cas/permstore:/cas/permstore \ --env CASENV_CASDATADIR=/cas/data \ --env CASENV_CASPERMSTORE=/cas/permstore \ --publish 8888:80 \ prebuilt:v1   Notice that the Base SAS License and the CAS License now report the newer date:        This confirms that we are indeed now using the newer license. (JAN 27 instead of JAN 06).   Updating the license for the BYO Image In the previous section, we have seen how storing the license in a mounted folder seems very practical at first, but can also lead to issues if one is not careful. Eventually, the elegant solution is also more complex, and pretty much involves version control of the files.   While this seems fast and practical, I personally dislike this approach. (This is merely a personal preference, and some colleagues vehemently disagree with me.) I dislike it because it leaves me at the mercy of the presence of this file in that folder. If someone was to accidentally delete that file, it would start impacting the users very quickly and it might take some time before you figure out what's wrong, find a backup of the folder and license, restore it, and get back to normal.   The steps in this section will illustrate what I personally consider to be a "safer" way of doing things when dealing with containers. You are entitled to disagree, as long as you read my post until the end before commenting. Dissenting opinions are welcome.  😉🙂   Unlike the Pre-Built image, the BYO image contains the SAS License, straight inside the image. The downside is that if someone was to steal the image, they'd also steal its license. The other downside is that I will have to create a new image each year, with the new license in it. But the upside is that the image is much more self-contained, self-sufficient, and portable, which to me, more than makes up for the drawbacks.   I am reminded of this colleague who downloaded the Pre-Built image from a Linux Server down to his laptop, only to find it "not working". It had slipped my colleague's mind that he was also supposed to download the license file, and mount it identically on his laptop. The fact that the wound was self-inflicted did not lessen the pain of wasting a couple hours. The reaction was more along the lines of "So much for the promise of 'build once, run anywhere!'".   Re-running the Build.sh script At this point, you might be tempted to re-run the initial build.sh script that you had used the create the first image. Although that could be an avenue, there are some serious considerations to take into account. Do you still have a copy of the RPMs that were used to build the initial image. If you did not follow my usual advice to create a mirror and build from it, let me save you some time: You do not (have a copy of the RPMs), and you won't be able to get one. If you don't have a copy of the original Mirror, you can build a new image, but chances are the new image will ALSO have a different version of the software. Are you sure you want to be changing 2 things (license, software version) at the same time? (Pro tip: You do not) You have to have same exact version of the Container Recipes code. Even if you can get the old version of the recipes, it might no longer work. (a change in Python2 packages has forced us to fix the last version of the recipes. Previous versions of the recipes do not have this fix in them, so you'd need to port that fix yourself) Even if you manage to get all of this sorted out, it's likely that the result will be quite inefficient: Your new image is likely to not have any layers in common with the old one, and therefore consume just as much space as the old one. (instead of leveraging the docker deduplication) So, to sum up: Can you re-run the build script? Probably. But should you? Probably not.   Creating V2 from V1 The method highlighted here might seem overly complicated at first, but it is quite important to be able to grasp it if you want to get a good understanding of Containers. What we will do is: Figure out what needs to happen to the V1 image to turn it into a working V2 that has a newer (updated) license file Transcribe that into a Dockerfile file (not a typo, that's what it's called) Build V2 from V1 so that the only difference is the license Confirm it all works before you announce success and disappear on a well-deserved vacation   Figure out which files need replacing in the V1 image At first, I thought I could easily do this without looking anything up. It turns that I was wrong. The following documentation pages ended up being quite useful: If you want to be able to snoop around in a fresh instance of the container, the following command can be quite useful:   ## start a "blank" container with a bash prompt: docker container run -it --entrypoint /bin/bash byo:v1   What this does is bypass the regular entrypoint, and "bashes" you into the container. Note that because of that, none of the services are started, and you might also be missing pieces that would have been performed by the regular entrypoint script.   But if you want to investigate the inside of the container, you can then run things such as:   find /opt/sas/ -'*.jwt' find /opt/sas/ -'*license*'     How to create a new image First, we need a working area and we should copy the files we need into it, while renaming them, for easier handling:   mkdir ~/image_v2/ cp /tmp/newlicense/SASViya*.txt ~/image_v2/license.txt cp /tmp/newlicense/SASViya*.jwt ~/image_v2/license.jwt   In order to create a new image, we simply need to create a dockerfile. Read the code and the comments carefully:   tee ~/image_v2/Dockerfile > /dev/null <<'EOF' ## The source image we want to start from: FROM byo:v1 ## copy the 2 license files to the inside of the image COPY ./license.txt /tmp/license.txt COPY ./license.jwt /tmp/license.jwt ## Execute the license Update routine (as the user 'sas') RUN su - sas -c "/opt/sas/spre/home/SASFoundation/utilities/bin/apply_license /tmp/license.txt /tmp/license.jwt" ## Delete the old license and replace with the new one: RUN su - sas -c "rm -f /opt/sas/viya/config/etc/cas/default/SASViya*_Linux_x86-64.jwt ; \ cp /tmp/license.jwt /opt/sas/viya/config/etc/cas/default/license.2020.01.21.jwt ; \ rm -f /opt/sas/viya/config/etc/cas/default/sas_license.txt ; \ ln -s /opt/sas/viya/config/etc/cas/default/license.2020.01.21.jwt /opt/sas/viya/config/etc/cas/default/sas_license.txt" ## Clean up those stray files we no longer need: RUN rm -f /tmp/license.txt /tmp/license.jwt EOF   Now that we have that file, we can run the following docker build command to create a second version of the image:   cd ~/image_v2/ time docker image build -f ./Dockerfile -t byo:v2 .   The output you get should look like this:   Sending build context to Docker daemon 16.38kB Step 1/6 : FROM byo:v1 ---> 1c1c66cdf6d5 Step 2/6 : COPY ./license.txt /tmp/license.txt ---> Using cache ---> edd3341b38eb Step 3/6 : COPY ./license.jwt /tmp/license.jwt ---> Using cache ---> d12d395fba6a Step 4/6 : RUN su - sas -c "/opt/sas/spre/home/SASFoundation/utilities/bin/apply_license /tmp/license.txt /tmp/license.jwt" ---> Using cache ---> db94abadb24c Step 5/6 : RUN su - sas -c "rm -f /opt/sas/viya/config/etc/cas/default/SASViya*_Linux_x86-64.jwt ; cp /tmp/license.jwt /opt/sas/viya/config/etc/cas/default/license.2020.01.21.jwt ; rm -f /opt/sas/viya/config/etc/cas/default/sas_license.txt ; ln -s /opt/sas/viya/config/etc/cas/default/license.2020.01.21.jwt /opt/sas/viya/config/etc/cas/default/sas_license.txt" ---> Running in c0a43094a5e9 Removing intermediate container c0a43094a5e9 ---> 80845a19260e Step 6/6 : RUN rm -f /tmp/license.txt /tmp/license.jwt ---> Running in 1d3267ff27fe Removing intermediate container 1d3267ff27fe ---> d1b11d43547e Successfully built d1b11d43547e Successfully tagged byo:v2 real 0m1.770s user 0m0.044s sys 0m0.042s   This process will not have impacted any of the existing running instance based off of the V1 image. And now, you can start testing and validating V2 privately. Your end users need not be informed there is a V2 image until you are 100% sure it's good to go.   Validating With the exact same syntax as earlier, save for the image name of byo:v2 instead of byo:v1, we restart our test container:   docker container run \ --interactive --tty --rm \ --publish 9999:80 \ byo:v2   On running our proc setinit code, we can see that the licences for both Base SAS and CAS have been properly updated:         At this point, we can safely start shutting off the old container Instances that were launched from V1 and start replacing them with V2 instances instead.   Playing Tag I usually don't venture too far out with tags. Mainly, I know I have not worked enough with containers to fully grasp image tagging best practices, and so I leave this to others. However, our license problem is presenting us with an interesting illustration of the power of tags.   (Obviously, for the Pre-Built image, this does not apply: we still only have one version of the image) Let's look at our BYO images:   docker image ls | grep byo   And we'll see:   REPOSITORY TAG IMAGE ID CREATED SIZE byo v2 7431acd8d684 57 seconds ago 10.5GB byo v1 f78122326319 8 hours ago 10.5GB   Anyone else than you will look at this and wonder what the difference is between V1 and V2. So let's add another round of tagging to make things clearer, for the good of future container-kind.   docker image tag byo:v1 byo:expires_20210106 docker image tag byo:v2 byo:expires_20210202 And now our images look like:   REPOSITORY TAG IMAGE ID CREATED SIZE byo expires_20210202 7431acd8d684 3 minutes ago 10.5GB byo v2 7431acd8d684 3 minutes ago 10.5GB byo expires_20210106 f78122326319 8 hours ago 10.5GB byo v1 f78122326319 8 hours ago 10.5GB   Much more meaningful. Isn't it?   Conclusion I have often stated in workshops the following: Bind mounts are for persistence, not convenience.   If you are externalizing a file as important as the license file simply because you think it will make updating it easier, you are opening yourself up for future human error.   If you own a car, you know that it is inconvenient to have to lock/unlock the car and start/stop then engine each time you need to use it.   And yet, you do it, because while leaving your car unlocked and the engine running might be convenient, there are risks associated with that, and most of us do not want to take those chances.   So do your future self a favor, and make it as difficult as possible for him/her to shoot himself/herself in the foot.   Search for more content from our group: SAS Global Enablement & Learning. ... View more

A brief history of VIRK   When we started working with Viya a few years ago, we felt that a simple and standard tool to stage the servers in preparation for the Viya deployment would be a good thing to have.   Since Ansible was the tool used to deploy Viya, it made a lot of sense to write this "tool to stage the server(s)" as an Ansible playbook as well.   We then quickly realized that if we found it to be a time-saver and a human-error-avoider(tm), then it was likely that our customers would too.   That is the how and why of how VIRK came to be.   Move over, VIRK, here comes Viya-ARK! However, in our excitement to share our work with others, we did not necessarily think about all the longer-term implications of our design choices, and just made it available. But to be honest, we were not sure if it actually would be useful to others or not. 😊   We've now decided that given the traction that VIRK has had with our colleagues and customers, it's time for VIRK to grow up, and get a fresh new look.   That look starts with, first and foremost, a new name: Viya-ARK.   The URL for it is therefore: https://github.com/sassoftware/viya-ark    Officially, I am required by law to inform you that ARK stands for Administration Resource Kit.   Un-officially, I think of it more as the Awesome Resource Kit. (You did not hear it from me and I will deny any knowledge of this if questioned. This post will self-destruct within 10 seconds of you reading it.)     What you need to know I know you are busy people, so I’ll give you what you need to know right here: VIRK will no longer be developed further, and will eventually be retired, once everyone has transitioned over to Viya-ARK. Viya-ARK is the new “branded” name of VIRK.     So, stop using VIRK, and start using Viya-ARK. Viya-ARK contains all the code-base that was in VIRK, and some new goodies too. (read the whole post if you like goodies).  Where VIRK used the concept of Git Branches to hold various versions of the software, Viya-ARK uses the concept of Releases. You’ll see, it's going to make things simpler. If you’ve read this far, then you know the most important points!   What you might want to know If you are still reading, then you probably have a few more questions. I'll try to answer them below.    How do I use Viya-ARK If you have used VIRK before, you will see that it is quite similar.    I will write a longer "howto" guide later, but the steps below should help you get started.    Assuming that you have an Ansible Controller ready, and want to check your environment for the various pre-requisites, you can run the following commands:   ## go to your home directory cd ~ ## install git and wget sudo yum install git wget -y ## clone the latest code from github git clone https://github.com/sassoftware/viya-ark.git ## Move into the the pre-install playbook folder cd ~/viya-ark/playbooks/pre-install-playbook/ ## execute the pre-req playbook, in check mode ansible-playbook -i ./pre-install.inventory.ini viya_pre_install_playbook.yml --check Of course, as a default, this will only run against the Ansible Controller itself, which is not necessarily what you want. You can start editing the pre-install.inventory.ini file, to add more of your future Viya servers.    What version do I have/use? In Viya-ARK, we are using the concept of "releases". (In my opinion, just another way of saying "version"). If you want to see which releases are available, you should head to the Releases page on Github. If you are curious, you can read about the versioning scheme we use, or see what changes occurred at which release   If you followed the steps above, you have git cloned the master branch, which essentially gives you the latest and greatest. If you want to know which version that is, you simply have to open up the file called CHANGELOG.md and the version at the top will be your current version:    cd ~/viya-ark head -n 5 CHANGELOG.md   As of today, I get the following:    # Changelog for SAS Viya-ARK <!-- LATEST RELEASE START --> ## Viya34-ark-1.1 - April 18, 2019 As you can see, the version I am using is Viya34-ark-1.1.    How do I get support? The best way to get support is to open a github issue. While you can open a ticket with SAS Tech Support (for example, if you don't have a Github account), be aware that the Support of Viya-ARK is not the same as the support for more traditional SAS Products: your questions and issues will be forwarded to the developers who have worked on Viya-ARK. Remember to include the Viya-ARK version number if ever you open a Gihub issue.    Using MMSU to start your Multi-Machine Environment.  Assuming that you have a deployed a Viya environment that spans multiple machines, you will probably want to check out the Multi-Machine Services Utility. Here are the high level steps:    ## Move Viya-ARK into the SAS Viya Playbook folder: mv ~/viya-ark ~/sas_viya_playbook/ ## go to the SAS Viya Playbook folder: cd ~/sas_viya_playbook/ ## Start your Viya environment ansible-playbook ./viya-ark/playbooks/viya-mmsu/viya-services-start.yml   Extra Goodie: The Viya Deployment Report! Have you ever wished you had a summarized report of your Viya Environment topology? No? Well, we are granting that wish anyways! You're welcome.   This is your reward for reading this far down in a pretty lengthy and technical post. Tell your friends!    First, let's look at how to run the report: ## go to the SAS Viya Playbook folder: cd ~/sas_viya_playbook/ ## Execute the Viya Deployment Report: ansible-playbook ./viya-ark/playbooks/deployment-report/viya-deployment-report.yml   Ok, fine, but what does that do? Well, it will generate a .html and a .yml file in the Viya Playbook directory. So that you can see what that result looks like, I'm attaching the results I obtained in one of my lab environments. Scroll to the bottom of this post and download the file called "reports.zip". One you have that, check out the .html file in it and how much information it contains!    Wrapping up  I hope this post was useful to you. Hopefully, it will make your transition from VIRK to Viya-ARK smooth and painless.  If you have further questions comment, feel free to post below. I'll do my best to answer in a timely manner. But for maximum visibility, you probably want to use your Github account and create an Issue in the Viya-ARK project.  ... View more

In your day job, how often do you to have to type the following words: docker run <something> I’m a relative newbie when it comes to Docker but I think that even as a newbie, there are some lessons to be shared. There are aspects of containers that are so important, and yet not so difficult to grasp, that I hope this post about SAS Viya and Containers will be useful to you, and allow me to share some of my early thoughts and insights. Before posting this here, I did share this content with some of my colleagues who are much more knowledgeable about containers than I am. And aside from rolling their eyes at my analogies and over-simplifications, their feedback was positive.   So, here are 5 simple things you should understand about containers. Containers are not magic Containers are not just a delivery model Containers, like cakes, have layers Containers, like cakes, can be homemade or store-bought Containers, unlike cakes, can be cloned quickly and infinitely I will develop these points and the ways they relate to Viya.   Containers are not magic I know, everyone likes magic. But behind a well-executed trick, there is often a lot work, preparation, and a little sleight of hand. In fact, while attending a beginner class on Docker and Kubernetes, I was surprised to learn that containers are not new, and that Docker essentially found a way to pull multiple technologies together into an elegant solution that has now become the standard way of doing containers. But, if your application is bad, putting it inside a container will not make it better. If your app does not scale, or if your app is stateful, putting it inside a container, although possible, might just make it behave even worse. Further to this, you will still be bound by the limits of physics: If the Server your container runs on has terrible IO performance, the containers that are running on it will not magically have amazing IO performance. Finally, if you are going to be using Containers in exactly the same way that you started using virtual machines fifteen years ago, there will be very little advantage, and possibly a lot of downsides. So, before you over-promise on what a container can do, check in with the experts. (Not me. A real container expert.)     Containers are not just a Delivery Model We, at SAS, often use the words “delivery model” or “deployment model". At Viya 3.3 we had one major delivery/deployment model, which was we affectionately refer to as "BareOS". That is to say, that if you had the right Operating System (Linux, at that time), Viya 3.3 could be installed. The implications being that you did not require any specific Infrastructure, but simply a Linux Server, physical or virtual, on-prem or on-cloud.  In September 2018, Viya 3.4 started supporting Windows and Containers, as “deployment model”. This sounds great at first glance: who wouldn’t want more choices in the way they the software gets delivered? But what those simplistic statements fail to convey is that not all “delivery models” are created equal, and don’t necessarily leave you with the same software behavior or capabilities. For example, although SAS supports running Viya on Windows as of September 2018, that is not true across the board for all SAS Viya offerings (e.g. SAS Visual Investigator is not yet supported on Windows). Furthermore, on Windows, only single-machine deployments are supported. (Not just CAS SMP, but the entire Viya platform on has to fit on a single server). So, although the answer to “Can Viya run on containers?” has changed from a no to a yes, this is a qualified yes. It’s a “Yes, but, it’s a single container image, and it’s a Programming-Only container, meaning that it’s only able to run SAS Studio, MVA SAS, and CAS, with possibly some Access Engines”. If someone simply says "Viya runs on Containers", they are not giving you the full picture and you should check that your specific requirements will be met by the container.   The reason I stress this point is because many might hear “delivery model” and think about something like Uber Eats. If you order a cake, you don’t care how it’s delivered, whether by car, foot, or bike. You only care about the time it takes for it to arrive, and that the cake gets there intact. But, you must realize that if you order a ten feet tall wedding cake, you might want to use the "truck" delivery model, rather than the "skateboard" delivery model. If the delivery model is to influence what kind of cake you can get, its flavor and how big it is, you will be a lot more careful about how it does get delivered. So, let me repeat it here again: The first Viya Container is a single-image, programming-only container. It will not be a full-fledged Viya deployment. And although you will be able to scale out, by running multiple instance of this Viya container, each one will work by default as an independent unit, which has both pros and cons. With that said, you can also bet on the fact that the offerings on containers will keep evolving, improving and expanding with time.       Containers, like cake, have layers The way that docker containers are built is by layering some changes. In a way, like some cakes. You’d start with a cake pan, obviously to contain the cake, and then add some layers. Batter, sugar, frosting, etc.. (Can you tell I don’t bake often?)   As of the 18th of September 2018, any SAS customer who licenses Viya 3.4, with their SAS Software Order, can download (aka “docker image pull”) and run ("docker container run") the Viya Programming-Only cake container image. (Beware, if you got your Viya 3.4 Order before that date, this won't work for you!).   To the average user, well, it looks simply like a cake. You don’t see the layers of a finished cake unless you cut it open. But, and this is where it gets interesting, if you need the cake to do a bit more, you can add more layers, on the top. You could, for example, add your own layer to the Viya Container-Cake, to overlay a Jupyter notebook on top. Now, although we can test and control the taste and look of the cake when it leaves SAS’s kitchen, there is no telling what adding more layers on top of that might do. There is also no way to test in advance all the possible combination of layers that could be added to a cake.       Containers, like cake, can be homemade or store-bought There is another aspect of containers that probably does not get enough publicity. If you come up with a new cake recipe, and this cake contains proprietary ingredients that only you sell, you have a couple options: You can simply manufacture the cake, and sell it in stores. Or, you could make the recipe available to the public. Then, if they get the ingredients, and follow the recipe, they should end up with the right-looking cake. This, however, is a potentially less reliable way of making that very same cake. For example, if you try to use a type of cake pan that is incompatible with the recipe, you might never get to the right cake. If you keep reading my posts and my container-related content, you’ll probably hear me differentiate between the “container recipe” and the “pre-baked container”. Keep in mind that they are slightly different beasts, and that their level of supportability will likely vary. For example, if you change the color of the tray in which you bake the cake, that's probably fine. But if you want to use a plastic container instead of a metal one, that probably will never work.       Containers, unlike cakes, can be cloned Every analogy must stop somewhere, and I’ve stretched mine to that limit. (In fairness, this is just a shortcoming on the part of the cake industry, in my opinion). Once you have obtained (or built) your cake (a Docker image), you can essentially clone it as many times as you want. When you “run an instance of a container”, you essentially start a clone of your container image. That running container instance will consume computing resources: CPU, Memory, IO, Network, etc.. So a container image , is really only using some space on a storage device.  I found the following on this website. I am paraphrasing: A container image is an inert, immutable, file that is essentially a snapshot of a container A running container instance is the result of running a specific container image This is an important aspect of containers: they are meant to be replicated. This will in turn influence how you choose to use it. Assuming that you have a Viya Programming-Only Container image, do you want to run one container for each user, or do you want to have a single running container, shared by all your users? There are pros and cons to each method, that are beyond the scope of this post.     Asking the right questions If you want to have something called "Viya on Docker", you need to ask yourself what the driving force behind this is. Is it because you are “moving to the cloud”? You can run Viya on IaaS very easily and successfully without using containers. Is it because you want Viya to scale infinitely and automatically? If so, be aware that it is not the current state of Viya, whether you are using containers or BareOS for that matter. Is it because you have a standard that everything must fit inside a container? There is a difference between do-able and advisable. Viya has many Stateful components, and fitting those into containers requires a lot of care. And I’m sure there are plenty of other reasons out there. It is up to you to learn more about the Viya container, and make sure that it is a good fit for your purposes. If you want to start learning more about SAS and Containers, this page should help getting you started.  ... View more

With the recent release of Viya 3.3, it is to be expected that many more existing SAS customers will get to embrace this exciting new platform. It’s also likely that fresh new customers will get to meet SAS through Viya alone. However, these greenfield customers will have one less choice to make, and so they are not the focus of my writing today.   Instead, I want to talk today about customers who have an existing SAS 9.X environment, who are now adding SAS Viya to their Analytics footprint. Therefore, if you are such a customer, or if you are one of the people likely to advise them, this post is written with you in mind.   If you are in the situation I describe, the one big question on your mind is likely to be: Should I run Viya on dedicated server(s), or should I run it on the same server(s) as my SAS 9.X environment? The question is applicable whether the servers involved are physical machines or virtual machines. For example, even if you have 2 VMs running on a single vSphere host, I will consider this to be 2 distinct and dedicated servers.   The short of it If you trust in the advice of this technical architect and don’t care about the details, then it’s going to be a short read for you: Overall, it will be better if you get new and dedicated hardware for the Viya environment.  Simple, eh?   The rationale of it Of course I’m not going to make such a claim without justifying it.   The above is my advice, and it’s based on my experience and my familiarity with SAS 9.X, Viya, customer interactions, and feedback from my colleagues who implement SAS Solutions at customers.   The rest of this post will go through the various considerations you should have in mind when making this decision. Either you will find my justifications obvious, or you might disagree and decide to not follow my advice, but at least you will have been informed of the potential pitfalls of not doing so.   Advantages of shared servers   – Cost It will come as absolutely no surprise to anyone that servers cost money. However, for the sake of completeness, I have to mention that NOT having to buy a new server, or NOT having to stand up a new Virtual Machine, or a new Cloud Instance will effectively save money. Duh.   – Early start So this one will not be surprising either, but even if a customer is willing to invest in new hardware, it can take time for this hardware to be ready. Although Cloud and Virtualization can speed up the process of obtaining fresh new servers, there are also a lot of customers who prefer physical machines (often for performance reasons). This may take a certain amount of time to be ready. But, obviously, it is true that nothing would be ready faster than something that is already up and running. 🙂   However, I don’t want to give the impression here that anyone can start installing Viya within a minute of connecting to a server. Regardless of how you obtained your servers and of whether they are shared of dedicated, there will be some critical prep work to execute before launching the Viya install. In fact, when I facilitate our internal Deployment workshops, I encourage my colleagues to worry more about the prep work than the actual deployment. For example, ensuring that ports are available and open across servers, users are pre-created, Linux packages are installed, and OS settings such as ulimits are properly configured are all key activities to ensure a smooth deployment.   Advantages of dedicated servers   – Having more hardware, the generic advantage I think we can all agree that if you had 2 laptops, one to run Excel, and another to run Word, you would get better performance than if you had both Word and Excel on the same laptop. What is this? You’re not convinced? I don’t blame you. There is an advantage, but it is so negligible that you would not notice it. But that’s because either of these programs only ever use a small fraction of your laptop’s capacity. (say 10% each). So even if you run both simultaneously, you still have 80% of the capacity available.  However, the cost of buying 2 laptops, and the impracticality of having 2 distinct computers, all of it to gain an imperceptible performance increase, is probably not something anyone would do.   However, you can’t and should not compare running SAS on a server and running Office on a laptop. Over time, SAS has made its analytics execute as fast as possible. This is possible due to the way SAS is designed to make use of all the computing resources at its disposal.   (quick aside #1: I started my career programming in SAS 6.12 on a PC, and I can tell you that while my programs were running on it, there were barely enough resources left for me to compose an e-mail. Oh, and when I could compose an e-mail, I’d put semi-colon at the end of each line.)   (quick aside #2: I know, you had nightmares about dropping your stack of punch cards on the floor. It’s not a competition.)   For this reason, and without any administrative overhead, it’s fair to say that running SAS 9.X and Viya on dedicated environments is likely to result in much better performance since you will have more hardware overall.   – Sizing concerns SAS offers its customers the option to perform a sizing exercise. This allows you to get an idea for the required size of a particular environment, given a particular amount of data, number of users, and desired execution time. Although sizing is more than just the number of cores (it’s also the network, memory, storage), we often use cores as a proxy for the rest.   What we typically see at customers is that their existing SAS 9.X environments are far from idle: over the years, with the addition of users and data, it’s not rare to see a high (75-99%) rate of utilization for SAS servers. Which is what you want: if you utilization was 10%, you’d probably be oversized.   But that means that, if the Viya sizing comes back with a requirement for a certain performance (say, as a short example, 16 cores, and 256 GB of RAM), then you would need to make sure that your existing SAS 9.X environment has that spare capacity to take on the additional Viya load.   It is possible that you are able to increase the size of your server, to add enough capacity to it, to account for the Viya footprint. This would require downtime to allow the server(s) to be upgraded, which would see the SAS 9.X environment being temporarily unavailable.   It’s possible that new Viya processing “replaces” existing SAS 9.X processing over time. In that case, the demand on resources should not really increase. However, it’s also possible that existing SAS 9.X workloads keep running unchanged, and the Viya environment is used for newer and additional processing.   – Accommodating distinct requirements on shared servers SAS 9.X and SAS Viya both have requirements in terms of Operating System, disk space, networking and OS settings. If you will install both on the same set of machines, you will have to carefully parse the requirements for each from the Documentation, and make sure that the single environment meets the requirements of both.   This might make things difficult, or plain impossible. Let me give a few examples: For many SAS 9.X products, the documentation asks you to set the sshd setting “maxstartups” to 1000. The Viya documentation asks you to set it to 100. What do think the number should for an environment that has both? The highest? The sum of both? The average? SAS 9.4 can be installed as any Linux user, part of any Linux group. So you might have deployed SAS 9.4 as a user called “sas”, which could be part of a group called “install”. However Viya requires that a user “sas” exist and that its primary group be “sas”. In such a situation, it would therefore become quite cumbersome to have to change the primary group of the account “sas” and if not handled properly, could have a very negative impact on the SAS 9.4 environment. Some of the static ports used in Viya are the same as those used in SAS 9.4. Therefore, one has to pick alternative values for these static ports before starting the Viya deployment. If your SAS 9.X is running on an OS that is not supported by Viya, you simply will not be able to run both of them on the same machine. Most of these are not showstoppers. But they complicate things and increase the potential for issues, not just on deployment day, but further down the road as well.   – Workload Management If you have ever worked with SAS Grid Manager, you probably have an appreciation for the concerns of Workload Management. Without specific workload management, everyone using a system has a full access to the environment on a first-in basis. And since anyone’s processing is liable to use up all the hardware resources, it can lead to unpleasant behavior. For example, a big but non-critical analytic job could use up 99% of the resources for a long period of time, therefore slowing down the legally-required regulatory report from the usual 5 minute to something like 1 hour.   These concerns would have to be addressed within your SAS 9.X environment and within Viya deployment. But, if both SAS 9.X and Viya share the same environment, then it needs to be managed across both worlds. Having SAS 9.X and Viya on dedicated server means that there is almost no way that newer Viya analytics workload can impact existing SAS 9.X workloads.  This alone, is likely going to save both the users and the administrators a lot of work.   Furthermore, even if you make the effort to actively manage your workload, there is no magic there: the solution to overloaded machines is to prioritize some workloads over others. You’d have to choose if you give priority to SAS 9.X or to Viya. Having them on separate and dedicated hardware means you don’t have to choose.   Conclusion I could probably go on, but I think I’ve hit the most salient points, and I don’t want to bury the message. Yes, SAS supports SAS 9.X and Viya on the same hardware and some customers will do this. Saving money on the purchase of newer hardware is always going to be appealing, I can’t deny it. But before choosing to do that, you should carefully weigh the real operational advantages and business value of having dedicated hardware. In my opinion, the amount of the hardware saving would have to be pretty high for it to yield a better TCO and overall business value. ... View more

2019.04.30 Update: Please read this announcement about the newer version of VIRK, Viya-ARK: https://communities.sas.com/t5/SAS-Communities-Library/Viya-ARK-the-Awesome-Resource-Kit/ta-p/552252    You are probably familiar with the expression "Many hands make light work." If not, you can probably guess the meaning: A task is easier when many people contribute to it. In that spirit, I am happy to report today that many hands have collaborated to the creation of VIRK, and I hope that many more will continue to do so in the future.   What's VIRK? VIRK - rhymes with "work" - stands for Viya Infrastructure Resource Kit. It is a collection of tools and utilities aimed at making SAS ® Viya ® infrastructure and readiness tasks easier, safer and faster. VIRK content will grow over time.   For now, the main tool available is an Ansible playbook that allows you to verify and/or configure your server(s) in preparation for a SAS Viya deployment. (The content of the playbook is based on the information from the SAS Viya Deployment Guide). In short, this playbook can perform in a couple minutes what could take a Linux admin up to a couple hours, prior to a SAS Viya deployment. And if there is ever a concern that the environment might have veered off the desired prerequisite state, you can easily rerun this playbook to either confirm or fix potential issues. In my series of YouTube videos on how to deploy SAS Viya, I did mention and use an early prototype of VIRK to perform the OS pre-reqs.   Where is VIRK? VIRK is hosted as a git repository on the SAS Software Github page. It is publicly accessible by anyone in the world. You can get VIRK and its documentation at https://github.com/sassoftware/virk. Note that their are multiple branches to reflect the multiple versions of SAS Viya. As of this writing (January 2018), the branch you would likely want to use is "viya-3.3."   See VIRK on GitHub   How can I use VIRK? Assuming that your Linux server can access the internet, and assuming that you already installed Ansible onto it, here are a few simple commands to get you started:   cd ~ sudo yum install git -y git clone https://github.com/sassoftware/virk.git --branch viya-3.3 cd ~/virk/playbooks/pre-install-playbook ansible-playbook -i pre-install.inventory.ini va_pre_install_playbook.yml --check   To unlock the full potential of this tool, you will have to update the inventory file to reflect your environment, and you should look at the default values of the variables (in virk/playbooks/pre-install-playbook/roles/virk.preinstall/defaults/main.yml) to make sure that they work for you.   Once you are more comfortable using the VIRK Pre-Install Playbook, you can remove the "--check" to let it change the things it wants to change.   Can I contribute to VIRK? Indeed you can! While VIRK is not officially supported by SAS Tech Support, its authors will do their best to address issues reported in Github, as well as incorporate pull requests. The main reason VIRK is available externally on Github is so that customers who find shortcomings or want to provide improvements can do so easily, quickly and for the benefit of the entire SAS Viya community.   Do I have to use VIRK? Absolutely not! This choice is entirely up to you. We understand that some Linux admins prefer to either do things manually, or to configure their own configuration management software. In such cases, they should simply consult the Deployment Guide and perform actions accordingly. You can find the SAS Viya Deployment Guide on the SAS Support site: support.sas.com/documentation/onlinedoc/viya/index.html   Remember, however, that you can always use VIRK in check mode (--check) to only confirm prerequisites. Check mode will not perform any changes to your environment.   Acknowledgements  A huge thanks to Thomas Pangborn and Sherrell Crenshaw for providing the elbow grease necessary for this content to be made available. ... View more