Building Docker Recipes and running SAS Viya Analytics in a Docker container
This post is intended to demonstrate the steps involved in building a SAS Viya programming only docker recipe and deploying that into a docker container. With the current release, any SAS customer who licenses SAS Viya, will have the ability to download the SAS Viya Programming-only SMP container offering with SAS Studio and run it in a docker container. This offering is called the Programming-only analytics container. It will not be a full-fledged SAS Viya deployment.
SAS Viya Programming-only analytics container offering with SAS Studio will have a recipe for how to efficiently build and deploy a docker container. This is the most flexible option as it will allow customers to layer their preferred OS and can pick and choose the components they would like to use to build their images and deploy the containers. To start, access the open source project on GitHub.
Docker
You can run docker in a windows, linux or a Mac environment. Make sure you have docker downloaded and installed and docker is started and running. If you are new to docker here is a good place to start: https://docs.docker.com/get-started/
Getting Started
Before you can install the SAS Viya Programming-only analytics container, you will need a valid SAS Viya Software Order. It does not matter what you have in your order, you could have VA, VS, VDMML or any other pieces: this recipe will only build and deploy the programming only environment in a container.
Download or clone the GitHub repository to your machine in a folder called /containers/poac. Notice that the repo creates a folder called sas-container-recipes-master. Change directory into sas-container-recipes-master. Notice the various folders in this directory, This will be your root folder. From the root of this folder we will be using the build.sh script to create the various docker recipes.
Copy the SAS_Viya_deployment_data.zip that came with your SAS Software order to the viya-programming/viya-single-container directory.
What is a SAS Viya Programming-only analytics container recipe?
A docker recipe allows you to build a set of docker images that represent a SAS Viya 3.4 usecase. A recipe provides end-to-end scenarios for exotic or otherwise advanced use-cases. Most users will not have a need for all of these, they can pick and chose what they want in their image. Recipes allow you to chose and build the images as you want. A base viya-single-container image is the starting point. Running the build.sh script will give you a SAS Viya 3.4 programming image. To this image you could add the addons to create custom images of your choice.
I guess you could use the analogy of a cake layering recipe here. Baking is not my forte so I may be using this poorly, but basically you select the layers you want in your cake, and “cook” them from the bottom layer up, building the cake layer by layer. A Programming-only docker recipe is just that. You build the base viya-single-container image and build it up by adding the other layers to it.
For example:
auth-demo: The auth-demo project will provide the ability to create a default user. It you specify nothing, then that user is “sasdemo”. If one provides specifics then the specific user that is created is whatever was provided. This would be useful for someone who would want to get a SAS Viya image up and running quickly and did not want to tye the container into LDAP, then one could use the auth-demo as a way to create a quick user that would be the CAS Admin user and the user for Jupyter.
auth-sssd: This provides a structure on how to hook in LDAP configuration that uses sssd and PAM
ide_jupyter_python3: This project allows you to add Jupyter Notebeook with Python3 to a SAS Viya programming image
access-xxxx addons: these projects allows you to configure the various SAS Access interfaces to the respective databases.
In this blog post we will be using the build.sh that exists in the root of the project to quickly build a set of docker recipes (images) that represent a SAS Viya 3.4 usecase.
Recipe 1: creating the viya-single-container (svp base image) with a default sasdemo user
This recipe will give you a viya-single-container image with svc-auth -demo: The auth-demo project will provide the ability to create a default user. It you specify nothing, then that user is “sasdemo”. If one provides specifics then the specific user that is created is whatever was provided. To build this image you would issue the following command.
./build.sh addons/auth-demo
The build step will take a while (about 30 min or so) while it downloads all the packages and installs. It will print what is happening to the console as well as to a file named build_sas_container.log. It will install ansible and other pieces that are needed in order for ansible to run. Once the playbook is done running, ansible is removed as it is no longer needed.
Once the image is built you could issue a “docker images” command to see the list of images that has been built. A base centos image gets built by default.
The svc-auth-demo project will provide the ability to create a default user. It you specify nothing during the docker run, then that user is “sasdemo”. This user is also the CAS Administrator.
To run this container issue the following command:
docker run –detach –rm –env CASENV_CAS_VIRTUAL_HOST=birdsgf23.unx.sas.com –env CASENV_CAS_VIRTUAL_PORT=8082 –publish-all –publish 8082:80 –name svc-auth-demo –hostname svc-auth-demo svc-auth-demo
You could provide another user instead of “sasdemo” by specifying the following arguments:
--env CASENV_ADMIN_USER=foobar --env ADMIN_USER_PWD=barbar
If you do that then the default user that would be created is “foobar” with a password of “barbar”. The “sasdemo” user will not be created in this case. Only the user “foobar” is created. For our example let us use these extra options.
A “docker ps” command shall tell you that this image is running on port 8082
If you want to see all the processes running in this container you can issue the following command:
docker exec -it svc-auth-demo ps -ef
You could also issue the following command to check the status of the Viya services running in this container:
docker exec --interactive --tty svc-auth-demo /etc/init.d/sas-viya-all-services status
At this point you could bring up SAS Studio with the following url:
http://<yourservername>:8082/SASStudio
You should be able to log in as user “foobar” and a password of “barbar”
Also log in to the CAS Administrator console as “foobar”
http://<yourservername>:8082/cas-shared-default-http/
and notice that “foobar” is the CAS Administrator.
Recipe 2: configure LDAP that uses sssd and PAM
This project allows us to create an image with sssd support and allowing a user to set the ADMIN_USER to a LDAP user.
You will need a sssd.conf file with all the custom ldap configurations placed in the auth-sssd folder. You could enable encryption by providing a sssd.cert file and placing that in the auth-sssd location.
In order to build this image simply issue the following command:
./build.sh addons/auth-sssd
This image will only build the base image with auth-sssd This may be desirable for security conscious customers that may decide to have only ldap users and not have local host account in the container. They may also choose to have one of the LDAP accounts to be the CAS Admininstrator.
To run the svc-auth-sssd container with a ldap user as a CAS Administrator, issue the following command. I want to make myself as an admin so I pass in my ldap userid as the CAS Admin in the option below:
docker run –detach –rm –env CASENV_ADMIN_USER=mevenk –env CASENV_CAS_VIRTUAL_HOST=birdsgf23.unx.sas.com –env CASENV_CAS_VIRTUAL_PORT=8083 –publish-all –publish 8083:80 –name svp_auth_sssd –hostname svp_auth_sssd svp_auth_sssd
Recipe 3: I want it all!
I am a user who wants it all. I want a base SAS Viya programming image with the default user, sssd and jupyter-python. To accomplish this I would issue a build command as follows:
./build.sh addons/auth-demo addons/auth-sssd addons/ide-jupyter-python3
The build.sh was setup so that you can pass in the collection of things you want to build. Running the above would create the following 4 images: viya-single-container, svc-auth-demo, svc-auth-sssd and svc-ide-jupyter-python3, where each one is based on the previous one.
Mae sure you have the custom sssd.conf file and the sssd.cert file copied in the auth-sssd folder before you begin building.
Since I want my image to have a collection of all of these things – I would only run my last container: svc-ide-jupyter-python3
Further, I would like my CAS Administrator to be a host user named “foobar” with a password of “barbar” .
Here is how my docker run command would look like:
docker run –detach –rm –env CASENV_ADMIN_USER=foobar –env ADMIN_USER_PWD=barbar –env CASENV_CAS_VIRTUAL_HOST=birdsgf23.unx.sas.com –env CASENV_CAS_VIRTUAL_PORT=8084 –publish-all –publish 8084:80 –name svc-ide-jupyter-python3 –hostname svc-ide-jupyter-python3 svc-ide-jupyter-python3
A “docker ps” should tell me that the svc-ide-jupyter-python3 image is running on port 8084
Testing with SAS Studio
Bring up SAS Studio: http://<yourservername>:8084/SASStudio/
Since we have sssd setup, you can log in as your ldap userid and password.
Create a cas connection and create a new cas session. For your cashost you can use localhost. But in the steps below I will also show you how to get the host name of the cashost.
CAS Administrator Console
Bring up CAS Administrator Console: http://<yourservername>:8084/cas-shared-default-http/ Log in as the CAS Administrator “foobar” and a password of “barbar”. This user was created in the previous step during the docker run step.
You will notice that the above code I ran in Studio as myself (mevenk) started a couple of cas sessions.
And notice that the CAS Administrator is “foobar”
In order to get the hostname for the cas host, click on “Runtime Environment” and scroll down to host. The cashost in this case is svc-ide-jupyter-python3
In defining the cashost, you can either use localhost or the cas hostname such as below
options cashost="svc-ide-jupyter-python3" casport=5570; cas;
Testing SAS Kernel with Jupyter
Bring up Jupyter notebook: http://<yourservername>:8084/Jupyter
Open the SAS kernel in Jupyter
The following piece of code will run the sas program in Jupyter Notebook.
ods html5 style=statistical; ods graphics / width=500 height=400; proc means data=sashelp.cars; run; proc sgplot data=sashelp.cars; histogram msrp; density msrp; run;
Testing python3 kernel with Jupyter
Now open a python3 kernel in jupyter and run the following cas action:
import swat
s = swat.CAS("svc-ide-jupyter-python3", 5570)
s.builtins.serverStatus()
import pandas as pd
import matplotlib.pyplot as plt
from IPython.core.display import display, HTML
%matplotlib inline
titanic3 = s.CASTable("titanic3", replace=True);
s.upload_file('http://biostat.mc.vanderbilt.edu/wiki/pub/Main/DataSets/titanic3.csv',
casout=titanic3)
type(titanic3)
titanic3.columnInfo()
For more about this example and the code – refer to the following documentation.
Note: The authinfo.txt needs to be in the users home directory if you want Jupyter to communicate with SAS *without* having to code user name and password in the SAS code. Jupyter runs as a specific user (Trusted) and thus taking you in that way. If you want a sense of security, you can provide the JUPYTER_TOKEN at “docker run” time which will make a user provide the token when going to the URL. By default this value is empty. More about this in my next blog.
At this point if you were to see the use sessions in the CAS Admininstrator Console – you should see the cas user sessions we created above in SAS Studio and python3 kernel in Jupyter.
Cleaning up and Starting again
The best part about containers is that they are lightweight and easy to use that I can cleanup and start all over again in minimal amount of time. The build scripts do not take long to build and starting a container is almost instantaneous. Here are some docker commands that you will be using:
# gives you a list of all docker images\containers docker images
# gives you a list of all running docker containers docker ps
# will stop all running docker containers docker stop $(docker ps -a -q)
# will remove a particular docker image # a "dockar images" command will get you the ImageID docker image rm <ImageID>
Closing Remarks
I hope you will all give this a try and post your comments here. Here are some additional reading, that you will find interesting.
- SAS Communities: Running SAS Analytics in a Docker container
- SAS Communities: You can have your cake and eat it too
- SAS for Containers
- Docker Toolkit for Data Scientists
