Private Docker destinations with SAS Open Model Manager
With SAS Open Model Manager 1.2 and SAS Model Manager 15.3 on SAS Viya 3.5, model containerization for Python and R models is supported. You must create a publishing destination and base image before users can publish Python or R models to a container destination.
In this post I will discuss what is required to create a publishing destination and a base container image for Python and R.
I should first state that the base set-up is required for both Python and R models to run in a Private Docker publishing destination.
Prerequisites and Overview
There are several prerequisites that must be in place before you can create the publishing destination and base container image. The following are the system requirements:
- Network connectivity to the Docker daemon must be provided. The Docker daemon (dockerd) must be listening on a TCP/IP port or socket.
- You need a Docker registry.
- You have downloaded the required files (python scripts) from GitHub.
- The Python3 and requests packages must be installed on the server where you will run the scripts provided on GitHub.
- You have a running SAS Open Model Manager container (or SAS Model Manager 15.3 environment).
I will guide you through completing these steps and creating the publishing destination and base container image.
Figure 1 provides an overview of the high-level steps to create the destination and run a Python3 scoring container with SAS Open Model Manager 1.2:
- Step 1: you must have a running instance of SAS Open Model Manager.
- Step 2: run the python script (create_privatedocker_destination.py) to create the publishing destination in SAS Open Model Manager. This uses the 'mmAuthorization.py' python script to retrieve an access token, using the access token to authenticate to the SAS Open Model Manager container and creates the publishing destination.
- Step 3: run the python script (create_python3_baseImage.py) to create the base container image. This uses the 'mmAuthorization.py' python script to retrieve an access token, using the access token authenticate to the SAS Open Model Manager container and retrieves the publishing destination information to create the base image.
- Step 4: publishing a model to the Private Docker destination will cause an instance of the scoring container to be created to run the model. The scoring container only runs for the duration of the scoring job (running the model).
Figure 1. Process overview
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
Note: The process is essentially the same for R, but in step 3 you need to run the ‘create_r_baseImage.py’ script specifying the publishing destination that you created in step 2.
Security concerns
There are security concerns with mapping the Docker daemon socket that are called out in the SAS Open Model Manager manuals. See the Docker Container Security Considerations section in the SAS Open Model Manager 1.2 Administration Guide.
There is additional information on the Docker web site. See the Docker documentation: Protect the Docker daemon socket.
Due to the security concerns it is recommended that a TCP/IP connection is used for the communication between the Docker daemon and the Docker run-time container, rather than using a UNIX socket connection.
To implement this, you must configure the Docker Daemon to listen on a TCP/IP port. The following steps can be used to enable the TCP configuration.
Configuring the Docker daemon using SYSTEMCTL
By default, if you have just installed Docker it will not be listening on any TCP port. Use the following steps to configure the Docker daemon (dockerd):
- Create a configuration file: startup_options.conf The file should have the following content.
This configuration uses port 2375, which is the default port used in the sample scripts.# /etc/systemd/system/docker.service.d/override.conf [Service] ExecStart= ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375 - Reload the unit files: sudo systemctl daemon-reload
- Restart the docker daemon with the new startup options: sudo systemctl restart docker.service
You can confirm that the Docker daemon (dockerd) is now listening on port 2375 using the following command:
sudo netstat -tulpn | grep LISTEN
Note: The example above is using an unencrypted connection. This is one method of configuration, you should confirm with your Linux administrator what is the appropriate configuration method for your environment.
Installing the Python3 and request packages
To create a publishing destination using the Python scripts provided on GitHub you need the 'Python3' and 'requests' packages installed on the server where you are going to run the scripts, in my case the Docker server.
Issue the following commands to install these packages.
sudo yum install -y python3
sudo pip3 install requests
Creating a local Docker Registry
To support the creation of the base container images you need a Docker Registry. You really shouldn’t use an open public Registry for this, such as Docker Hub. A better approach is to use a secure private Docker Registry. You can do this locally on the Docker server, as shown in Figure 1.
Working in my environment I used the following commands to create a local registry on my Docker server.
Step 1. Create the certificate and private key
Create the certificate and private key that we will use to secure the registry using the following command. You need to update the '-subj' parameter with your company details and the server name.
openssl req -newkey rsa:4096 -nodes -sha256 \
-keyout /registry/certs/server.key -x509 -days 365 \
-out /registry/certs/server.crt \
-subj "/C=US/ST=NC/L=Cary/O=Company/OU=Users/CN=server.company.com"
Step 2. Include the certificate in the Docker configuration
In this example we will use port '5001' for the Docker Registry. You need to create the folder for your certificate and copy the certificate created in step 1.
sudo mkdir -p /etc/docker/certs.d/server.company.com:5001/
sudo cp /registry/certs/server.crt /etc/docker/certs.d/server.company.com:5001/ca.crt
Step 3. Start the Docker registry
Use the following command to start a Docker registry.
docker container run -d \
--restart=always \
--name local_registry \
-v /registry/certs:/certs \
-v /registry/images:/var/lib/registry \
-e REGISTRY_HTTP_ADDR=0.0.0.0:5001 \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/server.crt \
-e REGISTRY_HTTP_TLS_KEY=/certs/server.key \
-p 5001:5001 \
registry:2
Note: Steps 1, 2 and 3 assume that you have created the following folders: ‘/registry/certs’ and ‘/registry/images’.
Step 4. Create a GUI for the local Docker registry
Having a GUI for the Docker Registry will make it easier to confirm the image creation and for debugging. Run the following command to create the GUI.
docker run \
-d \
--name local_registry_frontend \
-e ENV_DOCKER_REGISTRY_HOST=server.company.com \
-e ENV_DOCKER_REGISTRY_PORT=5001 \
-e ENV_DOCKER_REGISTRY_USE_SSL=1 \
-p 15001:80 \
konradkleine/docker-registry-frontend:v2
You can now test the set-up by pushing a small image to the registry. You can use the following commands to pull the ‘hello-world’ image and push it to the local registry.
docker image pull hello-world
docker image tag hello-world server.company.com:5001/testing/hello-world
docker image push server.company.com:5001/testing/hello-world
Creating the Publishing Destination and Base container image
With all the prerequisites completed you are now ready to create a publishing destination and base container image.
To do this we need to customize the sample scripts that are provided on GitHub. The following scripts are required:
- create_privatedocker_destination.py
- mmAuthorization.py
Depending on whether you are working with Python or R models, you will also need one or both of the following:
- create_python3_baseImage.py
- create_r_baseImage.py
You can use the following command to download the scripts from GitHub, it assumes that you have a ‘/working’ directory in your home folder.
BASE=~/working
CONTENT="https://raw.githubusercontent.com/sassoftware/open-model-manager-resources/master/addons"
curl -s -o "$BASE/#1" "$CONTENT/{create_privatedocker_destination.py,create_python3_baseImage.py,mmAuthorization.py,create_r_baseImage.py}"
Creating a Publishing Destination
To create a publishing destination the ‘create_privatedocker_destination.py’ script is used. This is used to create Private Docker destinations for both Python and R models. You need to edit the python script to provide the details for your environment. The following changes need to be made:
- viya_host = “<the hostname or IP address for you Viya environment>”
- port = ":<port_number_for_your_viya_environment>"
- admin_userId = "<SAS_admin_username>"
- user_passwd = "<admin user password>"
- dest_name = "<the destination name>"
- BASE_REPO_URL = "<your_server_name>:5001"
- DOCKER_HOST_URL = "tcp://<your_server_name>:2375"
Note: Container destination names have a maximum length of 128 characters. The name must start with a letter or an underscore. It cannot contain spaces, multi-byte characters, or special characters other than the underscore.
Figure 2 shows the create_privatedocker_destination.py script. As I am running everything on the same host I can use “localhost” for the ‘viya_host’ parameter. The publishing destination is called ‘pythonModels’.
Figure 2. Updating the create_privatedocker_destination.py script
If you inspect the image above, you will see that the ‘create_privatedocker_destination.py’ script has a dependency on the ‘mmAuthorization.py’ script. The mmAuthorization.py script is used to get an access token to authenticate to the Viya environment.
To create the destination, you need to issue the following command.
python create_privatedocker_destination.py
When a publishing destination has been created you should see an http response code of 201.
A word of caution here, this (<Response [201]>) just means that the publishing destination has been created, it does not mean that it is usable. The script does not test the validity of the parameters that you have provided for the Docker daemon or registry.
Creating a Base Image
Now that the destination has been created the base image can be created. To do this we need to customize the ‘create_python3_baseImage.py’ script, providing the following details:
- viya_host = “the hostname or IP address for you Viya environment”
- port = ":<port_number_for_your_Viya_environment>"
- user_id = "<SAS_admin_username>"
- user_passwd = "<admin user password>"
- dest_name = "<the destination name>"
For example, using the ‘pythonModels’ destination that I created above, my file looks like the following.
Figure 3. Updating the create_python3_baseImage.py script
Again, the create_python3_baseImage.py has a dependency on the ‘mmAuthorization.py’ script.
To create the Python3 base image, you need to issue the following command.
python create_python3_baseImage.py
To create an R base image, you first need to create a publishing destination as above. For example, I created a destination called ‘rModels’, then updated the ‘create_r_baseImage.py’ script as shown in Figure 4.
Figure 4. Updating the python create_r_baseImage.py script
To create the R base image, you need to issue the following command.
python create_r_baseImage.py
When the creation of the base image has been successful you will see a http response code of 201. For my environment it took around 1-minute 30s to create the Python3 base image and 3 minutes to create the R base image.
Finally, you can confirm that the images have been successful pushed to the local registry using the GUI that we installed. I have run both image creation scripts; in Figure 5 you can see an entry for the python3-base image and the r-base image. The GUI is using port '15001' (http://server.company.com:15001).
Figure 5. Images in the local Docker Registry
Publishing a Model
Once the publishing destination and base image(s) have been created you are able to use the publish model function. For my SAS Open Model Manager environment, I used the Quick Start tutorial to validate my environment for publishing a Python model.
The following screenshot shows the Private Docker publishing destinations that I created as well as the default CAS and maslocal destinations. See Figure 6.
Figure 6. Publishing a Python model
Conclusion
It might be stating the obvious but having all the prerequisites in place is key to being able to create the base container image. In my testing I found that being able to authenticate to the Docker Registry was key. I initially started by trying to use an unsecure registry, but this proved to be more difficult than you might think.
The best approach is to use a secure registry as shown here.
I hope this is useful and thanks for reading.
References
- GitHub resources
- Docker: Protect the Docker daemon socket
- SAS Open Model Manager 1.2 Administration guide: Configuring the Docker Daemon
