In my last community article I shared some insights about Deep Learning with SAS Viya on GPU Accelerated Docker Containers. SAS Viya is the perfect choice for developing and scoring deep learning models as long as you are not dealing with applications that need real-time performance.
SAS Event Stream Processing for Streaming Analytics
For real-time applications, such as analyzing video streams, you should turn to SAS Event Stream Processing.
Deep Learning models developed in SAS Viya can be easily exported in the astore format. You can then integrate the astore model into your SAS Event Stream Processing process. This is the approach behind most of my Computer Vision demos. Have a look at these two small code examples that demonstrate the process:
# Have a look at the save_to_astore function from SAS DLPy.
# This function will export your model as an astore.
# Assuming your trained model is called model
model.save_to_astore("/path/")
# In the next step we will use SAS ESPPy to load the astore model.
# We use a Publisher to send the model information into our ESP
# process and activate GPU processing with USEGPUESP.
publisher.send('i,n,1,"USEGPUESP","1"\n')
publisher.send('i,n,2,"ndevices","1"\n')
publisher.send('i,n,3,"action","load"\n')
publisher.send('i,n,4,"type","astore"\n')
publisher.send('i,n,5,"reference","model.astore"\n')
publisher.send('i,n,6,,\n')
publisher.close()
SAS Event Stream Processing on Edge
There are two different versions of SAS Event Stream Processing:
- SAS Event Stream Processing
- SAS Event Stream Processing on Edge
The second option is the perfect choice if you plan to use SAS ESP with Docker containers. It doesn’t come with unnecessary user interfaces but focuses on programmatical development. I usually use the Python interface.
Note: We will also use this version for deployment on edge devices such as NVIDIA Jetsons, which will be explained in my next community article.
SAS Event Stream Processing on Edge with GPU accelerated containers
If you take a look at the documentation of SAS ESP on Edge, you will see that Docker Deployments are officially supported. However, they currently do not support GPUs. A No-Go for scoring deep learning models in real-time.
To still support GPUs, we can create our own Docker Container. As described in my last article about SAS Viya, we are again using the NVIDIA Container Toolkit and a base image from NVIDIA that supports GPUs.
In addition to SAS Event Stream Processing on Edge we will install common Python packages that will help us developing our streaming applications. The most important one being SAS ESPPy, our Python interface to the SAS Event Stream Processing and OpenCV 4.4.0 for Computer Vision tasks.
Get your hands dirty
If you have a license for SAS Event Stream Processing on Edge, an NVIDIA GPU and are interested in deep learning, you can take a look at my GitHub repository to see how to create an executable container in a few steps.
SAS Event Stream Processing Docker Container with GPU acceleration
Once your container is up and running, feel free to browse my other GitHub repository to see what kind of applications you could develop.
Final Remark
Please note: This is my private work and not officially supported by SAS. Feel free to ask me any questions but don’t expect official support from SAS.
