Imagine you are responsible for manufacturing quality at your company. Wouldn't it be great to know what percentage of product contains flaws? Maybe replace human oversight with automation or collect data to perform a root cause analysis. SAS Analytics for IoT is a perfect solution to solve this or any Computer Vision (CV) problem. This comprehensive AI-embedded solution provides so many capabilities because it integrates a number of key SAS products (see the figure below).
Here we are going to simulate our manufacturing process using a Rubik's cube. Our cube represents a part which is being produced, and it is up to us to determine whether it was manufactured correctly. This will be accomplished using an image classification computer vision model. This article will cover the following topics:
Please refer to the github repository iot-image-classification-rubiks-cubes for more information and examples.
Before we get started please take a look at look at the following video which gives an overview of the process:
CV techniques provide the ability to acquire, process and analyze incoming images. This analysis produces numerical results in the form of predictions based on the classes we define. In this example, we need to create one CV model which will be used to tell if a Rubik's cube is solved or unsolved. We will also add in a third class which will represent any image where the cube isn't properly presented to the camera. We don't want to score images where there is no cube present or the camera can only see one side of the cube. Having an invalid case makes the predictions more accurate. Since we are using image classification without object detection our cubes must be correctly positioned within the camera frame in order to get accurate results. Here is an example of a correctly positioned cube:
Ideally, three sides of the cube should be visible to the camera and you would need two cameras to know for sure that the cube is correctly manufactured. However, for this example I am considering two visible sides valid, and simply holding the cube in my hand as I wait for the image to be scored.
The first step in the process is creating an awesome training dataset. When building your dataset remember these axioms:
It is recommended that no less than 1000 images be used for each class you intend to train and more images are always better. These images are also subdivided into two datasets. The first is for training and the second is a testing dataset. Therefore, an additional 200 images for testing is also recommended. Image size should be no less than 224 by 224 pixels. You may need to add your own images to the training dataset. For example, I was told that Rubik's cubes in Brazil have different colors. For my training, I collected 1000 images of the 3 classes, good, bad, and empty. Each class of images needs to be separated into its own sub directory.
Before we build an application that uses analytical models to predict outcomes, we need to train them. The training process for CV involves classifying images and separating these images into datasets that can then be fed into a machine learning model such as ResNet50, VGG16, or Darknet. This stage of the process is completed using the SAS Deep Learning Python (DLPy) package which provides the high-level Python APIs to the deep learning methods in SAS Visual Data Mining and Machine Learning (VDMML).
The previous diagram illustrates the flow for training a deep learning analytical model. Starting from the left we see that a Jupyter notebook is used to issue DLPy and SWAT commands which communicate with SAS Viya. For large jobs Viya has the ability to farm processing out to workers and take advantage of parallel processing. GPU support is also available. Deep learning algorithms are then invoked as each image is processed to create a portable analytics file called an .astore file. A typical training exercise contains these steps:
For more information about the DLPy high-level Python APIs which allows you to build deep learning models please see the following examples:
Streaming analytics is defined as the ability to constantly calculate statistical analytics on an incoming stream of data. In our case, that stream of data is the images coming from the camera. SAS Event Stream Processing (ESP), which is part of the SAS Analytics for IoT solution, provides the ability to deploy our newly trained analytical model, in the form of an .astore file, at the edge. With ESP you can ingest, filter and transform your data in stream.
Here we can see the flow of information through the system and highlight some key points:
As seen here, the ESP model which creates our detector is quite simple. All that is needed is a source window to inject images, 2 windows to load the CV model and a scoring window. Once the model is loaded and running a Jupyter notebook can be used to visualize the outputs from the imageScoring ESP window.
Using a Jupyter notebook we can use ESPPy to display ESP project information and subscribe to the imageScoring window to see the results. Let's take a look at the ESPPyRubiksControl Jupyter notebook which is located in the streaming directory. Using this notebook you can,
After the notebook is running in Jupyter connecting to an ESP server is a simple one line command. Once you have a connection to the server, you can query it for information about projects (running and stopped) and the server itself. You can also create, load, start, stop, and delete projects. Projects can be loaded from file paths, Python file objects, or URLs. Using the project.to_graph(schema=True) we can graphically see the currently active project.
Jupyter notebook and ESPPy can be used to show streaming statistics. For example, here is a bar which shows the recent Rubik's cube results:
The combination of ESP, ESPPy, Python and Jupyter notebook is quite powerful. As you can see, SAS Analytics for IoT provides you with all the tools you’ll need to quickly go from concept to production. Although this example consumed image data, you can use any type of data. This comprehensive solution also provides tools to maintain and govern model development, as well as everything you need to visualize and cleanse new data sources. Let’s see where your imagination takes you! I’d love to hear how you put SAS Analytics for IoT to work for your company.
Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.
If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website.
Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning and boost your career prospects.