Michael Gorkow | Data Scientist @ SAS Germany & CV-Enthusiast
Face Mask Detection using Computer Vision
Views
3,140
Due to the Corona crisis many governments decided that people should wear face masks when they’re in public areas such as public transporation, supermarkets, etc.
Closely related to my other story where I track social distancing rules this demo shows how Computer Vision can help us to identify whether people wear a mask or not.
In this example I train a Tiny YOLOv2 model to detect faces and masked faces on images. This model is then used in a SAS Event Stream Processing project to score frames from a video.
Such applications could be interesting if applied to camera streams, e.g. coming from CCTV.
Obtaining Training Images
We need a lot of annotated images to train Computer Vision models. Collecting these images might be easy but labeling them costs a lot of time.
I spent roughly two hours to find and label images of people wearing and not wearing masks using Google image search. I ended up having 300 labeled images of people wearing masks and 150 of people without masks.
That amount is far away from being the perfect size of a training data set but we can try to enhance it with data augmentation.
Data Augmentation
In the context of simple image classification where we have one label per image it is easy to perform data augmentation. You can crop, rotate, shear, etc. your image as you like it — the label stays the same.
However, if you’re trying to train an object detection model, this doesn’t work anymore because your labels (coordinates) will change if you transform the image. To overcome this challenge we will not only transform our image but also the corresponding labels.
I wrote a small example function in Python that will perform horizontal flipping, HSV transformations (hue, saturation and value) as well as creating mosaic images consisting of 4 randomly sampled images. The labels are changed accordingly.
Using my augmentation function, I grew my training set from 450 images to ~9000 images.
An example for a transformed training image looks like this:
Training the Object Detection Model
With the available training data the training process itself is straight forward and not really different to my other examples. I am again using the Python API from SAS called SAS DLPy. The training notebook can be found here.
First I load and display my training data:
Afterwards I split the data into a training and validation set and fire up the training on my GPU. I trained the model for 80 epochs and achieved a validation loss of ~2. Not perfect but enough for a small demo application.
Then the final model is exported in SAS binary format called “astore” which can easily be used in real time applications using SAS Event Stream Processing.
Applying the Model to Videos
The advantage of using YOLO as an object detection model is its capability of running in real time, e.g. using videos coming from CCTV.
I developed a simple pipeline in SAS Event Stream Processing using the Python API SAS ESPPy that loads my trained model (bottom part) and receives video frames (top part). The frames are then scored in the last box. The notebook can be found here.
The result of this pipeline is a scored frame with coordinates and classes for our detected objects. Using simple coding in OpenCV we can display and save the results in a file.
More useful applications could use the extracted objects to generate alarms or restrict access to people without masks.
Final Results
I used my webcam to make a short video of myself wearing various different masks. The model is able to distinguish between masked and unmasked faces indicating the prediction with green or red bounding boxes.
The whole project can be found in my GitHub repository.
