Why data quality really matters in analyzing sensor data
The influence of sensor data quality on data analysis
Article 1: Why data quality really matters in analyzing sensor data
One of the significant challenges facing the manufacturing industry, especially in Japan, is the shortage of human resources and how to transfer knowledge from skilled workers. This has increased the popularity of digital transformation (DX) and the Industrial IoT (IIoT). There is a growing need for automatic quality inspection of manufactured products and the use of computers to detect anomalies in manufacturing equipment.
Over the last few years, therefore, many companies have adopted new analysis systems using sensors and artificial intelligence (AI). These include both machine learning and deep learning systems. However, it is not always easy to derive meaningful insights from data analysis.
The problem may lie with the analysis—and many companies have called in consultancies to help. However, this is not the only problem. The reality is that many people do not realize that accurate sensor data is necessary for accurate data analysis.
Why is the manufacturing industry interested in AI-based sensor data analysis?
AI is promising in the manufacturing industry because it can handle cases that statistical methods such as threshold setting simply cannot address. One example is the replacement of sensory inspections (Case 1 in Figure 1). An inspector can compare the waveforms of normal and anomalous conditions on the screen of a measurement device. Statistically, however, it is not easy to distinguish minor waveform disturbances that occur at specific positions in the waveform from noise. AI is better able to make this judgment.
Another example is shown in Case 2 in Figure 1. The table shows the actual data for an inspection for small motors. There is a problem with the motors in #4 and #5. At this stage, however, the analysis shows that they are normal, because all measurement data are statistically within the thresholds. Here, AI may be able to identify defects that statistical methods will overlook.
Figure 1. Why is AI popular in the manufacturing industry?
Generally, there are two categories for sensor data analysis systems using AI: process systems and maintenance systems. Each has different monitoring targets and purposes (Figure 2). The primary purpose of data analysis for process systems is to improve yield and quality, and save energy. Typical applications for maintenance systems include reducing downtime, improving after-sales service, and managing safety.
Figure 2. Types of purposes of sensor data analysis using AI
Sensor data quality issues
At SAS, we work on many data analysis projects to realize smart factories. Most of the queries we receive on process systems are related to root cause analysis, quality prediction, and optimizing operating parameters. The main area of questions for maintenance systems is anomaly detection for manufacturing equipment (Figure 3).
Figure 3. The main areas of interest in smart factories
However, there are also cases where data analysis using sensor data does not give the desired results. There are many reasons for this, including sensor measurement errors.
Sensor data quality affects the results of data analysis
Data analysis begins with data collection. This means that good quality data are essential. Figure 4 shows the flow of a sensor data analysis system. System construction starts with data acquisition, then data storage and analysis, and finally automation of the whole system. Data quality depends on the first step, data acquisition.
Figure 4. The flow of a data analysis system using sensor data
Figure 5. The development flow of an AI model
Figure 5 shows the development flow of the AI model. The flow begins with business goal setting, moving onto data acquisition, feature extraction, and model development. All these steps require expertise, and each one is essential and complex. Some might suggest that model creation is the most difficult step. However, I suggest that the most critical steps are sensor data acquisition and feature extraction.
The bottom line, always, is that you can have the best model in the world, but if you feed it with poor quality data, then the quality of your results will also be poor. The quality of sensor data is critical to a reliable outcome from your analysis. My next article will discuss six ways to improve the data quality of your sensor data.
Related Articles
