The missing link in data quality: The importance of information exchange between domain experts
The influence of sensor data quality on data analysis
Article 3: The missing link in data quality: The importance of information exchange between domain experts
In a previous article(Article 1 and 2), I explained that data quality is crucial to analysis of sensor data. Without good quality data, it is impossible to get reliable insights. You could have the best model in the world, but if the data you are using are not reliable, then your outputs, and insights, will also be false.
I also explained that there are six main ways to improve the quality of your sensor data, starting with understanding what you want to monitor, then selecting the right sensor for the job, mounting it and installing it correctly, selecting the right devices to acquire data, and finally, selecting the right data for storage and use in analysis. These aspects are critical to data sensor quality, and they all require experience and expertise to get right.
The missing link
However, there is one final and crucial factor that cannot be overlooked. In my experience, the key to delivering good sensor data quality is the proactive information exchange among professionals (Figure 1). We could describe this as the importance of cross-industry collaboration or, more simply, IT and OT convergence.
There are three main groups who need to work together to deliver accurate and useful sensor data. The first group is the sensor measurement experts. These are the people who know about sensors, and their use. They are able to identify the right sensors and the right devices to use for data acquisition.
The second group is skilled people such as technicians and operators in the field. These people can supply information about the details of the object to be monitored, the manufacturing process, the work process, the details of abnormal conditions, and what mechanism causes the abnormality. Without this information, it is impossible for sensor measurement experts to make an accurate judgement of what is required.
The third group is the analysts and data scientists who will be analyzing the data, or overseeing the AI-based system that performs the analysis in practice. This group provides essential information about what they need to deliver accurate insights, including the type of data, frequency of collection, and details about its use. They also draw on the information from the field operators to identify the business questions that need answering.
Figure 11. Proactive information exchange between domain experts prevents an easy mistake
Conclusion
Sensor data quality is key when building a sensor data analysis system using AI. There are six ways to improve the quality of sensor data: understanding the object to be monitored (understanding how abnormal conditions arise), selection of the sensor, mounting position, installation method, choice of the data collection device, and the selection of data to be stored in the data lake. Many people focus on the data analysis methodology when building a sensor data analysis system using AI. However, I argue that we should also be aware of the data generation source because the data quality determines the outcome of the data analysis.
The bottom line is that the key success factor in building a sensor data analysis system using AI is to have a broad range of knowledge available to give an expert view from sensor data collection to data analysis. This will allow the project team to answer all the crucial questions about the process, and deliver a system that answers the necessary business questions.
Related articles
