Data Tracking and Tracing in the Metadata Driven Organization
By: Nora Burema-Bracho Surga (1) & Zéger Nieuweboer (2)
(1) Nora Burema-Bracho Surga works as an Information Expert for the Ministry of Infrastructure and Water Management, The Netherlands, Europe. Nora holds a bachelor degree (BA) in Modern Languages at the Universidad Metropolitana in Venuzuela and a master of science degree (MSc) in International Business at the Sheffield Hallam University in the United Kingdom. Linkedin profile: linkedin.com/in/nora-burema-bracho-4840a55
(2) Zéger Nieuweboer works as a Data Steward for the Ministry of Finance, The Netherlands, Europe. Zéger did his minor at the University of Illinois and holds a master of science degree (MSc) at the Wageningen University & Research in the Netherlands. He holds a professional degree in engineering (PDEng) at the Dutch School for Technological Design at Eindhoven Technical University. He is certified in planning and inventory management (CPIM) and data management (CDMP). Linkedin profile: linkedin.com/in/zegernieuweboer
Topics of Interest in this article: Data Stewardship, Case Study, Innovation Lab, Lessons learned.
Abstract
The investments in data tracking and tracing in a big data organization are needed for compliance to data ethics and GDPR. Data tracking and tracing starts with identification, the metadata of data. Data tracking and tracing is successful in a well-organized metadata driven company.
Introduction
For years I worked within the consumer goods industry, with a main focus at marketing and logistics. In the consumer good industry a goods tracking and tracing system is needed. Because of government legislation and consumer risk management. Later, I made a career switch to a big data organization within the Dutch government. In this organization, we can learn from the goods tracking and tracing proces of the consumer goods industry.
Logistics
It is obvious that data is needed to manage the physical goods flow. Inventory data, production data, sales data, customer data. In a data organization the main flow is not a physical flow but a data flow. The data needed to manage a data flow in an organizations is called business metadata.
Tracking and tracing
Data tracking is the downstream activity to determine the data products with a given origin. Data tracing is the upstream activity to determine the origin of a data product. Data lineage is a data process that roughly covers the data tracing and tracking form source to data warehouse. The downstream activity data tracking becomes more important in a GDPR compliance organization. To prevent deliveries of privacy sensitive data products which can conflict with data ethics.
Case Study Data Tracking and tracing
In a government organization a discrepancy in use of the personal diversity data was detected. Some business information products contains personal diversity data which were not in compliance with data ethics and GDPR. The first upstream activity was data tracing: which data warehouse from which source contains this obsolete personal diversity data. An handicap was that data lineage was right in place for only the new data environment. With a data retention period of ten years older data stocks must be analyzed manually. The second downstream activity was data tracking. Which other business information product contain the obsolete personal diversity data. The metadata repository was of big help in the tracing search. In the historic metadata versions of the business information products it became clear where the diversity data was used and on which privacy detail.
Business metadata standard
Data tracking and tracing is successful in a well-organized metadata driven organization. A new business metadata standard was developed in a big data organization of the Dutch government. To accelerate the development the learning and prototyping process was done in an innovation lab: “The Living Blue Lab”. The developed new business metadata standard consists of three levels:
- Business rules
- Business Information products
- Data building blocks
The business rules represents the tax guidelines and internal framework agreements. The business information products represents the key performance indicators of the data company. The data building blocks are the connection between he business information products and the data flow. As part of the data building blocks, the data quality rules are described.
Simultaneously a business metadata work process was designed and implemented in the learning and prototyping innovation Lab. De business metadata process consists of three stages in the workflow:
- Define business metadata by information experts
- Review business metadata by business stakeholders
- Publish business metadata in metadata repository
Lessons learned
The business metadata standard was tested in the learning and innovation lab “The Living Blue Lab”. Results from the use of the business metadata standard are:
- The business metadata standard and the use of the actual metadata repository are requiered for the compliance to the international data ethics standard GDPR.
- The business metadata standard improves the quality of the business information product by the validation with the business rules.
- The business metadata standard improves the quality of data by integrating data quality rules in the design process with data vendors and data customer
The biggest lesson learned is that deployment of a successful lab solution into the harsh reality of the regular organization requires excellent transition leadership. Bases upon the results, the new business metadata standard is deployed in the organization. The new business metadata standard is the foundation of the standard data management solution.
Conclusion
The investments in data tracking and tracing in a big data organization are needed for the compliance to data ethics and GDPR. Data tracking and tracing starts with identification, the metadata of data. Data tracking and tracing is successful in a well-organized metadata driven company.
Version: July 2021, The Netherlands
This article is submitted for the: Data Governance & Information Quality Conference (DGIQ) December 6-10, 2021, San Diego, Californië
Sources
Data Stewardship Second Edition, David Plotkin. 2021, Elsevier Academic Press, ISBN 9780128221327
Supply Chain Logistics Management, Fifth Edition, Donald Bowersox, David Closs and M. Bixby Cooper. 2019, McGraw-Hill Education, ISBN 9781260547825
DAMA-DMBOK, Data Management Body of Knowledge, 2ND Edition, DAMA International. 2017, Technics Publications, ISBN 9781634622349
