Regulatory Compliance Data Management

Over the last decade, there has been a significant rise in data analysis and reporting along with the emergence of big data. However, it often appears that little attention is given to ensuring data is validated for its quality and purpose. In New Zealand, many regions have geographically remote water assets. Therefore, regulatory compliance data often must pass through many remote control and telemetry components such as media converters, RTUs, PLCs and gateways before finally being stored on a database within a corporate network. It is critical that this data’s integrity can be verified from source to destination.

This paper analyses data management through the entirety of a data path. Management of all elements of a data path creates a data custody chain that allows the path to be traced end-to-end. This is critical to validating regulatory compliance data and reports. A data path begins with data generation and includes all data transfers and data storage. Data generation is instrument focussed and involves the correct calibration and setup of instrument parameters. Data transfer encompasses data scaling, the volume of data transfers and data transfer protocols. Data storage includes time stamping, time synchronisation, data compression methods, data types and sometimes multiple databases.

However, the initial setup or correction of a data custody chain is only one part of data management. This paper also explores the management of instrument and software modifications along the chain. Change management ensures transparency, traceability, and verifiability of modifications to data paths.

Ultimately, this paper tests the hypothesis that a report demonstrating consistently compliant results can be hiding wildly fluctuating non-compliant events. It will lead you to ask in future – what is the data path compliance validation behind this report? Are these reported results real?