Data Data Everywhere and Not a Digit Dropped - The Waikato District Council Experience

This paper outlines the Waikato District Council experience in modernising the collection and management of data for operations and compliance reporting. Like it or not, data (and in particular good quality data) is driving the water industry. Data requirements are on the increase: to prove compliance, to prove performance, to plan for the future and to reduce risk. It has now got to the point where a small rural council like Waikato District Council will typically handle in excess of 30,000 points of manual data points and tens of millions of SCADA data points each year. This data comes from various different agencies and needs to be reported to numerous different stakeholders, so it is no longer acceptable to rely on human-based systems and simple spreadsheets to manage operations and compliance. 

Every consent and every standard has evolving rules for analysis and a responsible operator needs to deploy a system that caters for this. Add greater sophistication in analytical methods and handling of time series data from SCADA systems, and the traditional methods for data management are becoming swamped, inefficient and ineffectual as a management tool. 

When looking to the future of data management, Waikato District Council (WDC) posed the following basic questions: what data is needed, where is that data coming from, where does it need to go to, what is the quality of the data, what is the chain of custody for that data, how is that data managed, what happens to legacy data, what systems are needed for today’s data and how to make provision for future data requirements? An outline of how the Waikato District Council addressed these questions and arrived at a cloud based system is mapped in this paper. A case study of a simple aerated pond-based wastewater treatment plant is presented, including operations and compliance reporting that involves SCADA data from two agencies (one internal and one external), one lab, and site personnel, complete with internal and external reporting structures.