Big Data

Outline for Discussion Paper on Initial Analytics Trial

Introduction

• The qldwater TRG acknowledged the impact that “Big Data” or “analytics” will have on the industry in the future and formed a sub-committee to investigate further involvement.
• Given the introduction of the DEWS KPI framework, there is likely to be an increased emphasis on analytics of water data in the near future and the industry should remain “ahead of the curve”.
• The sub-committee has agreed on the need for the industry to form a joint position on this issue but recognising the complexity and size of the industry has recommended an initial investigatory project.

Barriers to Analysing Queensland Water Industry Data

• Numerous data providers and range of reporting scales.
• Data could include KPIs, asset data, CAD, SCADA, GIS, sharing protocols, customer information.
• IP and privacy issues only starting to be recognised/resolved.
• Resolution of data architecture, warehousing, processing requirements and associated costs. 
• Variation in utility capacity for reporting and analysing data and digesting any information produced.

Proposed Trial

An initial trial investigation was recommended to explore the complexity and utility of water industry analytics in Queensland. Suggestions were assessed against the following criteria:

• Relevant question(s) posed that can be addressed using data from a range of sources and potentially adds value for a water utility
• No additional onerous reporting requirements for service providers.
• Analysis of data captured at higher than annual frequency but commonly recorded by utilities.
• Inclusion of a range of utilities and scheme sizes.

A project that met all criteria was analysis of water use at a community level compared with external drivers of water demand (e.g. climate, population, price and average income). Suggested initial data required to address this issue (over 3-5 years) is listed below along with the source of the data:

• daily water production/use per scheme (council service providers)
• daily rainfall, evapotranspiration or temperature (BOM)
• population size, average age and average income (ABS-census data)
• typical water bill (SWIM)
• Water Source - Bore, Dam, Desal (SWIM)
• Size of catchment (Qld Globe)

Potential questions that could be addressed through analysis of this data include:

• How does water use vary seasonally across the state and different water sources?
• What is the impact of rainfall and temperature on water use in different regions (annually and over several years)?
• Does population size and income impact on the above?
• Is the average amount paid for water correlated with any of the above factors?

Outputs

1. A discussion paper comparing each of the above factors to be reviewed by TRG ‘Big Data’ group and all councils volunteering their data (then potentially seeking broader industry comment).
2. Spreadsheets containing data sourced for all participants.
3. Analysis of data availability and feasibility of further trial of analytics of Queensland data.

First Steps

1. Seek volunteer councils of all sizes and request daily water production data since 2008 and any available demand management measures.
2. Collate data from other sources.
3. Cleanse and align available data.
4. Graph and present initial basic comparisons for further discussion in sub-committee.