This is the first of a series of blog articles focussing on data stewardship, the most complex but most valuable feature of developing data infrastructures across government, utilities and transport.
Matthew White, Senior Business Development Manager at 1Spatial kicks off the series by asking the question – How good is your data governance?
In recent months a number of strategies and studies have been published associated with data. These include the HM Government’s Industrial Strategy published in November 2017. The Industrial Strategy comments on why now is the right time to harness the growing momentum around data driven technologies, placing the UK at the cutting edge of trusted and innovative deployments of data.
The Open Data Institute (ODI) study opened a debate on our data infrastructure being as important as our physical infrastructure. The ODI commented that data is as important as our road, railway and energy networks and should be treated as such. The ODI continued to say that data infrastructure consists of:
- Data assets
- The organisations that operate and maintain them
- Guides describing how to use and govern the data.
Data infrastructure will only become more vital as our populations grow and our economies and societies become ever more reliant on getting value from data. For example, innovations such as the Internet of Things (IoT) and Machine Learning are enabling the use of massive amounts of data for the emergence of “digital twins”, creating living digital simulation models of physical infrastructure.
Similarly the National Infrastructure Commission (NIC) appointed Dr Peter Kawalek (Director of the Centre for Information Management at Loughborough University) and Ali Bayat (Alliance Manchester Business School at the University of Manchester) to explore the case for considering data as infrastructure. This was one of four pieces of external research commissioned to support the Commission’s study, Data for the Public Good. Data itself has become as important as physical infrastructure and must be managed for the best effect. Data infrastructure is critical at local, national and international scales, from helping make communities smarter, through to building resilience and responding to global challenges. Data infrastructure should be open for all to use and operate in standard and expected ways.
As commented by the ODI, a key strand to data infrastructure and therefore realising value from data, is having guides and processes describing how to use and govern data. The NIC’s research also sets out that the realisation of the value of data across the public realm depends on the use and governance of vast amounts of data. Efficient and effective data stewardship is fundamental to data infrastructures at local, national and international levels.
Data governance (also known as data stewardship) is the most complex but most valuable feature of a data infrastructure. The innovation examples described earlier demand that data, often coming from multiple sources is processed and available for modelling and analysis in near or real-time. Meeting these demands requires a different approach to data governance.
Raw data, for example from sensors and other sources need to be automatically ingested, processed and stored. Reliable data flow is key to doing anything with data.
Do you have reliable data extraction, transformation and loading (ETL) processes?
Is your data easy to access and analyse?
Automated rules and policies need to be applied to the data, flagging data as trusted or untrusted and recording the reason and source of the decision. The first rules produced are usually directed via effective data quality and data management rules for cleansing, validating, integrating, enriching and maintaining data assets. Both human and machine decision making requires the use of trusted data.
Do you apply data quality rules and policies?
Do you understand how trustworthy your data is?
There is a growing need across both public and private sector organisations to focus on data governance as the foundation for data infrastructure. Efficient and effective automated data governance will enable the realisation of the value of data for the benefit of the digital economy. This is becoming ever more important in a world where data-driven approaches to decision making and productivity are growing exponentially.