Using standards for improved data quality and data management
Why standards are critical for better data management
When it comes to systems, data, and working with data, standards are crucial. We discover this working across all our focus sectors – Government, Utilities, Transportation and Facilities Management.
Standards help with technical integration (e.g. file formats, APIs, etc) and behaviours and processes (e.g. many ISO Standards), but both help organisations or systems to work together safely and efficiently. Data standards make it easier to create, share, and integrate data by making sure there is a clear understanding of how data is represented and ensuring the data you receive is in a form or format you expect.
When developing a solution, it is often tempting to improvise a structure that meets your exact needs at the time, building it as and when required. However, this can quickly become unwieldly difficult to describe to other data users, and challenging to integrate with other sources.
Common data language
By utilising a standard, you are implementing a tried and tested infrastructure and common data “language” that you know others will be able to understand and use. It can help ensure things work seamlessly when they are plugged together.
At 1Spatial we have always supported a standards-based approach to help us build and work faster, to take advantage of specialised and expert knowledge, and to future proof what we develop. After all, standards also have their own lifecycle and roadmap, adapting and improving as technology evolves. Standards tend to be either defined formally, to force order on disorderly processes (e.g HTTP+HTML), or they emerge as de facto standards where something is so commonly used that it is treated like a standard without formal agreement (e.g. PDF files).
We have talked about the benefits of services in how we build systems, especially with our technology partners Esri and Safe Software, and you can read about that here.
How standards help improve Location Master Data Management (LMDM)
For all of the LMDM solutions that 1Spatial builds, there are common themes where standards help us.
- Simplify and Minimise Data Handling - The 1Spatial principle is to move the data as few times as possible, on each occasion performing as many of the transformation tasks as possible with the data in hand. This helps us to avoid unnecessary handling of data which can slow systems down, adds no value to the data, introduces extra cost in deployment and test and brings more “moving parts” which can be sources of error, duplication and data corruption. A pre-requisite to this step is having good tried and tested ways to share and interoperate with it. Data Standards help us to do this.
- Separation of Concerns - The 1Spatial principle here can be summarised as “every critical system, including GIS, should do one thing and every ‘thing’ should be done in the critical system”. Key system responsibilities such as data entry, validation or transformation should wherever possible be achieved using a single interconnectivity service which is able to perform the entirety of the action. Having complete services like this avoids complex interconnectivity between critical systems and enables the system behaviour to be thought about as a series of simple interconnectivity contracts. API standards help us to do this.
- Design for Testability and Reuse - The 1Spatial principle is to adopt a hierarchical testing approach, building up the entire system from its lowest constituent parts. With any solution that requires interconnectivity between critical systems, it is essential that it can be built up as a modular assembly of simpler components, each with defined responsibilities as described in Separation of Concerns, above. When systems are successfully tested in isolation, there is a greater chance integrated and interconnected system will behave correctly when combined to form a system of systems. Applying development standards such as good practice around risk reduction to data management and integration flows is paramount. For example, automated test suites for regression testing to achieve system-level CI (Continuous Integration) so that any changes to any part of the system can be immediately tested to confirm that nothing is broken.
There are plenty of standards available for use, some of them private and vendor-specific, others more widely adopted and vendor-agnostic.
Data management and the ISO family of standards
ISO (International Organisation for Standardisation) are an international body responsible for worldwide proprietary, industrial, and commercial standards.
You may be familiar with business standards such as ISO 9001 (Quality Management Systems) or 14001 (Environmental Management Systems) but there are thousands of ISO standards established for all manner of things – ISO 3103 defines a standardised way to brew a cup of tea!
At 1Spatial, as well as being certified for ISO 9001 and ISO 14001, which influence how we work, we have also employed several ISO standards when dealing with data:
- ISO 19107 (Spatial Schema): Geometries and topology within 1Spatial products are represented according to the models defined in ISO 19107 (Spatial Schema). We make use of geometry encoding and object interfaces, guaranteeing flexible and well-defined behaviour for a wide range of geometry and topology data. Validation according to these rules is provided as standard.
- ISO 19109 (Rules for Application Schema): ISO 19109 defines a standard method for defining application schema (i.e. user data models) in a consistent way and makes maximum use of the other TC211 standards for geometry, temporal data, metadata and other content. Standard representations of feature content, attribute structure, object association and inheritance are defined. 1SMS complies fully with the requirements of ISO 19109; this allows composition of rich expressive object models to describe application-defined features.
- ISO 19115/19139 (Metadata): ISO 19115 defines a standard conceptual model for the representation of metadata for geographic features and datasets. These models can be encoded and transferred using the XML representation described in ISO 19139. 1Spatial use elements of ISO19115 to describe quantitative dataset level data quality metadata for logical consistency and geometric and topological integrity.
- ISO 19136 (Geographic Markup Language): Geographic Markup Language (GML) is used to exchange geographic data on the Web. It provides facilities for defining custom application features, coverages, observations, and many other geographic phenomena and encoding these for exchange using standard XML/XMLSchema techniques. It is an important building block within the Open Geospatial Consortium reference architecture where it was originally developed, and it has now been standardised by ISO as ISO 19136 and is used in 1Spatial products.
The advantages of using Open Standards
Open standards have many advantages over private or vendor-specific standards.
There is often a misconception that “open” means less secure, however the opposite is often true. When developing solutions, there is no need to disclose sensitive datasets – the data structure is well known and so can be developed before introducing data.
In choosing an open standard over a proprietary or vendor-specific standard, you have the choice to move between a wide variety of tools, rather than being “held hostage” to one toolset for a long period of time. This can drive competition and enable access to the latest innovations in software solutions.
The Open Geospatial Consortium (OGC) is an international consortium of hundreds of businesses, government agencies and research organisations who work to create royalty free, publicly available, open geospatial standards. Their goal is to make geospatial (location) information and services Findable, Accessible, Interoperable, and Reusable (FAIR).
A number of OGC standards are implemented within the products that make up the 1Spatial Platform:
- Geographic Markup Language (GML): As mentioned earlier, GML is used to exchange geographic data on the Web. Both Oracle and 1Spatial sat on the revision working group for the current GML3 standard at OGC, where Oracle contributed key elements of the feature and geometry models and 1Spatial contributed the topology schema. GML is used within 1SMS for the exchange of feature data for editing using the 1Plan and 1Edit components.
- Web Map Service (WMS): WMS is a standard network service specification for accessing map images and limited feature information over the Web. Within 1SMS, WMS is used to provide access to user-supplied vector and raster map data.
- Web Feature Service (WFS): WFS is a standard query service specification for accessing geographic data encoded in GML or other formats such as GeoJSon over the Web. The majority of 1Spatial products provide the ability to create a data store to read and validate directly from a WFS.
These are often standards that have been adopted because of their popularity. They are frequently supported by open specifications, so even if they are owned by a commercial vendor, people can build robust systems that interoperate with them. We also find that these can often be industry specific, and it’s also possible that eventually these de-facto standards do evolve to become recognised open standards later in their lifecycle.
Building Information Modelling (BIM) involves a number of established and evolving standards, driven by a large global community of BIM specialists, software vendors, and projects.
The UK government originally introduced the BS/PAS 1192 series of standards for BIM. Many non-UK countries started to adopt PAS 1192 but suppliers felt that it was unfair to make them work to UK standards outside the UK. As a result, the international community approached ISO to request that the UK PAS 1192 series be elevated to an international level. ISO 19650 was born as a series of international standards that defines the collaborative processes for the effective management of information throughout the delivery and operational phase of assets when Building Information Modelling (BIM) is being used. The ISO 19650 series is based on the UK PAS 1192 series.
Various BIM and Land Management datasets exist within the overall BIM world which use various associated formats, such as:
- IFC – Industry Foundation Classes: Industry Foundation Classes (IFC) data model is essentially an interchange and exchange format, intended to describe architectural, building and construction industry data. IFC files are the main way to share BIM information between different CAD and BIM systems. IFC model files provide a standardised, digital description of the built environment, including buildings and civil infrastructure. The most important standard is the official International Standard ISO 16739-1:2018 Industry Foundation Classes (IFC) for data sharing in the construction and facility management industries — Part 1: Data schema. This is a popular format used in RealWorld4D.
- COBie - COBie (Construction Operations Building Information Exchange) is a non-proprietary data format. This is used to help capture and record important project data at the point of origin. COBie information may take several approved formats include spreadsheet, IFC file format, or ifcXML. COBie is based on IFC’s definitions, mapping the asset data to its rows and columns. We have used these standard formats in our recent work with the Environment Agency.
Standards help to interoperate data, systems and organisations. Some people may see them as constraining but by following standards you accelerate collaboration and integration and help future-proof systems.
To read more on where we have been working with our customers to use a standards approach in creating solutions, check out the following recent blogs: