Don’t let dirty data impact your network connectivity!
Prevent future errors with 1Data Gateway for ArcGIS and the UN Readiness App
Data quality issues can have unforeseen impacts on the migration of your source data to a Utility Network (UN) model. One small data error can have future implications, preventing you from successfully carrying out tasks such as running traces on your UN dataset.
In the following data journey, we will follow three main phases of having bad quality data if left ‘un-identified’ in your source dataset: The first phase detects a Problem which leads to an Impact and finally, we explore a Solution provided by the UN Readiness App deployed via 1Data Gateway for ArcGIS Pro.
Problem: A data error present in the utility network
A self-intersecting line is one example of a data quality issue, defined by Esri as a case where a linear network feature loops back and overlaps itself at the intersection of two vertices.
If this data quality issue is left "un-fixed" in your source data, this can have implications at various points of your data migration. As illustrated here, where a self-intersecting line can be seen to exist in your dataset.
Impact: Dirty area error is created if not fixed
The existence of this data issue in the UN has resulted in a dirty area error being identified while running a ‘Connected’ Trace.
The Error Inspector map has picked up multiple dirty areas in the UN dataset. Two specifically related to the quality of the geometric data associated with error 20: Self-Intersecting line, and error 25: Stacked point features.
Solution: Identify and Fix, using the UN Readiness app via 1Data Gateway for ArcGIS Pro
A pre-emptive method to combat such downstream impacts is to find and fix these data issues at the source level. The simplest way to achieve this is using the UN Readiness App, now available directly within ArcGIS Pro, so you don’t even need to switch applications!
1Data Gateway for ArcGIS Pro is a simple add-in that provides access to the UN Data Readiness Assessment. Once installed this becomes available under the 1Spatial ribbon display as shown above. The add-in allows you to run a series of network-specific data validation rules against your source data to ensure that the data quality standards required by the ArcGIS Utility Network are maintained.
Many of the rules present in the UN Readiness App are based on Esri’s “Dirty area” errors. For example Esri error 20: Self-intersecting line, or error 25: Stacked point features are related to the Data Readiness rules of ‘Geometry does not self-intersect’ and ‘point is not stacked’. Deploying this tool before attempting to start your migration process to the UN model works as a preventative solution for multiple future UN error-specific issues.
You can process source data via three different types of feature selection options; ‘All features’, the ‘Current Extent’ or selection via a self-defined polygon as demonstrated below which will only process a smaller subset of the source dataset to analyse its ‘Readiness’ for the UN.
You can track the progress of your submission while it’s running in ArcGIS Pro itself without the need to navigate to another platform. A finished submission is shown below, with the option to either Add Results to Map or Finish Submission.
After submission, the UN Readiness Non-Conformance Report is generated as a File geodatabase (File GDB) which can be added directly to the map.
In this example, concentrating on the self-intersecting line feature, we have identified a few non-conformances which can be viewed in the geospatial report available as a File GDB extracted from this tool. The File GDB can be overlaid on the original dataset as visible above, allowing you to navigate to the area of concern more easily for further analysis of the error and potential fixups of the data.
The Benefits of a Rules-based Approach
The culmination of this rules-based approach allows you to get a detailed geospatial report outlining the various data quality issues specific to the utility network such as ‘geometry does not self-intersect’, ‘Line has closing nodes’ and ‘point is not stacked’.
One major benefit of the tool being available to run in ArcGIS Pro are the reports being available on one central platform where you can easily navigate to the location of the errors and apply immediate fixes. Once fixed, the tool can be re-run on the ‘fixed’ dataset for verification of the applied fixes or even check previous records of failing features for comparison.
In addition, while the 1Data Gateway for ArcGIS Pro plug-in allows you to access previous data submissions, this information is also available to be viewed via the 1Data Gateway Interface providing details of the platform it was submitted in, users, dates, and the type of data that was uploaded as well as the status and outcome of the submission.
In conclusion, this data journey has investigated the benefits of using a rules-based approach deployed via 1Data Gateway for ArcGIS Pro with the primary goal being focused on the pre-emptive identification and fixing of your source dataset aiming to resolve any existing future Utility Network errors.
Don’t let dirty data impact your network connectivity!
In this webinar, we talk about how Data quality issues can have unforeseen impacts on the migration of your source data to a Utility Network model. One small data error can have future implications, preventing you from successfully carrying out tasks such as running traces on your UN dataset.