National Load Mode

The National Load Mode is used to process a large area by breaking it up into partitions and processing each of them independently. This brings several benefits:

• Scalability: In National Load Mode, 1Generalise is able to process huge areas (whole countries), without loss of performance and without risking running out of memory. The initial partitioning only needs to load partitioning features (usually roads). The rest of the data is generalised one partition at a time, and the size of these partitions is under user control.
• Resilience: If a failure occurs during the generalisation of a partition, only this partition needs reprocessing and the rest is not affected. In contrast, if the national load was processed in a single job, any failure would require reprocessing the whole country, which would waste vast amounts of time. Typically, a number of new issues, not necessarily hard to fix, are always found when the first national load runs are attempted.
• Speed: This strategy allows the use of parallel processing. Once the initial partitioning process is complete, the generalisation of each partition can be executed independently, in parallel using a grid of processing nodes. The grid can be extended at will to reach the speed required.

To deliver the correct results, the national partitions must tessellate the dataset and each partition must exactly follow the partitioning features from which it was formed, after they have been generalised. In order for the national partitions to be processed independently, their boundaries need to be fixed when they are processed, to avoid conflicts between adjacent national partitions. The generalisations of the features that form the partitions boundaries is therefore conducted before the final creation of the national partitions.

Note: Any generalisation that reduces the number of features is required on partitioning features (like collapsing dual carriageways to a centreline, or thinning a network), these must be performed before the National Load Partition Creation Subflows, described in section 4.3.3.4.

The partitioning is performed using features belonging to classes specified by the user. Once automatic partitions have been created, these are merged to form bigger partitions containing smaller automatic partitions.

Parameters are used to control the level of aggregation. The next important step is to break up the outer partition, which is often formed on the outskirts of the dataset, when the partitioning features do not reach the extent of the dataset.