Schema Transformation

Schema Transformation is a Session Task that reads input data classes and creates new objects in the output classes. This is achieved by copying the objects from the input classes, so the session will contain both the source and the target objects at the same time.

Schema transformation is designed for more advanced changes of schema, other than simple renaming. For more information on renaming schema, refer to Schema Mapping.

Note: Some conversions, such as using logic to analyse a value and split it across two attributes, will not be possible using Schema Transformation. For this, consider running Actions before or after the Schema Transformation task to operate on the data, using custom attributes added to the source or target data stores to temporarily hold these values.

Prerequisites

The Schema Transformation task requires the following:

A Source data store to read the objects from. This must be open in the session.
A Mapping data store containing the transformation definition. See Configuring a Mapping data store below for more.
A Target data store to write objects to. This must be open in the session.

Note: The class names from the Source data store must be different to the class names in the Target data store. If any class names are the same then change the name of the classes in the Input Mappings and use the changed name in the contents of the Mapping Data Store.

Configuring a Mapping data store

A Mapping data store can be configured using any data store format readable by 1Integrate, but are normally stored in a CSV file or a database table.

A mapping data store requires a strict structure of up to three tables with specific attributes which are listed below. These names can be set to the correct values in the input mapping.

ClassAttributeMapping

This is the main definition of how to create objects in target classes by copying feature attributes from the source classes.
CodeListMapping

Use this table, in addition to the Class Attribute Mapping, to convert specific attribute values in source codelists to attribute values in the target codelists. Codelist Mappings, are intended to be complete and can ‘replace’ the definition in the Class Attribute mappings rather than override specific values.
DerivedAttributeMapping

In addition to the above, use Derived Attribute Mapping to set values on target classes optionally depending on the source. For example, to set a type attribute based on the source class or to set an attribute to a constant regardless of the source. The Derived Attribute Mapping values are for specific targeted overrides that layer on top of class attribute mapping, as opposed to a replacement for them.

Note: CodeListMapping and DerivedAttributeMapping must use Source/Target class pairings present in ClassAttributeMapping

Mapping data store requirements

Note: Any named class must match the exact spelling and case of the classes in the Input Mappings of the respective data stores.

Class Name	Attribute Name	Type	Optional/Required	Description
ClassAttribute Mapping (Mandatory)	SourceClass	String	Required
	SourceAttribute	String	Required
	TargetClass	String	Optional	May be null/empty to indicate value is being discarded.
	TargetAttribute	String	Optional	May be null/empty to indicate value is being discarded.
CodeList Mapping (Optional)	SourceClass	String	Optional	May be null (match value from any class)
	SourceAttribute	String	Required
	SourceValue	Any	Required
	TargetAttribute	String	Required
	TargetValue	Any	Required
DerivedAttribute Mapping (Optional)	SourceClass	String	Optional	May be null (with SourceAttribute and SourceValue also null) to indicate to just set the target value.
	SourceAttribute	String	Optional	May be null (with SourceClass not null) to indicate to just set the target value on objects created from SourceClass.
	SourceValue	Any	Optional	May be null (with SourceClass not null) to indicate to just set the target value on objects created from SourceClass.
	TargetClass	String	Required
	TargetAttribute	String	Required
	TargetValue	Any	Required

Type Casting

Source Type	Can Automatically convert to
Any type (not including geometry)	String
Geometry	Geometry
Datetime, Integer64, String	Datetime
Boolean, Integer, Integer64, Double, String	Boolean (0=false) or (case insensitive "true"=true)
Integer, Boolean (0/1) Double, Integer64, String	Integer
Integer64, Datetime, Boolean, Integer, Double, String	Integer64
Double, Boolean, Integer, Integer64, String	Double

Mapping Examples

ClassAttributeMapping

Attributes				Notes
SourceClass	SourceAttribute	TargetClass	TargetAttribute	-
ROAD	Classification	Network_Feature	Type
ROAD	Width	Network_Feature	Span
ROAD	Name			In this example the attribute is not copied.

CodeListMapping

Attributes					Notes
SourceClass	SourceAttribute	SourceValue	TargetAttribute	TargetValue	-
	Responsible_County	CA	Responsible_County	Cambridgeshire	Any "Responsible_County" attribute value of "CA" will be converted to "Cambridgeshire", regardless of class. The source class name is missing which means that this codelist mapping can apply to any class with this attribute
ROAD	Status	Open	Active	True	Only for "ROAD", map Status="Open" to Active="True"
ROAD	Status	Closed	Active	False	Only for "ROAD", map Status="Closed" to Active="False"

DerivedAttributeMapping

Attributes						Notes
SourceClass	SourceAttribute	SourceValue	TargetClass	TargetAttribute	TargetValue	-
ROAD	Lane_Count	2	Network_Feature	Type	A_Road
ROAD	Lane_Count	6	Network_Feature	Type	M_Road
			Network_Feature	Import_date	2021	Set the value on all objects created in the class regardless of origin
RIVER			Water_Feature	Type	River	If it is from a "RIVER" class then set the "Water_Feature" "Type" to "River".

Using Schema Transformation in a Session

Schema transformation tasks are carried out as part of session.

Open a Session and select an Open Data Store task for both the Source and Target for the transformation.

Note: You can also use the Open Schema task in place of Open Data to open the target data store. This is useful to ensure that no target objects are accidentally loaded in the session.
Add a Schema Transformation task to add it to your session.
In the pop-up, select the three Data stores you will be using for the transformation.

You will need to select a Source, Target and Mapping Data Store, as in the example below:

The Select Data Stores... with data store examples selected
(Optional) In the session, add check rules tasks for the transformed data. Typically you will also add a Copy-To or Commit task for the Target Data Store, in order to write the transformed data out.
Run the session and your transformation will be applied.

Transform Schema in a session

Using non-spatial filtering in a Transform Schema task

You can utilise non-spatial filtering when using a schema transformation task, this allows you to filter any of the mappings based on parameters you define.

For more information on defining a non-spatial filter, please refer to Non-Spatial Filtering.

Example:

You have been supplied a data store containing the mapping definitions for a number of different data sets.

If there is a column on the mapping table defining which dataset that mapping applies to, then you can pass in the dataset name as a session parameter and have that control a non-spatial filter on the data store, which reads in only the rows (i.e. transformations) relevant for that data owner.