Schema Transformation

Schema Transformation is a Session Task that takes Read and Commit source classes and attributes, and creates new objects for the target classes and attributes. This means the Session contains both the source and the target objects at the same time.

Schema Transformation is designed for Schema changes that are more advanced than renaming. Renaming Schema can be done with standard Schema Mapping.

Note: Some conversions, such as using logic to analyse a value and split it across two attributes, will not be possible using Schema Transformation. For this, consider running Actions before or after the Schema Transformation task to operate on the data, using custom attributes added to the source or target Data Stores to temporarily hold these values, see Temporary Values.

Prerequisites

The Schema Transformation Task requires the following:

A Source Data Store to read the objects from. This must be open in the Session.
A Mapping Data Store containing the transformation definition. See Configuring a Mapping Data Store.
A Target Data Store to write objects to. This must be open in the Session.

Note: An Open Schema Task is commonly used to open the Schema for the Target Data Store, without loading the data.

Note: The class names from the Source Data Store must be different to the class names in the Target Data Store. If any class names are the same then change the name of the classes in Read and Commit and use the changed name in the contents of the Mapping Data Store.

Configuring a Mapping Data Store

A Mapping Data Store can be configured using any Data Store format readable by 1Integrate, but are normally stored in a CSV file or a database table.

Mapping Data Stores require a strict structure of up to three tables with specific attributes, which are listed below. These names can be set to the correct values in the Read and Commit mapping.

Selecting the Schema Transform Schema fixed schema type is designed to help this configuration process when defining Data Store.

ClassAttributeMapping

This is the main definition of how to create objects in target classes by copying feature attributes from the source classes.

Example

Attributes				Notes
SourceClass	SourceAttribute	TargetClass	TargetAttribute	-
ROAD	Classification	Network_Feature	Type
ROAD	Width	Network_Feature	Span
ROAD	Name			In this example the attribute is not copied.

CodeListMapping

Use this table, in addition to the Class Attribute Mapping, to convert specific attribute values in source codelists to attribute values in the target codelists. Codelist Mappings, are intended to be complete and can "replace" the definition in the Class Attribute mappings rather than override specific values.

Example

Attributes					Notes
SourceClass	SourceAttribute	SourceValue	TargetAttribute	TargetValue	-
	Responsible_County	CA	Responsible_County	Cambridgeshire	Any "Responsible_County" attribute value of "CA" will be converted to "Cambridgeshire", regardless of class. The source class name is missing which means that this codelist mapping can apply to any class with this attribute
ROAD	Status	Open	Active	True	Only for "ROAD", map Status="Open" to Active="True"
ROAD	Status	Closed	Active	False	Only for "ROAD", map Status="Closed" to Active="False"

DerivedAttributeMapping

In addition to the above, use Derived Attribute Mapping to set values on target classes optionally depending on the source. For example, to set a type attribute based on the source class or to set an attribute to a constant regardless of the source. The Derived Attribute Mapping values are for specific targeted overrides that layer on top of class attribute mapping, as opposed to a replacement for them.

Example

Attributes						Notes
SourceClass	SourceAttribute	SourceValue	TargetClass	TargetAttribute	TargetValue	-
ROAD	Lane_Count	2	Network_Feature	Type	A_Road
ROAD	Lane_Count	6	Network_Feature	Type	M_Road
			Network_Feature	Import_date	2021	Set the value on all objects created in the class regardless of origin
RIVER			Water_Feature	Type	River	If it is from a "RIVER" class then set the "Water_Feature" "Type" to "River".

Note: CodeListMapping and DerivedAttributeMapping must use Source/Target class pairings present in ClassAttributeMapping.

Mapping Data Store requirements

Note: Any named class must match the exact spelling and case of the classes in the Read and Commit Mappings of the respective Data Stores.

Class Name	Attribute Name	Type	Optional/Required	Description
ClassAttributeMapping (Mandatory)	SourceClass	String	Required
	SourceAttribute	String	Required
	TargetClass	String	Optional	May be null/empty to indicate value is being discarded.
	TargetAttribute	String	Optional	May be null/empty to indicate value is being discarded.
CodeListMapping (Optional)	SourceClass	String	Optional	May be null (match value from any class)
	SourceAttribute	String	Required
	SourceValue	Any	Required
	TargetAttribute	String	Required
	TargetValue	Any	Required
DerivedAttributeMapping (Optional)	SourceClass	String	Optional	May be null (with SourceAttribute and SourceValue also null) to indicate to just set the target value.
	SourceAttribute	String	Optional	May be null (with SourceClass not null) to indicate to just set the target value on objects created from SourceClass.
	SourceValue	Any	Optional	May be null (with SourceClass not null) to indicate to just set the target value on objects created from SourceClass.
	TargetClass	String	Required
	TargetAttribute	String	Required
	TargetValue	Any	Required

Type Casting

Source Type	Can Automatically convert to
Any type (not including geometry)	String
Geometry	Geometry
Datetime, Integer64, String	Datetime
Boolean, Integer, Integer64, Double, String	Boolean (0=false) or (case insensitive "true"=true)
Integer, Boolean (0/1) Double, Integer64, String	Integer
Integer64, Datetime, Boolean, Integer, Double, String	Integer64
Double, Boolean, Integer, Integer64, String	Double

Using Schema Transformation in a Session

Schema Transformation Tasks are carried out as part of a Session.

Open a Session and select an Open Data task for both the Source and Target for the transformation.

Note: You can also use the Open Schema task in place of Open Data to open the target Data Store. This is useful to ensure that no target objects are accidentally loaded in the Session.
Add a Transform Schema task to add it to your Session.
In the pop-up, select the three Data Stores you will be using for the transformation.

You will need to select a Source, Target and Mapping Data Store.

The New Transform Schema pop-up
(Optional) Add a Non-Spatial Filter to the Mapping Data Store.
(Optional) Check the Allow missing source classes option to allow the task to ignore missing source classes.
(Optional) In the Session, add Check Rules tasks for the transformed data. Typically you will also add a Copy-To or Commit task for the Target Data Store, in order to write the transformed data out.
Run the Session and your transformation will be applied.

Transform Schema in a Session

Non-Spatial Filter