Delimiter Separated Values (DSV)

The Delimiter Separated Values (DSV) data store is available for Reading and Writing data in 1Integrate.

A DSV file stores data as sets of values separated by a specified ‘separator’ character, with a new line character at the end of each line.

Each DSV file will be represented as a separate class, where attributes are defined by the DSV ‘columns’. Attributes are strings by default but can be overridden in the input mapping. Each line in the DSV will be read in as a separate object, this means that all the rows in the single DSV file must have the same number and order of values.

Read and Write Support

Read	Write		FME Required
Read	Commit	Copy To	FME Required

Data Support

Data	Support	Notes
Non-Spatial
2D
Measured
Heighted
3D		For more information about 3D data in 1Integrate, please refer to 3D Support.

Key Information

The DSV datastore preserves the order of class attributes from source data files. This allows 1Integrate to preserve the field order when exporting DSVs.
Geometry field(s) will always appear last in the list of class attributes. When reading data with geometries encoded as WKT, you may need to select Strip quotes from field.
Other than geometry fields, all field types in DSV files are detected as strings. You can correct this manually in the schema mapper.
1Integrate handles Multiple DSV files in one zip. When using this method the files will come through as separate classes.

Input and Output Parameters

Input Parameters

Parameter	Description
Source Files (.csv, .gz, .txt)	Drag and drop your file into the window, or click within the upload box and navigate to the file on your network. A .zip archive can be uploaded if more than one file is required.
Encoding	Controls which character encoding is used to interpret text attributes. If blank, the character encoding will be automatically detected.
Field names specified in file	If the field names (or column names) are specified in the file, this option ensures that the names will be extracted from the file. Otherwise, the columns of the DSV table are given default names (col0, col1, ... , colN).
Field separator	Define how the fields are separated in the file (a comma "," is used by default). Note: For tab delimiters, use `\t` as the field separator.
Number of header lines to skip	Enter the number of header lines to skip when reading the datastore. Note: This can be used if any header lines include information you do not want included as a column name.
Strip quotes from field	Some DSV files place quotation marks around all values they contain. If selected, quotation marks will be stripped from column values. Note: Some DSV files place quotation marks around all values they contain. If this option is selected, quotation marks will be stripped from column values. This is needed for comma separated data with geometries encoded as WKT.
Skip Duplicate delimeters	If selected, multiple contiguous delimiters are treated as a single delimiter.
Include Line Numbers	When enabled, a new attribute Line number will be included on each feature read that's automatically populated with the line number of said feature.
Coordinate Reference System	This parameter can be used to specify a Coordinate Reference System (CRS) code of the input data, if it cannot be deduced automatically. This will not re-project the data, but allows its current CRS to be defined (e.g to enable multiple data stores to open where one does not have a CRS defined). Note: Coordinate Reference System parameters are required in a session that will write data to Oracle. See Coordinate Reference System for more information on when this is required.
Spatial Data type	Select the data type present in your DSV file.
Spatial data fields	A comma separated list of the column names containing spatial data. The number of fields you will need to provide will be based on the Spatial Data Type you have selected.
Allow invalid geometries	Allows features with invalid geometries (e.g. duplicate vertices, self-intersections) to be imported. This is necessary if they need to be imported in order to be validated or corrected. If not ticked then features with invalid geometries are not imported and are logged as feature errors in the open data task.
Fix ring direction and inclusion errors	Polygon rings are assumed to be encoded using the right-hand rule, i.e. outer rings are anti-clockwise and inner rings are clockwise. If the service does not guarantee that encoding, then set this flag to have polygon rings automatically oriented on import to avoid later problems when processing the data. Note: This requires a small amount of additional processing so only set if the ring direction is not guaranteed.
For X,Y,Z,M data	If you have data that is recorded as X,Y,Z,M, you must opt to ignore either Z or M values by choosing to use X, Y and M or X, Y and Z only. Note: If the default is left on and X,Y,Z,M is uploaded then an exception will be thrown as 1Integrate only supports X,Y,M or X,Y,Z.

Output Parameters

Parameter	Description
Destination Files (.csv, .gz, .txt)	Results are available to download by clicking Download.
Output file extension (e.g. csv)	Set the file extension of the resulting file(s).
Encoding (e.g. UTF-8)	Controls which character encoding is used to interpret text attributes. If blank, the character encoding will be automatically detected.
Field names specified in file	If the field names (or column names) are specified in the file, this option ensures that the names will be extracted from the file. Otherwise, the columns of the CSV table are given default names (col0, col1, ... , colN).
Field separator	Define how the fields are separated in the file (a comma "," is used by default). Note: For tab delimiters, use `\t` as the field separator.
Quote fields	If selected, every field (including field or column names) will be quoted (by placing quotation marks around the values).
Coordinate Reference System	Required if the data store will be used in a session that will write data to Oracle. This can be used to specify the CRS of the output data if it cannot be deduced from the input data. This will not re-project the data, but allows it’s current CRS to be defined. See Coordinate Reference System for more information on when this is required.
Spatial data type	Select the data type present in your DSV file.
Spatial data fields	A comma separated list of the column names containing spatial data. The number of fields you will need to provide will be based on the Spatial Data Type you have selected.
Force geometry to be written with X, Y, Z, M	Select to output a fourth z or m ordinate for objects. Only takes effect if the geometry to be written has 3 ordinates.
For X, Y, Z, M data: Use zero rather than NULL or NaN when writing ignored ordinate	Select to write ignored values as zero in place of null/NaN (NaN is used for the PostGIS data store).