Importing data with the Konect CLI

The Konect CLI can import data into new or existing datasets from a number of different GIS formats including:

• Esri shape file

• MapInfo TAB file

• GeoPackage

As well as the text format:

• Geojson

• KML

• CSV

When importing from CSV only point geometries can be imported by specified longitude and latitude columns.

Geometry data for text formats will not be translated so need to be in WGS 84 coordinate system.

The Konect CLI will not import multi-geometry geometry types, and will skip records that are missing geometries.

Konect CLI import commands

The Konect CLI has the following data import commands

• csvimport

• geojsonimport

• gpkgimport

• kmlimport

• tabimport

• shpimport

For example a geojson import would use the following syntax

$ konect geojsonimport -k [API_KEY] -e [ENVIRONMENT_NAME] -d [DATASET_NAME] -f [FILE_NAME] [other parameters and flags]

Konect CLI import actions

The Konect CLI will process data for imports differently such as creating a new dataset, appending new records, updating existing records or merging specified fields into existing records. The action parameter specifies how the Konect CLI should process the data. The action parameter has the following possible values

• new

• replace

• append

• append_update

• update_matching

• update_nominated

New

The new action will create a new dataset, using the fields and data types found in the input data source, before importing all records. Textual formats are not strongly typed so it may be difficult for the importer to correctly guess field data types.

Replace

The replace actions will delete all records from the specified dataset before importing all records. This includes all attachments and any links to parent or child records.

Append

The append action will append all records to the specified dataset. If the input data source contains existing konect ids or system fields, these are will be ignored and the records appended without them.

When using the append action you can specify a unique id field using the --uif parameter. This is a special append action that will check to see if the record already exists in the dataset by using the specified unique id field. If the record doesn’t exist it is appended, if it does exist the record is skipped

Update

The update_matching and update_nominated actions will only update records in the specified dataset if a match can be found in the dataset. Matches can be done using the Konect Id by default of by specifying a unique id field with the --uif parameter.

When using the --uif parameter if the dataset contains more than 1 record with a given id, all records with the id will be updated. If the input data source contains more than 1 record with the same id, only the last record in the input data source will be kept.

When updating records only the fields that exist in the input data source will be updated in the dataset records, all other fields will be left alone. When using the update_nominated action you must specify what fields to update, any other fields that exist in the input data source will be ignored.

CSV files can be imported for update without records requiring geometries.

Append and update

The append_update action works the same as the append and update_matching actions, any records that can’t be matched will be appended. You can use the --uif parameter.

Multi table formats

The Konect CLI can import data from the multi table formats KML and GeoPackage when they contain more than 1 table. An additional parameter --layername specifying the table will need to be included.

The Konect CLI will display a list of available layer names that can be used if the specified import file is found to contain multiple tables.

Parent linking

The Konect CLI will allow imports to update parent links in all actions except new. The input data source is required to contain the system field _parent_id. This field will contain the Konect Id of the parent record to link to.

Import logs

The Konect CLI will generate a detailed log file after an import describing the action taken for every record in the import. The log file is a copy of the original import file in the same format but with the added fields

• _match_id - The value of the unique field id or konect id that matched this record on the server

• _konect_id - The konect id given to this record if it was new

• _result - The import status of this record such as inserted, updated, identical, skipped or error

• _message - A more detailed description of what action was taken

• _match_link_id - If linking to a parent this is the value used to match the parent konect id

• _link_result - If linking to a parent this is the linking status such as linked, skipped or error

• _link_message - If linking to a parent this a more detailed description of what linking action was taken

Konect CLI import parameters and flags

The following are all the parameters and flags available to the Konect CLI data exports.