Exporting data with the Konect CLI
- Michael Conlan
The Konect CLI can export the data and attachments from datasets or queries to a number of different GIS formats including:
Esri shape file
MapInfo TAB file
GeoPackage
As well as the text format:
Geojson
KML
CSV
When exporting to CSV all features are exported, but only point geometries will be included in the export using the columns X and Y. The X and Y columns will remain empty for all other geometry types.
Geometry data is exported to WGS 84 coordinate system.
Exports to the multi table formats KML or GeoPackage, will only export to a single table creating a new or overriding an existing file.
Exports that contain no records will fail.
Attachment fields
Attachment fields in Konect can contain more than 1 attachment, so when exported are normalised into their own export files per field. The attachment field files are exported in the same format as the dataset and into the same folder, named the same as the specified dataset file but concatenated with the attachment field’s export name.
The attachment field data contains the feature id, attachment id, attachment system fields and the name of the exported attachment file. Since the export is often a GIS format the feature geometry is also included as the geometry for each attachment record.
The actual attachments are downloaded into subfolders (created if they do not exist) in the same folder as the specified file, called photos and documents. If the attachment already exists in the folder the attachment is not downloaded again.
Photo attachments are encoded with the Konect metadata footer and any annotation rendered on top. The footer and annotation can be excluded during export. Any EXIF metadata found on the original photo is re-encoded into annotated photos.
Photos that are being exported to the same folder over multiple exports will not be overridden if the annotation changes between exports.
Attachment logs
An attachment log file for each attachment field that is exported will also be generated giving the details of how each attachment was dealt with during export. This file is generated in the specified export folder as a CSV file.
Transaction history
The Konect CLI can also export the entire transaction history for a dataset. Queries can be used, but unless a record matched the query through out it’s entire transaction history, it will be missing transactions. For this reason we do not recommending performing transaction history exports on queries. Transaction history exports do not include attachments, or attachment transaction history.
Konect CLI export commands
The Konect CLI has the following data export commands
csvexport
geojsonexport
gpkgexport
kmlexport
tabexport
shpexport
For example a CSV export would use the following syntax
$ konect csvexport -k [API_KEY] -e [ENVIRONMENT_NAME] -d [DATASET_NAME] -f [FILE_NAME] [other parameters and flags]
Konect CLI export parameters and flags
The following are all the parameters and flags available to the Konect CLI data exports.
Parameters and flags | Example value | Required | Definition |
|---|---|---|---|
-k, --key | XXXXXXXXXXXXX | Yes | A valid API key for the organisation with permissions on the specified environment |
-e, --env | "Environment name" | Yes | The environment to export data from |
-d, --dataset | "Dataset name" | Yes | The source dataset name or query name to export data from |
-f, --file | C:\export\file.ext | Yes | The destination file to export data to |
--fields | field1 field2 | No | The list of fields to export. If this parameter is omitted all fields are exported. The list of fields must use the field’s export names. Field names are separated by spaces. System fields are still exported by default, please see --excludesysfields. |
--listvalue | both | No | Specify how to export list field data from the dataset. Possible values are "unique", "display" or "both". unique - Export just the unique list values in the fields export name display - Export just the lists aliases to a field named field export name concatenated with ‘_Display_Value’ both - Export both the unique and display values to the specified field names. Defaults to unique if parameter is omitted. |
--excludesysfields |
| No | If this flag is on, do not export the Konect system fields: _id, _created, _created_by, _modified, _modified_by, _sync_date, _parent_id. |
--photos |
| No | If this flag is on export photo attachments. See attachments fields for more details. |
--excludephotofooter |
| No | If this flag is on do not render the Konect footer or photo annotation. |
--documents |
| No | If this flag is on export document attachments. See attachments fields for more details. |
--history |
| No | If this flag is on export the all transaction history for data in the specified dataset. Queries should not be used for transaction history. When exporting transaction history attachment fields will not be exported. |
--excludelogfile |
| No | If this flag is on do not write a log file. |
--logfilename | log.ext | No | The name of the log file. By default the log file is written to the same folder as the export. Use the --logfolder setting to change the folder where the log file is written. |
--logfolder | C:\export | No | The folder to write the log file. If --logfilename is not specified the default log file name is used which is based on the dataset name IMPORTLOG_[dataset name].ext. |
--proxyhost | hostname | No | The proxy host for any proxy settings |
--proxyport | 8080 | No | The proxy port for any proxy settings |
--proxyusername | john | No | The proxy username for any proxy settings |
--proxypassword | johnspassword | No | The proxy password for any proxy settings |