Konect Dataset Import Tool The KD Import Tool is used to automate the import and update of datasets from GIS files. Commonly you will use this tool to refresh Konect datasets with updated corporate data. An example of this may be where you have processes that keep the corporate property data refreshed daily and the daily refresh needs to be pushed to the dataset in Konect so that the most up to date information is available for field users.

WARNING when replacing existing datasets: KD Import will delete and replace any existing data in the Konect Dataset when the ‘replace_force’ action is specified. If existing records are linked to records in a child dataset, the links to the child records will be lost if the parent records are deleted. Please take care when deleting data with KD import. KD Import is a console or command-line program that is run from the DOS prompt. The Windows Task Scheduler can be used to automate the execution of such command-line programs. This means that KD Import can be run automatically at an appropriate time without requiring user intervention.

Prerequisites The KD Import Tool is a 64 bit .NET application. It has the following prerequisites

• 64 bit Windows operating system

• .NET framework 4.8 - https://dotnet.microsoft.com/download/dotnet-framework/net48

• The latest 64 bit C++ redistributable - https://support.microsoft.com/en-au/help/2977003/the-latest-supported-visual-c-downloads

Please download and install the above software packages.

Installation The KD Import Tool is a 64 bit .NET application. It has the following prerequisites

• 64 bit Windows operating system

• .NET framework 4.8 - https://dotnet.microsoft.com/download/dotnet-framework/net48

• The latest 64 bit C++ redistributable - https://support.microsoft.com/en-au/help/2977003/the-latest-supported-visual-c-downloads

Please download and install the above prerequisites. Copy the file ‘KonectImportTool.zip’ to a suitable location (eg create a folder called Konect Import Tool) with read/write access and extract the zip file. A folder structure as shown below will be created. DO NOT CHANGE THE FOLDER STRUCTURE. There are two files located in the Konect Dataset Importer folder that you will need to interact with:

kdimport.exe you will need to browse to this file when scheduling your exports in the Windows Task Scheduler. If running kdimport.exe from the DOS command line, you will need to type in the full path to this file or change your current directory to the folder where kdimport.exe is located.

kdimport.exe.config this file will need to be edited to set the location of the log file

Setting the Log Folder Open the file ‘kdimport.exe.config’ with a text editor such as Notepad. There is a setting called LogFolder

where the path of the folder containing the log file folder is set.

<add key="LogFolder" value="C:\TEMP\KONECT IMPORT LOG" />

Edit the default ‘value’ for this setting to the folder where the log file is to be written. Make sure that the

full folder path exists and that appropriate write privileges have been applied.

Note that each time kdimport is run, a folder is created in the log folder with a name that reflects the date/time that the program is run. This new folder will hold the log files for that run. For example, if the kdimport is run on 14-Aug-2017 at 3:03:38 PM then a folder with the name ‘2017_08_14_150338’ is created in the log folder. Two log files are generated for each run, one with general information called ‘kdimport.log’ and the other containing the import result for every record in the import file. This second file has a name of the form ‘IMPORTLOG_DatasetName.<import file type>’ and is the same type as the import file. For example, the import file is a TAB file and the destination dataset is DCDB, then the name of the log file is ‘IMPORTLOG_DCDB.TAB’.

Running Konect Dataset Importer KD Importer can be run in two ways:

1. From a Command Prompt – it is likely that this will be used to test KD Import.

2. From the Windows Task Scheduler – used to automatically run KD Import on an ongoing basis.

Command Line Structure The structure shown below is common to both methods of running KD Import. However, there are slight variations as to how the executable is referenced for each method. These variations will be shown in the more detailed notes that follow. Apart from the reference to ‘kdimport.exe’, the structure shown immediately below is common. -pin=account_PIN –action=action_name -dataset=dataset_name -file=import_filename

A detailed explanation of the command line switches is provided in the following section.

Command Line Switches

Switch Possible values Required Definition

-pin Your PIN Yes The account pin

-action new replace replace-force append append_using_uif append_update update_matching update_nominated

Yes Create a new dataset. Replace the records in an existing dataset. This will be the most commonly used value for the action switch. Replace the records in an existing dataset even if the dataset has a child dataset. Append records to an existing dataset. Append records to an existing dataset where a specified unique id field does not match. Appends records to an existing dataset where a specified unique field value DOES NOT MATCH and updates records in an existing dataset where a specified unique field value MATCHES. The dataset fields that are updated are those that match the fields in the import file. Updates records in an existing dataset where a specified unique field value MATCHES. The dataset fields that are updated are those that match the fields in the import file. Update nominated fields in records in an existing dataset. The –fields switch must be used in conjunction with this option.

-fields The fields to update

Only if the action is update_nominated

The list of fields that are to be updated in every record in an existing dataset. The field names are delimited by commas.

-uif The unique id field Only if the action is append_using_uif. Optional if the action is append_update, update_matching or update_nominated

The name of the unique id field. If not set then the default value is the konect system Id field.

-longlat The longitude and latitude field names

Required when importing a CSV file and the import action creates records. Optional when the import action updates records.

The two field names are separated by a comma.

-dataset KM Dataset Name Yes The name of the dataset.

-file Import filename Yes Specifies the import filename. The switch value must be the full pathname to the import

file. Files can be of type MapInfo TAB, CSV, ESRI Shape or GeoJSON.

-parentlink KM Parent Dataset Name

No The parent dataset name to link import records to. The input file must contain the field _parent_id. The switch is only available for actions: append, append_using_ufid, append_update, update_matching and update_nominated.

-user Konect_username No Specifies your konect username if authentication has been turned on for this account

-pwd Konect_password No Specified your konect password if authentication has been turned on for this account

-proxyhost proxy.domain.local No Specifies the proxy hostname if your organisation uses a proxy server

-proxyport 8000 No Specifies the proxy port if your organisation uses a proxy server

-proxyuser Proxy_username No Specifies the proxy username if your organisation uses a proxy server with authentication

-proxypwd Proxy_password No Specifies the proxy password if your organisation uses a proxy server with authentication

Note 1: If any of the switch values contain a space then the value must be enclosed in double quotes. For example, if the dataset name is ‘My dataset’ and the import file is ‘C:\TEMP\My import.TAB’ then the command line switches must be: -pin=pin_number -action=replace -dataset="My Dataset" -file="C:\temp\My import.TAB" Note 2: It is recommended that the command line be created/edited in a text editor (such as Notepad) and not an application such as Microsoft Word that uses its own set of characters. For example, in MS Word the double quotes character, “, is not recognised on the DOS command line. However, the double quotes character used in Notepad, ", will be recognised on the DOS command line.

Note 3:

If the import file contains no data the command line will error and fail.

Command Line Examples Shown below are some examples of how to use kdimport.

1. To create a new dataset called ‘TestDataset’ from the TAB file ‘C:\TEMP\TestData.TAB’: kdimport –pin=12345 –action=new –dataset=TestDataset –file=C:\TEMP\TestData.TAB

2. To create a new dataset called ‘TestDatasetCsv’ from the CSV file ‘C:\TEMP\TestData.CSV’ where the longitude and latitude of point objects are held in the columns ‘X’ and ‘Y’ (it is assumed that the X and Y values are in the WGS 84 coordinate system):

kdimport –pin=12345 –action=new –dataset=TestDatasetCsv –longlat=’X,Y’–

file=C:\TEMP\TestData.CSV

3. To replace all the records in an existing dataset called ‘TestDataset’ with data from the SHP file ‘C:\TEMP\NewTestData.SHP’: kdimport –pin=12345 –action=replace –dataset=TestDataset –

file=C:\TEMP\NewTestData.SHP

4. To replace all the records in an existing dataset called ‘TestDataset’ that has a child dataset with data from the SHP file ‘C:\TEMP\NewTestData.SHP’ (note that links to the child dataset will be lost): kdimport –pin=12345 –action=replace_force –dataset=TestDataset –

file=C:\TEMP\NewTestData.SHP

5. To append all the records in a TAB file called ‘C:\TEMP\ExtraTestData.TAB’ to an existing dataset called ‘TestDataset’: kdimport –pin=12345 –action=append –dataset=TestDataset –

file=C:\TEMP\ExtraTestData.TAB

6. Assume that an existing dataset called ‘TestDataset’ has a field called ‘MyId’ that has unique values. Assume that there is a TAB file called ‘TestDataWithMyId.TAB’ that has a field called ‘MyId’ and that we want to append every record in the TAB file to the dataset where the value of ‘MyId’ in the TAB file record is not already in the dataset: kdimport –pin=12345 –action=append_using_uif –uif=MyId –dataset=TestDataset –

file=C:\TEMP\TestDataMyId.TAB

7. Assume that an existing dataset called ‘TestDataset’ has a field called ‘MyId’ that has unique values. Assume that there is a TAB file called ‘TestDataWithMyId.TAB’ that has a field called ‘MyId’ and that we want to append every record in the TAB file to the dataset where the value of ‘MyId’ in the TAB file record is not already in the dataset and in addition we want to update every record in the dataset whose value of ‘MyId’ is in the TAB file. (Updating the record in the dataset means updating the fields that occur both in the dataset and the TAB file, that is, updating the matching fields.): kdimport –pin=12345 –action=append_update –uif=MyId –dataset=TestDataset –

file=C:\TEMP\TestDataMyId.TAB

8. Assume that an existing dataset called ‘TestDataset’ has a field called ‘MyId’ that has unique values. Assume that there is a TAB file called ‘TestDataWithMyId.TAB’ that has a field called ‘MyId’ and that we want to update every record in the dataset whose value of ‘MyId’ is in the TAB file. We don’t want to do any appending. (Updating the record in the dataset means updating the fields that occur both in the dataset and the TAB file, that is, updating the matching fields.): kdimport –pin=12345 –action=append_matching –uif=MyId –dataset=TestDataset –

file=C:\TEMP\TestDataMyId.TAB

9. Assume that there is existing dataset called ‘TestDataset’ that has been exported to a CSV file called ‘TestDataExport.CSV’. The CSV file will have the value of the konect Id for every record in the column ‘_id’. Assume that various records have been updated in the CSV file and that we want to update the corresponding records in the dataset. (Updating the record in the dataset means updating the fields that occur both in the dataset and the TAB file, that is, update the matching fields.): kdimport –pin=12345 –action=update_matching –dataset=TestDataset –

file=C:\TEMP\TestDataExport.CSV

10. Assume that there is existing dataset called ‘TestDataset’ that has at least two fields called ‘Field_1’ and ‘Field_2’ and that the dataset has been exported to a CSV file called ‘TestDataExport.CSV’. The CSV file will have the value of the konect Id for every record in the column ‘_id’. Assume that

various records have been updated in the CSV file and that we want to update only the fields ‘Field_1’ and ‘Field_2’ in the corresponding records in the dataset: kdimport –pin=12345 –action=update_nominated –fields=Field_1,Field_2 –

dataset=TestDataset –file=C:\TEMP\TestDataExport.CSV

Running from a DOS Prompt – Command Line

1. Open a Command Prompt window (enter ‘cmd’ in the Start Search box – Win 7/8 or Cortana – Win 10

and select Command Prompt from the presented list)

2. Change folders using the “cd” command so that the DOS prompt shows the path to the file

‘kdimport.exe’.

For example, in my case this is: C:\Konect Import Tool\Konect Dataset Importer

3. Either type the required command line directly at the DOS prompt or create the command line in a

text editor and then copy and paste it into the Command Prompt Window. In the example below, the

appropriate command line to run kdimport.exe has been created in Notepad++. The text is then

copied and pasted into the Command Prompt window before pressing the Enter button to run

kdimport.exe.

4. Press Enter to run the command line

Creating a Scheduled Task in Windows 1. Open Windows ‘Task Scheduler’ (type ‘task scheduler’ in the Start Search box – Win 7/8 or Cortana –

Win 10 and select Task Scheduler from the list.)

2. Click on the ‘Create Task…’ in the Action Panel and the ‘Create Task’ window will appear.

1. Fill in the Name

2. Fill in the Description if needed.

3. Edit the User or Group if required.

4. Select the ‘Triggers’ tab.

5. Select New to open the ‘New Trigger’ definition dialog

6. Set the time and frequency that the task is to run.

7. OK

8. Go to the ‘Actions’ tab

9. Select ‘New’

10. Set the Action to ‘Start a program’.

11. Browse to the location of ‘kdimport.exe’ and then select ‘kdimport.exe’.

12. Add all the command line arguments in the ‘Add arguments’ text box.

For example

-pin=123123 -action=replace -dataset="Parcel" -file=C:\Temp\Property.TAB

13. Press ‘OK’ to return to the Actions Tab

14. Press ‘OK’. You will return to the main Task Scheduler Window

15. Click on ‘Task Scheduler Library’

16. The new task will appear in the list of tasks in the Task Scheduler