Download pdf - QA4LiDAR version 1.4.5 User Manual · 4.12.5 Density / Resolution ... Spatial file corruption (las, shp, asc, tiff, tif, ecw, ... Total point count of all supplied LAS types matches

QA4LiDAR version 1.4.5 User Manual

Page 1 of 78

COOPERATIVE RESEARCH CENTRE for SPATIAL INFORMATION – 2

Project 2.09/2.15


Airborne Laser Scanning Compliance and Quality Assurance Tool: Development of a Standard Software Procedure and

Tool to Quality Assure Elevation Data

Software Developers: Think Spatial & CRCSI

2013-2016


Page 2 of 78

Revision History

QA4LiDAR Version

Date Reviewer Description of changes

1.0.18 17/09/2013 Jessica Keysers First Draft

1.0.29 29/11/2013 Jessica Keysers Document updates

1.1.6 29/04/2014 Jessica Keysers Software improvements

1.3.19 06/02/2015 Jessica Keysers Phase 3 development Beta release

1.3.26 15/06/2015 Jessica Keysers Use case updates

1.3.29 02/07/2015 Jessica Keysers Bug fixes and new LAS corruption check components

1.4.0 7/01/2016 Jessica Keysers Significant processing & efficiency improvements, Pacific Islands software update, upgrade to ArcGIS 10.3 (and hence LAS 1.4) and drop support for ArcGIS 10.1, new bathymetry coverage check

1.4.2 24/03/2016 Jessica Keysers New license model, upgrade to ArcGIS 10.4

1.4.5 14/09/2016 Jessica Keysers Bug fixes and minor improvements including the QA Report saving edits unless the check is re-run. Addition of new tile origin check.


Page 3 of 78

Table of Contents

List of Tables ........................................................................................................................................... 5

1. Introduction .................................................................................................................................... 6

1.1 Software Purpose .................................................................................................................... 6

1.2 What QA4LiDAR Does and Does Not Do ................................................................................. 6

1.3 Licensing .................................................................................................................................. 9

1.4 Requirements & Recommendations for using QA4LiDAR .................................................... 10

2. Installation Procedure ................................................................................................................... 12

2.1 System Requirements ........................................................................................................... 12

2.2 Installation ............................................................................................................................ 12

3. Using the QA4LiDAR Form Editor .................................................................................................. 13

3.1 Purpose and Importance....................................................................................................... 13

3.2 Forms in the Acquisition Process .......................................................................................... 14

3.3 Create a New Form ............................................................................................................... 14

3.4 Open an Existing Form .......................................................................................................... 14

3.5 Filling in the forms ................................................................................................................ 15

3.5.1 Fill in a Tender (*.ctf) Form ........................................................................................... 15

3.5.2 Fill in a Report (*.crf) Form ........................................................................................... 17

3.6 Save and Print a Form ........................................................................................................... 20

4 Using the QA4LiDAR Software ...................................................................................................... 21

4.1 Supported Data Formats ....................................................................................................... 21

4.2 Project Data Version Control ................................................................................................ 22

4.3 Dealing with Very Large Projects .......................................................................................... 23

4.4 Open a QA4LiDAR Project ..................................................................................................... 24

4.5 File & Help Menus ................................................................................................................. 25

4.6 SETUP: Dashboard ................................................................................................................. 25

4.7 SETUP: User Settings ............................................................................................................. 26

4.8 SETUP: Project Settings ......................................................................................................... 27

4.8.1 Inputs ............................................................................................................................ 27

4.8.2 Processing ..................................................................................................................... 27

4.9 CHECKS: Automated Checks ................................................................................................. 28

4.9.1 Survey Control ............................................................................................................... 29

4.9.2 Processing ..................................................................................................................... 30

4.9.3 Running Automated Checks .......................................................................................... 31


Page 4 of 78

4.10 CHECKS: Extent Check ........................................................................................................... 34

4.11 CHECKS: Visual DEM Check ................................................................................................... 36

4.11.1 Sampling ........................................................................................................................ 36

4.11.2 Multiple Users ............................................................................................................... 37

4.11.3 Check DEM Tiles ............................................................................................................ 38

4.11.4 Shortcut Keys ................................................................................................................ 42

4.12 OUTPUT: Report .................................................................................................................... 43

4.12.1 Presence & Reading ...................................................................................................... 44

4.12.2 Forms Report ................................................................................................................ 45

4.12.3 Classification Statistics .................................................................................................. 46

4.12.4 Survey Control ............................................................................................................... 47

4.12.5 Density / Resolution ...................................................................................................... 48

4.12.6 Flight Lines .................................................................................................................... 49

4.12.7 Vertical .......................................................................................................................... 49

4.12.8 Visual Checks (DEM) ...................................................................................................... 50

4.12.9 Output Supporting Information .................................................................................... 51

4.12.10 Printing the QA Report .............................................................................................. 53

4.13 OUTPUT: Map ....................................................................................................................... 54

5 Common Tasks .............................................................................................................................. 55

5.1 Approving the survey control design .................................................................................... 55

6 Troubleshooting ............................................................................................................................ 56

7 Future Improvements ................................................................................................................... 57

8 Glossary ......................................................................................................................................... 57

9 Appendices .................................................................................................................................... 61

9.1 Appendix 1 – How the Automated Checks are performed ................................................... 61

9.1.1 Project scan in ............................................................................................................... 61

9.1.2 Delivery Completeness & Spatial File Corruption ......................................................... 64

9.1.3 File Naming, Shapefile Attributes & Horizontal Coordinate System ............................ 65

9.1.4 Comparison of Report Form to Tender Form ................................................................ 66

9.1.5 Classification Statistics .................................................................................................. 66

9.1.6 Accuracy of Survey Control ........................................................................................... 67

9.1.7 Point Density & DEM Resolution................................................................................... 73

9.1.8 Flight Line Coverage ...................................................................................................... 75

9.1.9 Absolute & Relative Vertical Accuracy .......................................................................... 75


Page 5 of 78

9.2 Appendix 2 – How the Visual Checks are performed ........................................................... 78

9.2.1 Extent Checks ................................................................................................................ 78

9.2.2 Visual DEM Checks ........................................................................................................ 78

List of Tables

Table 1. ArcGIS 10.3 for Desktop hardware requirements ................................................................... 12

Table 2. Data formats supported by QA4LiDAR. ................................................................................... 21

Table 3. QA4LiDAR Visual Check shortcut keys..................................................................................... 42

Table 4. Survey control output shapefile fields added ......................................................................... 50

Table 5. Tile Index output shapefile fields added ................................................................................. 52

Table 6. Key term search rules (not case sensitive) .............................................................................. 61

Table 7. Tile name in file name search rules ......................................................................................... 63

Table 8. Openness rating scheme ......................................................................................................... 68

Table 9. Flatness rating scheme ............................................................................................................ 68

Table 10. Control density rating scheme .............................................................................................. 69

Table 11. Control weighted distribution rating scheme ....................................................................... 72

Table 12. Overall survey control rating scheme ................................................................................... 72

Table 13. Point density type definitions ............................................................................................... 73


Page 6 of 78

1. Introduction

1.1 Software Purpose This compliance and Quality Assurance tool for airborne LiDAR - QA4LiDAR - is an easy to use,

independent, standard compliance and quality assurance (QA) checking mechanism for airborne

LiDAR elevation data. By bringing together a set of common checks into one software package and

largely automating their implementation, QA4LiDAR simplifies the QA process and increases

efficiencies for both the contracting agency and LiDAR provider. Contracting agencies can use

QA4LiDAR to perform standard independent compliance and QA testing on their airborne LiDAR

data, to ensure providers are delivering data that meets the specification before the project is signed

off and completed. LiDAR providers can also use this tool and supply the standard report produced

by the tool to the contracting agency as part of the project delivery (in addition to their standard

processing and QA procedures).

QA4LiDAR is the only package that is focused purely on compliance and QA testing and provides a

standard set of checks and an associated compliance report for a suite of airborne LiDAR products.

The checks provided are the fundamental checks required, as opposed to a fully comprehensive set

of checks. The tool is based on the Australian Intergovernmental Committee on Surveying and

Mapping (ICSM) LiDAR Acquisition Specifications and Tender Template version 1.0 November 2010

(ICSM Template) and it’s equivalent for the Pacific, hence is specific to Australian, and Pacific

airborne LiDAR data collection. If a user does not have access to QA4LiDAR but wishes to perform

LiDAR validation, they can refer to Chapter 15 (Airborne LiDAR Acquisition and Validation) of the

TERN AusCover Good Practice Guidelines

http://data.auscover.org.au/xwiki/bin/view/Teams/GoodPracticeHanbook.

1.2 What QA4LiDAR Does and Does Not Do QA4LiDAR DOES enable checks for the following aspects of a LiDAR delivery;

Delivery Completeness & Spatial File Corruption

Spatial file corruption (las, shp, asc, tiff, tif, ecw, ESRI Grid)

Delivery completeness of elements requested as per tender form

Tile index coordinate origin

Delivery completeness for tiles as per tile index (Orthometric las, Ellipsoid las)

Delivery completeness for swath LAS as per flight line shapefile

Presence of WDP for full wave form LAS

LAS files are requested version and point data record format

File Naming, Shapefile Attributes & Horizontal Coordinate System

File naming for Australia and Pacific for file formats (las, asc, tiff, tif, ecw, ESRI Grid)

Shapefile attributes

Horizontal coordinate system of spatial files

Delivery completeness for tiles as per tile index (DEM, DSM, aerials, intensity)

Delivery Completeness of mosaics (DEM, DSM, aerials, intensity)

Validates LAS header information

http://data.auscover.org.au/xwiki/bin/view/Teams/GoodPracticeHanbook


Page 7 of 78

Tile size as requested

Swath LAS PSID validation

Comparison of Report Form to Tender Form

Report Form certified

Required elements reported as delivered

Horizontal coordinate system matches

Vertical Reference system matches

Compliance with on ground environmental conditions at the time of survey

Reported absolute vertical accuracy within specification

Reported relative vertical accuracy within specification

Minimum Bathymetry Coverage % within specification

Minimum Bathymetry Soundings within specification

Reported maximum scan angle within specification

Geoid model matches

Classification level matches

Classification Statistics

Classified AHD LAS point classification

Classified ELL LAS point classification statistics match AHD classification statistics

Unclassified LAS data swaths only contain class 0

Total point count of all supplied LAS types matches

Accuracy of Survey Control

Internal survey control distribution

Control density meets minimum requirements

Control collection methods suitable

Point Density & DEM Resolution

Pseudo pulse density meets NPS requirements (reports ground point and all point densities)

DEM resolution (asc, ESRI Grid)

Bathymetry coverage meets minimum tile % and sounding requirements

Flight Line Coverage

Gaps between flight lines

Scan angle recorded in points within acceptable range

Absolute & Relative Vertical Accuracy

Absolute (Fundamental) vertical accuracy of LAS and DEM

Supplemental vertical accuracy of LAS and DEM

Relative vertical accuracy of LAS

Extent Checks

Visually check the data covers the required extent

Visually check for internal voids in the DEM


Page 8 of 78

Visual Checks

Visually check the digital elevation model (DEM) for classification, relative vertical accuracy,

surface interpolation, systematic and other errors.

Tile management for multiple users

Other

Version control between subsequent deliveries

Enables the use of relative paths for data on external hard drives

QA4LiDAR does NOT replace existing LiDAR provider QA procedures, or replace contracting and

project management requirements. It also does NOT check the following aspects of a LiDAR delivery;

Corruption of aspatial data

Naming conventions other than Australian, Pacific & NZ NEDF

Folder structures

Horizontal accuracy

Every LAS point attribute

Point density at nadir

Spatial distribution of points

% flight line overlap

DEM hydro-flattening

Aerial or intensity imagery values

Digital Surface Model (DSM) accuracy

Contours; interval, topology, continuity, vertices or smoothness

Canopy Height Model (CHM) values

Foliage Cover Model (FCM) values

Waveform LiDAR other than presence of header link and WDP

Compressed LAZ format data

Photogrammetric LAS datasets

Visual Checks of LAS point data


Page 9 of 78

1.3 Licensing After installing and running the software you will see the registration screen (shown to the right).

Please provide your Machine ID to [email protected]. You will then be supplied with a license

key which must be entered in the License box to Register the software. If you re-install the software,

you will need to re-supply the same license key to the registration dialog.

QA4LiDAR leverages ArcGIS proprietary software. To utilise QA4LiDAR users must have at least an

ArcGIS 10.2-10.4 Basic license with the 3D Analyst and Spatial Analyst extensions. All ArcGIS

components remain under their original ESRI licensing system. QA4LiDAR also leverages the open

source spatial library GDAL, and the LAStools las2las, lasinfo and lasvalidate which are open-source

and LGPL (http://opensource.org/licenses/LGPL-2.1). These are packaged within the installation.

The Disclaimer of Warranty states that QA4LiDAR is provided "as‐is" and without warranty of any

kind, express, implied or otherwise, including without limitation, any warranty of merchantability or

fitness for a particular purpose. In no event shall the author of this software be held liable for data

loss, damages, loss of profits or any other kind of loss while using or misusing this software.

mailto:[email protected]

http://opensource.org/licenses/LGPL-2.1


Page 10 of 78

1.4 Requirements & Recommendations for using QA4LiDAR The requirements to use QA4LiDAR are:

Users MUST have at least an ArcGIS 10.2-10.4 Basic license with the 3D Analyst and Spatial

Analyst extensions.

To be processed as one project, every partial data delivery MUST be integrated into the one

overall project directory, overwriting superseded data. Full deliveries are stand-alone

project directories.

The project extent shapefile MUST be a single (can be multi-part) polygon.

The tile index MUST accurately represent the data, i.e. for every tile of data, there needs to

be a polygon in the tile index and vice versa.

Survey Control data should consist of Ground Control and Fundamental Vertical Accuracy

Check Points which MUST be supplied to QA4LiDAR as two separate shapefiles (see section

4.9). Users can also supply custom control and Supplemental Check Points if available.

The project extent, tile index and control data shapefiles MUST all have the same coordinate

system definition (as identified in ArcMap layer properties) for control checks to work.

If the NEDF naming conventions are NOT used;

o Key term rules listed in Table 6 MUST be used in file naming otherwise QA4LiDAR

will be unable to identify data.

o File naming of tiled data MUST match one of the tile name search rules in Table 7

for QA4LiDAR to identify tiled datasets.

The Output Folder selected;

o MUST have sufficient available disk space (DEM mosaic requires ~one twentieth the

size of the DEM, other outputs require a relatively small amount of space).

o MUST be located outside the project folder.

o As it is shared between users, if there will be multiple users of the project, it should

be located on a network drive so everyone can access it. If there will only be a single

user for the project, the Output Folder should be located on the local machine or

USB 3 connected external disk for increased speed.

The Working Directory selected;

o MUST have at minimum, the size of the classified AHD LAS dataset in available disk

space.

o MUST be located outside the project folder and separate to the output folder.

o As it is user specific, it should be located on the local machine or USB 3 connected

external disk for increased speed.

The location of project data MUST be appropriate to perform the Visual Checks with greater

than one user i.e. it cannot be on an external hard drive that is only accessible to one user.

File and folder names should NOT contain spaces (use underscores - this is common practice

for spatial data).

Project files should NOT be open in other software as access issues may occur.

Avoid opening instances of ArcGIS on the computer while QA4LiDAR is running as conflicts

may occur.

If data FAILs the corruption check in the first check group with an error (as opposed to a

warning), it MUST be fixed (or removed and the project rescanned) before running the

following checks or it may cause them to crash.


Page 11 of 78

The recommendations for using QA4LiDAR are:

The QA4LiDAR Tender and Report Forms are not required but they are recommended.

It is HIGHLY recommended to run the Delivery Completeness & Spatial File Corruption check

first to check and fix any corruption or delivery issues.

Topographic and bathymetric data should be run as separate projects.

If unclassified swath LAS and/or waveform LAS do not exist in your dataset, un-ticking the

sub checks within the first two check groups related to these data types is recommended to

save time.

The recommended naming convention is NEDF.

Attributes used in the relevant shapefiles should be the EXACT, case sensitive field names as

per the Tender Form for the shapefile attributes check to pass. However, if not this will not

adversely affect other checks and can be conditionally passed.

If processing data from and/or to external disk, a USB 3 connection (disk and port) is HIGHLY

recommended. If attempted over USB 2 or a network connection processing will be

extremely slow. Otherwise data should be processed to/from the local machine.

The use of long folder path names for project data, the output folder and the working

directory should be AVOIDED otherwise files may not be able to be written successfully.

Ideally less than 256 characters (this is common practice).

Once a QA4LiDAR project is created, the data within the folders should NOT be renamed or

rearranged as project links will be broken. The user is however able to rescan the project

directory to identify changes (refer to section 4.2).


Page 12 of 78

2. Installation Procedure

2.1 System Requirements QA4LiDAR is a standalone, application for Windows 7, 8 and 10. It will run on 32 bit and 64 bit

machines. It requires ArcGIS 10.2-10.4 and the Microsoft .NET Framework version 4.0 or higher to be

installed on the computer. A minimum of 100 MB of disk space is required for installation. When

processing data, additional disk space will be required for temporary files and output files (see

section 0). Hardware requirements can be considered the same as for ArcGIS 10.2-10.4 (Table 1).

Table 1. ArcGIS 10.3 for Desktop hardware requirements

CPU Speed 2.2 GHz minimum; Hyper-threading (HHT) or Multi-core recommended

Processor x86 or x64 with SSE2 extensions

Memory/RAM 2 GB minimum (for large datasets, the more RAM the better i.e. for a 1TB project, 16GB RAM is recommended)

Display Properties

24-bit colour depth

Screen Resolution

1024 x 768 recommended minimum at normal size (96 dpi)

Swap Space Determined by the operating system; 500 MB minimum.

Disk Space 2.4 GB. In addition, up to 50 MB of disk space may be needed in the Windows System directory (typically, C:\Windows\System32). You can view the disk space requirement for each of the 10.3 components in the Setup program.

Video/Graphics Adapter

64 MB RAM minimum, 256 MB RAM or higher recommended. NVIDIA, ATI, and Intel chipsets supported. 24-bit capable graphics accelerator OpenGL version 2.0 runtime minimum is required, and Shader Model 3.0 or higher is recommended. Be sure to use the latest available driver.

Networking Hardware

Simple TCP/IP, Network Card, or Microsoft Loopback Adapter is required for the License Manager.

2.2 Installation QA4LiDAR Form Editor

1. The QA4LiDAR Form Editor is a standalone installation.

2. Obtain the installation package from http://qa4lidar.com/downloads-and-user-guide/.

3. Run the QA4LiDAR Form Editor msi file to install the editor. The QA4LiDAR Form Editor setup

wizard will guide you through the installation.

4. New versions of the editor will overwrite existing versions when installed.

5. To uninstall use Control Panel > Programs and Features.

6. Any software updates will be available at http://qa4lidar.com/downloads-and-user-guide/.

QA4LiDAR

1. Obtain the installation package from the CRCSI.

2. Run the QA4LiDAR Setup msi file to install the tool. The QA4LiDAR setup wizard will guide you through the installation (click yes to all the windows that appear).

3. New versions of the tool will not overwrite existing versions when installed. 4. To uninstall use Control Panel > Programs and Features.

5. Any software updates will need to be supplied by the CRCSI.

**Please ensure that QA4LiDAR and the QA4LiDAR Form Editor are installed in separate folders (as

per the default installation directories). If the installation files are mixed within the same folder,

crossover may occur between the Forms and QA Report displays.**

https://desktop.arcgis.com/en/desktop/latest/get-started/system-requirements/arcgis-desktop-system-requirements.htm

http://qa4lidar.com/downloads-and-user-guide/

http://qa4lidar.com/downloads-and-user-guide/


Page 13 of 78

3. Using the QA4LiDAR Form Editor

3.1 Purpose and Importance QA4LiDAR is intended to complement the ICSM LiDAR Acquisition Specifications and Tender

Template. In order to check a LiDAR delivery against the specifications defined in the template,

QA4LiDAR requires a condensed form version of the template from which to automatically extract

project specifications. This is referred to as the Tender Form (*.ctf extension) and is completed by

the contracting agency. To be able to check project results against the specifications and data

delivered, QA4LiDAR also requires a similar condensed form version of the final project report from

which to automatically extract project results. This is referred to as the Report Form (*.crf extension)

and should be completed and certified by the LiDAR provider.

The information supplied in these forms MUST match the relevant information in the full tender and

project report. QA4LiDAR is able to run without the forms however they are recommended, as

without them, some checks will not be possible and no Pass/Fail results will be returned. If you do

not have access to the forms but have sufficient information, you can create the forms in hindsight

to obtain Pass/Fail results. If the forms are not used with QA4LiDAR, no Pass/Fail results will be

output and only the following checks and results ARE possible;

Corruption checks on spatial files

Tile index coordinate origin

All tiles for tiled datasets have been delivered

LAS version and PDRF

Swath LAS PSID validation

Presence of WDP for full wave form LAS

Validate LAS header information

Classification Statistics checks

Survey Control; Density & Distribution

Vertical Accuracy; Absolute and Supplemental accuracy of LAS and DEM,

Relative vertical accuracy of LAS

Flight Lines coverage raster

Density; point density statistics and rasters, DEM resolution, bathymetry

coverage statistics

Visual checks

The following checks are NOT possible without the forms;

All Pass/Fail results that are based on the specification

Delivery Completeness

File naming checks

Horizontal Coordinate System (HCS) of data matches the Tender Form

Shapefile Attributes

Tile size

Tender Form versus Report Form checks

Classification ‘Must Have’ and ‘Must Not Have’ point classes

Survey Control collection method

Scan angle of points


Page 14 of 78

3.2 Forms in the Acquisition Process The contracting agency fills out the Tender Form along with their customary version of the ICSM

LiDAR Acquisition Specifications and Tender Template and supplies these to the LiDAR provider.

Along with the LiDAR data delivery, the LiDAR provider completes the traditional project report and

the QA4LiDAR Report Form. The LiDAR provider and contracting agency are both able to run

QA4LiDAR. QA4LiDAR can also be run without supplying the forms as input. The Form Editor is

versioned to match the main software and ensure compatibility between the two. Hence if a new

version of QA4LiDAR is released, it may be necessary to regenerate forms with the equivalent

version of the Form Editor.

3.3 Create a New Form Run the QA4LiDAR Form Editor (a separate installation to the main QA4LiDAR

software).

Click on the button for the type of form you wish to create i.e. Tender or Report. Alternatively use

the File dropdown menu to choose New then New Tender Form or New Report Form. A new, blank

form will appear.

3.4 Open an Existing Form Within the QA4LiDAR Form Editor, from the File menu, select Open. Locate your Tender Form (*.ctf)

or Report Form (*.crf) file and select Open.


Page 15 of 78

3.5 Filling in the forms

Most elements on the forms have a Tool Tip , which if hovered over with the mouse, will provide

additional information about that element and how to address it on the form.

The Home menu provides a Zoom panel to navigate around the

form and a Print panel from which to preview and print the

form.

3.5.1 Fill in a Tender (*.ctf) Form

Firstly, filling in the project details section at the top of the form is recommended. This includes

contract number, project title, the date the contract was issued, who the tenderer is (who is

completing the form), the contracting company and their address.

Under the headings Datasets and Reports & Ancillary Information, tick the check boxes for the

elements you require. You will notice that some elements are ticked by default and cannot be un-

ticked. These elements (Report Form, Ground Control, FVA Check Points and Tile Index) are required

to use QA4LiDAR (the Report Form is required if using the Tender Form) and have default values

provided. You will also notice some elements are ticked by default but may be un-ticked if not

required. These elements (LAS, DEM, Project Report and Metadata) are the recommended base

products for every LiDAR delivery. Three of these elements appear in red meaning the form is not

valid, unless validated by supplying a data format or un-ticked.

*Note. If you are requesting multiple DEMs of different resolutions, the primary DEM should be

specified with the tick box and resolution slider (under the Coordinates & Accuracy heading), and

any additional DEMs should be specified under the Custom Datasets heading. Only the primary DEM

will be used in the checks.

For each element you tick, use the Format dropdown box to the right to specify the required

format(s) (where applicable). You are able to select more than one format per dataset where

required.

If you ticked Digital Elevation Model, use the Hydro-Flattening dropdown box below it to specify

whether hydro-flattening is required. If you ticked Contours, Ground Control, FVA Check Points, SVA

Check Points, Tile Index or Flight Trajectory, use the associated Attributes dropdown box to specify

which attributes you require for each dataset. You are able to select more than one attribute per

dataset. Field name attributes used in the shapefile should be the exact, case sensitive attributes

specified on the Tender Form so QA4LiDAR can locate the fields. They are not editable in an attempt

to standardise fields.


Page 16 of 78

There is a Custom Datasets option, for you to add additional required elements that do not appear

in the standard list. Add a Custom dataset and select or type the delivery element and format.

You can add multiple custom datasets. Custom elements will not be checked by QA4LiDAR but they

help provide a complete project summary for the contractor and provider.

Under the heading Other Requirements, use the dropdown boxes and slider bars to specify the

details for each element. If there are Environmental Conditions, please specify the page number at

which details can be found in the tender contract., The Class Requirements are defined using a grid

similar to an aeroplane seating chart. If there is no requirement for a class, leave it white (it will be

reported as an “Ignored” class on the QA4LiDAR QA Report). If you Must Have a class, click once on

its number (e.g. 2 - Ground) so it turns green (it will be reported as a “Required” class on the

QA4LiDAR QA Report). If you Must Not Have a class, click twice on its number so it turns red (e.g. 12

– “Overlap Points” which is red by default. It will be reported as an “Unwanted” class on the

QA4LiDAR QA Report). If a class above 12 (i.e. “Other as specified”) is selected as Must Have, a

definition for this class must be typed in the free text box that appears.

Under the heading Coordinates & Accuracy, use the dropdown boxes for Horizontal Coordinates,

Vertical Reference, and Geoid Model to specify your requirements. For Absolute Vertical Accuracy,

Relative Vertical Accuracy, and DEM Resolution use the slider bars to specify your accuracy (@ 95%

confidence interval), and primary DEM resolution requirements. The default values set are typical

values for a LiDAR survey.


Page 17 of 78

When your form is complete and valid (i.e. no red text), Save the CQT and print it to PDF (refer to

section 3.6) to supply it to the LiDAR provider. The CQT file can also be supplied to the LiDAR

provider if required. Saving the form will create a Last Save date stamp in the top right corner so the

user is aware of when the form was last edited.

To save time completing a Tender Form, you may open a previously filled in Tender Form from a

similar project, edit the values as required, and choose to Save As. If using this method, be careful to

correctly edit all necessary values.

3.5.2 Fill in a Report (*.crf) Form

Firstly, filling in the project details section at the top of the form is recommended. This includes

contract number, project title, the delivery date, delivery volume unique identifier, who the

contractor is (who is completing the form), the company contracted and their address, and the

contractor job number.


Page 18 of 78

Under the headings Datasets and Reports & Ancillary Information, tick the check boxes to confirm

the elements you are delivering for the whole project (even if supplying in partial deliveries). For

each element you tick, use the Horizontal Coordinates dropdown box to the right to specify the

horizontal coordinates used (where applicable). When you specify the horizontal coordinates for one

dataset, the rest will be populated with the same value to save time. Edit these pre-populated values

if required.

For Digital Elevation Model, Ground Control, FVA Check Points and SVA Check Points use the Vertical

Reference dropdown box to specify the vertical reference system for those elements.

There is a Custom Datasets option for you to add additional elements you are supplying that do not

appear in the standard list. Add a Custom dataset and select or type the delivery element and

format. You can add multiple custom datasets. Custom elements will NOT be checked by QA4LiDAR

but they help provide a complete project summary for the contractor and provider.

Under the heading Other Requirements, use the dropdown boxes and slider bar to specify the

details for each requirement. Indicate if you complied with the Environmental Conditions in general

for the project. If there was a small area affected differently or an issue that needs to be mentioned,

use the free text space provided to comment or reference the relevant page in the full project report

for further detail.

Under the heading Coordinates & Accuracy, use the dropdown boxes for Ground Control Method

and FVA and SVA Check Point Method to specify the methods used for data collection for these types

of survey control. Use the free text box associated with each to briefly explain how each type of

control was connected to the datum or refer to the relevant page in the project report. For Absolute


Page 19 of 78

Vertical Accuracy LiDAR Point Cloud, Absolute Vertical Accuracy DEM, and Relative Vertical Accuracy

LiDAR Point Cloud, use the slider bars to specify the accuracies achieved (@ 95% confidence

interval). Use the dropdown box for Geoid Model to select which geoid model was applied to the

LiDAR. Indicate whether any corrections additional to the geoid model were applied, the size of the

shift if a vertical constant shift was applied, and use the associated free text box to explain the

additional corrections or reference the relevant page in the project report.

Finally, certify the Report Form as a certificate of delivery.


Page 20 of 78

When your form is complete and valid (i.e. no red text), Save the CQR and print it to PDF (refer to

section 3.6) to supply to the contracting agency. The CQR file can also be supplied to the contracting

agency if required. Saving the form will create a Last Save date stamp in the top right corner so the

user is aware of when the form was last edited.

To save time completing a Report Form, you may open a previously filled in Report Form from a

similar project, edit the values as required, and choose to Save As. If using this method, be careful to

correctly edit all necessary values.

3.6 Save and Print a Form From the File menu or Home panel within the form you are working on, select Save. To save a new

version select Save As then name and choose the location for your new Tender (*.ctf) or Report

(*.crf) file and click Save. You will only be able to save if the form is valid i.e. all required areas have

been filled out and there is no red text remaining.

From the Home panel within the form you are working on, select Print to PDF. A Save As dialog will

appear for you to choose the output directory and file name. Once the PDF has been created, you

can open it in Adobe to print a hard copy. The PDF lists the sections of the form as bookmarks in the

left panel for easy navigation.


Page 21 of 78

4 Using the QA4LiDAR Software

4.1 Supported Data Formats QA4LiDAR supports the data formats listed in Table 2. It does NOT perform checks on other data

formats that may exist within project folders. To save processing time such files are ignored. The

reporting of file counts and sizes on the Dashboard only include the file formats listed below.

Table 2. Data formats supported by QA4LiDAR.

Data Type Supported Format/s

Point Cloud LAS (versions 1.0-1.4) Unclassified swath LAS waveform + WDP (only for presence)**

DEM, DSM ESRI GRID ESRI ASCII

Intensity Imagery ECW Tif & GeoTiff ESRI GRID ESRI ASCII

Aerial Imagery ECW Tif & GeoTiff

Contours, Flight Trajectory, Tile Index

Shapefile (preferred) GDB Feature Class ArcInfo Coverage MapInfo TAB

Ground Control Shapefile CSV

GPS Base-station Data Rinex v2.11 Rinex v3.0

Metadata ANZLIC XML NEDF XML PDF Word (.doc and .docx)

Project Report PDF Word (.doc and .docx) Excel (.xls and .xlsx)

QA4LiDAR Tender Form CQT

QA4LiDAR Report Form CQR

Other Any other data formats will not have any checks performed on them

* QA4LiDAR uses free LAStools and ArcGIS to handle LAS files. The LAS 1.4 standard states it has

"Backward compatibility with LAS 1.1 – LAS 1.3 when payloads consist of only legacy content”

(http://www.asprs.org/a/society/committees/standards/LAS_1_4_r11.pdf pages 2 & 3). ArcGIS 10.2

supports LAS versions 1.0, 1.1, 1.2, and 1.3. Additionally, LAS version 1.4 files that are 1.3 compliant,

containing point record formats 0 through 5 are supported. LAS version 1.4 files containing point

record formats 6 through 10 are not supported. Hence backwards compatible LAS 1.4 files (only

containing point record formats 0 through 5) are supported by QA4LiDAR, but those containing

record formats 6 through 10 are not supported. ArcGIS 10.3 and above support LAS version 1.4.

**If a project contains waveform LAS of a type other than unclassified swath, this may cause the

checks to fail as such data is untested.

http://www.asprs.org/a/society/committees/standards/LAS_1_4_r11.pdf pages%202%20&%203


Page 22 of 78

4.2 Project Data Version Control QA4LiDAR REQUIRES all project data and files to be within a single parent directory. The first time a

project is scanned into QA4LiDAR, the New Project option is used. If a subsequent (partial) delivery

is made for the project it must be placed within the existing project directory and the Open Project

and Rescan options used. Partial deliveries could occur for large projects delivered in stages, or for

re-supply of a subset of failed data (e.g. LAS files) by the provider. In the case of re-supply, QA4LiDAR

requires the user to remove the superseded data from the project directory and replace it with the

new data. It is recommended that a delivery number (e.g. 2) be included in the folder name for that

partial delivery element as explained in the following paragraphs. Retaining superseded data within

the project directory is NOT recommended as may significantly increase the size of the project and

slow down the scan and checks. When the user opens an existing project with changed files, they

MUST then select to Rescan Project Directory so the changes to the project directory can be

identified by QA4LiDAR.

QA4LiDAR does NOT record all previous (i.e. an entire history) of scans/rescans of a project

directory. It DOES retain record of the scan immediately prior to the current scan. Therefore, it is

able to detect changes in the project directory between two consecutive scans i.e. files that have

been added or removed from the project.

QA4LiDAR version control operates primarily for DEM files for the Visual Checks. The system uses

unique identifiers (hashes) based on the binary code within each DEM (and other raster) files as well

as DEM file names to manage versions of the DEM files in the project folder between two

consecutive scans. For LAS files, the scan/rescan only detects new or removed files based on file

creation/modification dates, file size, and file paths and file names. Hence you may want to use a

naming convention that includes the delivery number in the folder name, especially for data types

other than DEM such as LAS. Using this information, the program will inform the user of the total

number of files in the project, the number of new files, and the number of removed files between

consecutive scans.

From the binary DEM data, QA4LiDAR can detect if DEM data has changed between scans and

whether it is necessary to regenerate the DEM mosaic for Visual Checks. If a provider accidentally re-

delivered the same DEM files, the files will have the same unique identifiers in QA4LiDAR, so a new

mosaic will not be generated and files will NOT be reported as new/removed unless the pathname

was altered. For other file types, it is more difficult to detect change using only file paths and file

names. If for example the project LAS files were updated and re-delivered and replaced the old LAS

files in the project folder using the same name and path, they would NOT be detected as a change

and would not be reported as new/removed on the Project Summary however checks could be re-

run on them and may return different results. If the user wished QA4LiDAR to detect this change and

report it in the Project Summary, they could slightly alter the path name of the new LAS files by

adding a delivery number (while still removing superseded data).

Changes CANNOT be detected between non-consecutive scans. For example, if the provider

accidentally re-delivers the original data (scan one) as the second re-delivery (scan three), QA4LiDAR

will be unable to identify this as part of the scanning process. However, the error could be detected

by the user when automated checks provide the same results as the original delivery.


Page 23 of 78

If a LiDAR project is collected and delivered in stages, each subsequent delivery should be added to

the single parent project directory and the directory re-scanned. However, if it is a very large project

it may be more efficient to process the areas separately - refer to section 4.3. Sometimes in staged

deliveries, the entire project is re-delivered at completion. To check this final complete delivery is

the same as the previously scanned project of all integrated staged deliveries that passed QA, the

user can locate the final complete delivery at the exact directory location of the staged delivery

project folder (don’t delete the integrated staged delivery, just re-name/re-locate it first) and re-

scan the project folder. If the total number of files in the project matches and there are no new or

removed files reported, the final complete delivery can be considered the same. This avoids re-

running the full set of Automated Checks.

To perform Visual Checks on re-delivered DEM data the user would want to target only the DEM

tiles that had been re-delivered. As mentioned the DEM mosaic will be regenerated if DEM tiles have

changed, however the system will not automatically recognise which tiles these were as part of the

sample selection. The solution to this is to load the tile index with the first set of Visual Check results

and use the failed tiles as the flag for sample selection (refer to section 4.11.1).

4.3 Dealing with Very Large Projects Very large LiDAR projects are usually collected and delivered in stages. Rather than combining all the

data into a single project directory for QA4LiDAR, it is recommended that each stage be processed as

an individual project. Large datasets can take a very long time to process and ArcGIS processing tools

do not deal well with very large volumes of data.

The size that can be handled will depend on a lot of different variables. These include the

specifications of your computer (RAM etc.), other applications running at the time, the location of

project data (USB, network etc.), dataset properties i.e. products delivered, point density, tile size,

DEM resolution etc. Testing has revealed that the software can handle projects of 2,000 x 1km tiles

(it may handle larger projects). If there are areas a lot larger than this delivered, they may need to be

split into chunks, for example perhaps based on Local Government Areas. In order to process each

stage/area as an individual project, the relevant files specific to each stage/area will need to be

created i.e. an extent shapefile, tile index, control shapefiles, flight line shapefile, that only have data

for that area. The same Tender and Report Forms can be used for each stage/area.


Page 24 of 78

4.4 Open a QA4LiDAR Project Run the QA4LiDAR software. The splash screen then project screens shown below

appear. The user can choose to create a New Project (this means a new QA4LiDAR

*.cqp file) or if a project (*.cqp file) already exists, Open Project. The folder pointed

to, should be the top level folder containing your LiDAR project data.

*Note. It is highly recommended that the project data is located on the local machine or a USB 3

connected external hard drive so that the scan in of data is not occurring over a network or slow

connection. If using USB 2, the scan process will run extremely slowly. However, if the project

requires multiple users for the Visual Checks it will have to be on a network drive.

If the user selects to create a New Project, the project directory will be scanned. This scanning

process creates a catalogue of all the files in the project directory which is known as the QA4LiDAR

Project Database (*.cqp). The scan determines things such as the data type, whether a file is a tile,

what coordinate system a spatial file is from the file name (using NEDF conventions) and so on. If a

QA4LiDAR Project Database already exists at the new project location selected, the user can choose

to Open the existing project or Overwrite it. If the user chooses to overwrite, the existing QA4LiDAR

Project Database (including any results) will be deleted and replaced with the new scan results.

Otherwise the user can select to Open Project (an existing project).


Page 25 of 78

4.5 File & Help Menus The File dropdown menu allows the user to start a New project, Open an

existing project, Close the current project, Print Report Summary (the

standard QA4LiDAR QA Report), Print QA Errors (detailed information

about Failed checks), Save Results as XML, and Exit the software.

The Help dropdown menu provides

access to the PDF help documents

including the Quick Start Guide, the Shortcut Key Guide for Visual

checks, this User Manual, the Map Legend and the Video

Tutorials on YouTube. It also gives version information About the

software.

4.6 SETUP: Dashboard Once a project is open, the below Dashboard Project Summary page will appear. This Dashboard can

be accessed under the SETUP heading on the left menu panel. QA4LiDAR is designed to step through

the side panel menu in order starting from the Dashboard and progressing down through each

subsequent screen as per the following instructions.

The Dashboard provides the dataset path and ArcGIS version, results of the last scan giving a

breakdown of the number of files for each file type and the file size for each file type, whether there

are any active QA sessions, as well as the total number of files in the project and the total file size. If

a rescan has been performed, the number of removed files, new files, and changed DEM files are

displayed (these numbers only include the supported data formats as described in section 4.1 –

other formats are ignored). The rescan button is explained below. The QA4LiDAR status including

version number, ArcGIS license type and path of the tile index are also provided along with links to

the output files created by the software (once they are generated).

The user has the option to Rescan Project Directory. A user may choose to rescan the project

directory if changes have been made to the directory i.e. if new data has been added or superseded

data removed. An example of this would be a large project delivered in stages, or a subset of the

project updated and re-supplied by the provider. Rescanning a project identifies changes to the

project directory and is quicker than overwriting and starting the QA4LiDAR Project Database from

scratch. However, users should be aware that if they rescan the project and go directly to the Report

without re-running the checks, any results that appear in the Report may not be valid for the new

data until relevant checks are re-run.

QA4LiDAR stores relative paths even though it displays full paths. Therefore, it is possible for the

user to Open a project for which the drive letter or a top level folder name has changed e.g. because

the data is on an external disk. The Project Settings (section 4.8) should still appear correctly with

green ticks. However if not the user can rescan the project folder to update the QA4LiDAR Project

Database and will also have to resupply the extent, forms and control which will have warning

symbols next to them. If there are warning symbols and the user just resupplies these parameters

and doesn’t rescan the project, the QA4LiDAR Project Database paths will be different to the existing

data and the checks will fail.

Save Results As XML


Page 26 of 78

4.7 SETUP: User Settings Once a project has been opened or created, the User Settings, which are located under the SETUP

heading on the left menu panel, MUST be completed. The user’s full name is required and supply of

an email address is recommended. This information is used by the Visual Checks section to handle

multiple users. The user information is recorded per QA4LiDAR session within the project database

and can be changed during an active session. It is used to track tile sample selection per user and to

record which user checked each DEM tile, so that clarification of Visual Check error mark-up is

possible if required. It is also used to ensure that only one Automated Checks session is being run at

a time. These details are saved to the user profile for the computer/user and will appear by default if

they have been filled in previously.


Page 27 of 78

4.8 SETUP: Project Settings Once the User Settings are complete, the Project Settings, which are located under the SETUP

heading on the left menu panel, MUST be completed. Once complete, the project settings can be

saved to HTML using the Export button at the top of the screen. If a new project for the same data

needs to be started, the user can Import these saved settings to save time filling them in manually

again.

4.8.1 Inputs

A Project Extent polygon shapefile is REQUIRED, which should represent the specified capture

extent for the project and be in the SAME coordinate system as project data (as per the ArcMap

layer properties dialog). It is then optional, but highly recommended to supply the QA4LiDAR Tender

Form (*.cqt) and QA4LiDAR Report Form (*.cqr).

4.8.2 Processing

The Output Folder is where raster and shapefile outputs will be stored. If a check that produces a

raster or shapefile etc output is run a subsequent time, the existing raster or shapefile etc for that

check will be overwritten. Please ensure that the output location selected is writeable and has

enough free space for storage (the DEM mosaic requires about one twentieth the size of the DEM

dataset, while other outputs require a relatively small amount of space). The output folder is per

project. If there will be multiple users for the project, the output folder needs to be in a location

accessible by all e.g. a network drive. If there will only be one user, it is more efficient to locate the

output folder on the local machine or a USB 3 connected external drive.

The Working Directory is where temporary files created during processing will be stored. The

working directory is per computer/user. It is highly recommended that the working directory

selected is on the local machine (or at least a USB 3 connected external drive) so that temporary files

are not being created over a network or slow connection. If using USB 2 or a slow network

connection, processes will run extremely slowly. Please ensure that this working directory is


Page 28 of 78

writeable and has, at minimum, the size of the orthometric LAS dataset in available space. The

working directory is wiped every time a QA session is initiated, therefore DO NOT select a directory

containing other files or they will be deleted. If QA4LiDAR has been run previously, the working

directory defaults to the last working directory used on that computer.

The Mosaic Dataset option allows the user to point to the location of an existing ESRI DEM mosaic

dataset residing within a geodatabase if one has been delivered with the project or pre-generated by

the user from the project DEM tiles. This will save QA4LiDAR from needing to generate a DEM

mosaic dataset, however if one does not already exist, this box can be left blank and the mosaic

dataset generated later. If a mosaic dataset has previously been generated by QA4LiDAR for a

project, and there has been no change to the project DEM tiles since its creation, the mosaic dataset

directory will already be set to the mosaic.

4.9 CHECKS: Automated Checks The Automated Checks setup screen can be accessed under the CHECKS heading on the left menu

panel. It allows users to supply Survey Control variables, Processing variables, and select the checks

to be run. Certain information must be provided before the Automated Checks can begin.


Page 29 of 78

4.9.1 Survey Control

There are four Survey Control point types that can be input as shapefiles to QA4LiDAR;

1. Provider FVA Check Points (FVA CPs)

o These are part of the Independent Check Point network supplied by the LiDAR

provider and are used to assess the fundamental vertical accuracy of the survey.

They must be gathered internal to the project area and are often collected in

clusters on open, flat ground where there is a very high probability the sensor will

have detected the ground surface, for easy comparison to LiDAR ground points. In

QA4LiDAR they are used as part of the Survey Control Collection Method, Survey

Control Density checks, the Survey Control Distribution check, as well as the

Flatness, Openness & Absolute Vertical Accuracy checks.

2. Provider Ground Control (GC)

o Along with Base Station data, GC (also supplied by the LiDAR provider) make up

what is termed the Control Network. GC are high accuracy points (e.g. state

benchmarks) that are used by the provider to establish the datum in the survey area

and adjust the LiDAR horizontally or vertically. They can be internal or external to

the project and assess the variation of the reference surface and/or geoid model

across the survey area. In QA4LiDAR they are used for the Survey Control Collection

Method, Survey Control Density checks, the Survey Control Distribution check, as

well as for Flatness, Openness & Absolute Vertical Accuracy checks.

3. Custom (Purchaser control)

o This can be any other control (or shapefile point elevation data of known accuracy)

that the purchaser has access to. It will be used by QA4LiDAR for the Survey Control

Density checks, the Survey Control Distribution check, and the Flatness, Openness &

Absolute Vertical Accuracy checks. It should be internal to the survey area.

4. Provider SVA Check Points (SVA CPs)

o These are part of the Independent Check Point network supplied by the LiDAR

provider and are used to assess the supplemental vertical accuracy of the survey.

They must be gathered internal to the project area and located in a range of

different terrain types. In QA4LiDAR they are only used as part of the supplemental

Absolute Vertical Accuracy check.

*Note. The Survey Control Accuracy and Absolute Vertical Accuracy Checks can be run with just FVA

Check Points, with both FVA Check Points and Ground Control, with just custom (purchaser) control,


Page 30 of 78

or with all three of these control. If all three are supplied the checks will run once for FVA CPs and

GC in combination, and separately for custom control.

To add a Survey Control point shapefile, click the button beside the relevant type, navigate to

the location, and Open the shapefile. Alternatively, type the location and a green tick will appear if

the location typed can be found.

Next, use the Elevation field (ORT) drop down list to select the field in each shapefile which

represents AHD elevations of the control/check points. Do the same for Elevation Field (Ellipsoid) if

applicable or select N/A. Note. It is recommended these fields in the shapefile be of type double.

Finally, use the drop down list to select the Acceptability Rating you wish to use for the

control/check points. This rating is used in the Flatness and Openness part of the Accuracy of Survey

Control check and the Absolute Vertical Accuracy check. If the rating for Flatness or Openness is

unacceptable (based on your acceptability rating) for a control/check point, that point is NOT used

to test the accuracy of the Survey Control or the Absolute Vertical Accuracy of the data. BOTH

Flatness and Openness for a control point must be rated equal to or better than the user’s

acceptability rating for the point to be used. The lower the rating selected (i.e. closer to 'Poor'), the

more control/check points that will be used for that type. Refer to section 0 for the Flatness and

Openness rating methods. Some testing of your control may be required i.e. you may want to start

with a rating of ‘Poor’ to run the Accuracy of Survey Control or Absolute Vertical Accuracy check,

then examine the output control files for the Flatness and Openness rating results for each point,

then try changing the rating to suit your requirements. The more points you supply, the longer these

checks will take to complete.

For SVA CPs, instead of the Acceptability Rating, the final drop down box is for the Cover Type Field.

Select the field in your SVA CP shapefile that represents the land cover type to which each point is

associated. E.g. you may have a filed called ‘Class’ with attributes such as ‘Tree’, ‘Grass’ etc.

4.9.2 Processing

To add a Flightline Shapefile (polyline or polygon), click the button beside the box, navigate to

the location, and Open the shapefile. Alternatively, type the location and a green tick will appear if

the location typed can be found. Use the Point Source ID Field to select the field in the shapefile that

represents the flightline number (integer as per the point source ID attribute in the LAS files). The

flightline shapefile is used for the swath related LAS checks and is only required if your project has

swath data, or if you want the results of the pseudo pulse density check output as attributes in a copy

of the flightline shapefile.

If your LiDAR data is bathymetric, supply a Bathymetry coverage tile index polygon shapefile which

must be a 50m or 100m tile index, only covering the tiles to be included in the check (i.e. not

shoreline tiles that are partially land). Also use the Bathymetry Classes box to type and add (click

Add Class) the bathymetry point class/s to be used for the bathymetry coverage and relative vertical

accuracy checks from the orthometric LAS. If your data is topographic, leave these sections blank

and the default ground class 2 will be used for the relative vertical accuracy check. If your data is


Page 31 of 78

topographic but you wish to include different classes in the relative vertical accuracy check you can

override the default class by entering ‘bathymetry classes’ i.e. class 1 and class 2 (or any relevant).

The Raster Cell Size will be used to produce output rasters for the density checks as well as for data

processing during the relative vertical accuracy check. If a cell size greater than 5m is selected, the

relative vertical accuracy checks will NOT run. Use the slider bar to select the cell size you wish to

use. The default (and minimum) cell size is 2m, while the maximum cell size is 10m. Note. Part of the

relative vertical accuracy check process creates a boundary polygon of the data set with internal

holes. If the raster cell size is set to 2m and there are a lot of internal holes in a large dataset, this

output boundary polygon may exceed the maximum shapefile size of 2GB. To avoid this, increase the

raster cell size to 5m. However, also note that as the raster cell size is increased, the area of this

boundary polygon is slightly increased, so the final calculated accuracy result will be slightly lower.

4.9.3 Running Automated Checks

The checks are organised into groups. The tick boxes can be used to select which automated check

groups or individual sub checks you wish to run. The checks are listed in the order it is suggested

they be run in. When a new project is opened, the Delivery Completeness* & Spatial File

Corruption** check is the only one ticked. It is HIGHLY recommended that this check group be run

FIRST without ticking any of the following check groups. If the delivery is incomplete or corrupt, it is

likely subsequent checks will fail and significant time may be wasted running them. Hence if any part

of this check group fails this will HALT the QA Session so that if you had more checks queued to run

they will NOT run and SHOULD NOT be run until the failure issue is addressed. It is more efficient to

update/fix any delivery incompleteness or corruption in the data before running other checks. For

example, if any LAS files are found to be corrupt and they are left as corrupt in the project, they will

cause some of the following checks to CRASH. Please un-tick the waveform and swath related checks

(last 2 checks) within this group if those data types do NOT exist in your dataset as this will save

time.

*Note on delivery completeness. LAS are checked for delivery completeness in this group and other

tiled datasets and their mosaics in the next check group as missing LAS halt the QA, while there may

be legitimate reason for missing tiles for other data types.

**Note on corruption. There are 6 parts to the corruption check for LAS files; file readability,

number of points is not zero, file size, number of points is less than 100 per file, number of points is

less than 100 per flightline, and maximum number of point source IDs. The lasinfo tool is used to

check readability, number of points is not zero, and number of points per flightline which are the

core corruption checks. If these core checks fails it returns the message “Unable to open LAS file {file

path}, it is corrupt” or “Error: {file path} contains 0 point records. The file is considered corrupt” or

“Error: Flight line {x} contains less than 100 point records ({x} points). All files with this point source

ID are considered corrupt.”. In these cases, the user should remove these files or obtain new

readable versions with points before running other checks or they will crash. The other three parts

simply provide warnings to the user if the file size is greater than 2GB (ICSM specification requires

LAS files <2GB), if the number of points is less than 100 (potentially causes processing issues), or if

the number of point source ID’s per file exceeds QA4LiDARs expected maximum. The maximum

expected number of point source ID’s is 10 times the tile size in km, which is to ensure these


Page 32 of 78

numbers represent flight lines. If these warnings are given, the files can still be used in remaining

checks.

One exception to running Delivery Completeness & Spatial File Corruption check first, is if you only

wish to run the Accuracy of Survey Control check to validate the proposed Survey Control in the

planning stage of a project.

After running the checks in Delivery Completeness & Spatial File Corruption successfully, it is a good

idea to run both File Naming, Shapefile Attributes & Horizontal Coordinate System and

Comparison of Report Form To Tender Form checks before queuing the remaining checks to run. If

for example, there is a significant LAS or DEM file naming issue (i.e. tiles cannot be identified or AHD

and ELL files cannot be separated), some of the remaining checks may be unable to run properly

before this is addressed. It would be unfortunate if you had set all remaining checks to run over the

weekend but upon return found they were unable to run successfully. Again please un-tick the

swath related check (last check) within this group if that data type does NOT exist in your dataset as

this will save time.

When the first three checks have run successfully, the remaining checks can be run i.e. Classification

Statistics, Accuracy of Survey Control, Point Density & DEM Resolution, Flight Line Coverage, and

Absolute & Relative Vertical Accuracy. You may also select to Prepare Visual Checks which will

generate the detailed DEM extent shapefile and produce the DEM mosaic for the project. Again, if

you are working with a large dataset it may be wise to do this overnight or a weekend as it may take

some time.

There is no coupling between checks therefore checks are independent of each other. However, as

some checks use bits of the same code and outputs, if these outputs have already been generated

for one of the checks, subsequent checks will utilise them to save processing time. An example of

this is the generation of LAS files by flight line which is used for the flightline coverage raster, the

pseudo pulse density flight line rasters, and the relative vertical accuracy check.

To run a check, tick the box for that check and click Run. Multiple checks can be ticked and queued

to run sequentially. The Automated Checks QA Session running screen will appear and processing

for the check/s will begin. For information on the methods used to run the checks, refer to Appendix

1.

When a check is running, a blue circle will appear to the left of each check and a progress bar to

the right as each check is processing. A disk icon will appear to the left of a check when the

check is finished and the results are saving to the QA4LiDAR Project Database (or when previous

results are being removed). There is a stop button on the upper right hand corner of the QA

Session screen which allows you to abort the checks. There is also a back button that

allows you to return to the Automated Checks screen. When the check is complete and saved, if a

green tick appears, the check has completed successfully but data has not necessarily Passed

QA. A yellow warning means the check cannot be run, for example a corruption check cannot

run for tiff files if no tiff files exist. A red warning means an error occurred and the check did not

run successfully.


Page 33 of 78

You can find additional information in the log window at the bottom of the screen. Each time a QA

Session is run, the log is automatically saved to a text file in a folder called ‘Logs’ in the output

directory. Each log file is named with the date, time, and project folder name. When the QA Session

has completed, you can click the Report screen to see the results of that check (refer to section 0).

The QA4LiDAR QA Report information will be generated each time you click on Report and the

output tile index shapefile will be updated only if a new QA Session has been run.


Page 34 of 78

4.10 CHECKS: Extent Check After the Automated Checks have been completed, the user can progress to the Extent Check. If a

DEM mosaic dataset was not provided on the Project Settings page, the user can either Return to

Settings to provide an existing DEM mosaic dataset (supplied with the delivery or pre-generated

from project DEM tiles), or choose to Generate Mosaic Dataset. If the user chooses to generate a

mosaic dataset, a loading dialog will appear while the dataset is generated from the project DEM

tiles. As the tiles are loaded into the mosaic they will colour green in the loading dialog. If a project is

being re-opened, a mosaic dataset already exists and there has been no change to the project DEM

tiles since its creation, this screen won’t appear, as the existing mosaic will be used to avoid wasting

time re-generating one.

Once the mosaic dataset has been generated, the below continue screen appears. The user can

either select Continue to load the Check Extents map using the basic extents, or they can tick the

box to Generate detailed DEM extents (LAS extents will be basic either way - explained below). If

you are working with a large dataset with many tiles, the basic extents can take time generate. The

detailed DEM extent is optional as this can take a long time to generate for a large dataset. The same

loading dialog as when generating the mosaic will appear, followed by the Check Extents map.

Extents for the LAS and DEM data are generated while this loading dialog is visible.

The LAS shapefile displayed in the Check Extents window is created by extracting the ArcGIS LAS

Dataset LAS file extents. Hence, most polygons will be square (i.e. the tile size) while any boundary

tiles partially filled with data will be rectangular (smaller than the tile size). The polygons will NOT

show internal holes in the data (the flight line coverage raster could be analysed for this purpose).

The basic DEM shapefile is extracted from the DEM mosaic footprint. Again, this extent will NOT

represent internal holes. To identify internal holes in the DEM, the user should tick to Generate the

detailed DEM extents (or alternatively, could analyse the Point Density & DEM Resolution

automated check output density rasters).


Page 35 of 78

The Check Extents map, shows the DEM boundary extent (hatched brown), the LAS boundary extent

(hatched yellow), the original project extent (light blue outline), and the detailed DEM extent if

generated (navy outline called “No Data – Extents” circled in red in the image below). The detailed

DEM extent only displays areas of no data in the DEM, so if it doesn’t seem to appear on the map,

your DEM is free of ‘No Data’ holes. You are required to investigate the data and determine if the

DEM and LAS extents cover the original project extent adequately, and if detailed DEM extents were

generated whether there are any unacceptable voids in the DEM. You can turn layers on and off in

the table of contents, zoom with the mouse wheel, right click a layer name in the table of contents

and zoom to layer, and add data if required.

The window will initially be zoomed to the extent of the data; hence if it does not appear to be

zoomed to the project extent, there may be a coordinate system issue with one or multiple LAS or

DEM tiles. If the extent of the data appears acceptable, select the relevant Yes radio button. If the

project extent is not fully within the LAS and DEM boundaries, select the relevant No radio button

(as per the below image), and briefly describe the problem in the box provided. If detailed DEM

extents have been generated and any existing no data holes are acceptable, select the relevant Yes

radio button (as per the below image). If any existing no data holes are unacceptable, select the

relevant No radio button. Select Save and move to the Visual DEM Check.


Page 36 of 78

4.11 CHECKS: Visual DEM Check

4.11.1 Sampling

The Visual QA Session screen can be accessed by going to the Visual DEM Check under the CHECKS

heading on the left menu panel. It allows users to choose whether to visually check all DEM tiles or a

sample of DEM tiles. There are two options available for selecting a sample. You can either load a

pre-flagged tile index shapefile, or type the percentage of tiles to check. If typing a value, a random

sample of tiles equivalent to the percentage entered will be highlighted on the tile map. If you wish

to check all DEM tiles, type the percentage as 100.

Two forms of “intelligent sampling” based on the user’s knowledge of the survey area are enabled;

the first is the pre-flagged tile index and the second the ability to manually select/unselect tiles on

the map. Zoom tools are available to assist the second option. The user may want to target tiles in

coastal areas, with steep terrain, known water bodies, or heavily vegetated areas. If a user manually

selects/unselects tiles, the sample percentage is updated to match.

To use the first option and load a pre-flagged tile index, the user can browse for a polygon shapefile

of pre-flagged tiles and select the flag field. The flag field should be an integer field of 0s and 1s

where 0 are tiles you don’t want to sample and 1 are tiles you do want to check as part of your

sample. The import of these flagged tiles is a once off update of the sample. The user is then able to

manually change the sample tiles by clicking them however will not be able to revert to the imported

selection unless they re-import the pre-flagged tile index. Only those tiles in the pre-flagged index

that actually match the project tile index and that have not already been selected for sampling by

another user will be selected. If a tile index that does not match the project tile index is imported, a

warning will be displayed.

To enable pre-flagged tile selection for multiple users, you are required to create and import a pre-

flagged tile index per user with the relevant (different) tiles flagged, or import the same tile index

multiple times using different pre-flagged fields for each user.

For multiple users, tile sample selection is based on the user information. Tiles in the sample

selection will be coloured by user. Users will not be able to change the sample selection of other

users. If a user/s has already been allocated tiles to check, and a new user selects to sample 50% of

tiles, they will be allocated 50% of the remaining tiles available. This is to avoid tile collision i.e. two

people checking the same tile.

To perform Visual Checks on re-delivered DEM data the user would want to target only the DEM tiles

that had been re-delivered. As mentioned, the DEM mosaic will be regenerated if DEM tiles have

changed, however the system will not automatically recognise which tiles these were as part of the

sample selection. The solution to this is to load the tile index with the first set of Visual Check results

and use the failed tiles as the flag for sample selection (refer to section 4.11.1).

When you are happy with your sample selection click Continue.


Page 37 of 78

4.11.2 Multiple Users

QA4LiDAR supports multiple users for the Visual Checks if the project data and output folder are

located on a network drive that multiple users can access. The speed of your network will determine

the responsiveness of the Visual Checks when there are multiple users. The conventions for Visual

Checks with multiple users are as follows;

Only one user can run the Automated Checks at a time

The Visual Checks cannot be run while the Automated Checks are running

Multiple users can perform Visual Checks at the same time

The project data and output folder must be on a network drive accessible by all users

The Visual Checks step each user through the tiles in their sample selection (unless they

manually navigate off this sample) and they display all tiles checked by all users with the

green ‘checked’ border

QA Sessions have an expiry time in case of crash so you don’t get locked out

The Dashboard displays any Active Sessions that are running

The Visual QA Session sampling selection screen displays the tiles allocated to each user

The Visual Checks Minimap displays the tiles checked by each user

The name of the user who checked each tile is written to the output tile index


Page 38 of 78

4.11.3 Check DEM Tiles

The Visual Checks screen appears next. If it seems to freeze part way through loading, click the

refresh button. It consists of a table of contents, a menu panel, and the map window. The DEM

mosaic is displayed in the table of contents and map window using bilinear interpolation for display.

QA4LiDAR zooms to the first tile to be checked, which is outlined with a black square and divided

into quadrants. It also displays a portion of the surrounding tiles so that boundary issues and cross

tile errors can be identified. Other tiles are outlined with a grey border. There is a Refresh button to

refresh the Visual Checks screen in case it slows down or freezes. If you zoom out too far the DEM

won’t display. The following section explains how to use the Visual Checks interface to check DEM

tiles, including a table of shortcut keys (section 4.11.4) to use for efficiency.

There are a number of ways to navigate through the tiles. The Previous and Next buttons in the

menu panel can be used, or the shortcut keys for these which are “f” and “g” respectively.

Alternatively, clicking Mark as “Checked” will move you to the next tile or the shortcut key for this

which is “spacebar”. It is also possible to click on part of one of the visible surrounding tiles to move

to that tile or use the arrow keys to navigate to surrounding tiles. Scrolling the mouse wheel zooms

in and out to change the scale. There are also Zoom buttons which allow fixed zoom in, fixed zoom

out and fit to tile. When checking a sample of tiles using the Previous, Next and Mark as “Checked”

forms of navigation, QA4LiDAR will step you through the sample. If the manual forms of navigation

are used i.e. clicking on a surrounding tile or using the arrows to navigate, the user may navigate off

the sample. However if they go back to using Previous, Next and Mark as “Checked” they will be

taken back to checking the sample selected. If the user selected sample tiles on the Visual Checks

initiation screen manually, navigation is in order of tile selection.

If you are reviewing someone else’s Visual Checks the buttons Next Error and Clear Errors are useful.

The Next Error button will take you to the next error that has previously been marked up so you can

Table of Contents Menu panels Map window


Page 39 of 78

check if the mark up is correct. If it is not correct you can use Clear Errors to clear the error in a

particular tile quadrant or all errors within the tile.

The DEM symbology can be changed using the Colour Ramp button. When clicked, a symbology

panel appears on the right hand side from which the user can select from a range of standard

symbology, including elevation symbology (Elevation #1 and Elevation #2). There is a toggle box at

the top to Stretch to Display Extent so that greater colour variation can be seen in the tile. It is also

possible to Set NODATA Colour at the bottom of this panel. You may wish to set this to a contrasting

colour to the symbology so areas of no data can be easily identified (e.g. pink as below). The chosen

symbology is retained as part of the user settings profile so that when the user navigates away from

and back to the Visual Checks their setting remains.

Error mark up for DEM tiles is broken into 3 parts; location (tile quadrant), error type (relative

vertical accuracy, classification, interpolation, systematic errors, follow up, and other), and error size

(small, medium, or large). These qualities should help identify the errors later on when referring to

the tile index. To check each DEM tile for errors, examine the tile and if an error is found, right click

on the quadrant containing the error or use the shortcut keys “1, 2, 3, 4” which apply to the

quadrants as below. The quadrant will be highlighted red and an error dialog will appear. The error

dialog can be repositioned if required or the “ESC” key used to exit it.

3 4

1 2


Page 40 of 78

To mark the error/s, select the appropriate error type/s and size using the mouse or the shortcut

keys “q, w, e, r, t, y” for the errors and “a, s, d” for the sizes. Then select Save. Errors can be

removed by selecting Cancel or the “ESC” key instead of save or later by right clicking the quadrant

again and un-selecting the error/s. Multiple error types can be recorded against a tile and/or

quadrant although only one size of error per quadrant. When finished checking a tile click Mark as

“Checked” or use the “spacebar” key which will change the tile boundary from black to green to

signify that it has been checked and move you to the next tile. For tiles that have already been

checked (green boundary) you can also Mark as “Unchecked” or use the “spacebar” key which will

turn the boundary back to black but NOT remove the errors (use Clear Errors) or move you to the

next tile. The errors are written to the output tile index shapefile upon generation of the QA Report.

There is also a Screenshot button to capture a PNG format image of the current tile, which is

automatically saved to a folder called ‘Screenshots’ in the output directory. The screenshots are

named with the user, date, time and tile number so they can easily be identified. They may assist in

reporting DEM errors to the provider.

If you have chosen to check a sample of tiles and are finding a lot of errors in the sample, you can

opt to Change Sample and increase the percentage of tiles you are checking. When you click the

change sample button you will be taken back to the Visual Checks setup screen where you can adjust

the sample percentage and continue back to the Visual Checks screen.

It is possible to produce a hillshade of the DEM to assist with error identification. Click the Hillshade

button in the menu panel. The Hillshade dialog asks you to specify the Azimuth, Altitude, and Z

Factor you wish to use to create the Hillshade. The default values are the defaults used by ESRI.

When you Generate the Hillshade, it will appear in the table of contents and map window and be

saved to the output directory in the folder of the user who produced it within the VisualChecks

folder. The symbology for the Hillshade cannot be changed, however you can turn its display on and

off using the table of contents tick box.


Page 41 of 78

To further assist error identification, it is possible to add other data, namely aerial photographs using

the Add Data button . The intention here is NOT to replicate ArcGIS. The add data button is

primarily intended for aerial photographs. Spatial Database Engine (SDE) is NOT supported. If vectors

are loaded their symbology cannot be changed. A workaround for these things is to use layer files.

You can also use the View Profile button to draw a profile on the DEM in the map window. It is

possible to draw a single straight line profile as shown below, or a multi-line profile with vertices.

Only the start point ‘A’ (the left of the profile graph), and end point ‘B’ (the right of the profile graph)

will be marked. Once the line is drawn, the profile graph appears at the bottom of the map window.

Units are in metres. The Hide button can be used to close the profile.

The progress of checked tiles can be viewed using the Minimap. The Minimap displays the tile index

in grey outline and colours the tiles (in a different colour for each user) when they have been

marked as checked. If you hover the mouse over tiles in the Minimap, you can see which user has

checked each tile. The Minimap can be printed to PDF using the Print button in the Minimap

window.


Page 42 of 78

To exit the Visual Checks screen there is a Finish QA button which moves you back to the Dashboard.

When you return to the Visual Checks, the user settings are retained. Hence if you have the same

user settings set, you will be taken back to the tile you were checking, the symbology will remain as

set, and any loaded data will persist. In order to see the results of the Visual Checks in the output tile

index, the user MUST generate the QA4LiDAR Report so that the tile index is updated.

4.11.4 Shortcut Keys

Table 3 outlines the shortcut keys that are available for the Visual Checks. They are intended to allow

the majority of the DEM checking to be done via the keyboard to make the process efficient.

Table 3. QA4LiDAR Visual Check shortcut keys

Action Shortcut Key

Tile

Nav

igat

ion

Previous tile f

Next tile g

Left tile ← arrow key

Above tile ↑ arrow key

Right tile → arrow key

Below tile ↓ arrow key

Change scale scroll mouse wheel

Erro

r M

ark

up

Mark as Checked spacebar

Mark as Unchecked spacebar

Open error dialog for NW quadrant 1

Open error dialog for NE Quadrant 2

Open error dialog for SW Quadrant 3

Open error dialog for SE quadrant 4

Escape from error dialog ESC

Relative vertical Accuracy error q

Classification error w

Interpolation error e

Systematic error r

Follow Up error t

Other error y

Small size error a

Medium size error s

Large size error d

Save error/s Enter


Page 43 of 78

4.12 OUTPUT: Report The QA4LiDAR QA Report can be accessed under the OUTPUT heading on the left menu panel. When

the Report is clicked, the messages “Updating tile index shapefile...” followed by “Generating Report

Data” and “Report Data Generated” appear while the report is generated and tile index updated

with the latest check results. The tile index will only be updated once for every QA Session run. The

Report can be generated in sections, after each check is successfully run. You can run one or multiple

checks at a time and see the results for those checks on the QA Report without having to run all

checks. The QA4LiDAR and ArcGIS versions are printed at the top of the report, along with the

project Contract Number and Project Title which are extracted from the Tender Form, and the

Project Directory. If no forms exist, the contract number and project title will be blank. A date and

time stamp is also printed at the top right of the report every time it is generated.

Before a check has completed, the Compliance for that check will state PENDING. The types of

compliance stated on the Report are explained at the top of the report. The user is able to change

the compliance found by QA4LiDAR by hovering the mouse over the particular compliance and using

the drop down box that appears. For example, a user may change compliance from FAIL to CPASS if

the data was very close to passing and is deemed acceptable. If a change is made, an asterisk will

appear next to the new compliance value meaning “* : The compliance value has been changed by

the user and is different to the value originally assigned by QA4LiDAR.” If a check has failed, the

element will be highlighted in red. The user can left click on the highlighted/failed element for more

information about the failure or to comment on the result e.g. why it was changed from FAIL to

CPASS. Left clicking on any non-red element will allow a comment to be made if required. The

detailed fail information is saved to a QA Errors PDF, while any comments are printed under the

relevant results table on the QA Report PDF. Changes to compliance and any comments added are

saved, unless the check is re-run, in which case any manual changes are overridden.


Page 44 of 78

By hovering over any entry in the left hand column of each table, a tool tip will be displayed which

gives a brief explanation of what each check does. For more information on how the checks are

performed, see section 9.1.

The sections of the Report are presented in the same order as the suggested running order of the

Automated Checks (as listed on the Automated Checks screen). The results in each section are

explained under the relevant headings below. If a user wishes to re-run a set of checks, they may

click the blue button directly beneath the check results which will return them

to the Automated Checks screen with the relevant checks ticked. All the user needs to do then is

click Run, and the checks will be re-run. Users should be aware that existing results for a check will

be overwritten if the check is re-run.

4.12.1 Presence & Reading

This section displays results for the Delivery Completeness & Spatial File Corruption and File

Naming, Shapefile Attributes & Horizontal Coordinate System checks. The Check column states the

check performed, the File Type column states the data format the check is applicable to where

possible, the Compliance column gives a PASS/FAIL statement, the Problem column states why the

element failed (it can only display one problem per check), and the Number of Failed Files column

states the number of failed files out of the total number checked. Any file types not applicable to the

project are removed from the results. The green button appears below the table

and links to the output location of the tile index shapefile. This is a copy of the original tile index with

many additional fields populated with QA results. If any row in the table is left clicked and comments

recorded, the comments appear below the table for the check and format and are colour coded to

match the compliance. Comments can be left for any row in any of the results tables.


Page 45 of 78

4.12.2 Forms Report

This section displays results for the Comparison of Report Form to Tender Form checks. The Form

Element column states the elements of the forms checked, the Compliance column gives a PASS/FAIL

statement based on whether the information in the two Forms match, the Specified column states

the value found on the Tender Form, the Reported column states the value found on the Report

Form, and the Additional Corrections column is only applicable to the Geoid Model. If there were no

Forms supplied to QA4LiDAR, these results will all be recorded as N/A. The section also displays a

PASS/FAIL for whether the supplier certified the form for the delivery.


Page 46 of 78

4.12.3 Classification Statistics

This section displays results for the Classification Statistics checks. There may be up to 4 tables in

this section depending on the data in your project. The main table consists of results for ORT LAS

(orthometric) data. The Required/Unwanted column states the classes that were highlighted green

(Required), red (Unwanted) or white (Ignored) on the Tender Form, the Compliance column gives a

PASS/FAIL statement based on the existence of these classes in the classified ORT LAS data, the ORT

Point Class column states the class number and description, the Point Count column gives the

number of points found in each class, the % Points column gives the percentage of points found in

each class out of the total, the Z Min column gives the minimum elevation value found in each class,

and the Z Max column gives the maximum elevation value found in each class. Ignored classes,

which are those not specifically required or unwanted in the dataset, are reported with ‘N/A’

compliance.

If your project contains ellipsoid LAS data there will be a second results table for this. It consists of

the same fields except the Point Class field is named Ellipsoid Point Class. The compliance column

gives a PASS/FAIL statement based on whether the Point Count for each class matches the Point

Count for the same class in the ORT LAS results.

If your project contains ellipsoid swath LAS files there will also be a results table for this data. It will

only have entries for each class that is present in the data. As there should only be class 0 points in

unclassified swath LAS data, class 0 will PASS and any other classes present will FAIL.

Finally, if there is more than one type of LAS file in your project, there will be a comparison table

showing the total point count for each LAS dataset and providing a PASS if the point counts match,

or a FAIL if the point counts differ. There should be the same number of points in each LAS dataset.


Page 47 of 78

4.12.4 Survey Control

This section displays results for the Accuracy of Survey Control checks. It is split into two tables. In

the first table, the Check column states the check performed, the Compliance column gives a

PASS/FAIL statement based on the minimum number of required points for ICSM Density (PASS/FAIL

results are not applicable for QA4LiDAR Distribution), the FVA Check Points Found (Minimum

Required) and Ground Control Points Found (Minimum Required) give the number of points found

followed by the minimum required in brackets (based on standard ICSM requirements), and the

Survey Control Rating column provides the QA4LiDAR rating given by the check.

*Note. The Pass/Fail result for the ICSM Density check are based on the ICSM minimum point

requirements, whereas the Control Density Rating is based on QA4LiDAR’s own rating system as

described in Appendix 1. As the ICSM minimum requirements are quite high, it is possible for the

density to FAIL with a strong rating. In such a case, the user may opt to conditionally pass the Survey

Control.

In the second table, the Delivery Element column states the type of control point, the Method

column states the method used to collect ground control as reported on the Report Form, and the

ORT Connection column states how the ORT connection was established as reported on the Report

Form. The user needs to decide whether these explanations are acceptable.


Page 48 of 78

4.12.5 Density / Resolution

This section displays results for the Point Density and DEM Resolution checks. It is split into three

tables, the first of which is for LAS point density results. The point type is stated in the Delivery

Element column, the Compliance column gives a PASS/FAIL statement based on a comparison of the

Required NPS column whose value is obtained from the Tender Form and the calculated Pseudo

Pulse Density column result (the average result for all flight lines). Also reported are the calculated

All Point Density and Ground Point Density and the Number of Failed Flight Lines (as the compliance

is for Pseudo Pulse Density which is calculated per flight line). The pseudo pulse density value for

each flightline can be found in the output flightline shapefile if an input shapefile was provided.

The second table is for DEM resolution results stated in the Delivery Element column, the

Compliance column gives a PASS/FAIL statement based on a comparison of the Required Resolution

column whose value is obtained from the Tender Form and the Found Resolution(s) column. The

number of Failed Tiles out of the total number of DEM tiles is also reported. If there are multiple

DEMs of different resolution in the delivery, the DEMs that don’t match the required resolution will

appear as FAILs. This can be adjusted to a CPASS.

The third table is for bathymetry coverage results. The point type is stated in the Delivery Element

column, the Compliance column gives a PASS/FAIL statement based on a minimum % of tiles having

a minimum number of soundings as requested on the tender form. Also reported are the Required

Minimum Soundings, the Required Minimum % of Tiles, and the % Results or percentage of tiles

having the minimum required number of soundings as found by QA4LiDAR.

Green buttons and appear below the tables when the

checks have been successfully run. When clicked, these links open the folder containing the output

ground point density raster and all point density raster. These output rasters are generated at the

cell size chosen on the Automated Checks set-up screen and therefore density values within the

rasters are NOT per square metre. They are per square cell value chosen (default and minimum is

2m). They give a good indication of the variation of density across the project and can be further

processed by the user outside of QA4LiDAR to achieve density per metre squared if desired. They

can also be used to find internal voids in the data.


Page 49 of 78

4.12.6 Flight Lines

This section displays results for the Flight Line checks. This check only applies to the classified ORT

LAS points stated in the Delivery Element column, the Compliance column gives a PASS/FAIL

statement based on a comparison of the Required Relative Vertical Accuracy value which comes

directly from the Tender Form and the Relative Vertical Accuracy Statistics value which is calculated

by the check. If there is <2% flight line overlap, the check will not run and the compliance will be

N/A.

The second table is for the scan angle results for classified LAS as stated in the Delivery Element

column. The Compliance column gives a PASS/FAIL statement based on a comparison of the

recorded scan angles within the LAS point data to the allowable Maximum Scan Angle value which

comes directly from the tender form. The Number of Failed Tiles out of the total is also given.

A green button appears below the table when the Flight Line Coverage

part of the check has been successfully run. When clicked, this link opens the folder containing the

output flight line coverage raster. The flight line coverage raster output is generated at 1m cell size

(based on the pseudo pulse density flight line rasters which are required per metre) so may appear a

bit pixellated however can be useful in highlighting internal voids in the data.

4.12.7 Vertical

This section displays results for the Absolute & Relative Vertical Accuracy checks. The Delivery

Element column states whether the check has been performed on the LAS or DEM, the Survey

Control column states which Survey Control has been used in the check, the Compliance column

gives a PASS/FAIL statement based on a comparison of the Required Absolute Vertical Accuracy

column value which comes directly from the Tender Form and the Absolute Vertical Accuracy Results

column value which is calculated by the check. The Acceptability Rating that the user selected for

each type of control on the Automated Checks set-up screen and an Acceptable FVA Check Points

Count (i.e. the number of FVA Check Points used in the absolute vertical accuracy check) are also

stated.

If supplemental check points were supplied, a similar results table exists for them. However, there

are no Compliance, Required Absolute Vertical Accuracy, Acceptability Rating, or Acceptable FVA

Check Points Count fields as these are not relevant. There is a new Land Cover Type column which

states the attribute from the class field i.e. Tree, Grass etc. that the calculated accuracy relates to.


Page 50 of 78

A green button appears below the table when the check has been successfully

run. When clicked, this link opens the folder containing the output FVA Check Points and Ground

Control shapefiles. These are copies of the original files with four additional fields;

Table 4. Survey control output shapefile fields added

Field Name Description

OPENNESS User acceptability rating for openness

FLATNESS User acceptability rating for flatness

LAS_DIFF The height difference between LAS and each control point in metres

DEM_DIFF The height difference between LAS/DEM and each control point in metres

*Note. If the user checked the accuracy of the survey control before acquisition (i.e. LAS data were

not present), the “OPENNESS” and “FLATNESS” fields are generated in the output control shapefiles

and all populated with the value “STRONG”. These values of strong are not accurate ratings; they are

simply for programming purposes so that all points are used in the Accuracy of Survey Control check.

4.12.8 Visual Checks (DEM)

This is the only section of the report for which the compliance and comments are not automatically

updated or re-set when visual checks are re-run. The section displays results for the Extent &

Horizontal Coordinate System, and Visual checks. The first two tables provide the results of the

visual extent checks; first whether the data extents were deemed valid and if not, the details of the

issue, and second if detailed DEM extents were produced, whether any internal voids are acceptable

or not. The third table is the results of the Visual DEM checks. The DEM Error Type column states the

error type found, the Compliance column states PASS unless tiles have been found with an error in

which case it will be FAIL, and the Number of Errors is also reported. You can click any FAIL lines to

see the tile name and quadrant with the error.


Page 51 of 78

4.12.9 Output Supporting Information

A number of supporting data files are generated by QA4LiDAR and saved to the Output Folder

chosen by the user. After the data has been generated, the links on the QA Report turn green and

when clicked, open the folder location of the data. Links also appear on the Dashboard. The user can

then open the data in a GIS. Within the output folder chosen, QA4LiDAR generates a folder with the

same name as the project. Within this project output folder are the following;

Logs folder

QA4LiDAR.Checks folder

QA4LiDAR.References folder

Screenshots

Visual Checks folder

Tile Index shapefile

The ‘Logs’ folder contains the text log files automatically saved when automated checks are run. The

files are named with the date, time and project name. One log file is saved per QA Session run.

The ‘QA4LiDAR.Checks’ folder contains a number of spatial output files generated by the checks

including; survey control points with additional fields added by QA4LiDAR as part of the flatness and

openness check (in the ‘PointStatistics’ folder and ‘AbsoluteVerticalAccuracy’ folder), some of the

datasets created in the Survey Control check (convex_hull, difference, extent_dissolved, and

extent_mean_center in the ‘ControlDistributionStatistics’ folder), the flight line coverage raster

(‘FlightLineCoverage’ folder), the point density rasters and pseudo pulse flightline results

(‘PointDensity’ folder), the bathymetry coverage results (‘BathymetryRaster’ folder) with the results

field being “GRIDCODE” in the shapefile, and the flight line area polygon (‘RelativeVerticalAccuracy’

folder). Not all processing dataset are output to the user as some are created in memory.

The ‘QA4LiDAR.References’ folder contains the flight line rasters produced by the Point Density

check which are used by the Flight Line Coverage check.

The ‘Screenshots’ folder stores the PNG screenshots captured in the Visual Checks part of the

software. Images are stored named by user, date and time.


Page 52 of 78

The ‘VisualCheck’ folder stores the extents, DEM mosaic dataset and user related information used

by the Visual Checks so they do not have to be regenerated.

The copy of the ‘Tile index’ is saved directly to the Output Folder with the original tile index name.

The fields in the output tile index include;

Table 5. Tile Index output shapefile fields added

Field Name Description

CQTILENAME A QA4LiDAR generated tile name field (from the tile index polygon locations)

map_code The code used to display the colours in the QA map – refer to the map legend

corrupt The per tile Pass/Fail result for the corruption check

naming The per tile result for the naming check with any dataset that failed listed

hcs The per tile result for the HCS check with any dataset that failed listed

las_header The per tile Pass/Fail result for the LAS header check

scan_angle The per tile Pass/Fail result for the scan angle check

den_all_pt The per tile results for all point density

den_grd_pt The per tile results for ground point density

cls_x_pts The number of points in the class “x” within the tile, only classes present in the dataset are represented

cls_x_min The minimum Z value in the class “x” within the tile, only classes present in the dataset are represented

cls_x_max The maximum Z value in the class “x” within the tile, only classes present in the dataset are represented

set_lasahd If AHD LAS were requested, whether each tile for that dataset was Supplied, Multiple Supplied, Not Supplied or the check remains Pending/Not Supplied*

set_lasell If ELL LAS were requested, whether each tile for that dataset was Supplied, Multiple Supplied, Not Supplied or the check remains Pending/Not Supplied*

set_dem If DEM was requested, whether each tile for that dataset was Supplied, Multiple Supplied, Not Supplied or the check remains Pending/Not Supplied*

set_dsm If DSM was requested, whether each tile for that dataset was Supplied, Multiple Supplied, Not Supplied or the check remains Pending/Not Supplied*

set_rgb If Imagery was requested, whether each tile for that dataset was Supplied, Multiple Supplied, Not Supplied or the check remains Pending/Not Supplied*

set_int_a If all return intensity imagery was requested, whether each tile for that dataset was Supplied, Multiple Supplied, Not Supplied or the check remains Pending/Not Supplied*

set_int_f If first return intensity imagery was requested, whether each tile for that dataset was Supplied, Multiple Supplied, Not Supplied or the check remains Pending/Not Supplied*

set_int_l If last return intensity imagery was requested, whether each tile for that dataset was Supplied, Multiple Supplied, Not Supplied or the check remains Pending/Not Supplied*

err_q1 The per tile quadrant 1 results for the Visual Checks**




reviewer The user name of the person who checked the tile with an error


Page 53 of 78

*There is no easy way of telling whether 0 results is due to a pending check or 0 files existing, hence

"Pending/Not Supplied". If there is at least 1 file present for a certain delivery element, all

subsequent tiles will correctly be reported as "Not Supplied".

**A cell is attributed with comma separated numbers that represent the error types, plus a letter for

error size e.g. err_q1 = “2,5,M” means quadrant 1 of that tile has classification and follow up errors

of medium size. The codes are as follows;

o 1 = Relative Vertical Accuracy error type

o 2 = Classification error type

o 3 = Interpolation error type

o 4 = Systematic error type

o 5 = Follow Up error type

o 6 = Other error type

o S = Small size error

o M = Medium size error

o L = Large size error

4.12.10 Printing the QA Report

When all necessary results are displayed on the QA Report and any applicable compliances changes

have been made, the QA Report can be printed to PDF. It is possible to hide sections of the QA

Report that may not be applicable to the project. To hide a section left click on the blue heading bar.

Any hidden sections will not appear in the printed PDF. To Print the QA Report go to the File menu

Print Report Summary. A Save As dialog will appear for you to choose the output directory and file

name. Once the PDF has been created, you can open it in Adobe to print a hard copy. The PDF lists

the sections of the report as bookmarks in the left panel for easy navigation.

It is also possible to print a detailed list of all the QA errors/fail results. To do so go to the File menu

Print QA Errors.


Page 54 of 78

4.13 OUTPUT: Map The QA results are also represented visually in the QA Map. It highlights whether tiles were delivered

and passed or failed automated and visual checks, providing a quick and basic visual review. If you

require a more detailed view of the QA errors results as they apply to the tile index, please add the

tile index to ArcMap (or a GIS) and review it there. Tiles in the QA4LiDAR QA Map are coloured

according to the below flow chart which can also be found via the software Help menu under Map

Legend.


Page 55 of 78

5 Common Tasks

5.1 Approving the survey control design The ICSM Template states that “The proposed check point survey design must be submitted with the

quotation, and approved by the Contract Authority prior to implementation.” The Contract Authority

can use QA4LiDAR for this approval if they obtain shapefiles of the proposed Survey Control (Ground

Control and FVA Check Points) and run the Accuracy of Survey Control check in the planning stage of

the project.

To do so;

1. Create a pseudo project folder to scan into QA4LiDAR containing the project extent

shapefile, the ground control point shapefile and the FVA check point shapefile.

2. In QA4LiDAR create a New Project and choose the pseudo project folder you created to scan

in.

3. On the Project Settings screen, provide the Project Extent shapefile, an Output Folder and a

Working Directory. Remaining elements can be left blank.

4. Proceed to the Automated Checks screen and supply the FVA Checkpoints shapefile and

Ground Control shapefile. There is no requirement to provide elevation fields or

acceptability ratings for the control files for this task.

5. Tick only the Accuracy of Survey Control check group and select Run.

6. When the check has run, go to the Report screen and scroll down to the Survey Control

section. Hide all other areas on the report and print to PDF.

*Note. The Pass/Fail result for the ICSM Density check are based on the ICSM minimum point

requirements, whereas the Control Density Rating is based on QA4LiDARs own rating system as

described in Appendix 1. As the ICSM minimum requirements are quite high, it is possible for the

density to FAIL with a strong rating. In such a case, the user may opt to conditionally pass the Survey

Control.

**Note. The “OPENNESS” and “FLATNESS” fields generated in the output control shapefiles in this

case are all populated with the value “STRONG” which is not an accurate rating as there are no LAS

files within the project. It is simply done for programming purposes so that all points are used in the

Accuracy of Survey Control check.


Page 56 of 78

6 Troubleshooting To prevent errors occurring while running QA4LiDAR, please follow the requirements and

recommendations outlined in section 0. If an error does occur, the error message and information

provided in the QA Session log file and QA Report will help identify the problem. Some investigation

of the project data causing the issue may be required. If no problem with the data can be identified,

try exiting and restarting QA4LiDAR and your computer before running the check again. Ensure

project files are not open in other software, and close any instances of ArcGIS open on the computer

while QA4LiDAR is running. You may also try rescanning your project directory or renaming/deleting

the ‘QA4LiDAR Project Database.cqp’ and clearing the output folders and starting again. If the error

still occurs, contact the CRCSI to report the problem. Any updates to QA4LiDAR and the QA4LiDAR

Form Editor will need to be supplied by the CRCSI. These can be downloaded and installed as

required and will provide bug fixes and new functionality.

Things to watch out for when running QA4LiDAR;

Ensure the Requirements and Recommendations for using QA4LiDAR as outlined in section 0

have been followed.

Ensure the computer in use has continuous access to the ArcGIS license and extensions.

Ensure the computer in use is not set to hibernate after a period of time or overnight while

QA4LiDAR is processing.

Ensure project datasets are tiled as per the ICSM specification i.e. origins that align with the

zero origin of the MGA Zone, on a whole metre coordinate value of the south west corner of

each tile.

The ESRI error type of "Error accessing tool ....." (as seen in the QA4LiDAR log file), is due to

ArcGIS not releasing a Geoprocessing tool correctly. If a check runs into this error,

subsequent checks within the same QA Session are likely to encounter the same error.

o It can be overcome by closing QA4LiDAR, re-opening the project, and re-running the

check.

The “Unexplained ESRI operating system error” (as seen in the QA4LiDAR log file which is

ESRI error 99999) is an operating system error or generic ESRI error for which the cause is

unknown.

o It can be overcome by closing QA4LiDAR, re-opening the project, and re-running the

check.

Ensure large datasets are split into manageable chunks e.g. of ~2,000 x 1km tiles.

*Note. An ESRI ArcObjects bug has been found with ArcGIS version 10.2.0.3348 (no service

pack) in which ecw files are deemed corrupt despite being able to be opened and viewed fine

in ArcMap* To fix, please update ArcMap to 10.2.1.


Page 57 of 78

7 Future Improvements Future improvements may be made to QA4LiDAR by implementing additional checks to extend the

capability of the software and by altering programming methods to improve efficiency. In addition, if

changes are made to the ICSM Template, QA4LiDAR should be adjusted in line with these changes.

For example, changes to the way in which point density should be measured, or changes to the

understanding of the use of overlap points etc. It may also become relevant for QA4LiDAR to be

applicable in New Zealand and hence incorporate the differences in their acquisition template.

8 Glossary

Term Description Example

Absolute Vertical Accuracy

Also known as Fundamental Vertical Accuracy is the vertical accuracy in open terrain tested to 95% confidence (normally distributed error) of the elevation data when compared to survey control points (FVA Check Points and Ground Control).

<= +/- 30cm. 95% confidence interval (1.96 x RMSE)

Aerial Imagery Or Aerial Photography can be coincident or non-coincident orthorectified imagery in 3 (RGB) or 4 (RGB + infrared) bands.

Bilinear Interpolation

Linear interpolation performed in two directions (e.g. X & Y) so as to be a distance weighted average of the four nearest values.

Classification Level

LiDAR point classification Levels 0 to 4 - refer to pages 10-11 of the ICSM LiDAR Acquisition Specifications and Tender Template Version 1.0, November 2010.

Level 0-4

Contours Vector representation of topography/bathymetry where continuous curved lines represent constant ground height values above a certain datum e.g. AHD.

Control Density The number of FVA Check Points and Ground Control in

the project area. QA4LiDAR uses minimum number requirements as well as a rating determined by the example formula (e=density, n=number pts, a=area).

𝑒 =𝑛

𝑎

Control Distribution

The spread of FVA Check Points and Ground Control across the survey area. QA4LiDAR uses a three part rating system (see Appendix 1).

Surv

ey

Co

ntr

ol

Ground Control (GC)

High accuracy GC (e.g. state benchmarks) are used to establish the datum in the survey area. They can be internal or external to the project and assess the variation of the reference surface and/or geoid model across the survey area.


Page 58 of 78

FVA Check Points (CPs)

FVA CPs are used to assess the vertical accuracy of the survey. They must be gathered internal to the project area and are often collected in clusters on open, flat ground for easy comparison to LiDAR ground points.

Digital Elevation Model (DEM)

A raster representation of the topography/bathymetry where cell values represent ground heights above a certain datum e.g. AHD.

Digital Surface Model (DSM)

A raster representation of the earth’s surface including objects such as trees and buildings where cell values represent object heights above a certain datum e.g. AHD.

Ellipsoid Or reference ellipsoid is a mathematically defined

surface that provides a simplified approximation of the geoid for coordinate system definition.

GRS80

Environmental Conditions

There may be certain environmental conditions imposed for a LiDAR data capture such as;

Cloud and fog free between the aircraft and ground

Floodplain/wetland data must be captured during times of base-flow and outside of significant surface inundation due to natural events and /or regulated environmental flows

Coastal surveys (areas under tidal influence) should be flown within 2 hours either side of low tide to minimise the effect of standing water or wave action

Flights should not be undertaken during periods of heavy smoke haze

Flatness How flat the area around each control point internal to the project area is. QA4LiDAR uses the heights of surrounding LiDAR ground points to determine this.

Flight Line Coverage

How well the survey area is covered by the flight lines. There should not be any gaps between flight lines due to the aircrafts flight path.

Flight Trajectory Or flightlines describes the aircraft flight path, usually represented by a line shapefile.

Geoid Model The geoid is an equipotential surface (surface to which

gravity is always perpendicular) that coincides with mean sea level (if at rest). A geoid model is an irregular 3D representation of the geoid which defines zero elevation and is used as a surface from which to measure elevations.

AUSGeoid09

Horizontal Coordinate System

A system which allows determination of the horizontal position of a point on earth.

GDA94


Page 59 of 78

Intensity Imagery An image created from the LiDAR intensity values i.e. the return strength of the laser pulse for every point, which looks like black and white aerial photography. It is based in part on the reflectivity of the object struck and is a substitute for aerial imagery when none is available.

LAS The common LiDAR data exchange file format – refer to the ASPRS format specifications.

http://www.asprs.org/Committee-General/LASer-LAS-File-Format-Exchange-Activities.html

LAS Dataset A LAS Datasets is an ESRI/ArcGIS format that stores reference to one or more LAS files on disk.

LiDAR Light Detection and Ranging remote sensing technology that measures distance using a laser and produces a point cloud of elevation data.

National Elevation Data Framework (NEDF) naming convention

Developed to provide easy ingestion into the NEDF-Portal – refer to pages 23-29 of the ICSM LiDAR Acquisition Specifications and Tender Template Version 1.0, November 2010.

http://www.icsm.gov.au/elevation/LiDAR_Specifications_and_Tender_Template.pdf

Nominal Pulse Spacing (NPS)

The target number of outbound LiDAR pulses over a given area set prior to data collection. As unsuccessful pulses can’t be measured, this is simulated in QA4LiDAR using last return and excluding data gaps to get a measure of ‘Pseudo Pulse Density’.

2

Openness How open the area around each control point internal to the project area is. QA4LiDAR uses the classifications of surrounding LiDAR points to determine this.

Orthometric (ORT)

The curved-line distance following the earth’s gravity field from the geoid to the point of interest. Orthometric heights can be used to predict and measure direction and rate of fluid flow.

AHD (normal-orthometric)

Po

int

De

nsi

ty

All Point Density

The number of successful ground and non-ground point returns (1st, 2nd, 3rd AND last return) over a set area (e.g. more points returned in vegetated areas due to the presence of 2nd & 3rd returns).

2.42 pts/m2

Ground Point Density

The number of successful ground point returns (1st, 2nd, 3rd OR last return) over a set area, which equates to removing all non-ground points from the point density (e.g. a typical ground point density required to generate a DEM is 2 points per square metre).

1.92 pts/m2

First (or last) return point density

The number of successful 1st (or last) returns over a set area which could be ground or non-ground (e.g. only by examining first or last return or pulse density will you find areas of greater density in a project).

2.04 pts/m2

Points at The number of successful ground and non-ground point 2.87 pts/m2












Page 60 of 78

Nadir returns (1st, 2nd, 3rd AND last return) over a set area at nadir (middle 10% of swath width).

Pulse density / Pseudo Pulse Density

The number of outbound pulses (not necessarily successful returns) over a set area. This is simulated in QA4LiDAR using last return and excluding data gaps to get a measure of ‘Pseudo Pulse Density’.

2.04 pts/m2

Relative Vertical Accuracy

The accuracy of LiDAR data between flight lines i.e. how well the flight lines align with each other.


Resolution The grid cell size of the DEM generated from LiDAR. 1m

Root Mean Square (RMSE)

A measure of the differences between values predicted by a model or an estimator and the values actually observed.

RMSEz = √[∑(zdata i – z check i)2 /n]

Scan Angle The maximum scan angle or Field of View (FOV) is the angular extent measured in degrees of the view surveyed by the sensor.

40°

Supplemental Vertical Accuracy

Absolute vertical accuracy achieved within land cover categories outside of bare open ground. Land cover categories specified in SVA Check Points shapefile.


Tile Index A polygon shapefile based on standard state indexes with an origin that aligns with the zero origin of the MGA Zone. The index defines how the LiDAR data is cut into tiles and supplied. The origin of the tile must be placed on a whole metre coordinate value of the south west corner of each tile. The tile name must be included as an attribute in the Tile Index file.

1km x 1km tiles based on MGA coordinates

Triangulated Irregular Network (TIN)

A vector based representation of a surface made up of irregularly distributed nodes and lines with three-dimensional coordinates (x, y, and z) that are arranged in a network of non-overlapping triangles.

http://www.icsm.gov.au/elevation/ICSM-GuidelinesDigitalElevationDataV1.pdf

http://www.asprs.org/a/society/committees/lidar/Downloads/Vertical_Accuracy_Reporting_for_Lid

ar_Data.pdf

http://www.icsm.gov.au/elevation/ICSM-GuidelinesDigitalElevationDataV1.pdf

http://www.asprs.org/a/society/committees/lidar/Downloads/Vertical_Accuracy_Reporting_for_Lidar_Data.pdf

http://www.asprs.org/a/society/committees/lidar/Downloads/Vertical_Accuracy_Reporting_for_Lidar_Data.pdf


Page 61 of 78

9 Appendices

9.1 Appendix 1 – How the Automated Checks are performed This Appendix provides details of the methods used by QA4LiDAR to perform the automated

compliance and QA checks. QA4LiDAR is coded primarily in C# and uses ArcGIS (ArcObjects and

geoprocessing tools etc.) and open source lastools.

9.1.1 Project scan in

During project scan in, QA4LiDAR identifies the files that have been delivered by first identifying the

file extensions and searching the file names using key term rules (see below example and Table 6). If

the files cannot be identified from extension and file name, the search is extended progressively up

the folder structure until the full directory path name is searched if necessary. This uses the

prerequisite that tiled datasets must have their tile name within the file name (see Table 7 for rules

used). As files are identified, they are added to the QA4LiDAR Project Database (.cqp). If a file cannot

be identified, it is marked as ‘unassigned’. If duplicate datasets are found a dialog pops up to warn

the user and instruct them to remove one of the duplicates, then try continuing the check or abort.

File identification example File path 1 e.g.: C:\Dataset\LAS\C1\e123n4567.las Using the set of key term rules in the table below, as the file's extension is ".las" QA4LiDAR will use

the LAS extension based rules to check the file path. First it will check for a match with

classified/unclassified LAS by checking if the file path contains any of the classified/unclassified LAS

rules. As the file path contains "C1", this is a match and the "Classified LAS" data type is assigned to

the file. The rules are not case sensitive. As there is no datum reference in the path it will be deemed

an ellipsoid LAS file as per the rule set.

File path 2 e.g.: C:\Dataset\LAS\e123n4567.las No match would be found in the LAS rule set, as there is no key term for classified/unclassified, so no data type can be assigned to this LAS file. Table 6. Key term search rules (not case sensitive)

Data Type Key Term Rules File name or path contains term (not case sensitive) using any of the rules:

"\xxx\", "-xxx", "_xxx", "-xxx_", "_xxx-", "xxx_" LAS (.las) Classified: "cl”, "c1", "c2", "c3", "c4" or "class"

OR Unclassified: "unc" or "raw"

(AND) *Type: "mkp”

AND *Tidal Datum: "lat", "mlw", "mhw" or "hat"

OR ^Orthometric Datum: "ort", "ahd", "msl", "nzv", New Zealand local vertical datums; “otp”, “akl”, “mot”, “gis”, “nap”, “tar”, “wel”, “nel”, “lyt”, “dun”, “dbl”, “blu” or “sti”

OR Ellipsoid: anything that is NOT one of the other datum options


Page 62 of 78

*Datasets found with these key terms are ignored in all but the first two check groups. ^Note that only a single orthometric datum is permitted within a project.

ESRI Grid (.esrigrid fake extension given to ESRI Grid folder for QA4LiDAR)

Type: “dem”, “bat” (bathymetry), “mix” (mixed bathy/terrain) or *“dsm” OR

*Intensity Imagery: “int” or “dim” AND

Return Type: “first”, “last” or anything that is NOT one of the first or last return options is deemed all returns



(AND) *Mosaic: “mosaic” (if not a mosaic refer to Table 7 for tile rules)


ESRI ASCII (.asc)

Type: “dem”, “bat” (bathymetry), “mix” (mixed bathy/terrain) or *“dsm” OR







ECW (.ecw)

*Aerial Imagery: “rgb” OR




GeoTIFF (.tiff) or TIF (.tif)

*Aerial Imagery: “rgb” OR



Page 63 of 78


Shapefile (.shp)

Contours: "contour" Flight Trajectory: "traject", “flightline” or “flight_line” Tile Index: “tileindex”, “tilelayout”, “tile index”, “tile layout”, “tile_index”, “tile_layout”, “tile-index” or “tile-layout” Survey Control: "control"

MapInfo TAB (.tab) Contours: "contour" Flight Trajectory: "traject", “flightline” or “flight_line” Tile Index: "tileindex", “tilelayout”, “tile index”, “tile layout”, “tile_index”, “tile_layout”, “tile-index” or “tile-layout”

ESRI Geodatabase (.gdb)

Contours: "contour" Flight Trajectory: "traject", “flightline” or “flight_line” Tile Index: "tileindex", “tilelayout”, “tile index”, “tile layout”, “tile_index”, “tile_layout”, “tile-index” or “tile-layout”

Microsoft Excel (.xls)

Report: "report" Tidal Data: "tide" or "tidal"

Microsoft Excel (.xlsx)

Report: "report" Tidal Data: "tide" or "tidal"

Adobe PDF (.pdf)

Report: "report" Metadata: "metadata"

Microsoft Word (.doc)


Microsoft Word (.docx)


Extensible Markup Language (.xml)

Metadata: "metadata”

Comma Separated Values (.csv)

Survey Control: "control"

Table 7. Tile name in file name search rules

Data Type Tile Name Search Rules File name contains one of

All tiled datasets i.e. LAS, DEM etc

Rule Example

e<EEE>n<NNNN> e342n5820

<EEE>-<NNNN> 342-5820

_<EEE><NNNN>_ _3425820_

<EEE><NNNN> 3425820

<EEE>_<NNNN> 342_5820

e<EEEEEE>n<NNNNNNN> e342000n5820000

<EEEEEE>-<NNNNNNN> 342000-5820000

_<EEEEEE><NNNNNNN>_ _3420005820000_

<EEEEEE><NNNNNNN> 3420005820000

<EEEEEE>_<NNNNNNN> 342000_5820000


Page 64 of 78

9.1.2 Delivery Completeness & Spatial File Corruption

The checks in this group programmatically ensure that spatial data are not corrupt and that the

delivery is complete i.e. all files requested have been delivered, the tile index coordinate origin is

placed on a whole metre coordinate value that aligns with the zero origin of the MGA Zone, all tiles

for tiled datasets are present (LAS are checked in this group and other tiled datasets and their

mosaics in the next check group as missing LAS can halt the QA), if swath data was requested there

is a swath LAS for every flight line in the shapefile, and if waveform LiDAR was requested checks

there is WDP file for every waveform LAS. This also checks that the required elements on the Tender

Form are selected on the Report Form. A failure in any one of these checks will halt QA4LiDAR and

require action by the user before the automated checks proceed. This is to ensure time is not wasted

running checks over data that is corrupt or incomplete. When checking all tiles for tiled datasets are

present, only missing tiles in classified LAS datasets will halt the process as there may be legitimate

reasons for missing DEM etc tiles e.g. in coastal areas, hence data types other than LAS are part of

the second check group.

QA4LiDAR uses the Tender Form to identify all the required deliverables. It then checks these

requirements against the data identified in the scan in process. For tiled datasets it uses the tile

names within the file names to match each file to a tile in the tile index.

Each different spatial data file type has a different corruption test generally involving checking the

header information matches the specification for the file and/or data within it. There are six parts to

the LAS files corruption check;

1. File readability (core)

a. The lasinfo tool is used to check readability and if a file is corrupt, the message will

read “Unable to open LAS file {file path}, it is corrupt”. In this case the user should

obtain new readable versions of these files before running other checks or they will

crash.

2. Number of points is not zero (core)

a. The lasinfo tool is used to check that the number of point records is not zero and if

there are zero points, the message will read “Error: {file path} contains 0 point

records. The file is considered corrupt”. In this case the user should obtain new

readable versions of these files before running other checks or they will crash.

3. Number of points in flightline is less than 100 (core)

a. The lasinfo tool is used to check if the number of point records in a flightline is less

than 100 and if it is, the message will read “Error: Flight line {PSID} contains less than

100 point records ({x} points). All files with this point source ID are considered

corrupt.”. In this case the user should confirm with the provider that the flightline is

valid and if not have data re-delivered, as the flightline has very few points and will

likely cause checks to crash.

4. Number of points is less than 100 (warning)

a. The lasinfo tool is used to check if the number of point records in a file or for a PSID

is less than 100 and if it is, the message will read “Warning: {file path} contains less

than 100 point records. The file is considered corrupt”. In this case the user should

consider removing such files from the delivery as they are suspected to cause issues

running other checks.


Page 65 of 78

5. File size (warning)

a. This part simply provides a warnings to the user if the file size is greater than 2GB

(ICSM specification requires LAS files <2GB). If this warning is given, the files can still

be used in remaining checks.

6. Point source ID (warning)

a. This part simply provides a warning to the user if the number of point source ID’s

per file exceeds QA4LiDARs expected maximum. The maximum expected number of

point source ID’s is 10 times the tile size in km, which is to ensure these numbers

represent flight lines. If these warnings are given, the files can still be used in

remaining checks however the files should be investigated to ensure PSID does

represent flight lines.

Shapefile corruption checks for the 4 essential files (shp, shx, dbf and prj) as well as a geometry

check. The ASCII corruption check compares the number of rows and columns specified in the

header, to the actual number in the data and if these match the data pass the corruption check. ESRI

Grid, ECW and TIFF files are opened in ArcGIS and if the raster properties cell size X or Y value can be

read successfully the files pass the corruption check.

*Note. An ESRI ArcObjects bug has been found with ArcGIS version 10.2.0.3348 (no service pack) in

which ecw files are deemed corrupt despite being able to be opened and viewed fine in ArcMap* To

fix, please update ArcMap to 10.2.1.

9.1.3 File Naming, Shapefile Attributes & Horizontal Coordinate System

The checks in this group programmatically ensure that all tiles for other tiled datasets (not LAS) and

any mosaics are present, that the file naming conventions and file formats are as specified, that the

attributes included in shapefiles are as specified, that the LAS headers are valid, that the definitions

for horizontal coordinate system in the data match the Tender Form, that the tile size used matches

the size requested on the Tender Form, and that the point source ID (PSID) for all points in each

swath is valid if swath LAS were delivered.

If one of the NEDF file naming conventions was specified in the Tender Form, file naming is checked against this specification (as defined by the relevant ICSM Template) by running files against a regular expression of the relevant convention i.e. using required characters, position of characters, order of characters etc.

File naming convention example The Australian NEDF naming convention for classified LAS point clouds (as per the Aus ICSM Template) is: ProjectNameYYYY-CL-DAT_xxxyyyy_zz_wwww_hhhh.las

The equivalent regular expression used is: \b[A-Za-z0-9]+[0-9]{4}-C[0-9]-(ELL|AHD)_[0-9]{7}_[0-9]{2}_[0-9]{4}_[0-9]{4}

Spatial files with required attributes as per the Tender Form are checked for these attribute fields.

The lasvalidate tool is used to validate the LAS header information for each LAS file (this includes

whether coordinate reference system information is present). The horizontal coordinate system

definition in spatial files is compared to that required on the Tender Form. For files such as LAS that

do not tend to have the horizontal coordinate system defined within the file or an associated

projection file (.prj), the file name and path name are checked for key terms such as “GDA”,


Page 66 of 78

“MGA55” or just an MGA zone number such as “55” if the NEDF naming convention has been used.

The horizontal coordinate system can also be visually checked at a later stage as part of the Extent

Check. The tile size of the polygons in the tile index is calculated and checked against the requested

size. The lasinfo tool is used to check that that there is only one valid (non 0) point source ID (PSID)

for all points in each swath if swath LAS were delivered.

9.1.4 Comparison of Report Form to Tender Form

This check compares the equivalent information on the Tender and Report Forms to ensure what has

been reported by the provider matches the specification. The elements checked include;

Report Form certified

Form Required Elements (i.e. the number of requested datasets matches)

Horizontal coordinate system

Environmental conditions (yes/no)

Absolute vertical accuracy

Relative vertical accuracy

Maximum scan angle

Geoid model

Classification level

Vertical Reference system matches

Minimum Bathymetry Coverage

Minimum Bathymetry Soundings

9.1.5 Classification Statistics

This check uses the ‘lasinfo’ tool. It first gets the list of required and must not have classes from the

Tender Form. It then checks the classes contained in each ORT LAS file and determines whether each

file passes or fails based on the list of required classes. Using the ArcGIS Point File Information tool,

the Z minimum and Z maximum are recorded for each class in each LAS file. The project wide

maximum and minimum for each class are then determined and reported. The number of points per

class and the percentage of points per class are also calculated and reported. If ellipsoid LAS are

present, the check is repeated for that dataset however the Pass/Fail criteria is based on whether

the Point Count for each ellipsoid class matches the Point Count for the same class in the ORT LAS

results. The lasinfo tool is also run on unclassified swath LAS, if they exist, to check the classes are all

0. If there is more than one type of LAS file in the project, the total point count between classified

(orthometric and ellipsoid) and unclassified data is compared returning a PASS if the point counts

match, or a FAIL if the point counts differ.


Page 67 of 78

9.1.6 Accuracy of Survey Control

This check ensures that the control points used by the LiDAR provider were collected to a minimum

standard. This check can be run in the planning stages of a project to test the contractors plan for

the control survey, if this is done the flatness and openness part is NOT run. If the check is run after

data supply when there are LAS files in the project folder, the flatness and openness part IS run. The

control should have their datum established independently of the LiDAR dataset. There are four

Survey Control point types that can be input as shapefiles to QA4LiDAR;

1. Provider FVA Check Points (FVA CPs)

o These are part of the Independent Check Point network and are used to assess the

fundamental vertical accuracy of the survey. They must be gathered internal to the

project area and are often collected in clusters on open, flat ground where there is a

very high probability the sensor will have detected the ground surface, for easy

comparison to LiDAR ground points. In QA4LiDAR they are used as part of the Survey

Control Collection Method, Survey Control Density checks, the Survey Control

Distribution check, as well as the Flatness, Openness & Absolute Vertical Accuracy

checks.

2. Provider Ground Control (GC)

o Along with Base Station data, GC make up what is termed the Control Network. GC

are high accuracy points (e.g. state benchmarks) that are used by the provider to

establish the datum in the survey area and adjust the LiDAR horizontally or

vertically. They can be internal or external to the project and assess the variation of

the reference surface and/or geoid model across the survey area. In QA4LiDAR they

are used for the Survey Control Collection Method, Survey Control Density checks,

the Survey Control Distribution check, as well as for Flatness, Openness & Absolute

Vertical Accuracy checks.

3. Custom (Purchaser control)

o This can be any other control (or shapefile elevation data of known accuracy) that

the purchaser has access to. It will be used by QA4LiDAR for the Survey Control

Density checks, the Survey Control Distribution check, and the Flatness, Openness &

Absolute Vertical Accuracy checks. It should overlap with the survey area.

4. Provider SVA Check Points (SVA CPs)

o These are part of the Independent Check Point network and are used to assess the

supplemental vertical accuracy of the survey. They must be gathered internal to the

project area and are located in a range of different terrain types. In QA4LiDAR they

are only used as part of the supplemental Absolute Vertical Accuracy check.

*Note. The Survey Control Accuracy and Absolute Vertical Accuracy Checks can be run with just FVA

Check Points, with both FVA Check Points and Ground Control, with just custom (purchaser) control,

or with all three of these control. If all three are supplied the checks will run using each type of

control against each type of dataset.

Before testing the Accuracy of Survey Control, if LAS data has already been collected and is present

in the project folder, QA4LiDAR determines how open and flat the area surrounding each control

point is, and hence its suitability for use in the network. This is done for all control points despite

clustering, using ArcGIS methods. However, if LAS data are not present (i.e. you are checking the


Page 68 of 78

accuracy of the survey control before acquisition), “OPENNESS” and “FLATNESS” fields are generated

in the output control shapefiles and all populated with the value “STRONG”. These values of strong

are not accurate ratings; they are simply for programming purposes so that all points are used in the

Accuracy of Survey Control check.

1. The ‘openness’ of the area around each control point internal to a convex hull minimum

bounding geometry of the project extent, is rated.

a) This is done by determining the percentage of non-ground classified (i.e. vegetation or

building) LiDAR points (out of all LiDAR points) within a 10m radius from the control

point. Ratings are based on the table below.

b) Also, the classification of non-ground points are checked and if there are any class 5

(high vegetation) or class 6 (buildings) within the 10m radius, that control point cannot

be rated better than ‘Weak’.

c) If LiDAR points are only classified as ground (class 2) and unclassified (class 1) a rating

will still be given however QA4LiDAR will not be able to take into account whether there

are class 5 and 6 points.

Table 8. Openness rating scheme

Openness Rating % of non-ground points within 10m of control point

Poor >50

Weak 25 – 50

Average 15 – 25

Good 5 – 15

Strong <5

2. The ‘flatness’ of the area around each control point internal to a convex hull minimum bounding

geometry of the project extent, is rated.

a) The rating uses only the LiDAR points classified as ground within a certain radius of the

control point.

b) QA4LiDAR first counts the LAS points within a 1m radius of the control point, if there are

less than 6 points the radius is increased by 25% and it searches again. This continues

until at least 6 points are found.

c) QA4LiDAR then determines the height differences of these ground points to the control

point and averages these as absolute values. Flatness is rated for each control point

according to the table below.

Table 9. Flatness rating scheme

Flatness Rating Average Height difference of ground points to control

point (cm)

Poor >8

Weak 6-8

Average 4-6

Good 24

Strong <2


Page 69 of 78

3. If the ‘flatness’ or ‘openness’ rating is unacceptable for a control point, that control point is not

used in the Accuracy of Survey Control check. Whether a control point is unacceptable is up to

the user. The user specifies a rating for each type of control in the Survey Control section of the

Automated checks setup screen. Both flatness and openness for a control point must be rated

equal to or better than the user’s acceptability rating for the point to be used. The lower

the acceptability rating selected by the user (i.e. closer to 'Poor'), the more control points that

will be used for that type of control. These ‘flatness’ and ‘openness’ ratings are recorded in the

output copies of the control data so the user can investigate the results. If a point falls outside

the area of the LAS data it is given ‘flatness’ and ‘openness’ ratings of poor.

There are then three aspects to this check;

1. Collection Method

a) The QA4LiDAR Report Form is checked for the following information which is supplied to the

user (the user requires some knowledge of control surveys to determine whether the

methods and explanations provided denote adequate survey technique);

i. Method used to collect GC

ii. Explanation of GCP connection to orthometric datum

iii. Method used to collect FVA CPs

iv. Explanation of FVA CP connection to orthometric datum

v. Method used to collect SVA CPs

vi. Explanation of SVA CP connection to orthometric datum

2. Control Density

a) QA4LiDAR checks whether an adequate number of points (GC and FVA CP) were collected

for the survey area i.e. the density of the Survey Control. The survey area is determined from

the original extent polygon supplied by the user. FVA CPs must be internal to the survey area

and GCs may be internal or external. The density of the Survey Control is then rated in two

modes. A Pass/Fail is given based on the following minimum points per square kilometre;

vii. 0 - 100km2: ≥5 FVA CPs + minimum 3 GC

viii. 100 – 400km2: ≥20 FVA CPs + minimum 5 GC

ix. ≥400km2: 20 FVA CPs + 1 FVA CP for every 50km2 over 400km2 + minimum 5 GC

b) In addition, a score as per the table below is provided for the density of points per square

kilometre (e). For this score only FVA CPs and GCs internal to the survey area are counted.

The density is given by;

𝑒 =𝑛

𝑎 where a = project area (km2)

n = number of internal FVA CPs & GCs

Table 10. Control density rating scheme

Score Point Density

1 - Poor 0 - 0.002

2 - Weak 0.002 - 0.005

3 - Average 0.005 - 0.01

4 - Good 0.01-0.02

5 - Strong ≥0.02


Page 70 of 78

3. Distribution

The distribution of FVA CPs and GCs internal to the survey area is rated. The ICSM Template (2010)

defines the requirement as “the distribution of FVA Check Points must be established to adequately

cover the full extent of the survey area, and be representative of the project area landscape”. This is

determined employing ArcGIS methods for the following three part algorithm;

a) Buffer:

This is a ratio of the project area (blue area in diagrams) compared to the area covered by

the FVA CPs and GCs (dots in diagrams) and determines the spread of points across the

project area.

i. The area per FVA CP/GC (b) is calculated using;

𝑏 =𝑎

𝑛 where a = project area (m2)

n = number of FVA CPs/GCs

E.g. a = 5911647m2

n = 6

b = 985274m2

ii. The FVA CPs/GCs are spatially buffered with a radius (r) in m of;

𝑟 = √𝑏

𝜋 where b = area per FVA CP/GC calculated above

E.g. r = 560m

(i.e. radius of grey buffer circles)

iii. The area of buffered points is spatially subtracted/erased from the original survey

extent (resulting area in m2 = g, orange area in diagram) and a numerical value for

the spread (s) is determined using;

𝑠 = 𝑎−𝑔

𝑎 where a = project area (m2)

g = subtracted area (m2)

0 ≤ s ≤ 1

E.g. g = 1472550m2

s = 0.75


Page 71 of 78

The closer ‘s’ (“Buffer value”) is to 1, the stronger the spread of Survey Control

points. However, this does not account for all types of distributions and is combined

with the following two scores (parts B and C) for a more accurate representation of

distribution.

b) Minimum Bounding Geomtery:

i. This is the percentage of the project area outside the minimum bounding geometry

of the FVA CPs/GCs which determines how well the FVA CPs/GCs cover the

extremities of the project extent.

ii. A minimum bounding geometry of the FVA CPs/GCs is created using a convex hull

E.g. the yellow area

iii. The minimum bounding geometry is erased from the project area (left with blue

area above)

iv. The area in m2 of the remaining project area is calculated after the erase (blue area)

v. This remaining area is divided by the original project area

vi. The result is rounded down to one decimal place and subtracted from 1, to get a

number between 0 and 1. This is the “Min Bound value”.

c) Centroid:

i. This is a ratio of the distance between the extent centroid and the FVA CP/GC mean

centre, to the longest axis of the project area (in the X or Y direction to keep things

simple) which determines whether the FVA CPs/GCs are weighted to one side of the

project extent.

ii. The project area centroid (blue point below) is found

iii. The point mean centre (red point below) is found

iv. The distance between the two points (blue and red) is measured in metres

v. The longest axis distance of the project area is retrieved from the layer properties in

metres i.e. top-bottom or right-left (arrow above)

vi. The distance from centroid to centre is divided by the project area axis distance


Page 72 of 78

vii. The result is rounded to the nearest decimal place, subtracted from 0.5 and

multiplied by 2 (if the value before rounding is >0.5, the value is limited to 0.5 to

prevent the final number going above 1). This is the “Centroid value”.

d) Final Weighted Distribution (d), the weighting values were determined through testing of

numerous different datasets.

= (Buffer value * 0.4) + (Min Bound value * 0.4) + (Centroid value *0.2)

Table 11. Control weighted distribution rating scheme

Score Distribution (d)

1 – Poor 0 - 0.4

2 - Weak 0.4 – 0.5

3 - Average 0.5 – 0.6

4 - Good 0.6 – 0.8

5 - Strong ≥0.8

e) The results of the second (Survey Control point density) and third (distribution) Survey

Control checks are then combined into an overall rating for the Survey Control by averaging

the two scores (i.e. numbers 1-5), and reported along with the minimum points Pass/Fail.

Table 12. Overall survey control rating scheme

Survey Control Rating Minimum Points Average Score

Poor PASS / FAIL 1

Weak PASS / FAIL 1.5 – 2

Average PASS / FAIL 2.5 - 3

Good PASS / FAIL 3.5 – 4

Strong PASS / FAIL 4.5 - 5


Page 73 of 78

9.1.7 Point Density & DEM Resolution

If the Absolute & Relative Vertical Accuracy check has already been run, QA4LiDAR uses the flight

line LAS datasets created to complete this check. However, to avoid coupling between checks, if they

do not already exist, this check will create them. This check uses ArcGIS tools to ensure that the

point density of the LAS data, coverage of bathymetry data, and resolution of the raster data meets

the specification. The DEM resolution check simply looks at the cell size attribute for the ORT data.

Three types of point density are computed by QA4LiDAR; All Point Density, Ground Point Density,

and Pseudo Pulse Density. The density types are defined in the table below. Bathymetry coverage is

also calculated.

Table 13. Point density type definitions

Point Density Type Definition

All Point Density The number of successful ground and non-ground point returns (1st, 2nd, 3rd AND last return) over a set area (e.g. more points returned in vegetated areas due to the presence of 2nd & 3rd returns).

Ground Point Density The number of successful ground point returns (1st, 2nd, 3rd OR last return) over a set area, which equates to removing all non-ground points from the point density (e.g. a typical ground point density required to generate a DEM is 2 points per square metre).

First (or last) return point density

The number of successful 1st (or last) returns over a set area which could be ground or non-ground (e.g. only by examining first or last return or pulse density will you find areas of greater density in a project).

Points at Nadir The number of successful ground and non-ground point returns (1st, 2nd, 3rd AND last return) over a set area at nadir (10% of swath width).

Pulse density The number of outbound pulses (not necessarily successful returns) over a set area. This is simulated in QA4LiDAR using last return and excluding data gaps to get a measure of ‘Pseudo Pulse Density’.

For All Point Density, values are reported per tile and for the overall project, as well as a density

raster provided as output. Statistics include internal gaps due to water etc. There will be higher

density found in overlap areas unless overlap points have been removed from the dataset. The

process is as follows;

1. A single LAS Dataset is created for all classified ORT LAS tiles

2. The LAS Point Statistics as Raster tool is used to convert the LAS Dataset to a single

raster using the POINT COUNT method and the cell size set by the user in the Automated

Checks setup screen. This raster gets saved as output for the QA4LiDAR user. It should

be noted that the density values in the output raster are per the cell size specified,

hence if this was not 1m, the density values are not per metre squared.

3. Any ‘no data’ values are then converted to zeros so gaps are included in the density

statistics


Page 74 of 78

4. Density statistics are extracted per tile using the Zonal Statistics as Table tool and a

statistics type of MEAN

5. The per tile statistics (as floating point values) are divided by the cell size squared to get

the number of points per square metre which are stored per tile

6. Ideally statistics would exclude cells on project boundaries so that cells partially filled

with data do not skew results, however to simulate this, outlier removal based on

standard deviation is performed before averaging the per tile density values to find the

overall project All Point Density per metre squared. Note. A maximum of 1 outlier will be

removed if there are less than 10 tiles.

The same rules and process apply to the determination of Ground Point Density aside from one

additional step. After the first step of creating a single LAS Dataset, this is converted to a LAS Dataset

Layer to filter the LAS to only the points classified as ground i.e. class 2. All other steps are

performed the same.

For Pseudo Pulse Density, values are reported per flight line. Statistics exclude internal gaps due to

water etc. As statistics are calculated and reported per flight line, the issue of overlap points is

negated. The process is as follows;

1. The ‘lasinfo’ tool is used with the histogram option to determine the point source IDs (flight

line numbers) within each LAS tile

2. The ‘las2las’ tool is then used to split each LAS tile per flight line using the point source IDs

and generate many smaller new LAS files for every flight line in every tile

3. ArcGIS is then used to create a LAS Dataset for each flight line (point source ID) from the

split LAS files

4. This is followed by LAS Point Statistics as Raster to generate a raster per flight line with the

method set as PULSE COUNT (last returns) and a cell size of 1m

5. As gaps are to be excluded from this density value, the raster properties statistics mean

(which excludes no data) is retrieved. This negates the issue of tiles on project boundaries

that may be partially filled with data. The result is the mean Pseudo Pulse Density for the

flight line.

6. The Pseudo Pulse Density values per flight line are compared to the NPS value specified on

the Tender Form and each flight line attributed a Pass or Fail.

7. The Pseudo Pulse Density values per flight line are written to the output flightline shapefile.

For Bathymetry coverage, results are reported per bathymetry coverage tile and overall. The

process is as follows;

1. The user supplies a 50m or 100m ‘Bathymetry coverage tile index’ shapefile only covering

the tiles to be included in the check, along with the class/s to be used.

2. A single LAS Dataset is created for classified ORT LAS tiles

3. This is converted to a LAS Dataset Layer to filter to only the class/s specified

4. The LAS Point Statistics as Raster tool is used to convert the LAS Dataset Layer to a single

raster using the POINT COUNT method and the cell size as the coverage tile size, snapping to

the location of the coverage index.

5. The Raster to Polygon tool is used to convert the raster to polygon

6. Then the Spatial Join tool is used to join the point count attributes from the resulting

polygon to the coverage tile index


Page 75 of 78

7. The % of tiles that have the minimum required number of soundings is then determined and

reported along with a pass/fail against the requirements

9.1.8 Flight Line Coverage

This check ensures that there are no gaps between parallel flight lines i.e. there is complete

coverage, by producing a raster identifying the number of flight lines covering each cell. It also

checks that LAS point scan angles are within the specified range.

The lasinfo tool is used to return a histogram of the scan angles of all the point data records for the

AHD LAS and the results are compared to the maximum scan angle requested on the Tender Form. If

any are outside the requested range, that tile fails.

Using the pseudo pulse density flight line rasters created by the density check or creating them if

they don’t already exist, along with ArcGIS tools QA4LiDAR does the following;

1. First performs an “is null” operation to convert the flight line rasters into pixels where 0

values represent data and 1 values represent NO data

2. Next it reverses this by reclassifying the “is null” rasters to pixels where 0 values represent

NO data and 1 values represent data

3. Finally it sums these 0 and 1 flight line rasters using a union operation to produce an output

raster showing how many flight lines cover each area

As the flight line coverage rasters are based on the pseudo pulse density flight line rasters, they are

produced at the same resolution (1m). Flight line coverage rasters can be quite pixelated due to cells

with zero density however they give a sufficient representation of coverage if viewed at a reasonable

scale.

9.1.9 Absolute & Relative Vertical Accuracy

This check ensures that the absolute vertical accuracy of data relative to control is within

specification, and that the relative (or internal) vertical accuracy of the data between flight lines is

acceptable. If supplemental check points were supplied it also checks the absolute vertical accuracy

of relevant land cover classes using this data.

Absolute Vertical Accuracy (Fundamental)

Before using the control points to test the vertical accuracy of the LiDAR, QA4LiDAR determines how

open and flat the area surrounding each control point is, and hence its suitability for use in testing.

This is done for all control points despite clustering, using ArcGIS tools.

1. The ‘openness’ of the area around each control point internal to the convex hull minimum

bounding geometry of the project extent (this includes FVA CP, GC, and any purchaser control

provided) is rated. If results already exist from the Accuracy of Survey Control check they are

used. Otherwise the exact same method as mentioned in section 0 is used to generate results.

2. The ‘flatness’ of the area around each control point internal to the convex hull minimum

bounding geometry of the project (this includes FVA CP, GC, and any purchaser control provided)

is rated. If results already exist from the Accuracy of Survey Control check they are used.

Otherwise the exact same method as mentioned in section 0 is used to generate results.

3. If the ‘flatness’ or ‘openness’ rating is unacceptable for a control point, that control point is not

used to test the absolute vertical accuracy of the data. Whether a control point is unacceptable


Page 76 of 78

is up to the user. The user specifies a rating for each type of control in the Survey Control section

of the Automated checks setup screen. Both flatness and openness for a control point must be

rated equal to or better than the user’s acceptability rating for the point to be used. The lower

the acceptability rating selected by the user (i.e. closer to 'Poor'), the more control points that

will be used for that type of control.

4. Absolute vertical accuracy of LAS;

a) This check is performed for both orthometric and ellipsoid LAS if applicable.

b) It is also performed for each control file input to QA4LiDAR based on the ORT and

ellipsoid height fields selected in the Survey Control section of the Automated checks

setup screen.

c) TINs are created from the ground point LAS data within a 10m radius of each acceptable

(in terms of openness and flatness) control point.

d) The height of each TIN at each control point location is extracted and the height

differences computed between each LAS TIN and control point.

e) These height differences are the result of “z data i – z check i” in the below formula.

f) The absolute vertical accuracy is calculated at 95% confidence interval as a function of vertical RMSE in line with the ICSM Guidelines for Digital Elevation Data v1.0 as follows;

i. Compute the vertical RMSEz = sqrt[Σ(z data i – z check i )2 /n]

ii. Compute Accuracyz = 1.9600 x RMSEz = vertical accuracy at 95 percent confidence level.

iii. Report Accuracyz

g) The computed accuracy values are compared to the specified and reported accuracies (separately for AHD and ellipsoid values if applicable) to determine a pass or fail for each type of control.

5. Absolute vertical accuracy of the ORT DEM;

a) If a Tender Form exists, QA4LiDAR uses the DEM of the resolution specified in the form.

If no DEM exists at the specified resolution, it uses the finest existing resolution DEM

delivered. If there is no Tender Form, QA4LiDAR checks how many DEMs have been

supplied and if there is only one it runs the checks on that DEM, otherwise if there are

multiple it uses the finest resolution DEM.

b) This is only performed for ORT as DEMs are always orthometric.

c) It is performed for each control file input to QA4LiDAR based on the ORT height fields

selected in the Survey Control section of the Automated checks setup screen.

d) Bilinear interpolation (using the four closest cell values including the value of the cell the

point is within) of the DEM occurs to extract a value at each acceptable control point.

e) The height differences are computed between the DEM and associated control points.

f) These height differences are the result of “z data i – z check i” in the below formula.

g) The absolute vertical accuracy is calculated at 95% confidence interval as a function of vertical RMSE in line with the ICSM Guidelines for Digital Elevation Data v1.0 as follows;

i. Compute the vertical RMSEz = sqrt[Σ(z data i – z check i )2 /n]

ii. Compute Accuracyz = 1.9600 x RMSEz = vertical accuracy at 95 percent confidence level.

iii. Report Accuracyz

h) The computed accuracy values are compared to the specified and reported accuracies to determine a pass or fail for each type of control.


Page 77 of 78

Supplemental Vertical Accuracy

If SVA Check Points were supplied, the Absolute Vertical Accuracy check runs again, just using the

SVA Check Points. The openness and flatness parts of the check are NOT run as do not apply. Points

4 & 5 (LAS and DEM accuracy) are run and results are summarised based on the categories in the

Cover Type Field specified on the Automated Checks setup screen.

Relative Vertical Accuracy

If the Point Density & DEM Resolution check has already been run, QA4LiDAR uses the pseudo pulse

density flight line LAS Datasets already created to complete this check. However, to avoid coupling

between checks, if they do not already exist, this check will create them. The following steps are

undertaken using ArcGIS tools;

1. Each flight line LAS Dataset is converted to a LAS Dataset Layer to filter datasets to just

ground points (class 2).

2. These are then converted to flight line DEMs of ESRI Grid format using the cell size the user

selected on the Automated Checks setup screen. If the user chose a cell size greater than

5m, this check will not run as the cell size is too coarse to determine relative vertical

accuracy.

3. Next a check is performed to determine the amount of overlap between flight lines. If there

is less than 2% total overlap in the dataset, this check will not run as the overlap is

insufficient to determine relative vertical accuracy.

a) A boundary polygon is created for each flight line raster

b) All flight line boundary polygons are merged retaining overlaps (saved to output

folder)

c) The merged boundary is intersected with itself. The output is a polygon that

represents just the overlap areas. The total overlap area is calculated and divided by

two to remove the double ups.

d) The total area of the merged polygon including overlaps is calculated, and the

percentage of overlap area (using the divided by two result) out of total area is

computed.

e) The overlap areas must be at least 2% of the total area (with overlaps) for QA4LiDAR

to proceed with the relative vertical accuracy check.

4. If the overlap was greater than 2%, the flight line rasters are mosaiced using a MEAN mosaic

operator and the same cell size and spatial reference as the inputs

5. Next all flight line DEMs are subtracted individually from the mean mosaic DEM

6. The resulting flight line difference rasters are then converted to absolute values

7. All the flight line absolute difference rasters are then mosaiced using a MAXIMUM mosaic

operator and the same cell size and spatial reference as the inputs

8. The overlap area polygon created as part of the 2% overlap test is then used to extract

values from the maximum mosaic. The output raster is the maximum difference raster in

just the areas of flight line overlap and the raster cells values represent the result of zdata i –

zcheck i in the below formula.

9. The relative vertical accuracy is calculated at 95% confidence interval as a function of vertical

RMSE in line with the ICSM Guidelines for Digital Elevation Data v1.0 as follows;

a) Compute the vertical RMSEz = sqrt[Σ(z data i – z check i )2 /n]


Page 78 of 78

b) Compute Accuracyz = 1.9600 x RMSEz = vertical accuracy at 95 percent confidence

level.

c) Report Accuracyz

9.2 Appendix 2 – How the Visual Checks are performed This Appendix provides details of the methods used by QA4LiDAR to create the data displayed in the

Visual checks.

9.2.1 Extent Checks

The basic LAS and DEM extent shapefiles used in the Extent Check do not show internal holes. The

LAS shapefile is created by extracting the ArcGIS LAS Dataset LAS file extents. Hence, most polygons

will be square (i.e. the tile size) while any boundary tiles partially filled with data will be rectangular

(smaller than the tile size). The polygons will not show internal holes in the data. The basic DEM

shapefile is simply extracted from the DEM mosaic footprint. This extent does not represent internal

holes in the data however to identify internal holes, the user can tick the optional detailed extents

check. To produce the detailed DEM extent the following ArcGIS process is used.

1. The Is Null tool is applied to each DEM tile to create binary rasters of data/NODATA. Is Null

determines which values from the input raster are NoData on a cell-by-cell basis and returns

a value of 1 if the input value is NoData and 0 for cells that are not.

2. Mosaic all the resulting rasters into one output raster.

3. Convert the mosaic raster to polygon but DO NOT simplify polygons as may lose small voids.

4. Display the final extent polygon with the original project extent shapefile for the user to

check. Display the GRIDCODE = 1 (no data) values from the extent polygon with outline only

(no fill) and about 2 in thickness so small internal voids can be seen.

9.2.2 Visual DEM Checks

A mosaic dataset with footprints and overviews is created from the DEM tiles so that a consistent

symbology can be displayed across tiles.