Terrestrial Photogrammetry &

Laser Scanning

-Mapping Science Overview -

Stuart Robson s.robson@ucl.ac.uk

Introduction to Terrestrial Photogrammetry

Principles of terrestrial photogrammetry The digital image and small format cameras

Coordinate systems, resection, intersection and bundle adjustment

Uncertainty and camera calibration

Example applications Stereo, multi-photo, panoramic

Photogrammetry with Targets Coded targets and example applications

From Leonardo to Laussedat

da Vinci c.1480 The appearance of points and lines to

the eye perspective geometry

First used in mid 19th century from balloons (Laussedat - 1885) and for surveys of

buildings (Meydenbauer)

VEXCEL ULTRACAM D 8 sensors delivering an image of 11,500 x 7,500 pixels

Flight over UCL, 4cm pixel foot print with Applanix 510 IMU Microsoft 3D model the worlds largest 3000 cities in the next 5 years

.. to today

Measuring with light

Kodak DS460 (~1995) 6MP

Hasselblad H3D

(~2007) 39MP pixels

Some close range imaging systems

GIS INCA 3 metric RolleiMetric6008

Digital 39MP

The GSI

ProSpot

projection

system

AXIOS 3D CamBar

Light photons

incident on the

sensor

material are

collected to

produce an

electrical

signal at each

pixel

Analog image

voltage and

timing signals

produced by

reading the

signal

produced at

each pixel in

turn.

Analog image

signal

quantised into

individual

pixels by

analogue to

digital

converter

Digital image

data in

computer

readable form

Distance

I A/D

Converter

(typically 8 bit, but

10, 12 16 and 32

bits possible) Distance

gv

0

2 5 5

Analog signal Digital representation

Digital Image Acquisition

Two principal types of image sensor

Interline transfer, is derived from the TV and broadcast standards where the

array produces an interlaced image to

minimise the quantity of data

transmitted whilst maintaining the 25

frames per second necessary to avoid

perceptible image flicker.

Method is limited in that the odd and even lines represent two

different periods in time

Frame transfer sensors are organised such that the light sensitive regions are

also used to transfer charge and the

image is read-out as a single frame.

Method depends on an independent storage and readout zone or a

mechanical shutter to prevent light

reaching the sensor whilst the

image information is read out.

Sensor

Element

Row Bus

Column Bus

Horizontal Scan Register

Vertical S

can R

egiste

r

Output

Amplifier

Video Out

Digital Image Sensors (CCD and CMOS)

Bucket Array Analogy

Photons

Gauge

Conveyors

Conveyor

It is conventional for a photogrammetric image coordinate system to

have an origin at the centre of the image format, coinciding with the

optical axis of the lens in an ideal central perspective projection. Image

sensor arrays are highly regular structures, which given consistent

electronic signal timing, provide an excellent image coordinate system.

x photo co-ordinate axis

y photo co-ordinate axis true origin

Y pixel axis (0,0)

(0,0)

Digital image

Y pixel size

X pixel size false origin

X pixel axis

However, the

sensor scanning

process

conventionally

reads from the top

left corner of the

array, line by line,

towards the

bottom right

corner. A simple

2D transformation

is therefore

necessary to

obtain the familiar

photo coordinate

system.

Image Coordinate Systems

Sub-pixel Location of a Circular Target Image on a Dark

Background

Pixel

Value

Threshold

Pixel

Value

Pixel

Value

Pixel Number

Pixel Number

Centroid

Sub pixel location

T

0

0 0

255

255 255

Intensity

A/D conversion

One or more cameras probing a single point

Y

X

Z

(Xo, Yo, Zo

w,f, k)

P

yx

z

X

Y

Z

camera 1 camera 2

OnlinesystemPrinzip.ppt

probe

object

Two or more cameras measuring multiple points

Right image Left image

Stereoanordnungen.ppt

b

Y

X

Z

General case: A Multi-photo Network

mehrbild1.ppt

Fundamental angular uncertainty

The ellipses represent the uncertainty of positions determined by the intersection of direction observations from two camera positions.

?

Essentials camera calibration

Essentials bundle adjustment (network adjustment)

Scale

Network of mechanically unconnected cameras

If the network geometry is strong enough (# images and intersecting rays per feature point) it is possible to determine parameters describing the systematic distortions in the camera(s) used. This process is termed self-calibration and allows the use of a wide variety of imaging sensors.

A carving from Sumatra

Examples of similar outputs to the aerial photogrammetry case with automated area and feature based matching techniques

Simple stereo pair

-200.00 0.00 200.00 400.00 600.00 800.00 1000.00

-200.00

0.00

200.00

400.00

Automatically generated contoured surface

Orthophoto one of the original images draped over the 3D surface model

Automated surface measurement examples

F. Guerra (Italy), C. Baletti, D. Miniutti The Arena of Verona

Instituto Universitario di Architettura di Venezia

A traditional stone by stone output based on multi-image registration, followed by stereo plotting into a CAD package Automation difficult, but possible with less structured outputs

Set of stereo images from a petro-chemical plant survey

Following a network adjustment, data might be manually plotted, or

extracted automatically based on edge and feature extraction coupled with

expected component geometries e.g cylinder reconstruction from tangents

and centre lines

A portion of the as built model (isometric view) Example CAD output, modelled into PDS/PDMS

Panoramic imaging - Optag: infrastructure tracking

system

Combined panoramic photogrammetry and radio frequency tagging real time photogrammetric panoramic camera

far field radio tag system

integrated together to track individuals within an airport environment

Photogrammetry with Targets

Targets provide a unique feature that is purpose designed to produce a signature image on a sensor

Automatically measured based on image scanning at a specified threshold

Used for resection of camera, locations of targets or probe systems

Variety of coding techniques

Note - similar basis to surveying with targets, but machine readable numbering offers

many advantages

Automatic processing with targets

Measure images: identification of known targets and

measurement of other imaged targets

Location of cameras: given appropriate spatial

information the location and orientation of each camera is

determined at the time the image was taken.

Identification and location of targets: given camera

orientations the identities and locations of new targets

are established.

Compute parameters of interest: for example the attitude

of the object, change in shape or motion parameters.

Re-compute the solution for the next set of images:

Using, for example, target tracking to enable a very rapid

update of the parameters of interest.

Sett

ing

up

Repe

ate

d

Inf

ormation

ava

ilable a

t ca

mera

syst

em

Significant research on automation: red light green light systems

R&D with NASA Langley: Stretched lens array

Monitoring 3D change during a structures test R&D with UCL Mech. Eng Oil Rig components

R&D with NASA Langley: Parachute flight performance

Dimensional and Accuracy Control Automation for shipbuilding

Photogrammetric edge measurements in multiple images to 3D reconstruction

Medical: surface measurement in support of optical tomography

An infant born after 24 weeks gestation

(~6 months)

Phot

ogra

mmetr

ic

surf

ace

Validation

CT S

can

Medical Physics, Computing Science and Geomatic

Engineering

Terrestrial Laser Scanning

Principles of laser scanning Time-of-Flight, Phase & Triangulation systems

FoV, scan pattern, specifications

Data acquisition Error sources, surface effects, range

Data processing Registration, points or triangles? surface-

growing, thinning, building CAD models

Applications

Overview of laser scanning systems

3D laser scanners record three-dimensional coordinates of numerous points on an object surface in a relatively short period of time.

A laser beam is projected onto the surface of the object to be measured and the horizontal angle, vertical angle and range are recorded to deliver 3D information.

Accuracies are between several tens of mm and a few cm, depending on object surface properties, instrument design and the range to the object from the scanner.

Applications City Modelling & Urban Planning Architecture & Facade Measurement Tunnel Surveying Archaeology & Cultural Heritage Documentation Topography & Mining Process Automation and Robotics Scene Acquisition for Virtual Reality Reverse Engineering

What makes it possible? - the semiconductor laser

Laser scanners use small semiconductor (diode) lasers to convert a pulse of electrical energy into a pulse of optical energy with high efficiency and high reliability.

A laser diode is a small cube of semiconductor material with two flat and parallel faces which form the mirrors of the laser cavity.

Light generation takes place in the very narrow active region ~ 1 m thick

The divergent laser radiation emitted by the semiconductor is collected by a collimating lens to form a narrow beam.

Laser wavelengths used in scanning are in the infrared (invisible) and green (visible) part of the spectrum (1 mm to 700 nm).

Regulations require manufacturers to certify each laser product as Class I (least hazardous), II, III, or IV, depending on the characteristics of the laser radiation emitted (http://www.fda.gov).

Laser Scanning system examples

Range measurement

Pulsed or phase measurement

Full waveform

Triangulation systems

FoV, scan pattern

Typical specifications

Total stations > direction and range

Time of flight laser scanning

Laser scanning systems emit a laser signal (1) to record the position of a point in object space.

Scanning of the laser beam (2) is achieved using one to two reflective surfaces (3) which are linked to accurate angle motors and angle encoders to allow changes of the deflection angle in small increments.

In addition, the entire instrument may be rotated to achieve a complete 3-dimensional point coverage (4). Alternatively a second mirror may be used

Angle encoders deliver the direction of the beam

The method used to measure range depends on the accuracy and distance capability required of the device. Example: Riegl (www.riegl.co.at)

Pulsed time of flight measurement

Time of flight sensors derive range from the time it takes light to travel from the

sensor to the target and return.

A laser diode sends a pulsed laser beam to the object. The pulse is diffusely

reflected by the surface and part of the

light returns to the receiver.

The time that light needs to travel from the laser diode to the object surface and back

is measured and the distance to the object

calculated using an assumed speed of

light.

Pulse-type time of flight systems are typically used over ranges of several

metres to several hundred metres.

The accuracy of these sensors is typically limited by the accuracy with which the time

interval can be measured, and the rise

time of the laser pulse.

Operation of a pulsed laser distance meter An electrical pulse generator periodically

drives a semiconductor laser diode

sending out light pulses, which are

collimated by the transmitter lens.

Via the receiver lens, part of the echo signal reflected by the target hits a

photodiode which generates an electrical

receiver signal.

The time interval between the transmitted and received pulses is counted by means

of a quartz-stabilised clock frequency.

The calculated range value is fed

into the internal microcomputer

which processes the measured data

and prepares is for range (and

speed) display as well as for data

output.

It is possible to the select different

data processing algorithms,

according to the prevailing

conditions and requirements

HDS2500 (Leica) ~1998

40 x 40 degree field of view

1000 points per second

Produces a 3D point cloud

Single point accuracy of 6mm

Uses two rotating mirrors with their axis of rotation set at 90 degrees to each other

Includes a digital camera designed to work as a viewfinder for the system

Data in point cloud coded according to the strength of the laser return signal

http://www.ascscientific.com/cyrax.html

Leica ScanStation C10 (2009)

Range up to 300 m 90% (134m 18%)

Laser Class 3R (532nm)

Field of View 270 x 360

Measurement capability 6 mm (single shot)

2 mm (surface average)

Spot diameter 4.5mm to 7mm

Angular accuracy 12 arc

Measurement rate 50 000 pts/sec

Additional Capabilities Integrated camera 2K x 2K pixels

Laser plummet

Dual axis compensators

Target capability to 2mm std dev.

http://www.leica-geosystems.com/en/HDS-Laser-

Scanners-SW_Leica-ScanStation-C10_79411.htm

Phase / modulated beam range measurement

The "modulated-beam" sensor also uses the time light takes to

travel to the target and back, but

the time for a single round-trip is

not measured directly.

The strength of the laser is rapidly varied to produce a

signal that changes over time.

The time delay is indirectly

measured by comparing the

signal from the laser with the

delayed signal returning from

the target.

Given several frequency modulations it is possible to

compute the number of full

wavelengths to the target (cycle

ambiguity) and to add these to

the offset T1

Waveform offset

(Thiel & Wehr, 2004)

Phase shift distance measurement

B = brightness A = amplitude f = phase d = range lmod = modulation wavelength

Kahlmann, Remondino, Ingensand (2006)

Z&F 5006i

Range up to 79 m 90%

Laser Class 3R (visible)

Field of View 310 x 360

Measurement capability 0.7mm rms at 10m (20%)

0.4mm rms at 10m (100%)

3.5mm rms at 50m (20%)

1.8mm rms at 50m (100%)

Spot diameter 3mm at 1m

Beam divergence 0.22mrad

Measurement rate 508 000 pts/sec

Additional Capabilities Tilt compensation

http://www.zf-laser.com/e_imager5006.html

Metris MV330 / 350 from MetricVision (USA), now Nikon

Field of View

90 x 360 degrees

Multiple lasers (Class 1)

2 visible lasers to point and focus

1 infra laser for time-of-flight distance measurement

Two range options - 30m, 50m

Several measurement modes

from 4000 pts/sec with 0.3mm typical accuracy

to 2 pts/sec with 102 mm at 10m

Accuracies are achieved through the use of beam modulation and extensive signal processing

www.nikonmetrology.com/large_volume_metrology/laser_radar

Full waveform scanning (after Riegl 2008)

Riegl VZ400 (2009)

Range up to 500m 80%, (160m 10%)

Laser Class 1 (near IR)

Field of View 100 x 360

Measurement capability

5 mm (accuracy)

3 mm (precision)

Beam divergence 0.3mRad

Angular resolution 1.8 acr

Measurement rate

125 000 to 42 000 pts/sec

Additional Capabilities

Fitting for Nikon digital camera

Laser plummet

Dual axis compensators

Target capability to 2mm std dev.

GPS receiver

Triangulation

A light spot or stripe is projected onto an object surface and the position of the spot on the object is recorded by a CCD cameras.

The angle of the light beam leaving the scanner (a) is internally recorded.

The fixed separation (D) between laser source and camera is known from calibration.

The direction of the reflected

laser spot (b) is computed by

measuring the location of its

image on a sensor array (P1)

The distance from the object to

the instrument is geometrically

determined using a, b and D.

The diagram shows that a

second spot at a differing range

would yield P2 and a different

value for b.

Example -Minolta VIVID 910

Object distance (range) 0.6m to 2.5m

The object is scanned by a plane of

laser light which is swept across the

field of view by a mirror, rotated by a

precise galvanometer

Reflected light from each scan line is

observed by a single frame, captured

by the CCD camera to provide over

300,000 vertices per scan

Scanning field of view depends on

interchangeable lenses used

Data captured in 2.5 seconds

A (24-bit) colour image is captured at

the same time by the same CCD to

provide RGB information for each 3D

data point

http://www.konicaminolta.eu/index.php?id=2079

Object coverage for Minolta Vivid 910

Lens Near field (mm)

(@ 0.6 m)

Far field (mm)

(@2.5m)

max depth resolution

Tele: 25mm 111 x 84 x 40 460 x 350 x 130 0.039 mm

Mid : 14mm 196 x 153 x 70 830 x 622 x 220 0.068 mm

Wide : 8mm 355 x 266 x 92 1200 x 903 x 400 0.090 mm

A triangulation scanning system reaches 3D point standard deviations of less than one millimetre at very close range (less than 2 meters).

The accuracy depends on the length of the scanner base, the optics used and the object distance.

With a fixed base length, the standard deviation of the distance measurement will increase in proportion to the square of the distance.

Error sources

Broadly similar to total station with REDM

Non-optical

Vibration

Air turbulence

Mechanical error

Human error

Optical

Speckle, signal buried in noise

Spot size

Range shift and noise: laser light surface penetration

Range artefacts: edge and reflectance jumps

Strength of laser return signal

Level of background illumination

Reflectivity of surface / colour

Angle of incidence

Calibration of instrument

Reflectivity of Various Surfaces / Materials

The amount of light that is returned from a target's surface is characterised by the reflection coefficient r.

For a diffusely reflecting target, the maximum value of r is 100 %.

For mirror-like or retro reflecting targets, the (theoretical) value of reflectivity can exceed 100 %.

The reflection coefficient also depends on the wavelength.

Diffuse reflection:

The signal is reflected omni-directionally according to Lambert's cosine law

Specular reflection:

The angle of the reflected beam with respect to the targets surface is equal to the angle of incidence. Incident beam and reflected beam lie in

the same plane.

Retroreflection:

The retroreflected beam is returned in the same direction from which the incident beam came. This property is maintained over a wide range

of directions of the incident beam

Maximum Range versus Target Reflectivity

The maximum range achievable with a laser

scanner depends strongly

on the reflectivity of the

target.

Range performance (as specified by RIEGL) is

given for a diffusely

reflecting (lambertian)

target with a reflectivity of

80 percent.

For a target of different reflectivity, the maximum

range can be found with

the range correction factor

as given in the diagram.

Maximum Range as a Function of Visibility

For long range systems the maximum range achievable

with a laser rangefinder

depends strongly on the

meteorological visibility.

Range performance can be given with respect to a

meteorological visibility of 20

km (clear air).

At lower visibility, the maximum range is reduced due to the

atmospheric attenuation

according to a range reduction

factor

Laser scanning acquisition and processing

Acquire scan data from multiple view points

Multiple clouds of 3D points, each point with its own sources of error scanning method, scan spot size, object surface qualities, colour information

Clean and Register data together

initial cleaning to remove any data that may obstruct the registration process

Register with Common points or mathematical fitting based on surface similarities

A single point cloud with overlapping areas and data of varying degrees of quality

Convert data into a model suited to final purpose

Points, triangles, mesh or NURBS model

Web delivery

Sharing information between institutions

Archive

Data processing steps Cleaning

The Cleaning Process involves removing unnecessary, unwanted, or bad data from the component image. Although cleaning of the data can occur throughout

the process the initial cleaning should remove any data that may obstruct the

alignment process.

Alignment

The alignment process transforms one image into position relative to another image. The scanning process results in several view oriented image of physical 3

dimensional object. The multiple view orientated scans (component images) can

be aligned until a completed 3d object is created (composite image). Overlapping

data between the component images is used to align them together.

Editing

The Editing process includes many methods of manipulating spatial (xyx), colour (rgb), and normal (ijk) data. Measurement data is edited to improve the quality,

filter data, enhance the colour, or segment the data into structures.

Hole Filling

The Hole Filling process creates new data within a hole. A hole is essentially a region where no measurement data exist. Holes are filled by blending new data with the surrounding data.

A single scan (Leica HDS 2500)

A second scan position

Registration Coordinate Systems

Individual scan data are based on a coordinate system defined according to the orientation of the scanner

Parameters of a 3D similarity transformation (3 translations and 3 rotations) are required to register data from two or more independent

scans

A minimum of 3 common points between scans are required

Registration Sources of common data

Control targets, identifiable in the scan

Must locate physical targets in the scan volume, positioned so as to provide sufficient common points

Optionally use a high definition scan to find target centre

Possible to link to external coordinate system through a target survey

Key advantage is that control targets provide clearly identifiable common points

Common natural features

Rely on natural features of interest

Natural features typically identified after scanning

Transformation parameters computed by minimising computed discrepancies between surfaces from different

scans

using iterative closest point

least squares surface shape matching

Dependant on appropriate features being available

The combined (registered) view

Engineering Applications

Natural Features

Natural Features Grimes Graves

Film Special Effects

Architectural Applications

Rialto Church, San Bernardino, California

Model created with a Leica HDS scanner, then modelled in CloudWorx and Autocad

Rialto Church, San Bernardino, California

Adding Colour..

Example - Riegl Z420i (2005) Measurement Accuracy 12mm (topo mode), 5mm (survey mode)

Red, Green and Blue Lasers

Optical combination

Schematic for a colour triangulation system

Combination of imaging and laser scanning

Carpiniana - the Italian delegate to World Summit Award in the e-Science category

Arius3D Foundation System

RGB colour from three lasers,

80mm spot diameter

100mm sampling interval

maximum dimensional error 25mm

Scanning cross section ~ 0.6 x 0.8 m

Arius im

ages c

ourt

esy o

f R

OM

http://www.arius3d.com/

Egyptian childs skull Arius 3D scanner

Laser scanning - Summary

Directly acquire a set of 3D surface measurements from a single instrument position provided that the surface concerned will reflect a laser beam

Distance measurement principle based on either on time of flight, phase, or triangulation

Data acquired in a regular fashion

Multiple scans required to overcome object occlusions

Registration between multiple scans required, either by use of physical targets, or through matching common surface features

Scanning systems tend to be built for specific purposes, e.g Cyrax 2500 or Minolta Vivid

Generate massive quantities of data which require significant post processing to produce a surface model