Wavelet-based analysis for object separation from ... - ITC · Wavelet-based analysis for object separation from laser ... yˆ F ou rie t ans fm y f,y j, k ... Wavelet-based analysis

Wavelet-based analysis for object separation from

laser altimetry data

Tsolmon Amgaa

February 17, 2003

Wavelet-based analysis for object separation from laser altimetry data

By Tsolmon Amgaa

Thesis submitted to the International Institute for Geo-information Science and Earth Observation in partial fulfilment of the requirements for the degree of Master of Science in GeoInformatics Degree Assessment Board Chairman: Prof.Dr. Alfred Stein External examiner: Dr.Ir. R.F. Hanssen Supervisor: Prof.Dr. Alfred Stein Second supervisor: Wim Harry Bakker M.Sc.

INTERNATIONAL INSTITUTE FOR GEO-INFORMATION SCIENCE AND EARTH OBSERVATION ENSCHEDE, THE NETHERLANDS

Disclaimer This document describes work undertaken as part of a programme of study at the International Institute for Geo-information Science and Earth Observation. All views and opinions expressed therein remain the sole responsibility of the author, and do not necessarily represent those of the institute.


iv

Abstract

Many physical processes, signals or other data are in fact ‘objects’ living on different scales. To mathematically describe such objects, an appropriate representation needed which is capable to capture spatially localized features of an object on coarser resolutions as efficiently as in finer resolutions. Wavelet analysis is most recent analyzing tool, which can combine both global and local analysis. This thesis examines the potential of applying the wavelet-based analysis for object extraction from laser scanning data. This was achieved by modeling the objects present in the scene and extracting their distinct features by wavelet based analysis. Several wavelet transforms were studied and tested such as standard pyramidal discrete wavelet transform, translation invariant wavelet transform, fast lifted wavelet transform, and continuous wavelet transform.

A number of experiments were carried out on complex urban scene. The general approach for the experiments is to apply the wavelet transform to decompose the original image into detail and approximation subimages, perform some operations to highlight the features which can distinguish the desired objects, and inversely transform the subimages back into single image. Tests on edge extraction, 3D edges, reconstruction of objects by 3D edges locally, inverting the wavelet coefficients have been performed.

The overall conclusion of the thesis is that wavelet based analysis has a good potential for object separation from laser data.

Keywords: wavelets, wavelet transform, laser scanning, object extraction, object separation


v

Acknowledgements

I would like to express my gratitude and appreciation to my supervisor Prof. Alfred Stein for guiding me throughout my work, for his support and help. His constructive criticism, suggestion and comments were extremely valuable for me.

Manny thanks to Mr. Wim Harry Bakker, my second supervisor for his valuable advises, and help. His guidance and support where extremely useful from the very beginning of my work.

I had a great chance to have three supervisors. I am indebted to Mr. Gerrit Huurneman for his trust and belief in my ability to carry out this research, for his guidance at the initial stage of this research.

I also want to acknowledge individuals who directly or indirectly contributed in my work. My special thanks to:

Prof. John van Genderen for his help and encouragement throughout my study in ITC. Studying in ITC was a big dream for me until I first stepped my foot in Netherlands.

Prof. Amgalan Bayasgalan who greatly supported and promoted me for studying in ITC.

Mr. Ishjamts Ulemj an experienced ‘waveleteer’ for his ideas and suggestions. With him I had a strong backup in wavelet theory.

Mr. Qingming Zhan for sharing his experience and his data. He spared me a lot of data preparation time.

Mrs Tumentsetseg Shaviraachin an ITC alumni, who first handled me the ITC booklet.

To my classmates, fellow students and my friends who made my stay in ITC enjoyable.

Last, but not least, my family for their encouragement and understanding during my study here in ITC.


vi

Contents Abstract .................................................................................................................................... iv

Acknowledgements ................................................................................................................... v

Mathematical conventions ...................................................................................................... ix

List of abbreviations................................................................................................................xi

Chapter 1 ................................................................................................................................... 1

Introduction .............................................................................................................................. 1

1.1 Introduction....................................................................................................................... 1 1.2 Problem definition ............................................................................................................ 1 1.3 Objective........................................................................................................................... 2 1.4 Research questions............................................................................................................ 2 1.5 Research limitations.......................................................................................................... 3 1.6 Prior work ......................................................................................................................... 3 1.7 Research approach............................................................................................................ 4

1.7.1 Experiments with mother wavelet ............................................................................. 4

1.7.2 Experimenting with wavelet transform ..................................................................... 4

1.7.3 Feature extraction ...................................................................................................... 4

1.7.4. Clustering algorithm selection.................................................................................. 5

1.8 Structure of thesis ............................................................................................................. 5

Chapter 2 ................................................................................................................................... 6

Laser scanning .......................................................................................................................... 6

2.1. Principles of laser scanning and system design............................................................... 6 2.1.1 Laser scanning ........................................................................................................... 6

2.1.2 Positioning and orientation system............................................................................ 7

2.2 Extended capabilities ........................................................................................................ 7 2.3 Laser altimetry data and its processing............................................................................. 8 2.4 Application of laser scanning in urban environment........................................................ 8

Chapter 3 ................................................................................................................................... 9

Wavelet theory .......................................................................................................................... 9

3.1 Simple example of Haar wavelet transform ..................................................................... 9 3.2 Fourier transform............................................................................................................ 10 3.3 Wavelets: The state of the art ......................................................................................... 11


vii

3.3.1. Wavelets definition................................................................................................. 12

3.3.2. Continuous wavelet ................................................................................................ 12

3.3.3 Wavelet properties ................................................................................................... 13

3.3.4 Discrete Wavelet Transform.................................................................................... 14

3.3.5 Scaling function....................................................................................................... 16

3.3.6 Multiresolution approach......................................................................................... 17

3.3.7 Fast wavelet transform............................................................................................. 18

3.3.8 2D Wavelet Decomposition..................................................................................... 19

3.3.9 DWT with Lifting scheme....................................................................................... 21

Chapter 4 ................................................................................................................................. 24

Object separation methodology............................................................................................. 24

4.1 Modeling......................................................................................................................... 24 4.2 General approach............................................................................................................ 25 4.3 Feature extraction ........................................................................................................... 26

4.3.1 Multiscale edges ...................................................................................................... 26

4.3.2 Smooth planar surface ............................................................................................. 28

4.4 Clustering........................................................................................................................ 28 4.5 Software.......................................................................................................................... 28 4.6 Translation invariance of wavelet transform .................................................................. 29

Chapter 5 ................................................................................................................................. 31

Experimental results and discussion..................................................................................... 31

5.1 Laser data........................................................................................................................ 31 5.2. Results of experiments................................................................................................... 32

5.2.1 Description of the planar surface property .............................................................. 32

5.2.2 Approximation of the height image as a DTM........................................................ 33

5.2.3. Extracting building edges by hard thresholding ..................................................... 34

5.2.4. 3D edges by inverse wavelet transform.................................................................. 35

5.2.5. Local reconstruction of objects from 3D edges with interpolation ........................ 36

5.2.6 Interpolating of building walls ................................................................................ 37

Chapter 6 ................................................................................................................................. 40

Discussion ................................................................................................................................ 40

6.1 Why are the overall test results unsuccessful? ........................................................... 40

6.2 Wavelet transforms..................................................................................................... 40


viii

6.3 Building custom wavelet ............................................................................................ 41

6.4 Testing different wavelets........................................................................................... 41

6.5 Data resolution issue................................................................................................... 41

6.6 Rotation invariance problem....................................................................................... 41

6.7 Is it possible to use wavelet based texture segmentation methods? ........................... 42

6.8 Summary..................................................................................................................... 42

Chapter 7 ................................................................................................................................. 43

Conclusions and recommendations....................................................................................... 43

7.1 Conclusions .................................................................................................................... 43 7.2 Recommendations........................................................................................................... 44

References................................................................................................................................ 45


ix

Mathematical conventions

This thesis and reference pages use certain mathematical conventions.

General notation Interpretation

Fourier analysis

�+∞

∞−

−= dtetfF tj πωω 2)()(

Fourier analysis of a signal

sn Signal in time f(t) Signal in time

e Natural exponent ω Frequency to analyze T Period

�−∗ −= dtetwtfS tjπωτωτ 2)()(),( Short Time Fourier Transform (STFT)

ω Frequency parameter τ Translation parameter

w(t) Windowing function

Continuous wavelet transform

dta

bttf

abaW � �

�

��

� −= ∗ψ)(1

),(

Continuous wavelet transform

),( baW Continuous wavelet coeficients

t Continuous time

ψ(t) ∈ L2(R) Wavelet function

( ) ℜ∈>��

��

� −= baa

bta

tba ,0 ,1

)(, ψψ

Family associated with the one dimentional wavelet indexed by 2,0 ℜ∈> ba

a Scale

b Translation L2(R) Set of signal of finite energy

∞<= � ωωωψ

πψ dC2

)(ˆ2

Admissibility condition

ψ̂ Fourier transform of ψ

kjf ,,ψ Scalar product of Function f(t) with wavelet function kj ,ψ


x

Discrete wavelet transform

Zja j ∈= 2 Dyadic scale

Zkkj ∈= 2b Dyadic translation

�=kj

kj tkjt,

, )(),()( ψγϕ Scaling function or father wavelet

),( kjγ Scaling function coefficients


xi

List of abbreviations

Abbreviation Description

WT Wavelet Transform

CWT Continuous Wavelet Transform

DWT Discrete Wavelet Transform

IDWT Inverse Discrete Wavelet Transform

DDWT?? Discrete Dyadic Transform

DWF Discrete Wavelet Frames

DWP Discrete Wavelet Packets

SWT Discrete Stationary Transform

MATLAB Software brand name from Mathworks Inc

YaWTB Yet another wavelet toolbox

TIN Triangulated Irregular Network

DTM Digital Terrain Model

DSM Digital Surface Model

LASER Light Amplification by Simulated Emission of Radiation

1D 1 Dimensional

2D 2 Dimensional

3D 3 Dimensional

GIS Geographic Information System

LIDAR LIght Detection And Ranging

INS Inertial Navigation System

GPS Global Positioning System

FOV Field Of View

IFOV Instantaneous Field Of View

STFT Short Time Fourier Transform

FT Fourier Transform

WFT Windowed Fourier Transform

FIR Finite Impulse Response

QMT Quadrature Mirror Filters

NaN Not available


1

Chapter 1

Introduction

1.1 Introduction

Wavelet analysis has enjoyed a tremendous attention and success over the last decade, and for a good reason. Almost all signals encountered in practice call for a time-frequency analysis, and wavelets provide a very simple and efficient means of performing such an analysis. The main contribution of the wavelet field as such has been to bring together a number of similar ideas from different disciplines and create synergy between these techniques. The result is a flexible and powerful toolbox of algorithmic techniques combined with a solid underlying theory. The theoretical developments of wavelets have been largely completed over the last two decades. The period of growth and intuition is becoming a time of consolidation and implementation. Wavelets have been making an appearance in many pure and applied areas of science and engineering. Object extraction object recognition with its many varied computational problems will not be an exception to this rule. The question is not “if” the wavelet can succeed, but “when” such achievement would take place. In this research work report we present the results of our research work exploring possibilities and potential of wavelet-based analyses for object extraction from laser altimetry data (further laser data). This chapter gives reasoning for research work, defines objectives and research questions as well as research limitations. Then, it continues with a description of prior work, methodology of this structure, research tools to be utilized and finally gives the overview of the structure of the thesis.

1.2 Problem definition

The last achievement in laser scanning technology expands its application frontiers. Although it was not initially developed for automated building extraction, the high density of sampling with high geometric accuracy extends the its potential for application in automatic extraction of buildings from laser data. Object extraction from digital signals and imagery, however is by far not an easy task. What is seemingly easy for humans is not often easy for machinery. The reason is that human interpretation based on multiple cues such as lightness, texture, color, shape, and association is very difficult to implement in computer. It is even more difficult to implement if the number of cues to be used is limited by the nature of data. This situation is familiar to object extraction from laser data. Experience from previous efforts towards building extraction suggests that photogrammetry and laser scanning both have a supplementary role and must be incorporated in object extraction process [19]. Research towards utilizing laser data with maximum efficiency to fully realize it potential however is still needed. Therefore we concentrate our research activities to utilize the laser data only.


2

Although the laser data provides accurate points with high spatial frequency, breaklines are not explicitly present in the data [35]. The two main reasons behind this are poor accuracy in planimetry relatively to the height measurements accuracy, different reflectance properties of material in complex urban area. This makes it difficult to extract objects with explicit boundaries. A possible solution to overcome this problem is to extract objects by segmenting the laser data. By segmentation of laser altimetry data we understand a partition of original data into a background and significant natural and man made objects. The segmentation of laser data has often been performed with using an external data source like 2D GIS data or an image, which is not considered as an alternative solution for this research work. In this research we employ the concept of object separation, which is broader than segmentation. It does not achieve image segmentation in a single go. Instead, we use different methods for extracting different objects, which are finally added to produce the final output. Overlap between objects is permissible. Hence we obtain more degrees of freedom over the number of possible algorithms and techniques to be used.

1.3 Objective

The objective of research work is to assess the potential of wavelet analysis for object extraction from laser data. To do so we will use wavelet transform. With wavelet transform we transfer the laser height image in to space-scale domain, perform different operations to highlight the properties of desired objects and transform it, back to original domain with inverse transform.

1.4 Research questions

To achieve the objective of this research we define the following research questions as a guideline for our research efforts:

�� How do we recognize different objects from laser data and which interpretation elements can be modeled by computer?

�� Which features can we extract from laser data by wavelet-based analysis?

�� In which domain does the separation between objects and background exist?

�� Which wavelet transforms are useful to apply?

�� How should we choose the best mother wavelet, and what is the effect of such choice?

�� Can we design a custom mother wavelet, which can be targeted to extract specific object?

�� How to extract edges with wavelet analysis and how to apply such method for our task?

�� Can we benefit by using texture segmentation methods?

�� How to overcome translation and rotation invariance?

�� Which clustering algorithm is most appropriate to use?

�� What is the effect of input data resolution for object separability?


3

1.5 Research limitations

This research work does not intend to develop a fully automatic object extraction system or algorithm. Adaptability of our research for scenes with complex building is considered to be of secondary importance. Finally, we do not intend for an in-depth look for all the routines of wavelet theory or segmentation. Therefore, mathematical proofs are not given. More importantly, we remain on the application side of wavelet theory.

1.6 Prior work

The name “wavelets'' was introduced in the late nineteen seventies by Morlet, a French geophysicists [46]. The seismic data he was studying typically exhibited rapidly changing frequency contents for which Fourier analysis did not suffice as an analyzing tool. For this reason he investigated the existence of functions, with sufficient compact support in both the time and the frequency domain, and called these wavelets. Grossman (1980) has provided the mathematical basis of Morlets ideas, which has triggered the start of the construction of a complete mathematical framework, nowadays known as wavelet theory, and a still rapidly growing number of applications of this framework [36].

Wavelet theory is not really a whole new theory. Rather, it is the result of synergy and generalization of ideas and concepts known in several fields like geophysics, signal (and image) analysis and compression, physics and mathematics (statistics). Some essential results in the progression of this field were achieved by the cooperation of Ingrid Daubechies (1992) and Stephane Mallat (1991). This research resulted in a comprehensive mathematical framework for signal analysis with applications for instance in physics, speech processing, image coding, image recognition and segmentation, denoising, density estimation etc.

Because of novelty of wavelets only few educational literatures is available. The majority of the articles and books are written by mathematicians, that is difficult to understand without strong mathematical background. As a consequence, wavelet theory and its application is not fully understood and appreciated by vast majority of scientists. The fact that very little research work being done on application of wavelet in remote sensing indicates that many chances exist for research opportunity.

The only paper on wavelets application in object extraction from laser data was published by T.Thuy [58,59]. In his work, he used a method proposed by Mallat and Zhong [31,32,33] for extracting multiscale-edges by detecting local modulus maxima in 2D dyadic wavelet transform. The idea behind his research is that edges, which are consistent over scales, are those of objects. This work demonstrated that the multiscale approach has a good potential for separation of objects at different scales. Some problems of increasing edge ambiguity at coarse scales were encountered. The solution to this problem was proposed by using existing 2D GIS data.

Besides the application of wavelet analysis, many alternative methods exist for object extraction, proposed by different authors. Most of these are based on the simple fact that objects (buildings, trees) are higher than the terrain. Some of these methods extract digital terrain model (DTM) from digital surface model (DSM) and apply a morphological filter [61]. Others apply filter with various window sizes, assigning the probability of the pixel being object or terrain based on window size and values of highest and lowest points within that window [38]. The idea of using various window sizes can be seen as a multiscale approach. An interesting idea of detecting buildings by looking at the consistency of the shape of building over several successive horizontal slices of DSM belongs to Q.Zhan [67]. Another alternative solution was found by applying statistical texture segmentation methods for laser scanner data [14,28].

Summarizing the previous research works, we can conclude that research work on object extraction from laser data triggered by advent of laser scanner technology is booming. The potential of laser scanning for urban area steel needs to be realized in applications such as 3D city modeling, and 2D


4

GIS database update. There is still enough room for creativity and testing new ideas and tools such as wavelet analysis.

1.7 Research approach

A detailed description of the methodology is given in chapter four. In principle, our approach follows the classical object extraction scheme, which basically relies on clustering of features where object separation exists. For the propose of estimating the “best” possible features we will work with artificial datasets, with some real world object properties. For example some scenarios with buildings with different roof types on inclined and flat terrain are tested. The successive methods will be tested next with a real world dataset. Consequently, these methods will be refined and improved. The main framework of this research comprises a number of experiments, listed below. 1.7.1 Experiments with mother wavelet

In fact, the decomposition results depends on the choice of analyzing wavelet i.e. it’s corresponding filter that are used. The choice of mother wavelet depends whether one needs to obtain better resolution in time or frequency. The design and proper choice of the wavelet function for diverse tasks comprises a considerable part of wavelet research. We will test a number of different predefined mother wavelets as well as designing the specific wavelets that can be targeted to separate objects such as trees and buildings. 1.7.2 Experimenting with wavelet transform

A number of alternative wavelet transforms, can be used. These are:

�� Continuous wavelet transform (CWT)

�� Discrete wavelet transform (DWT)

�� Discrete dyadic wavelet transform (DDWT)

�� Tree structured wavelet decomposition

�� Discrete wavelet frames (DWF)

�� Adaptive discrete wavelet packets

�� Wavelet transforms with lifted scheme

All transforms are explained in some detail in chapter three. While some are designed for a specific application in mind, we need to explore and if necessary test them all. Such experiments however are restricted by software availability capable of doing such analysis with images.

Special attention is given to the wavelet transform using the lifting scheme, as it offers perspective for the processing of raw laser data. 1.7.3 Feature extraction

Feature extraction is an essential part of research. This is due to the fact that wavelet coefficients that are generated as a result of the wavelet transform cannot be directly used as features in clustering algorithms. Thus, feature extraction can be viewed as an interface layer between the wavelet transform and a clustering algorithm. Any method that is used for feature extraction process, should produce


5

features that are acceptable to the chosen clustering algorithm, but not at the expense of decreasing the information content of the features themselves. In addition, we would expect the feature extraction algorithm to be consistent among the pixels within a class but not to possess a high degree of discernability between classes. 1.7.4. Clustering algorithm selection Finally, after extracting features that can differentiate objects, the original data are segmented by using classifiers. The multivariate k-means clustering algorithm and the watershed region-growing algorithm are of primary interest. The choice of a classification algorithm and the number of possible features to be extracted offers some degrees of freedom for the research work.

1.8 Structure of thesis

The thesis is organized into six chapters, which can be described as follows: Chapter 1. Introduction - gives an overview of thesis work, problem definition, research objectives and questions, prior work, methodology, and thesis structure. Chapter 2. Laser scanning - provides brief information about laser scanning technology, an overview of applications, its present status and future expectations. Chapter 3. Wavelet theory – introductory part of the wavelet theory, its application in image processing, wavelet transforms and wavelet design. Chapter 4. Object separation methodology – description and justification of the selected approach for object separation. Chapter 5. Experimental results – presentation of intermediate and final results, findings and experience. Chapter 6. Discussion – summary of experiments and research questions answered Chapter 7 Conclusions and recommendations- summarizes the conclusions drawn from the research work, gives recommendation for further work.


6

Chapter 2

Laser scanning

2.1. Principles of laser scanning and system design

Laser scanner is an active sensing systems using laser beam as the sensing carrier. All laser systems measure the distance between the sensor and the illuminated spot on the ground. Airborne laser scanning systems use a combination of three advanced technologies:

�� Laser rangefinders LIDAR (Light Detection and Ranging)

�� highly accurate inertial navigation systems (INS)

�� and the global positioning satellite system (GPS). By integrating these subsystems in to a single instrument mounted in a small aircraft or helicopter, it is possible to rapidly acquire sub randomly distributed 3D point clouds of the terrain beneath the flight path of the aircraft.

2.1.1 Laser scanning

The ranging unit comprises the emitting laser and the electro-optical receiver. Apertures of these devices are sharing the same optical path. This assures that emitted laser beam is always in field of view (FOV) of the receiver. The narrow divergence of the laser beam defines the instantaneous field of view (IFOV), which is typically 0.3-2mrad. Due to narrow IFOV of the laser, the optical beam has to be moved across the flight direction by scanning mechanism. The second dimension is realized by forward motion of airplane [60].

A laser (light amplification by simulated emission of radiation) is a powerful highly directional optical light beam, which is often highly coherent both in space and time. However, for laser scanning only high collimation and optical power of laser are required.

In range measurements with laser, two major principles are applied: the pulsed ranging and phase difference between transmitted and received signal also referred as continuous wave laser. Although different physical effects are used, both principles measure the traveling time of a signal. Most of

figure 2.1: Principle of laser scanning


7

current laser rangers are pulse based. Pulse lasers are mostly solid-state, which produces high power output.

The travel time (t) is covered to distance (R) from the plane to the ground based on speed light (c).

ctR ×=21

The range resolution is determent by time resolution t∆

tcR ∆=∆21

,

and can achieve sub decimeter level.

Typically, a laser ranger works on wavelength 800 to 1600nm (x10-9) infrared pulses, which is invisible to humans. However the eye safety issue is still big concern in lower wavelengths 800-1000nm. The reflectivity property of the object surfaces is directly affected by the choice of wavelength. These and other factors causing loss of laser power, influence the maximum range distance, which can reach up to 1600m. The sampling pattern is determent by the laser ranger design and plays an important role during processing stages. The across-track sampling spacing depends on the pulse repetition frequency and scanning mechanisms of the scanner. There exist three types of scanning mechanism: oscillating mirror, Palmer scan, fiber scan and rotating polygon [60]. The along-track spacing is determent by aircrafts speed and the link frequency. 2.1.2 Positioning and orientation system

The distance measurements are converted to geographic coordinates and elevations for each laser pulse by combining the distance data with information on the position of the airplane at the time and the direction in which the laser pulse was fired. The position of the aircraft during the entire flight path is recorded by using a differential Global Positioning System. The direction of the laser pulse is established by using an Inertial Navigation System (INS), which measures the orientation of the airplane. All these devices must be precisely synchronized to get readings from devices at the same time. All this information together with calibration data is processed to geocoded 3D point data with x,y,z coordinates in a local coordinate system.

2.2 Extended capabilities

Recent laser altimeter systems are capable of measuring multiple returns from each laser pulse. The footprint of the laser beam may vary from few centimeters up to 2.5 m depending on the height of aircraft and aperture of LIDAR. If this wide laser pulse reflects from more than one feature, distances to the multiple features can be measured. This property can be beneficially utilized for DTM generation, forestry applications and object extraction.

Another supplementary information is provided when the electro-optical receiver of laser scanner records not only the traveling time but also the intensity of reflected laser beam. The reflectance can be seen as an image in a very narrow wavelength (near infra-red) band. This information can contribute to classification algorithms, enabling the separation of vegetation.


8

2.3 Laser altimetry data and its processing

Although the technology of laser scanning is in a fairly mature state of the art while, the processing of airborne laser scanner data still is in an early phase of development [3] . What very much remains is the development of algorithms and methods for interpretation and modeling of laser scanner data, resulting in useful representations and formats for an end-user.

The processing of raw laser scanner data (hereafter laser data1) often aims at either removing unwanted measurements, either in the form of erroneous measurements or objects, or modeling data given specific model. Removing unwanted measurement is referred as filtering, finding specific object is known as classification and the generalization of classified objects is referred as modeling [3]. Filtering, classification and modeling are thus defined according to aim and not to method.

Usually the laser data needs to be processed with algorithms, which came from different application like image processing, and statistics. Often these algorithms require the input data to be a regular grid of samples. However some information is lost from original sub-randomly distributed 3D point clouds if the data are interpolated in-to a regular grid. The loss of information can be significant, especially if multiple reflections were registered, since the points with similar xy coordinates but with different elevation poorly represented in regular grid.

Although most of the applications require special algorithms and strategies for classification and interpretation of laser data, one task remains common for them: that is separation of objects from the ground surface. In context of this research work it can be said that finding the separation between objects and terrain using wavelet analysis is the main objective of this research.

2.4 Application of laser scanning in urban environment

It is common to new technologies that their technical potential soon opens up a new area of application. Laser scanning is expanding its application area beyond the DTM generation. A particularly interesting new application of laser scanning concerns the automatic extraction of buildings and built up areas for 3D city-modeling purposes. Buildings and constructions masking the ground, were originally considered as obstructions to be removed in the DTM generation. In the meantime, the recognition of building objects has become an important independent task. In the built-up areas, many laser point’s lie on top of building structures. With high density of sampling, the vertical geometric distribution of the raw laser data allows the delineation of buildings in very close approximation. This opens new potential for automatic extraction of buildings from laser data.

1 After interpolation of laser data in to regular grid we obtain an image referred as height or range image, laser altimetry image and DSM. For the sake of simplicity we use ‘laser data’ throughout this thesis.


9

Chapter 3

Wavelet theory

This chapter is devoted to the theory of wavelets, wavelet decomposition, multiresolution analysis and the wavelet lifting scheme. To give a full overview of wavelet theory would require more than a single book. We will limit ourselves to the basics concept of wavelet analysis, which provides a basis for the application of wavelet in object extraction from laser data. The interested reader is referred to [34, 40,56] for a more in-depth look at theory of wavelets. The theory of wavelets is difficult to understand without a strong mathematical background. However, it can be explained in an intuitive way that is understandable for the readers who are new to this subject. As an introduction to the wavelet, a simple example of the Haar wavelet transform will be presented. Starting with the basic Fourier transform, the introduction will continue by a discussion of its shortcomings, including the Heisenberg uncertainty principle. The principles of wavelets and wavelet transforms are explained by the continuous wavelet transform (CWT). Furthermore, properties of wavelet and scaling functions are explained and continued by a discussion on discretization of the wavelet transform. Finally, the discrete wavelet transform (DWT) will be introduced as well as issues of its practical implementation for images and a background for fast lifted wavelet transform will be discussed.

3.1 Simple example of Haar wavelet transform

Consider the numbers x2i and x2i+1 and think of them as two neighboring samples of the real axis. The values of x2i and x2i+1 have some relationship, which we would like to take advantage of. We first apply a simple linear transform, and replace x2i and x2i+1 by their average a and difference d:

2122 ++= ii xx

a

ii xxd 212 −= +

We have not lost any information because for given a and d we can always recover the xi and xi+1 pair as:

22

dax i −=

212

dax i +=+


10

If values of x2i and x2i+1 are almost similar, then the expected absolute value of the difference d will be very small and can be neglected, and these two numbers can be represented by their average a. This simple observation serves as key behind the Haar wavelet transform (fig. 3.1). Similarly, we can apply this pair-wise transform for signal sn of 2n sample values sn,l:

}20|{ ,n

inn lss ≤≤=

The input signal is split into two signals: with 2n-1 averages a and details d. Given average and detail, one can restore the original signal sn . We can think of the averages an-1 as a coarse resolution representation of the signal sn and of the differences dn-1 as the information needed to go from the coarser representation back to the original signal.

Figure 3.1 : Haar wavelet transform

We can apply the same transform to the coarser resolution representation an-1 and split it in to a coarse signal an-2 and detail dn-2 , each of these containing 2n-2 samples. At the end a hierarchal set of approximations and details emerges.

3.2 Fourier transform

It is well known from Fourier theory that a signal2 f(t) can be expressed as the linear combination of two basic functions sine and cosine, with different amplitude phase and frequencies:

2 The signal is any complex valued function of time f(t). However equations for the time variable can be equally valid for other generic cases, e.g. spatial variable.


11

��∞

∞−

∞

∞−

+= dtttfjdtttfF )sin()()cos()()( ωωω . (2.0)

This can be also expressed in an exponential form:

�+∞

∞−

−= dtetfF tj πωω 2)()( (2.1)

where 1−=j and e denotes the natural exponent,

tjte tj ωωω sincos += (2.2) with an inverse transform defined as:

ωω πω deFtf tj�

+∞

∞−

= 2)()( (2.3)

where f(t) is the signal in time, and F(ω) is the corresponding frequency function and ω is the frequency to analyze. Fourier coefficients, represent the spectral components of f(t). The complex exponential functions at different discrete frequencies of 2πjk/T are not compactly supported in time since they extend to infinity. This means that a Fourier transform has only a frequency resolution but not a temporal resolution.

If we want to have both temporal and frequency resolution, then time-frequency joint representation is required. To get such a representation, we cut the signal into several parts and then analyze these parts separately. This technique is known as Windowed Fourier transform and Short Time Fourier Transform (STFT):

�−∗ −= dtetwtfS tjπωτωτ 2)()(),( (2.4)

ωττωτ πω

τ ω

ddetwStf tj2)(),()( −= ∗� � (2.5)

where w(t) is a windowing function, ω and τ are frequency and translation parameters respectively, * denotes complex conjugation, and S(τ,ω) is the STFT of f(t) at frequency ω and translation τ. Note that for each frequency ω, time localization is obtained through segmenting f(t) by w(t-τ), the windowing function centered at t=τ. The Fourier transform of this segmented signal then provides the frequency localization. The problem with this approach is that it provides a constant resolution for all frequencies since it uses the same window for the analysis of the entire signal. Narrow windows however correspond to wider frequency bands, resulting in to a poor frequency resolution. On the other hand, using a wider window we have a good frequency resolution at the expense of the time resolution. This can be explained by Heisenberg’s uncertainty principle, which states that it is impossible to know the exact frequency and exact time of the occurrence of this frequency in a signal. In other words, a signal simply cannot be represented as a point in the time-frequency space [42].

3.3 Wavelets: The state of the art

The wavelet transform (WT) or wavelet analysis is probably the most recent solution to overcome the shortcomings of the Fourier transform. In wavelet analysis the use of a fully scalable modulated


12

window solves the signal-cutting problem. The window is shifted (or translated) along the signal and for every position the spectrum is calculated. This process is repeated many times with a slightly shorter (or longer) window for every new cycle. In the end a collection of time-frequency representations of the signal, all with different resolutions emerges. Because of this collection of representations we speak of a multiresolution analysis. For wavelets, we usually do not speak about time-frequency representations but about time-scale representations; scale being in a way the opposite of frequency, because the term frequency is reserved for the Fourier transform. Let’s now formulate the WT in mathematical language. 3.3.1. Wavelets definition A wavelet ψ(t) is a function in the form of wave that has effectively a limited extent. It has an average value of zero. The wavelet function can be defined with any function ψ(t) that satisfies following conditions: 1. Its square integral is finite –the wavelet must have a finite energy (small).

�+∞

∞−

∞<dtt 2)(ψ (3.1)

This can be summarized by saying that the ψ(t) ∈ L2(R), functions with finite squared integral. 2. The integral of wavelet ψ(t) equals zero and therefore it must be oscillatory. In other words it must be a wave.

�+∞

∞−

= 0)( dttψ (3.2)

3.3.2. Continuous wavelet Let us denote a variable size windowing function as a wavelet function ψ(t) with two parameters a and b as scaling and translation parameters respectively. Then the translation and scaling of the wavelet function can be expressed as :

ψ(t-b) and ��

��

�

at

aψ1

respectively.

Then the combined translated and scaled version is obtained from a prototype function, the mother wavelet ψ(t), as

( ) ℜ∈>��

��

� −= baa

bta

tba ,0 ,1

)(, ψψ (3.3)

Now the STFT formula (2.4) can be rewritten as an inner product of f(t) with the scaled and translated versions of the basis functions ψ (psi):

( ) dtttfbaW

dttntranslatioscaletfntranslatioscaleW

ba�

�∗=

=

)()(),(

),,()(),(

,ψ

ψ (3.4)

the expansion of which equals:

dta

bttf

abaW � �

�

��

� −= ∗ψ)(1

),( (3.5)


13

where * denotes complex conjugation.

From an intuitive point of view, the CWT computes a “resemblance index” or correlation between signal and the analyzing wavelet at position b and scale a. If the signal has a spectral component that corresponds to the current scale, the product of the wavelet with the signal at the location where this spectral component exists gives a relatively large value. If the spectral component that corresponds to the current scale is not present in the signal, the product value will be relatively small, or zero. Scaling a, as a mathematical operation, either dilates or compresses a signal. Larger scales correspond to dilated (or stretched out) signals and small scales correspond to compressed signals3. By applying the inverse wavelet transform, the function f(t) can be synthesized from the wavelet coefficients:

21 ),()(

adbda

abt

baWCtfa b

⋅��

��

� −= � �− ψψ

(3.6)

where W(a,b) is the continuous wavelet transform (CWT) of f(t), and Cψ is an admissibility constant, which satisfies the following condition (admissibility condition):

∞<= � ωω

ωψπψ dC

2^

)(2 (3.7)

which depends on ^

ψ , being the Fourier transform of ψ.

If ^

ψ is continuous, condition (3.7) is reduced to:

�+∞

∞−

=⇔= 0)(0)0(^

dttψψ (3.8)

Meaning: ψ(t) has zero mean. It can be proven that the admissibility condition is necessary to ensure that the wavelet transform is invertible [63]. 3.3.3 Wavelet properties Compact support. The condition of (3.1) implies that the basis functions are non-zero only on a finite interval while the sinusoidal basis functions of the Fourier transform are infinite in extent. Localization. The wavelet function must have localization both in frequency and time. In other words the wavelet basis functions must have zero average (3.2) to allow the WT to efficiently represent functions or signals, which have localized features. There is one more condition, which is the regularity condition stating that the wavelet function should be smooth and concentrated in both time and frequency domains. We will explain this using the concept of vanishing moments.

3 Warning: the understanding scale in wavelet is completely opposite to which we use in map scales, large scales means less detail, lower frequency components, smaller scales means more detail. This is something that reader should keep in mind throughout this thesis.


14

If we expand the wavelet transform (3.2) into the Taylor series at t = 0 until order n (let b = 0 for simplicity) we get:

��

�++�

�

��

�= ��=

)1(!

)0(1

)0,(0

)( nOdtat

pt

fa

aWpn

p

p ψ (3.9)

here f(p) stands for the pth derivative of f and O(n+1) means the rest of the expansion. Now, if we define the moments of the wavelet by Mp,

�= dtttM pp )(ψ (3.10)

then we can rewrite (3.9) as a finite development (3.11)

��

�+++++= ++ )(

!)0(

...!2

)0(!1

)0()0(

1)0,( 21

)(3

2

)2(2

1

)1(

0nn

n

n

aOaMn

faM

faM

faMf

aaW (3.11)

From the admissibility condition we already have that the 0th moment M0 equals zero. Therefore the first term in the right hand side of (3.11) is zero. If we now manage to make the other moments up to Mn zero as well, then the WT coefficients will decay as fast as an+2 for a smooth signal ƒ(t). This is known in the literature as the vanishing moments or approximation order. If a wavelet has N vanishing moments, then the approximation order of the wavelet transform is also N. The moments do not have to be exactly equal to zero, a small value is often good enough. The regularity condition allows us to analyze small-scale fluctuations by ignoring regular polynomial components of the signal [21]. 3.3.4 Discrete Wavelet Transform Neither the FT, nor the STFT, nor the CWT can be practically computed by using analytical equations, integrals, etc. To make digital computations possible, a signal is sampled and in that case, a signal is a time series i.e., a function of discrete time variable. For that the CWT must be discretized. Discrete wavelets are not continuously scalable and translatable but can only be scaled and translated in discrete steps. This is achieved by modifying the wavelet representation (3.3):

��

��

� −= jk

j

jkj sskt

st 00

0

,

1)(

τψψ (3.12)

Figure 3.2: Localization of the discrete wavelets in the time-scale space on a dyadic grid.

Although it is called a discrete wavelet, it normally is a (piecewise) continuous function. In (3.12) j and k are integer scaling and translation factors respectively and s0 > 1 is a fixed dilation step. The translation factor τ0 depends on the dilation step. The effect of discretizing the wavelet is that the time-scale space is now sampled at discrete intervals. If we choose s0 = 2 and τ0 =1 then sampling of the


15

frequency axis corresponds to dyadic sampling (fig 3.2) and scaling and translation parameters obtain the form:

Zkjka jj ∈== ),( ,2b ,2 , (3.13)

Now, if we apply these to (3.3), then we obtain the discrete wavelet:

Zkjktt jjkj ∈−= −− ),( ),2(2)( 2/

, ψψ , (3.14)

and by applying this wavelet to (3.2) we obtain the discrete wavelet transform (DWT) and this will result in a series of wavelet coefficients kjC , and also referred to as wavelet series

decomposition(3.15).

dtttfC kjkj )()( ,, ψ�+∞

∞−

= (3.15)

Due to the discretization the inverse DWT (IDWT) is not directly available. The discrete wavelet must satisfy some additional constraints, which will be described further. The different DWT schemes can be obtained depending on which constraints are satisfied. Generally, it can be subdivided in to redundant and non-redundant types: 1. Redundant if a digitized signal consists on N samples then a redundant DWT maps it on M>N samples. For this DWT it is necessary and sufficient that the energy of the wavelet coefficients must lie between two positive bounds, i.e.

22

,,

2)(),()( tfBtftfA

kjkj ≤≤� ψ (3.16)

where 2

f is called the energy of f(t), denotes scalar product, and 0<A≤ B<∞ ; A,B are

independent of f(t) and called frame bounds, and a tight frame when A=B. The family of basis functions ψj,k(t) with j,k ∈Ζ , i.e. frames are generalization of a basis. 2. Non-redundant. For removing redundancy, orthogonality of wavelet basis is required. The discrete wavelets can be made orthogonal to their own dilations and translations by special choices of the mother wavelet, which means:

�� ==

=otherwise

nkandmjifdttt nmkj 0

1)()( *

,, ψψ (3.17)

At same scale at different scales

Orthogonal wavelets are called orthonormal if it’s normalized to have the norm in L2


16

� ==∀ 12dtiii ψψ

The orthonormality of the wavelets has a very important mathematical and engineering consequence: any continuous function may be uniquely projected onto the wavelet basis functions and expressed as a linear combination of the basis functions.

This function can be reconstructed by summing the orthogonal wavelet basis functions ψj,k(t), weighted by WT coefficients kjC , :

��=j k

kjkj tCtf )()( ,, ψ (3.18)

We will now discuss a particular wavelet transform introduced by Mallat [34]. It is based on a discrete wavelet frame with a dyadic decomposition. The advantage of this scheme is that it can be implemented by applying Finite Impulse Response (FIR filters).

3.3.5 Scaling function

Let us define the frequency axis from time-frequency plane, which we are covering with wavelet analysis as a frequency spectrum. It is apparent that it is impossible to cover the whole frequency spectrum with only wavelets. Because every time we stretch the wavelet in the time domain with a factor of 2, its bandwidth is halved. In other words, with every wavelet stretch we only half the remaining spectrum. This means that we need an infinite number of wavelets. The solution to this problem is not to try to cover the spectrum all the way down to zero with wavelet spectra, but to use a so-called scaling function ϕ(t) or father wavelet. The scaling function was introduced by Mallat. Because of the low-pass nature of the scaling function spectrum it is sometimes referred to as the averaging filter.

Figure 3.3 : How an infinite set of wavelets is replaced by one scaling function. If we consider the scaling function as a signal with a low-frequency spectrum, then we can decompose it in wavelet components and express it as:

�=kj

kjkj tt,

,, )()( ψγϕ (3.19)

Where kj ,γ are the scaling function coefficients. The low-frequency spectrum of the scaling function

allows us to state some sort of admissibility condition similar to (3.7)

� =1)( dttϕ (3.20)

which shows that the 0th moment of the scaling function can not vanish.


17

Since the scaling function ϕ(t) neatly fits into the low-frequency spectrum left open by the wavelets, the expression (3.18) uses an infinite number of wavelets j (figure 3.3).

The scaling function )(tφ and its associated mother wavelet function )(tψ , which are needed in the construction of a complete basis, must satisfy the two-scale difference equations:

�

�

−=

−=

k

k

ntngt

ntnht

)2()(2)(

)2()(2)(

φψ

φφ (3.21)

where the coefficients )(nh and )(ng satisfy the following:

)1()1()( nhng n −−= (3.22)

The coefficient h(n) in (3.21) have to meet several conditions for the set of basis wavelet functions in (3.14) to be unique orthonormal, and have certain degree of regularity.

Similar to the construction of )(, tkjψ , a family of orthonormal basis )(, tkjφ can be obtained through

translation and dilation of the kernel )(tφ .

Zkjktt jjkj ∈−= −− ),(),2(2)( 2

, φφ (3.23)

3.3.6 Multiresolution approach

A series of nested subspaces jV , which is spanned by the orthonormal basis Zktkj ∈ ),(,φ , forms a

multiresolution space.

The subspace jW , which is spanned by the orthonormal basis Zktkj ∈ ),(,ψ , is the complementary

space of jV in the subspace 1−jV (fig. 3.4)

jjj WVV ⊕=−1 (3.24)

The subspaces jV and jW are called the approximate and residue spaces respectively at resolution j .

Figure 3.4 : Schematic representation of multiresolution subspaces


18

Now in (3.18) fix j and sum according to k a detail Dj is the function:

�∈

=Zk

kjkjj tCtD )()( ,, ψ (3.25)

and after summation on j, the signal is seen to equal the sum of all details:

�∈

=Zj

iDtf )( (3.26)

For a reference scale J, two sorts of details exist. Those associated with indices j ≤ J corresponding to the scales k=2j ≤ 2J which are the fine details. The others, which correspond to j > J, are the coarser details. We can group these later details in to:

�>

=Jj

jJ DA (3.27)

which defines what is called an approximation of the signal f(t). Hence

�≤

+=Jj

jJ DAtf )( (3.28)

the signal is sum of all finer details and its approximation at Jth scale. Which is same as (3.24).

3.3.7 Fast wavelet transform

For the discrete case, the coefficients )(nh and )(ng in (3.21) play an important role because the continuous forms of )(tφ and )(tψ can be neglected, and the coefficients can be directly applied to the discrete signal using the following iterations:

�

�

−=

−=

+

+

kjj

kjj

nkgkcnd

nkhkcnc

)2()()(

)2()()(

1

1

(3.29)

where )(nc j and )(nd j are the coefficients at resolution j .

Thus, for a J –level discrete wavelet decomposition of the given coefficients )(0 na , a series of

coefficients, { )(ncJ , )(nd J , …, )(1 nd }, is obtained. Because of the orthonormality of the wavelet transform, the number of coefficients after the decomposition is equivalent to that before decomposition. The synthesis of the signal from the wavelet coefficients obeys the following:

�� −+−= ++k

jk

jj nkgkdnkhkcnc )2()()2()()( 11 (3.30)

If )(~

nh and )(~

ng are defined as

)()( ),()(~~

ngngnhnh −=−= , (3.31)

It is convenient to view the decomposition (3.29) as passing a signal )(kc j through pair of filters H

and G with impulse responses )(~

nh and )(~

ng , and downsampling the filtered signals by factor of


19

two. The pair of filters H and G correspond to the halfband lowpass and highpass filter respectively, and are known as quadrature mirror filters (QMF) in the signal processing literature.

When the signal is convolved with a highpass and lowpass filters, the resulting high and lowpass signals can be subsampled by a factor of 2, without loss of information. The convolution (and subsampling) is repeated iteratively on the lowpass signal. The result of this process is a pyramid of signals with successive frequency content, which is called standard wavelet decomposition (fig 3.5). The reconstruction procedure is implemented by upsampling the subsignals )(1 kc j+ and )(1 kd j+ by

inserting zeroes between neighboring samples and filtering with )(nh and )(ng , respectively and adding these two signals together.

Figure 3.5: Wavelet decomposition through filtering; Lo_R and Hi_R denotes convolution and 2↑ (2↓)downsampling (upsampling) by factor of two.

Alternatively, the decomposition can be iterated not only on the lowpass but on the high pass signals as well. This results in a binary tree of signals, which is called tree-structured wavelet or wavelet packet decomposition [63]. The tree structured scheme without subsampling is known as discrete wavelet frames (DWF). When the decision whether of decomposition of subimage does not depend on it is high or low frequency component, but subject of some criteria such as energy such schema is called adaptive discrete wavelet packets (DWP).

The filtering approach is most widely used because it is easy to understand and familiar in a signal processing context. The results of the decomposition depend on the filters (or corresponding wavelets) that are used. The design and proper choice of wavelet functions for diverse tasks comprises a considerable part of the wavelet research efforts. It is not possible to find a set of quadrature filters for every wavelet function. In practice, it are the filters that are constructed in such a way that their corresponding functions have the desired wavelet properties. A very elegant alternative for constructing and computing DWT's has recently been proposed by Wim Sweldens [62]: the so called lifting scheme which we will discuss later.

3.3.8 2D Wavelet Decomposition

So far we have discussed only wavelet analysis of a one-dimensional signal. Now we will explain how wavelets work for the 2D case. It is easy to extend the above 1-D derivations to the 2-D case. As we already stated 1D analysis is based on one scaling function ϕ and one wavelet ψ. The usual 2D wavelets are defined as tensor products of 1D wavelets: ϕ(x,y)= ϕ(x)ϕ(y) is the scaling function and ψ1(x,y)= ϕ(x)ψ(y), ψ2(x,y)= ψ(x)ϕ(y), ψ3(x,y)= ψ(x)ψ(y) are the three wavelets. Examples of the 2D wavelet given in fig(3.6)


20

Figure 3.6 : 2D coiflet wavelet

We can easily adapt pyramidal DWT for 2D case. The 2-D filter coefficients can be expressed as:

)()(),( ),()(),(

),()(),( ),()(),(

lgkglkhnhmgnmh

lgkhlkhnhmhnmh

HHHL

LHLL

====

(3.32)

where the first and second subscripts denote, respectively, the lowpass and highpass filtering along the row and column directions of the image.

As far as computation is concerned, due to the separability of the filters, the wavelet transform can be implemented (convolution and downsample) along the rows and columns separately. Fig. 3.7 shows how to implement the wavelet decomposition and reconstruction of an image. Fig. 3.8 gives two practical examples of wavelet decomposition.

Figure 3.7 : 2D Decomposition (top) and reconstruction (bottom) schemes for DWT with QMF. Lo_R and Hi_R denotes convolution and 2↑ (2↓)downsampling (upsampling) by factor of two.


21

a)

b)

Figure 3.8 : a) Typical organization of the detail images with DWT, b) example of DWT of image with circle; please note the directionality of the detail images (column, row, diagonals)

3.3.9 DWT with Lifting scheme The ‘‘lifting scheme’’ is a new approach for the construction of families of wavelets that are independent of the Fourier transform [17]. Constructing wavelets using lifting consists of three simple phases or stages (fig. 3.9):

Figure 3.9: Wavelet lifting scheme 1. Split, This stage is also known as lazy wavelet transform and simply splits the signal into two subsets. Nothing prevents from splitting the signal in to two subsets, therefore it can be done even on the data with irregular sample location.

2. Predict calculates the wavelet coefficients (high pass) as the failure to predict the odd set based on the even (fig. 3.10. b).

)( 111 −−− −= jjj evenPoddd

The construction of prediction operator P is based on some model of the data, but not on the data itself, for example linear interpolation prediction (fig. 3.10).


22

a)

b)

Figure 3.10: linear prediction of odd samples based on even 3. Update, updates the even set using the wavelet coefficients to compute the scaling function coefficients (low pass).

)( 111 −−− += jjj dUevens

We need U update operator because we need to preserve some scalar quantity like average (fig. 3.11.a) of the original over the successive scales.

a)

b)

Figure 3.11: Update stage The advantages of lifting are numerous:

1. Lifting allows for an in-place implementation of the fast wavelet transform, a feature similar to the fast Fourier transform.

2. It is particularly easy to build wavelet transforms that map integers to integers using lifting.


23

3. Lifting enables the construction of wavelets entirely in the spatial domain, i.e., without making use of the Fourier transform. This means that it can be used to build wavelets that are not necessarily translates or dilates of one function. These wavelets are known as ‘‘second-generation wavelets’’ and typical examples are wavelets adjusted to weight functions, irregular samples, the sphere, or manifolds.

4. Every transform built with lifting is immediately invertible where, the inverse transform has exactly the same computational complexity as the forward transform

5. Lifting allows for adaptive wavelet transforms, i.e., one can start the analysis of a function from the coarsest levels and then build the finer levels by refining only in the areas of interest.

These properties promise a good potential for its application in image analysis, especially for laser data because we would prefer to work on laser scanning point cloud data rather than the resulting resampled image. By now we have reviewed the essentials of the wavelet theory. The next chapters will be dealing with application of wavelet analysis for object extraction from laser data. But before we start we need define what is signals, image, and laser data.


24

Chapter 4

Object separation methodology

This chapter explains our methodology for object extraction from laser data. There are a number of different approaches in object extraction. Our choice is limited by the nature of the data and the target objects to be extracted. Some methods within pattern recognition are designed to find a match between unknown parameterized objects and the existing object models in the database. Application of these methods for object extraction from laser data is limited due to complexity of object forms to be extracted. In this research we use a classical object extraction scheme of feature extraction followed by clustering.

First we define and describe how to extract features that are generic to our objects. Then the resulting features will be used for clustering in feature space to define the number of classes. The final output will be a segmented image, where each pixel is assigned to a segment, which is either an object or part of an object.

The accuracy of segmentation largely depends on features that can distinguish the objects. In the following sections we will discuss how this can be possibly achieved. We start from modeling of the objects to be extracted. Then a possible alternative of research software to be used is discussed, followed by description of the data.

4.1 Modeling

We will start from a generic question: “how do we recognize objects from the laser data?” From the variety of features, which characterize the objects we select only those, that can be extracted from laser data. With laser data, where we only have elevation measurements, the number of possible features will be very limited. We will model the object’s features and, if possible, extract them by wavelet-based analysis. The following diagram shows objects and their features that can be extracted from such data. (figure 4.1) The superclass of geographic objects can be divided into man-made and natural objects. A Man-made object have a consistent structure, which follows some geometry constraints. We will rely on this property to distinguish man-made object. The building objects can be described as follow:

�� The building extent is limited by sharp edges

�� a man made object is build from smooth planar surfaces and can be described by piecewise linear function

�� buildings are objects with height


25

�� the size of a building has a certain range

-S ize-T extu red-E leva ted-R e flec tance (IR )

Tree

-P lanar S urface-E leva ted-E dges-S ize

b u ild in g

-S m ooth su rface

M an m ad e ob ject N atu ra l o b ject

G eo g raph ic O b ject

-H orizon ta l-G round leve l

R oad

-N o re flec tion

W ater b od ies

Figure 4.1: Object model

The road objects can be described by:

�� having a horizontal smooth surface

�� having linearity and consistency of width

�� being a ground level object (not always)

In contrast to man-made objects, natural objects are created by natural force. Therefore it is hard to find uniform properties at the higher object hierarchy.

Tree objects can be modeled by:

�� being elevated objects

�� having a high height variation in its local neighborhood, thus textured in the height image

These properties cannot explicitly separate all objects, because of the complex urban situation, diversity of building design etc. From all the possible objects we have selected only buildings and trees, as they have a good potential to be extracted from laser data. Therefore, our research work will be guided towards extraction of trees and buildings.

4.2 General approach

The main objective of this research work is to assess the potential of wavelet-based analysis for automatic object extraction from the DSM in order to produce a DTM. This means that we are not interested in rebuilding the objects as they are. Only a simple indication or representation of an object


26

is sufficient. The 3D city modeling is a far more complicated task and will not be dealt with in this thesis. It is difficult to extract both objects and DTM in one go, because we do not know the DTM contribution present in elevation at any arbitrary point belonging to certain object. This can be only calculated by interpolating the DTM after removing object pixels. Therefore, we should roughly detect where the objects are, exclude them from the DSM, and run interpolation to calculate an approximation of the DTM. Finally, the DTM approximation will be extracted from the DSM, followed by filtering and refining the object’s exact location from the residuals. This procedure is shown in figure 4.2. In the following section this will be explained in more detail.

Object extraction schema

-Size constraint-Shape

Refinement

DSM

Exclude object pixel fromDSM

-Multiscale Edge-Wavelet coeficients-Elevated

Feature extraction

-2D DWT-2D CWT-Wavelet Lifting

Wavelet transform

-classification

Clustering

Approximation ofobjects

Approximation ofDTM

Interpolate DTM

DTM substractionfrom DSM

DTM

Buildingobj

Figure 4.2: Object extraction schema

Since the description of wavelet transforms can be found in the previous chapter let’s start from the feature extraction methodology.

4.3 Feature extraction

4.3.1 Multiscale edges Stephane Mallat and Sifen Zhong [33] carried out the initial research on multiscale edge detection using wavelet transforms. These techniques found their applications in pattern recognition and object recognition tasks. It was found that multiscale edges in wavelets can be extracted either by zero-crossing or local modulus maxima methods. The idea behind these methods is that the indication of sharp discontinuities are present in their wavelet coefficients over several successive scales in the form of local maxima or zero crossing. In this research we will employ a simple thresholding method, which can be described as follows.


27

The multiscale edge is related to sharp discontinuities, which are preserved over several scales of decomposition and produce large wavelet coefficients. By applying a threshold on wavelet coefficients we can preserve only sharp discontinuities while smoothing the edges that are not important for object extraction. We can derive and apply this threshold on all (horizontal, vertical, diagonal) detail images for every scale. In hard thresholding the absolute values of all wavelet coefficients dj,k are compared to a fixed threshold λ. If the coefficient is less than λ, the coefficient is replaced by zero:

��

��

≥

<=

λλ

kjkj

kj

kj dd

dd

,,

,

, ,

,0 (4.1)

Finally with the inverse transform we will restore only the edges that are higher than a given threshold.

Let us define that the building must have a wall with at least 2m high walls in order to be called as such. Then we can calculate what will be the threshold value at each scale. The result of applying such thresholding method on an artificial dataset is shown below (fig 4.2).

Figure 4.2: a) test data with blocks b) reconstructed 3D edges after applying threshold

The left block with 1m elevation and some simulated spike noises on the original data where dissolved after applying threshold. This shows the robustness of this method to noise.

a)

b)


28

The resulting edges do not satisfy or prerequisite for object extraction yet. However this result can be directly used for edge-based segmentation. The distinction between different edges is a good finding from this experiment. 4.3.2 Smooth planar surface

This property can be analyzed within some neighborhood of the pixel. The problem is that the result of such analysis will be dependent upon neighborhood size (e.g. window size), and becomes highly parameter dependent. A desireable method should use various window sizes. This can be achieved by multiresolution approach of a wavelet analysis, where the size of neighborhood is determined by scale. Smaller scale means small neighborhood size, large scales means larger neighborhood size. This suggests that wavelet analysis will be suitable tool for describing planar surfaces. Although the wavelet-lifting scheme was developed for image coding we believe that it can contribute to our analysis.

The fast lifted wavelet transform with first order polynomial fitting is a perfect tool for analyzing planar surface. Low absolute values of wavelet coefficients in all three-detail images will indicate the planarity of surface. This property can be used as one of features in final clustering. However this approach will not differentiate the objects by its height. Clearly the roads and building roofs are in the same situation, both have smooth surfaces. This suggests that incorporating height information will help us to separate between ground level objects from those with height. 4.3.3 Building height Probably the most widely used concept in building extraction methods from laser data relies on a simple fact that buildings are higher than their neighborhood. However as with the case with surface planarity description, determining the size of the neighborhood is problematic, and it cannot be a fixed size. Therefore multiscale approach has been employed by various authors [58,59,30] and proven it superiority over single scale approach. Clearly wavelet analysis with its multiscale approach should give an advantage. Although the height information is crucial for detecting buildings, the absolute value of height is meaningless for object extraction until the DTM contribution is taken into account. Therefore the relative height of a building is more important than the absolute value of height.

The most common approach employed by various researchers is extracting DTM from DSM and process the residual further in order to extract objects. We also are going to take an advantage of this method, but with application of wavelet analysis. We will derive an approximation of objects, by means of any indication that they are objects with height. Then these candidates of objects will be excluded from DSM followed by interpolation, which will give us an approximation of DTM. This approach is easier from implementation perspective because the DTM is less complex than the trees buildings and other man made structures. A number of different alternatives and possible solutions together with their implementation are described in the chapter five.

4.4 Clustering

Finally, with all features extracted, we need to combine them to work out estimates of where the objects are. One possible solution is to use multifeature clustering methods, like the k-mean clustering method. It is expected that features will form separate clusters corresponding to different objects. However there will be some ambiguity due to overlap between classes. We also think that in order to combine different features we need to parameterize them. For instance tall buildings are by no means “better” buildings than shallow ones.

4.5 Software

There are a number of possible wavelet analysis softwares. A short list of available wavelet software packages for Matlab is shown below:


29

Table 4.1 Wavelet toolboxes for Matlab and their features

1D analysis 2D analysis Transformations Software name

CWT DWT SWT* CWT DWT SWT* Additional features

Mathworks Wavelet toolbox1

Yes Yes Yes No Yes Yes packet analysis 1D soft/hard thresholding

YaWTB2 Yes Yes No Yes Yes No 3D CWT, CWT on sphere, wavelet frames, packet

RICE university wavelet toolbox2

No Yes Yes No Yes Yes Soft/hard thresholding

Wavelab 8022 Yes Yes Yes No Yes No interpolating wavelet transforms, cosine packets, wavelet packets, matching pursuit

SWT* stationary (translation invariant) wavelet transform 1 commercially available 2 free package This is not a complete list of wavelet softwares. Here we list only the complete wavelet toolbox packages for Matlab. There are some stand-alone packages, which where developed for specific projects and applications. Also there is another family of powerful software designed for S plus or R packages, which have similar capabilities, but we do consider them. Instead we will stick to the Wavelet Toolbox for Matlab.

There is one software worth mentioning here, the Liftpack. This software was developed by Wim Sweldens [62] and performs the DWT with lifting scheme. Because of a number of advantages of the lifting scheme, such as proper boundary handling by second generation wavelets, supporting irregular data structure (NaN is allowed), and working on spatial domain instead of frequency domain. The potential of the lifted wavelet transform in laser image analysis still remains to be explored. In this research we use IMLAB software with the Liftpack’s source code embedded.

4.6 Translation invariance of wavelet transform

After having established the software to be used we need to focus on which wavelet transform do we need to use for our experiments. The main criterion for such decision in object recognition task is translation and rotation invariance property, which is explained below.

The translation invariance problem occurs because the wavelet transform of a signal and of a time-shifted version of the same signal are not simply shifted versions of each other. The lack of translation invariance together with rotation invariance is the key drawback of DWT in feature extraction, object detection and pattern recognition application. Here we concentrate only on the translation invariance problem. More on rotational invariance problem will be given in chapter six. In figure 4.3.a one of the edges was not detected in the wavelet coefficients, if we would translate this image by one pixel to the right then we would lose the existing one and gain the one, which did not appear here. This problem can be overcome by calculating and retaining wavelet coefficient at every possible (integer) translation of the convolution filters i.e. the redundant transform. In the Matlab SWT function is called Discrete Stationary Transform. The trick is to upsample the original filter at scale J


30

for the next successive scale J+1, instead of downsampling the details and approximation at current scale J. More detail explanation how it works can be found on Matlab Wavlet toolbox manual. The resulting image of this transform is shown in figure 4.3.b, where both edges are shown. At this point we can state that this wavelet transform has more desirable results than a standard DWT. Therefore we will mainly use this transform for upcoming experiments.

Figure 4.3: a) DWT of a cube

b) translation invariant DWT


31

Chapter 5

Experimental results and discussion

5.1 Laser data

Before we start discussing experimental results the reader must be familiarized with test dataset. Since we are working with the generic case it was decided that any laser altimetry data with resolution of at least 1m would be sufficient for this work. The LIDAR image data was shared with the ITC graduating PhD student, Qingming Zhan [67], which was in turn provided by Dutch Rijkwaterstaat. The data is in raster format with 1m ground resolution (standard Dutch DSM, AHN) [1], which covers an area of 3 by 3km area southeast of Amsterdam. The area of covers a number of residential, commercial areas as well parks, lakes and canals, which contributes to the complexity of the scene. There are a number of complex structures like bridges over canals, underpasses, car parking garages with connecting road that makes object extraction even more challenging task.

As the wavelet transform is a computationally expensive transform we have subtracted a test bed dataset of 256 by 256 pixel. This subset was chosen to represent the most difficult cases, which may occur during processing. It includes both tall an short buildings surrounded by trees, a raised road with underpasses, and a parking garage with a road connection (fig 5.1)

Figure 5.1: Test dataset


32

5.2. Results of experiments

In this chapter we discuss the results of experiments. Each discussion item describes motivations behind each test, details of the test and comments on results. The experiments are:

1. description of planar surface

2. image approximation as a DTM

3. extraction of building edges by hard threshold

4. 3D edge detection by inverse wavelet transform

5. local reconstruction of objects from 3D edges with TIN and krigging

6. interpolation of building walls

7. reconstruction with inverted edge coefficients

5.2.1 Description of the planar surface property

In this experiment we deal with the question of how we can possibly describe the planar property of a surface. As it was stated earlier in chapter three, the wavelet analysis calculates the “resemblance index” between signal and wavelet base. Clearly we should take an advantage of this property wavelet transform. We can use the wavelet, which can test the planarity of surface by fitting first order polynomial. The Cohen-Daubechies-Feauveu biorthogonal (2,2) wavelet is a good candidate for such analysis.

As a part of this research work, extensive exploration of wavelet transforms and their capabilities were proposed. Therefore, we will use Wavelet Lifting Scheme, which was briefly described in chapter three.

Using the free image processing software IMLAB we can perform WT with the lifting scheme. The result of this transform is shown in figure 5.2

Figure 5.2: Result of Lifting Scheme WT; two levels of decomposition


33

The large wavelet coefficients of the detail image are related to sharp discontinuities caused by objects. In contrast, small coefficients describe the surface planarity, especially those, which are man-made objects such as roads, building roofs. The advantage of this transform is that the coefficients will be low even on inclined planes, and thus purely describe planarity. This transform computes the failure to linearly predict the value of the pixel based on values on two neighboring pixels in four directions: column, row, and both diagonals.

Although the described method depicts the surface planarity, this feature cannot separate ground level objects from those, which have a height, flat building roofs from roads for instance. Therefore this feature alone cannot have a decisive role in object extraction. 5.2.2 Approximation of the height image as a DTM

By applying wavelet decomposition, we compute details and the approximation of a signal. The approximation of a signal at scale J is a smooth version of a signal at scale J-1. The natural question of whether we can use the approximation of the signal as a terrain profile may arise. The idea behind this question is self explanatory in figure 5.3

Figure 5.3: Signal and 5th level approximation of a signal

We use reverse biorthogonal wavelet rbio6.8. The smoothed version of the signal after five levels of decomposition is shown in figure. 5.4.b. This choice of wavelet is motivated by the necessity of smoothing out the signal as quickly as possible, in other words to extract as much information into detail images as possible. Because we believe that information about objects like sharp discontinuity will be transferred in to detail image.

Figure 5.4: original height image (a), and it’s approximation at level 5 (b)

a) b)


34

After subtracting it from a DSM and applying a threshold on the difference value we are left out with an indication of the objects. However we did not detect buildings with large structure, we could only detected edges of these buildings. Using approximation of a higher scale (6) did not improve the result; instead we detect some artifacts caused by scaling wavelet (fig 5.5.a).

Figure 5.5: Binary image of residuals after subtracting the approximation from DSM a) after 5th b) 6th levels of decomposition

Four important conclusions can be drawn here:

1. Approximation of the DSM at large scales (5,6) cannot represent DTM. We cannot generate DTM by only smoothing DSM. Big building structures do not decay easily and separation between them and terrain do not present in larger scales.

2. The DTM contribution is always present in approximation and cannot be distinguished

3. With increasing scale the scaling function introducing the artifacts

4. The information lost by smoothing is stored in wavelet detail coefficients.

Hence, it will be more promising to experiment with these coefficients. 5.2.3. Extracting building edges by hard thresholding

In this experiment we use thresolding method, which is widely applied in image compression. Two methods for thresholding wavelet coefficients exist: hard and soft thresholding. The policy of hard thresholding is keep or kill. The absolute values of all wavelet coefficients are compared to a fixed threshold. This technique generates discontinuities after rebuilding the signal from the thresholded coefficients. Second Method Proposed by David Donoho shrinks all the wavelet coefficients towards the origin and does not generate discontinuities and is better applicable for image compression [56]. In this test we will use first method since our task is to detect sharp discontinuities, those of which related to objects. Also we are going to use Haar wavelet as a base or mother wavelet, as it is better suited for box-like shaped signal [26], which is the case with laser data in urban area. We start with assumption that the buildings have a discontinuity of at least 2m. A test dataset was produced with cubes of 1,2,3m height respectively. Also we include in to test objects with steep slope with 45 degree and 26 degree, where the first is considered as an object while the second is not considered as such. We decompose the test image up to scale J=4 and try to detect the threshold value, where the objects can be separated. As the scale increases the separation between wavelet coefficients related to objects become increasingly less distinct from those of non-object coefficients. For example at the scale of 4 the difference between 1m box and 2m height object coefficients is only 0.25. After finding threshold value for each scale we preserve only the coefficients with absolute value higher than

a) b)


35

the threshold; the rest of coefficients are assigned to zero. The approximation is also assigned to zero, because we do not want to have DTM contribution. Then we apply the inverse wavelet transform with preserved coefficients. The resulting image is shown in fig 5.6

Figure 5.6: Edges of buildings after hard thresholding As we can see, we have successfully detected most of the building edges. We could apply a higher threshold, which are explicitly related to the building structures but the complexity of building structures to be extracted does not allow us to do that. For example the large rectangular building structure in the left corner is a car-parking garage, which has the road driving in to it. Therefore we loose one side of the building if we apply higher threshold. The result of this experiment inspires us for extending our exploration in this direction. The extension of these edges to 3D edges will be discussed in the next experiment. 5.2.4. 3D edges by inverse wavelet transform

In this experiment we try to rebuild building or any other representation, which can represent buildings from the edges. The idea is that if the signal synthesis is achieved by adding details such as:

S=A4+D4+ D3+….+D1 �≤

+=Jj

JJ DA (5.1)

where J-is a reference scale

Figure 5.7 : a) Signal in profile

b) Signal reconstruction


36

At higher scales (j=4,5,6) we should have enough information transferred to detail from which we can reconstruct buildings without ever being concerned about the approximation. We simply neglect the approximation (AJ=0) because the approximation keeps storing the terrain height contribution. In this case we use all detail coefficient that fall in to the edge region, because now we know where the buildings are, therefore the edges will be reconstructed without artifact caused by thresholding. The output image is quite close to the results of previous test. The problem occurs with building interior pixels. They never yield coefficients large enough to be restored from the detail coefficients. Moreover, with approximation assigned to zero these edges are restored symmetrically to the zero height (fig 5.7). This shows that we need something more than the edge, which will be discussed in the next test. 5.2.5. Local reconstruction of objects from 3D edges with interpolation

In this test an attempt has been made to reconstruct buildings from the 3D edges by using interpolation. We will use approximation of the signal but only for the edge pixels. The main idea is to reconstruct the buildings only locally and subtract them from DSM. The result of this test is shown below (fig 5.8)

a)

b)


37

Figure 5.8: a) Objects edges restored; b) Interpolation with krigging; c) TIN interpolation from 3D edges

The idea of reconstructing building only locally failed, because you can always find some objects like trees nearby building. Therefore this method cannot be reliable. Subtracting fully interpolated data from DSM does not make sense because it still contains DTM values. An attempt to fix this problem was made in the next experiment. 5.2.6 Interpolating of building walls

Our main problem is still the extraction of some indication that the objects that we are aiming for are elevated objects. The idea in this test is to reconstruct the sharp discontinuities with the amplitude of height difference; building walls for instance. As it’s shown in figure 5.6 we have the reconstruction of the wall but symmetrically to the zero axis. A solution was found by assigning negative height values to zero and multiplying the positive ones with the factor of two. The resulting image depicts the sharp discontinuities, which are associated with objects (fig 5.9.a). This image pixels where converted to point dataset

a)

c)


38

Figure 5.9: a) building walls extracted and it’s b) linear interpolation, light tone indicates the elevation increment

and interpolated using linear interpolation. Now it is possible to apply some threshold value in height to extract only the objects with height, because the data does not contain any DTM value. This will perfectly work with simple-structured buildings. But with complex buildings, which are stepwise, this will fail only within the edges, which are located within the interior of buildings. In fact, we know already where the object edges are, so this should not cause a big problem. The result of such experiment is shown in figure 5.9.b. From this result we discover that complex objects such as tunnels bridges, which were not concerned in our approach, cause serious problems. The idea, which could work in some other simpler case, did not work in a complex situation.

5.2.8 Inverting wavelet coefficients sign

The last idea is that there are still some possibilities for manipulating wavelet coefficients. For example we can invert the sign of the wavelet coefficient and multiply it by some number, in such a way that after applying the inverse transform the coefficient will override the approximation value by neglecting its contribution at the next scale. The idea is to remove the building pixels by multiplying the edge coefficients with some big negative number and reconstruct the signal. The reconstructed signal was subtracted from the DSM thus neglecting the DTM values down to zero and leaving only the building and other objects. This was successfully implemented first with test data with various blocks. However this approach did not work with the real dataset. Again, the problem occurred with large building structure. We could only detect the wide rim around the building. This happened because we have only zero or very small wavelet coefficients at the interior pixels of the buildings with a flat roof. Therefore, it could not overrule the approximation of a signal. This is the case with the buildings at lower right and lower left (figure 5.10)

b)


39

Figure 5.10 : a) Object pixels extracted

b) terrain element coefficients inversed

It is also possible to approach this problem from the opposite view. If it is not possible to get rid of building roofs, then we can benefit from it if we invert the sign of wavelet coefficients of those pixels, which belong DTM. Once again this approach did work with test data but failed with the real data set (fig 5.10.b). As we can see from the image we could neglect the value of the DTM but in this case building roofs and road surfaces where in the same situation where both had low wavelet coefficients. It is still possible use this output, because if we clip the DSM by this output and run interpolation on the resulting dataset we will reconstruct roads, as both sides, of the road will be preserved while buildings will be removed.


40

Chapter 6

Discussion

As a concluding remark to the work which have been done, the following chapter will give a summary of experiments and research question that have not been answered so far. 6.1 Why are the overall test results unsuccessful?

The main reason why the test results are not impressive is related to the complexity of objects that we were aiming to extract. In particular the features that we believed to explicitly separate objects where ambiguous and where overlapping between the classes of objects. For example building roofs and roads both have planar surfaces; bridges and buildings both have sharp edges and could be described as elevated objects. Moreover, there are objects, which are difficult to model at all. A car-parking garage connected to the road from one side is one of such exceptional cases. The most difficult objects to extract turned out to be large structures with relatively small height. The reason behind it is that the height values cannot be used without subtracting a DTM from it. In this specific case, the separation between large objects and terrain don not preserve the higher scale (j=4 and greater), the walls decay very quickly and it becomes very difficult to separate them from the terrain element such as small hill or elevated road. The main strength of wavelet analysis, which is its multiresolutional approach, could not contribute much to our task. Although we could not separate objects explicitly, some of methods used in this work can be used in simpler situations. It’s worth mentioning that the test area where selected to represent the most difficult situation. Finally, we should go back to our model and combine all the features to produce a final result. This was not implemented due to time limitations. 6.2 Wavelet transforms

All the test work has been done using DWT and its translation invariant version known as discrete stationary transform. There exist some other variation of DWT like tree-structured wavelet, discrete wavelet packet. These transforms differ from standard DWT by their redundant decomposition of detail wavelet coefficients, which has found its application in texture recognition. The main stronghold of DWT is its application compression and de-noising. Continuous wavelet transform where found to be more useful for pattern recognition, feature extraction and detection. Continuous wavelet transformation tools are mostly available for signal processing but not for image processing. Currently only two softwares YaWTB for Matlab and Cri-tech’s Psilets can perform 2D CWT. A free clone of Psilets CWT was found for Matlab. However the inverse transform is not available because the Mexican hat wavelet family is not neither orthogonal nor biorthogonal. The CWT with Mexican wavelet is shown below (fig 6.1)


41

Figure 6.1: CWT with Mexican hat wavelet transform. a) low scale coefficients image

b) large scale coefficients image

The Mexican hat wavelet does not have scaling function. It will be interesting to test some of our ideas with this wavelet transform. Another big advantage of CWT is the ability to use custom wavelet with minimum requirement. The next section will discuss the issue of custom-built wavelets. 6.3 Building custom wavelet

It is easy to build wavelets for minimum condition (admissibility condition) but if more interesting properties like existence of scaling function are needed, the building the wavelet is difficult. For DWT orthogonality or biorthogonality of the mother wavelet is required. Very few wavelets can satisfy these conditions and constructing a new wavelet for DWT is a difficult task and outside of scope of this research. 6.4 Testing different wavelets

It may seem that all tests where carried out using predominantly the Haar wavelet. It will be fare to mention that some of test where tested with different mother wavelets from Wavelet toolbox wavelet library. It was found that with wavelets which are not symmetric such as Daubechies, Symlet, Coiflet, some biorthogonal spline family wavelets may not be suitable for object extraction in some cases, bearing in mind that in our application case the location of feature is highly important. Application of these types of wavelets shifts the position of objects. Also it was noticed that during decomposition the scaling function forces the approximation data to comply with its waveform. This means that at large scales 5,6 the approximation is highly distorted by scaling function. 6.5 Data resolution issue Testing with different resolution data was planned to test our working solution for object extraction. Since this task was not achieved kind of testing did not take place. Based on the object separability in wavelet coefficients it can be said that the scales beyond fourth do not contribute much for objects detection in our case. It becomes impossible to differentiate object features from DTM in the higher scales. 6.6 Rotation invariance problem

It was observed during tests with wavelet coefficients thresholding that the threshold estimates are differ according to analyzing direction. This can be explained by rotation invariance problem in wavelet analysis.


42

Rotation invariance occurs when one desires that the outcome of analysis is not influenced by orientation of input image. One possible solution for extracting rotation invariant feature is to use an isotropic wavelet such as Mexican hat wavelet, the second derivative of Gaussian function. However, this wavelet cannot be implemented in DWT, it can only be used in CWT. At some point it may well be that rotation invariance is not such important and desirable property of wavelet transform for our application. Unlikely to some pattern recognition and texture recognition applications it is hard to imagine that characteristics of different building structure could be rotationally invariant. 6.7 Is it possible to use wavelet based texture segmentation methods?

A lot of time was spent at the beginning of this research work trying to utilize texture segmentation methods for object separation. The wavelet based texture segmentation methods work in such a way that they extract some features and compare those with an existing texture feature database. It was finally concluded that these methods could not be used to solve our problem. Firstly, because smooth surfaces are not considered as a texture, the texture is defined as a local variation of intensity. Secondly, building a database of all possible texture features of man-made objects is an ambiguous task.

6.8 Summary

From the critical point of view one would notice that most of our experiments involves computation or comparison within single level of decomposition separately. Further these layers combined by inverse transform resulting a single output image. This can be treated as a serious drawback a step out from multiresolutional approach. It was consistently stressed that the separation between objects and terrain does not exist in the large-scale approximation. However, we must say the separation between objects may not exist explicitly in all scales, because objects ‘live’ with its associated scale. This has been clearly shown during our experiments. We can state that the dilemma of choice between ‘spatial’ resolution versus ‘scale’ resolution still needs an appropriate compromise. In case with buildings we do get an indication, as it is objects with height in larger scales, but we do not have any idea about its extent. Alternatively when we get good spatial resolution and we know the spatial extent of the building by extracting the edges but then we don’t know any idea about its behavior in a larger neighborhood i.e. we don’t have good frequency resolution. These statements suggest that further research work efforts need to be put on defining objects ‘behavior’ over several successive scales. Again for the building this would mean that building as well as other elevated objects could be distinguished in higher scales. By refining the detail information such as edges at lower scales we can separate the building from other elevated objects. It can be seen that in our object hierarchy in the object model does not incorporate scale information.


43

Chapter 7

Conclusions and recommendations

7.1 Conclusions

The conclusions drawn during this research work could be formulated as follows: 1. The challenge to be met in this research work was to extract the objects with application of wavelet based analysis from the laser data only. Although not fully successfully, the test results show that wavelet analysis has a good potential for object extraction. Even though we failed with some exceptional and difficult cases, in overall this technique can be applied in other simpler cases. 2. The main stronghold of wavelet analysis is its multiresolutional analysis property. However this did not play a decisive role in our research. Because, with only elevation data available, object separation occurs only in height differences but not in absolute height values. Therefore we can recognize edges of objects but not interior pixels. 3. We proposed to separate objects using some common features such as height, smoothness of the surface, height, size etc. The idea was to find a generic solution for separating object by combining these features. However, we have discovered that dealing with these features only is not sufficient for separating objects explicitly. Classes of objects are mutually overlapping. It is necessary to find other features, which are still common to most of objects or incorporate additional information such as reflectance, and first and last pulse reflection data. These additional data are becoming standard products with the most recent laser scanners. 4. The wavelet technique succeeded in the domain of digital signal processing, coding and compression. While its application in digital image processing, it is still a new research area. The number of popular literature as well as software is limitedly available. Some transforms like the continuous wavelet transform, which is better suited for pattern recognition and feature extraction, is still unavailable. Therefore the idea of designing tailor-made wavelets for certain type of object was not accomplished. 5. We have to accept that the complexity of urban scene. Therefore diversity of objects will not allow performing object extraction fully automatically using wavelets only. There will be always some other operation to perform it either manually or automatically.


44

7.2 Recommendations

Here we will list some potential research areas, which was not explored during this work or need more exploration. 1. Wavelet transform with lifting scheme is one of interesting topics. Its ability work on spatial domain and irregular data structures support may have a good potential for application in laser data. 2. The 2D continuous wavelet transform (CWT) is a better tool for feature extraction, pattern recognition applications. It will be interesting to experiment with this tool for object extraction. 3. Also the continuous wavelet transform requires minimum condition for wavelet base to be used. The research question of building tailor made wavelet for extracting specific objects is still valid and needs to be explored. 4. One possible alternative of utilizing multiresolution analysis in object extraction from laser data is using data fusion. Theoretically it is possible to use known DTM from coarser scale and subtract it from the approximation of DSM at respective scale of decomposition. For example AHN 5 meter DTM can be extracted from 5th scale approximation of 1m resolution DSM.


45

References

[1-50]

1. ANH: Actual Height model of the Netherlands.

2. Ackermann, F., Airborne laser scanning-present status and future expectations. ISPRS Journal of Photogrammetry and Remote Sensing, 1999. 54: p. 64-67.

3. Axelsson, P., Processing of laser scanner data-algorithms and applications. ISPRS Journal of Photogrammetry and Remote Sensing, 1999. 54: p. 138-147.

4. Balmeli, L. and A. Mojsilovic. Wavelet Domain Features for Texture Description, Classification and Replicability Analysis. in IEEE International Conference on Image Processing (ICIP). 1999.

5. C.Chandler, R., et al. Autonomous Agent Navigation Based on Textural Analysis. in 2002 Florida Conference on Recent Advanced in Robotics. 2002. Florida International University.

6. C.Valens, The Fast Lifting Wavelet Transform. 1999. 22.

7. Chang, T. and C.-C.J. Kuo, Texture Analysis and Classification with Tree-Structured Wavelet Transform. IEEE Transactions on Image Processing, 1993. 2(4): p. 429-441.

8. Cheikh, F.A., A. Quddas, and M. Gabbouj, Multi-level Shape Recognition based on Wavelet-Transform: Modulus Maxima. p. 5.

9. Chen, J., et al., Adaptive image segmentation based on color and texture. p. 4.

10. Claypoole, R., et al., Nonlinear wavelet transforms for image coding via lifting. IEEE Transactions on Image Processing, 1997: p. 24.

11. Claypoole, R., et al., Nonlinear wavelet transforms for image coding. 1997: p. 9.

12. Dai, X., Improve Classificaion of Remote Sensing Data for Land Use/Cover Classification. 2003: p. 10.

13. Diaz, M., Wavelet Features for Color Image Classification. p. 4.

14. Elberink, S.O. and H.-G. Maas, The use of anisotropic height texture measures for the segmentation of airborne laser scanner data. IAPRS, 2000. XXXIII: p. 7.

15. Epinat, V., et al., A wavelet characterization of high-resolution NDVI patterns for precision agriculture. JAG, 2001. 3(2): p. 121-132.

16. Fatemi-Ghomi, N., Performance measures for Wavelet-based Segmentation Algorithms, in Centre for Vision, Speech and Signal Processing

School of Electronic Engineering, Information Technology and Mathematics. 1997, University of Surrey: Guildford, Surrey. p. 187.


46

17. Fernandez, G., S. Periaswamy, and W. Sweldens, LIFTPACK: A Software Package for Wavelet Transform using Lifting. p. 13.

18. G.Priestnall, J.Jaafar, and A.Duncan, Extracting urban features from LIDAR digital surface models. Computers, Environment and Urban Systems, 2000. 24: p. 65-78.

19. Gamba, P. and V. Casella, Model independent object extraction from digital surface models. International Archives of Photogrammetry and Remote Sensing, 2000. XXXIII(Part B3): p. 312-319.

20. Gendron, P., Introduction to Wavelet Methods. 1998.

21. I.M.Dremin, O.V.Ivanov, and V.A.Nechitailo, Wavelets and their applications. Achievements in Physical Science, 2001. 171(5): p. 465-501.

22. J.Harding, D., Principles of Airborne Laser Altimeter Terrain Mapping. 2000.

23. Jawerth, B. and W. Sweldens, An overview of wavelet based multiresolution analyses. 1993: p. 37.

24. Leguizamon, S. Characterization of texture in remotely sensed images by using the wavelet transform. in 4-th Internaional Symposium on High Mountain Remote Sensing Cartography. 1996. Karistad-Kiruna-Tromse.

25. Livens, S., Image analysis for material characterisation. 1998: Antwerpen. p. 5-19.

26. Luo, A., E.P. Lam, and A. Amer, Texture Classification Using Wavelets and Vector Quantization: Santa Cruz. p. 6.

27. Luo, J., Wavelet-based Texture Classification. p. 9.

28. Maas, H.-G., The potential of height texture measures for the segmentation of airborne laserscanner data. 1999: p. 8.

29. Maas, H.-G. and G. Vosselman, Two algorithms for extracting building models from raw laser altimetry data. ISPRS Journal of Photogrammetry and Remote Sensing, 1999. 54: p. 153-163.

30. Madchakham, S., P. Thitimajshima, and Y. Rangsanseri. Edge detection in speckled SAR images using wavelet decomposition. in 22nd Asian Conference on Remote Sensing. 2001. Singapore: Centre for Remote Imaging, Sensing and Processing (CRISP), National University of Singapore; Singapore Institute of Surveyors and Valuers (SISV); Asian Association on Remote Sensing (AARS).

31. Mallat, S., Wavelets for a Vision. Proceedings of the IEEE, 1996. 84(4): p. 604-614.

32. Mallat, S. and W.L. Hwang, Singularity Detection and Processing with Wavelets. IEEE Transactions on Image Processing, 1992. 38(2): p. 617-643.

33. Mallat, S. and S. Zhong, Characterization of Signals from Multiscale Edges. IEEE Transactions on pattern analysis and machine intelligence, 1992. 14(7): p. 710-732.

34. Mallat, S.G., A Theory for Multiresolution Signal Decomposition: The Wavelet Representation. IEEE Transactions on pattern analysis and machine intelligence, 1987. 11(No.7): p. 674-693.


47

35. McIntosh, K. and A. Krupnik, Integration of laser-derived DSMs and matched image edges for generating an accurate surface model. 2000, Science Direct Web Editions.

36. Misiti, M., et al., Wavelet Toolbox. First printing ed. 1996, Natick: The MathWorks, Inc.

37. Mojsilovic, A., M.V. Popovic, and D.M. Rackov, On the selection of an optimal wavelet basis for texture characterization. IEEE Transactions on Image Processing, 2000. 9(12): p. 2043-2050.

38. Morgan, M. and K. Tempfli, Automatic building extraction from airborne laser scanner data. International Archives of Photogrammetry and Remote Sensing, 2000. XXXIII(B3): p. 616-623.

39. N.M.Astaf'eva, Wavelet analysis: basic theory and some applications. Achievements in Physical Science, 1996. 166(11): p. 1145-1170.

40. Ogden, R.T., Essential Wavelets for Statistical Applications and Data Analysis. 1997, Boston, Basel, Berlin: Birkhauser. 206.

41. Petzold, B., P. Reiss, and W. Stossel, Laser scanning-surveying and mapping agencies are using a new technique for the derivation of digital terrain models. ISPRS Journal of Photogrammetry and Remote Sensing, 1999. 54: p. 95-104.

42. Polikar, R., The wavelet tutorial, Part III. 1996.

43. Polikar, R., The wavelet tutorial, Part I. 1996.

44. Polikar, R., The wavelet tutorial, Part II. 1996.

45. Polikar, R., The wavelet tutorial, Part IV. 1998.

46. Polikar, R. The Story of Wavelets. in IMACS/IEEE CSCC'99. 1999.

47. Prince, S., et al., Texture analysis using wavelet transform. 2001.

48. Ranchin, T. and L. Wald, The wavelet transform for the analysis of remotely sensed images. International Journal on Remote Sensing, 1992. 14(3): p. 615-619.

49. Randen, T. and J.H. Husey, Filtering for Texture Classification: A Comparative Study. IEEE Transactions on pattern analysis and machine intelligence, 1999. 21(4): p. 291-310.

50. Schutte, K., Knowledge Based Recognition of Man-Made Objects. 1994, Enschede: GIP-GEGEVENS KONINKLIJKE BIBLIOTHEEK, DEN HAAG. 116.

51. Zhan, Q., M. Molenaar, and K. Tempfli, Building extraction from laser data by reasoning on image segments in elevation slices. 1998: p. 2.

52. Sebe, N. and M. S.Lew, Wavelet based texture classification. p. 4.

53. Tuceryan, M. and A. K.Jain, Texture Analysis, in The Handbook of Pattern Recognition and Computer Vision, C.H.Chen, L.F. Pau, and P.S.P.Wang, Editors. 1998, World Scientific Publishing Co.,. p. 207-248.

54. Unser, M., Texture Classification and Segmentation Using Wavelet Frames. IEEE Transactions on Image Processing, 1995. 4(11): p. 1549-1560.


48

55. Uytterhoeven, G. and F.V. Wulpen, WAILI-Wavelets with Integer Lifting. 1999. 52.

56. Uytterhoeven, G., et al., WAILI: A Software Library for Image Processing using Integer Wavelet Transforms.

57. Vidakovic, B. and P. Muller, Wavelets for kids. 1991, Durham: AMS Subject Classification: 42A06, 41A05, 65D05. 26.

58. Vosselman, G. and H.-G. Maas, Adjustment and filtering of raw laser altimetry data.

59. Vu, T.T. and M. Tokunaga, Designing of wavelet-based processing system for airborne laser scanner segmentation. p. 5.

60. Vu, T.T. and M. Tokunaga. Wavelet and scale-space theory in segmentation of airborne laser scanner data. in 22nd Asian Conference on Remote Sensing. 2001. Singapore: Centre for Remote Imaging, Sensing and Processing (CRISP), National University of Singapore, Singapore Institute of Surveyors and Valuers (SISV), Asian Association on Remote Sensing (AARS).

61. Wehr, A. and U. Lohr, Airborne laser scanning-an introduction and overview. ISPRS Journal of Photogrammetry and Remote Sensing, 1999. 54: p. 68-82.

62. Weidner, U. and W.Forstner, Towards automatic building extraction from high-resolution digital elevation models. ISPRS Journal of Photogrammetry and Remote Sensing, 1995. 50(4): p. 38-49.

63. Wim, S., Building your own wavelets at home.

64. Wouwer, G.V.d., Wavelets for Multiscale Texture Analysis, in Departement Natuurkunde. 1998, Universiteit Antwerpen: Antwerpen. p. 24.

65. Xie, Z.-Y. and M. Brady, Texture Segmentation Using Local Energy in Wavelet Scale Space. p. 304-313.

66. Zhan, Q., M. Molenaar, and K. Tempfli, Hierarchical image object-based structural analysis toward urban land use classification using high-resolution imagery and airborne LIDAR data. p. 251-255.

67. Zhan, Q., M. Molenaar, and K. Tempfli, Finding spatial units for land use classification based on hierarchical image objects. 1998: p. 3.

[51-67]

Documents

Wavelet-based analysis for object separation from ... - ITC · Wavelet-based analysis for object separation from laser ... yˆ F ou rie t ans fm y f,y j, k ... Wavelet-based analysis