Embed Size (px)
Citation preview
Studies in Computational Intelligence 545
Agnieszka Lisowska
Geometrical Multiresolution Adaptive TransformsTheory and Applications
Studies in Computational Intelligence
Volume 545
Series editor
Janusz Kacprzyk, Polish Academy of Sciences, Warsaw, Polande-mail: [email protected]
For further volumes:http://www.springer.com/series/7092
About this Series
The series ‘‘Studies in Computational Intelligence’’ (SCI) publishes new devel-opments and advances in the various areas of computational intelligence—quicklyand with a high quality. The intent is to cover the theory, applications, and designmethods of computational intelligence, as embedded in the fields of engineering,computer science, physics and life sciences, as well as the methodologies behindthem. The series contains monographs, lecture notes and edited volumes incomputational intelligence spanning the areas of neural networks, connectionistsystems, genetic algorithms, evolutionary computation, artificial intelligence,cellular automata, self-organizing systems, soft computing, fuzzy systems, andhybrid intelligent systems. Of particular value to both the contributors and thereadership are the short publication timeframe and the world-wide distribution,which enable both wide and rapid dissemination of research output.
Agnieszka Lisowska
Geometrical MultiresolutionAdaptive Transforms
Theory and Applications
123
Agnieszka LisowskaInstitute of Computer ScienceUniversity of SilesiaKatowicePoland
ISSN 1860-949X ISSN 1860-9503 (electronic)ISBN 978-3-319-05010-2 ISBN 978-3-319-05011-9 (eBook)DOI 10.1007/978-3-319-05011-9Springer Cham Heidelberg New York Dordrecht London
Library of Congress Control Number: 2014932122
68-02, 68U10, 68W25
� Springer International Publishing Switzerland 2014This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part ofthe material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission orinformation storage and retrieval, electronic adaptation, computer software, or by similar or dissimilarmethodology now known or hereafter developed. Exempted from this legal reservation are briefexcerpts in connection with reviews or scholarly analysis or material supplied specifically for thepurpose of being entered and executed on a computer system, for exclusive use by the purchaser of thework. Duplication of this publication or parts thereof is permitted only under the provisions ofthe Copyright Law of the Publisher’s location, in its current version, and permission for use mustalways be obtained from Springer. Permissions for use may be obtained through RightsLink at theCopyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law.The use of general descriptive names, registered names, trademarks, service marks, etc. in thispublication does not imply, even in the absence of a specific statement, that such names are exemptfrom the relevant protective laws and regulations and therefore free for general use.While the advice and information in this book are believed to be true and accurate at the date ofpublication, neither the authors nor the editors nor the publisher can accept any legal responsibility forany errors or omissions that may be made. The publisher makes no warranty, express or implied, withrespect to the material contained herein.
Printed on acid-free paper
Springer is part of Springer Science+Business Media (www.springer.com)
There are 10 types of people in this world,those who understand binaryand those who do not.
Foreword
I had the pleasure, and privilege, to get acquainted with Agnieszka’s work in 2005as a reviewer of her Ph.D. thesis. It was like a continued thrill to read that work, tosee a fascinating area being developed another step further. I was first thrilled tosee the wavelets, and thus the multiresolution analysis, enter into signal processingin the 1980s. The second thrill followed soon, in 1993, when D. L. Donoho,I. M. Johnstone, G. Kerkyacharian, and D. Picard wrote their pioneering paper onnonparametric density estimation by wavelet thresholding. It was clear that thewavelets must find their way into image processing, and soon we had more thanthat. In late 1990s geometric wavelets were introduced—ridgelets of EmmanuelCandès, wedgelets of David L. Donoho, and curvelets of both of them. Agnieszkafollowed the lead and introduced second-order wedgelets in 2003 to make themlater the main topic of her thesis (at about the same time, platelets and surflets wereintroduced by other researchers). Of course, the story did not end then.
Geometric multiresolution transforms, these early ones and those laterproposed, are either adaptive or nonadaptive depending on the way the imageapproximation is made. In her book, Agnieszka focuses on the adaptive approach,actually on multismoothlets, i.e., vectors of smoothlets (both of her own inven-tion), although shown within a broader context. A short account of all of theadaptive and nonadaptive approaches is given along with a discussion of theirrespective ranges of applicability.
The core of the book is divided into two parts. In the first, the MultismoothletTransform is introduced in detail, while in the second, its Applications are thor-oughly described. It is a whole which is not only highly original but, as the readerwill surely agree, the one of a great practical value. A truly illuminating andvaluable read, and written in a very clear and lucid style.
Warsaw, November 2013 Jacek Koronacki
vii
Preface
Modern image processing techniques are based on multiresolution geometricalmethods of image representation. These methods are known to be efficient insparse approximation of digital images. There is a wide family of functions that areused in such a case. All these methods can be divided into two groups—theadaptive ones, like wedgelets, beamlets, platelets, surflets, or smoothlets, and thenonadaptive ones, like ridgelets, curvelets, contourlets, or shearlets. This book isdevoted to the adaptive methods of image approximation, especially tomultismoothlets.
Besides multismoothlets, a few new ideas were introduced in this book as well.So far, in the literature the horizon class of images has been considered as themodel for sparse approximation. In this book, the class of blurred multihorizonwas introduced, which is used in approximation of images with multiedges.Multismoothlets assure the best approximation properties among the state-of-the-art methods for that class of images. Additionally, the semi-anisotropic model ofedge (or multiedge) representation was proposed. It was done by introduction ofthe shift invariant multismoothlet transform. It is based on sliding multismoothletsintroduced in this book as well.
The very first definition of this book is a monograph treating about multi-smoothlets and the related methods. However, the book is presented in anaccessible fashion for both mathematicians and computer scientists. It is full ofillustrations, pseudocodes, and examples. So, it can be suitable as a textbook or asa professional reference for students, researchers, and engineers. It can be treatedas a starting point for those who want to use geometrical multiresolution adaptivemethods in image processing, analysis, or compression.
This book consists of two parts. In the first part the theory of multismoothlets ispresented. In more details, in Chap. 2 the theory of smoothlets is presented.In Chap. 3 multismoothlets are introduced together with the methods of theirvisualization. In Chap. 4 the multismoothlet transform and the discussion about itscomputational complexity are presented. In the second part of this book, theapplications of the smoothlet and multismoothlet transforms are presented.In consecutive Chaps. 5–7 the applications to image compression, denoising andedge detection are presented, respectively. The book ends with conclusions andfuture directions.
ix
This book would not have been written without the support of many people.I would like to thank Prof. Jacek Koronacki for writing the foreword, Prof.Wiesław Kotarski for the help and support, Krzysztof Gdawiec for good proof-reading and suggestions, and all my colleagues. I also would like to thank LynnBrandon from Springer for the endless help in the publishing process and anon-ymous reviewers for the precious remarks and suggestions, which improved thequality of this book. Finally, I would like to thank my family and all my friends forbeing with me.
Sosnowiec, May 2013 Agnieszka Lisowska
x Preface
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 State-of-the-Art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.4 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.5 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Part I Multismoothlet Transform
2 Smoothlets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.2 Image Approximation by Curvilinear Beamlets . . . . . . . . . . . . . 182.3 Smoothlet Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.4 Image Approximation by Smoothlets . . . . . . . . . . . . . . . . . . . . 212.5 Sliding Smoothlets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.6 Smoothlets Sparsity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3 Multismoothlets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.1 Multismoothlet Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.2 Multismoothlet Visualization . . . . . . . . . . . . . . . . . . . . . . . . . 283.3 Image Approximation by Multismoothlets . . . . . . . . . . . . . . . . 313.4 Sliding Multismoothlets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.5 Multismoothlets Sparsity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4 Moments-Based Multismoothlet Transform. . . . . . . . . . . . . . . . . . 394.1 Fast Wedgelet Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404.2 Smoothlet Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424.3 Multismoothlet Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . 434.4 Computational Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . 45References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
xi
Part II Applications
5 Image Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535.1 Binary Images. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
5.1.1 Image Coding by Curvilinear Beamlets . . . . . . . . . . . . . 545.1.2 Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5.2 Grayscale Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575.2.1 Image Coding by Smoothlets . . . . . . . . . . . . . . . . . . . . 575.2.2 Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
6 Image Denoising . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 676.1 Image Denoising by Multismoothlets . . . . . . . . . . . . . . . . . . . . 686.2 Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
7 Edge Detection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 837.1 Edge Detection by Multismoothlets . . . . . . . . . . . . . . . . . . . . . 84
7.1.1 Edge Detection by Multismoothlet Transform . . . . . . . . 847.1.2 Edge Detection by Sliding Multismoothlets . . . . . . . . . . 847.1.3 Edge Detection Parameters. . . . . . . . . . . . . . . . . . . . . . 85
7.2 Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
8 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 978.1 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 978.2 Future Directions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
8.2.1 Fast Optimal Multismoothlet Transform. . . . . . . . . . . . . 988.2.2 Texture Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . 988.2.3 Image Compressor Based on Multismoothlets. . . . . . . . . 998.2.4 Hybrid Image Denoising Method . . . . . . . . . . . . . . . . . 998.2.5 Object Recognition Based on Shift Invariant
Multismoothlet Transform . . . . . . . . . . . . . . . . . . . . . . 99References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Appendix A. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
Appendix B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Appendix C. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
xii Contents
Chapter 1Introduction
Abstract In this chapter, the motivation of this book was presented based on thehuman visual system. Then, the state-of-the-art review was given of the geometricalmultiresolution methods of image approximation together with the contribution ofthis book. The chapter ends with the outline of this book.
More and more visual data are gathered each day, which has to be stored with moreandmorememory space. So, the data have to be represented as efficiently as possible.The efficiency is related to a sparsity in an obvious way. The sparser representation isused, the more compact image representation is obtained [1]. It is known that humaneye perceives the surrounding world in geometrical multiresolution way [2]. So, anefficient image representation method should be geometrical and multiresolutional.In fact, more information is available about the human eye-brain system. This knowl-edge can be useful in definition of functions family that can be used efficiently inimage representation.
Many such families of functions have been defined in the recent years. Theyare commonly called as “X-lets”. All of them arose as generalizations of the well-known wavelets theory [3]. It is known that wavelets are characterized by locationand scale. “X-lets” are characterized, additionally, by orientation. However, the setof these features is not yet closed. Functions can be also characterized by curvatureor blur. This issue is discussed in more details in this chapter.
1.1 Preliminaries
The growth of image-based electronic devices in these days caused that imageprocessing became very important and omnipresent. Indeed, collected data haveto be optimally coded in order to preserve a disc space. Usually, because imageacquisition methods are not perfect, image quality has to be improved by denoising,deblurring or inpainting. In order to analyze further an image content, it has to besegmented. All these tasks can be performed in different ways, depending on the
A. Lisowska, Geometrical Multiresolution Adaptive Transforms, 1Studies in Computational Intelligence 545, DOI: 10.1007/978-3-319-05011-9_1,© Springer International Publishing Switzerland 2014
2 1 Introduction
application. There are many approaches that are used in such cases. They can besummarized as follows.
• Morphological approach—allows to perform image processing for geometricalstructures. By defining a structuring element one can define basic operations, likeerosion or dilation. They are further used in definitions of opening and closingtransforms. These transform play a crucial role in objects segmentation [4].
• Spectral analysis—Fourier and spectral methods used to be seen as the most pow-erful ones in image representation. They can catch changes of signal in differentlocations. Theyweremainly applied to linear filtering or image compression (JPEGalgorithm for instance) [5].
• Multiresolution methods—mainly wavelets-based methods. Wavelets play a cru-cial role in image approximation. They can catch changes of a signal in dif-ferent locations and scales (and directions in the case of the recent methods).The most commonly used applications include denoising and image compression(JPEG2000 algorithm for instance) [1].
• Stochastic modeling—used for images with a stochastic nature, like images ofnatural landscapes. These methods are based on Bayesian framework. They areusually used in image denoising [6].
• Variationalmethods—are considered as thedeterministic reflectionof theBayesianframework in the mirror of Gibbs’ formula in statistical mechanics. They are used,among others, in image segmentation or restoration [7].
• Partial Differential Equations—very successful approach to image representation,since PDEs are used to describe, simulate and model many dynamic phenomena.They are used, among others, in image segmentation, denoising or inpainting [8].
Some of these approaches are intrinsically interconnected. It means that a givenproblem can be described in equivalent ways by different approaches. The use of aconcrete one depends on the application.
This book is devoted to geometrical multiresolution methods. The motivation ispresented in Sect. 1.2.
1.2 Motivation
The construction of an efficient image representation algorithm cannot be done with-out the knowledge of the way in which the human visual system perceives an image.It is known that the human eye-brain system can transmit from an eye to the brainonly 20 bits per second [9]. Todays compression standards, for instance JPEG2000[10], use some tens of kilobytes for a typical image to code it. Since one needs onlya few seconds to observe an image, less than a kilobyte should be thus enough tocode this image. There is, therefore, a plenty of room for improvements in the codingtheory.
So, it is known that an improvement can be made. But the question arises howto do it? The answer is given, once more, by the research in neuropsychology and
1.2 Motivation 3
Fig. 1.1 The features differentiable by a human eye, from the left location, scale, orientation,curvature, thickness, blur
psychology of vision [2]. As follows from the experiments in that areas, a humaneye is sensitive to location, scale, orientation, curvature, thickness, and blur [11]. Allthese features are presented in Fig. 1.1. Everyone can check that all of them are easyto observe. Another feature that is perceived by a human eye is color. But note thatnot every human eye can perceive differences in some colors (like in the case of adaltonist).
Let us note that in nearly every image there are edges of different locations, scales,orientations, curvatures, thickness, andblur. In fact, usually, the combinations of thesefeatures are present. For instance, there are many edges that are of high curvature andare blurred or are of different thickness. Since a human eye is more sensitive to edgesthan textures [11], the former ones should be represented in the best possible way.
In this book, the smoothlet and multismoothlet transforms are presented (togethercalled shortly as the (multi)smoothlet transform). A multismoothlet is a vector ofsmoothlets and a smoothlet is the generalization of a wedgelet—a function definedto represent edges efficiently [12]. Both transforms are defined in order to rep-resent edges in an adaptive geometrical multiresolution way. An example of a(multi)smoothlet transform is presented in Fig. 1.2. As one can see, the transformdifferentiates all visual features mentioned above, that is (see Fig. 1.2): (1) location(caught also by wedgelets [12]), (2) scale (caught also by wedgelets), (3) orientation(caught also by wedgelets), (4) curvature (caught also by second order wedgelets[13]), (5) thickness (caught also by multiwedgelets [14]), and (6) blur (caught alsoby smoothlets [15]).
Examples of an image approximation by different adaptive methods are presentedin Fig. 1.3. In more details, a sample image is presented in Fig. 1.3a. It represents themultismoothlet consisting of three curvilinear blurred edges. As one can see, such animage can be represented by the only onemultismoothlet (Fig. 1.3a) or 52 smoothlets(Fig. 1.3b) or 148s order wedgelets (Fig. 1.3c) or 151 wedgelets (Fig. 1.3d) givingcomparable PSNR quality. The increase of the functions number is substantial. Letus note that the inverse tendency is not the same. Indeed, the sharp edge that can berepresented by only one wedgelet can also be represented by only one smoothlet.
Shift invariant versions of the (multi)smoothlet transform are also presented inthis book. The idea standing behind them is to free the representation from a quadtreerelation. In such a transform (multi)smoothlets are defined on any supports insteadof the ones based on the definition of a quadtree partition. It means that the support
4 1 Introduction
Fig. 1.2 The features caught by (multi)smoothlets: (1) location, (2) scale, (3) orientation, (4)curvature, (5) thickness, (6) blur
Fig. 1.3 Image approximation by a 1 multismoothlet, b 52 smoothlets, c 148s order wedgelets,d 151 wedgelets. PSNR of all images equals to 35d B
1.2 Motivation 5
(a) (b) (c)
Fig. 1.4 Edge representation: a isotropic, b semi-anisotropic, c anisotropic
may be placed anywhere within an image and may be of any size (though a squareis assumed in this book). Such an approach leads to many consequences. The onlybad consequence is that the transform is no longer fast (due to a huge size of thedictionary—much more locations are to be computed than in the quadtree-baseddictionary). The good consequence is that an edge can be represented inmore efficientway than in the case of a quadtree-based transform.
Let us note that a shift invariant (multi)smoothlet transform leads to a semi-anisotropic representation. Indeed, this is something between an isotropic represen-tation (represented by adaptive methods) and an anisotropic one (represented bynonadaptive methods). In the former case, supports of functions representing edgesare fixed. In the latter case, supports are adapted to an edge. In the case of the shiftinvariant (multi)smoothlet transform supports adapt only partially. Let us see theexample presented in Fig. 1.4. The isotropic representation is shown in image (a). Asone can see it is rather not optimal. The locations of supports are determined by aquadtree partition. The peak of the edge cannot be therefore represented efficientlyon this level of multiresolution. Further quadtree partition is required for this seg-ment of the edge. The anisotropic representation is presented in image (c). As onecan see it is very efficient, since the supports are well adapted to the edge. Finally,the semi-anisotropic representation, proposed in this book, is presented in image (b).The supports are defined like in adaptive methods but can adapt to the edge in aquite flexible way. The application of the shift invariant (multi)smoothlet transformto edge detection is presented in this book as well.
So far, the class of imagesmodeled by horizon functions has been commonly usedin the literature [12, 13, 16–29]. A horizon function models a black and white imagewith a smooth horizon. This model, though very popular, is rather more theoreticalthan practical. In fact, an edge present in a real image can be of different shape,blur, and multiplicity. So, in the paper [15], the class of blurred horizon functionswas proposed. This class enhances the commonly used model by introducing blurto the horizon discriminating black and white areas. In this book, the wider classof images is proposed. This is the class of blurred multihorizons. A blurred mul-tihorizon is a vector of blurred horizon functions. It represents a blurred multipleedge. Such a model is thus more practical than the commonly used class of horizonfunctions.
The multismoothlets proposed in this book were designed to represent blurredmultiedges efficiently. As was proven in this book, both theoretically and practi-
6 1 Introduction
cally, they give nearly optimal representation of images from the class of blurredmultihorizons. For comparison purposes, let us note that great majority of thegeometrical multiresolution methods proposed so far have been defined to be nearlyoptimal in the class of horizon functions and they fail to be nearly optimal in theproposed wider class. On the other hand, special cases of multismoothlets are stillnearly optimal in the appropriate subclasses of the blurred multihorizon class.
1.3 State-of-the-Art
The one of the recently leading concepts in image processing is sparsity. Sparsitymeans that, using usually an overcomplete representation, the main information ofa signal or an image is stored in a small set of coefficients. In other words, one canrepresent an image by a small number of atomic functions taken from a dictionary.Two main drawbacks have to be addressed with this approach. The first one is howto define a good dictionary? And the second one is how to choose the optimal rep-resentation of a given image, having a good dictionary? Of course, it is not possibleto define a universal good dictionary. Depending on the class of images differentdictionaries, frames, or bases have been proposed in these days. They are calledshortly as “X-lets”. They are also described further in this section. Then, having agood dictionary, the optimal or a nearly optimal solution can be found on differentways. For unstructured dictionaries, the methods based on a greedy algorithm andl1 norm minimization were proposed [30–32]. For both structured and unstructureddictionaries the methods based on dictionary learning were developed [33, 34]. Onthe other hand, for the dictionaries used in quadtree-based image representations,in other words for highly structured dictionaries, a CART-like method can be used[35]. Finally, some of dictionaries are related with their original methods of signalrepresentation.
Because geometry of an image is the most important information from the humanvisual system point of view, geometrical multiresolution methods of image repre-sentation are commonly researched in these days. There is a wide family of suchmethods. This family can be divided into two groups. The one is based on nonadap-tive methods of computation, with the use of frames, like ridgelets [36], curvelets[37], contourlets [38], shearlets [39, 40], and directionlets [41]. In the second groupapproximations are computed in an adaptive way. The majority of these represen-tations are based on dictionaries, examples include wedgelets [12], beamlets [42],second order wedgelets [43], platelets [29], surflets [16], and smoothlets [15]. How-ever, the adaptive schemes based on bases have been also recently proposed, likebrushlets [44], bandelets [45], grouplets [46], and tetrolets [47]. More and more“X-lets” have been defined.
The nonadaptive methods of image representation are characterized by fast trans-forms and are based on frames. Such an approach leads to overcomplete represen-tations. However, the M-term approximation of these methods is better than that
1.3 State-of-the-Art 7
of wavelets, what follows from the research [3, 48]. The family of these methodsconsists of many known functions. Ridgelets are defined as directional wavelets,they are used to represent line discontinuities instead of point ones as it is in thecase of wavelets [3, 36]. Curvelets are defined as a tight frame used to representsmooth objects having discontinuities along smooth curves. They are used in imagecompression, denoising, segmentation, and texture characterization [3, 37, 49, 50].Contourlets define something like a discrete version of curvelets, which is simplein implementation. It is based on a double filter bank structure by combining theLaplacian pyramid with a directional filter bank. Contourlets are used in image com-pression and denoising [38]. Shearlets are defined as a family of functions generatedby a generator with parabolic scaling, shearing and translation operator, in the sameway as wavelets are generated by scaling and translation [40]. They are used in edgedetection [51]. Directionlets are related to an anisotropic multidirectional lattice-based perfect reconstruction and a critically sampled wavelet transform [41]. Theyare used in image compression [52].
The adaptivemethods of image representation are based on bases and dictionaries.Themethods based on bases are relatively fast, since they are usually implemented ina multiresolution filterbank way. The best known functions are as follows. Brushletsare constructed as an adaptive basis of functions, which are well localized in onlyone peak in frequency. They are used in image compression [44]. Bandelets aredefined as an orthonormal basis used in an approximation of geometric boundaries[53]. They are used in surface compression [54]. Grouplets are defined as orthogonalbases. They are defined to group pixels to represent geometrical regularities. Theyare applied to image inpainting [55]. Tetrolets realize the adaptive Haar wavelettransform performed on specific domains. The domains are of tetromino shapes.They are used in image coding [47]. There is also a number of methods that arebased on wavelet transforms computed adaptively along edges and used in imagecompression [56, 57].
The adaptive methods based on dictionaries were seen as the ones with a sub-stantial computational complexity. However, recent research have supported quitefast algorithms [19, 24, 26]. Since that the methods can be also used in real timeapplications. It is very important, since the methods have many applications in imageprocessing. They are used in image compression [15, 16, 20, 27, 28, 58, 59], objectdetection [17], denoising [14, 18, 22, 23] and edge detection [21, 25, 29, 42].
The scheme of generalization dependencies among all adaptive methods is pre-sented in Fig. 1.5. As one can see, the theory started from the introduction ofa wedgelet [12]. Shortly after that, the generalizations known as a second-orderwedgelet [43], a platelet [29] and a surflet [16] were proposed. The second-orderwedgelet is defined basing on a conic curve beamlet instead of a linear one [13]. Inthe platelet a linear color approximation is used instead of a constant one. The surfletwas extended to higher dimensions (two, three or more), but without any practicalapplication of that. Additionally, the surflet is based on a polynomial beamlet. Manyyears later a smoothlet was introduced [15]. It is based on any curve beamlet (how-ever, the conic curves have been used in practice). Additionally, the definition of the
8 1 Introduction
Fig. 1.5 The scheme of generalization dependencies among adaptive geometrical multiresolutionmethods of image representation
smoothlet has introduced the new quality to that area—the smoothlet is a continuousfunction (only some cases are not continuous). It can adapt thus to blurred edges withany degree of blur. It substantially improved the denoising results in comparison tothe other methods. It is important to mention that, in some cases, the smoothlet isalso the generalization of the platelet and the surflet. Since the smoothlet can adaptlinearly to an image, it can be a special case of the platelet. However, the plateletwas defined to represent blurred areas around a sharp edge, whereas the smooth-let was defined to represent constant areas around a smooth edge. The smoothletis also a generalization of the surflet defined for dimension equal to two. Recently,the definition of multiwedgelet was also introduced [14]. As the name suggests, itis defined as a vector of wedgelets, based on a multibeamlet. Such a construction isuseful in multiple edge detection, for instance, in approximation of edges of differ-ent thickness. Finally, the definition of a multismoothlet was presented in this book.It is a generalization of all the adaptive methods described above (although two ofthem only partially). It is a vector of continuous functions, which can be useful inrepresentation of multiple edges with different degrees of blur and any curvature.
Finally, beyond all the mentioned approximation methods, let us note that theinterest of the research community in sparsity lead to the new sampling theory,called compressed sensing [60, 61]. It is an alternative to the Shannon samplingtheorem. Compressed sensing paradigm allows to represent compressible signals ata rate below the Nyquist rate, which is used in the Shannon’s theorem. It opens quitenew possibilities in the area of sparse image representation [1, 48].
1.4 Contribution 9
1.4 Contribution
To summarize, the main contribution of this book can be pointed out as follows:
• Amultismoothlet and its transformwere introduced. Themultismoothlet is definedas a vector of smoothlets. It is also a combination of a smoothlet [15] with amultiwedgelet [14]. Two methods of multismoothlet visualization were supportedas well.
• The shift invariant version of the multismoothlet transform was proposed. It leadsto the semi-anisotropic model of edge representation. Such approach was furtherapplied to edge detection.
• The multismoothlet transform was applied to image denoising, leading to results,which are better than the ones of geometrical multiresolution state-of-the-artmethods.
• The new future directions were presented, including pattern generation, optimizedcompression or object recognition.
1.5 Outline
This book consists of two parts. In the first one, the theories of (multi)smoothletsare introduced. In the second one, the applications of (multi)smoothlets to imageprocessing are presented. In more details, the book is organized in the followingway.
In Chap.2 the definition of a curvilinear beamlet is presented, followed by thedefinition of a smoothlet. Then, the application of smoothlets to image approximationis presented. A sliding smoothlet is also defined. Then, the sparsity of smoothlets isdiscussed in details, in the mean of the Rate-Distortion dependency and theM-termapproximation.
InChap.3 the definition of amultismoothlet is presented. Then, two differentwaysof its visualization are proposed. After that the method of image approximation bymultismoothlets is described. A slidingmultismoothlet is introduced as well. Finally,the sparsity of multismoothlets is discussed.
Chapter4 is devoted to computational details. In the first order, themultismoothlettransform is described in details. Then, its computational complexity is discussed.The running times are given as well.
Chapter5 starts the second part of this book. It is devoted to image compression.It consists of two sections—the one related to binary edge images and the second onerelated to still images. In the first section, the curvilinear beamlets-based compressionscheme is described. In the second section, the smoothlets-based compression schemeis presented. Both sections end with numerical results of the benchmark imagescompression.
10 1 Introduction
Chapter6 is related to image denoising. First, the method of denoising, basedon the multismoothlet transform is presented. Then, the numerical results of thebenchmark images denoising are described.
Chapter7 treats about edge detection. First, the twomethods of edge detection aredescribed—the one based on the multismoothlet transform and the second one basedon the shift invariant multismoothlet transform. The chapter ends with numericalresults of edge detection from the benchmark images.
Finally, in Chap.8 the concluding remarks and the future directions are presented.The book ends with appendices including: the set of benchmark images, the bottom-up tree pruning algorithm and the method of smoothlet visualization.
References
1. Mallat, S.: AWavelet Tour of Signal Processing: The SparseWay.Academic Press, USA (2008)2. Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a
sparse code for natural images. Nature 381, 607–609 (1996)3. Welland, G.V. (ed.): Beyond Wavelets. Academic Press, Netherlands (2003)4. Soille, P.: Morphological Image Analysis. Principles and Applications. Springer, Heidelberg
(2010)5. Gonzales, R.C., Woods, R.E.: Digital Image Processing. Prentice Hall, Upper Saddle River
(2008)6. Won, C.S., Gray, R.M.: Stochastic Image Processing. Springer, New York (2004)7. Vese, L.A.: Variational Methods in Image Processing. Chapman and Hall/CRC Press, Boca
Raton (2013)8. Aubert, G., Kornprobst, P.: Mathematical Problems in Image Processing: Partial Differential
Equations and the Calculus Variations. Springer, New York (2002)9. Gabor, D.: Guest editorial. IRE Trans. Inf. Theory 5(3), 97–97 (1959)10. Christopoulos, C., Skodras, A., Ebrahimi, T.: The JPEG2000 still image coding system: an
overview. IEEE Trans. Consum. Electron. 46(4), 1103–1127 (2000)11. Humphreys, G.W.: Case Studies in the Neuropsychology of Vision. Psychology Press, UK
(1999)12. Donoho, D.L.: Wedgelets: nearly-minimax estimation of edges. Ann. Stat. 27, 859–897 (1999)13. Lisowska, A.: Geometrical wavelets and their generalizations in digital image coding and
processing. PhD Thesis, University of Silesia, Poland (2005)14. Lisowska, A.: Multiwedgelets in image denoising. Lect. Notes Electr. Eng. 240, 3–11 (2013)15. Lisowska, A.: Smoothlets: multiscale functions for adaptive representations of images. IEEE
Trans. Image Process. 20(7), 1777–1787 (2011)16. Chandrasekaran, V.,Wakin, M.B., Baron, D., Baraniuk, R.: Surflets: a sparse representation for
multidimensional functions containing smooth discontinuities. In: IEEE International Sympo-sium on Information Theory, Chicago, New Orleans (2004)
17. Darkner, S., Larsen, R., Stegmann, M.B., Ersbøll, B.K.:Wedgelet Enhanced AppearanceMod-els, pp. 177–184. In: Proceedings of the Computer Vision and Pattern RecognitionWorkshops,IEEE (2004)
18. Demaret, L., Friedrich, F., Führ, H., Szygowski, T.: Multiscale wedgelet denoising algorithms.Proceedings of SPIE, Wavelets XI, San Diego 5914, 1–12 (2005)
19. Friedrich, F., Demaret, L., Führ, H., Wicker, K.: Efficient moment computation over polygonaldomains with an application to rapid wedgelet approximation. SIAM J. Sci. Comput. 29(2),842–863 (2007)
References 11
20. Lisowska, A.: Second order wedgelets in image coding. In: Proceedings of EUROCON ’07Conference, Warsaw, Poland, pp. 237–244 (2007)
21. Lisowska, A.: Geometrical multiscale noise resistant method of edge detection. Lect. NotesComput. Sci. Springer 5112, 182–191 (2008)
22. Lisowska, A.: Image denoising with second order wedgelets. Int. J. Signal Imaging Syst. Eng.1(2), 90–98 (2008)
23. Lisowska, A.: Efficient denoising of images with smooth geometry. Lect. Notes Comput. Sci.Springer, Heidelberg 5575, 617–625 (2009)
24. Lisowska, A.: Moments-based fast wedgelet transform. J. Math. Imaging Vis. Springer 39(2),180–192 (2011)
25. Lisowska, A.: Edge detection by sliding wedgelets. Lect. Notes Comput. Sci. Springer, Hei-delberg 6753(1), 50–57 (2011)
26. Romberg, J.,Wakin,M.,Baraniuk,R.:Multiscalewedgelet image analysis: fast decompositionsand modeling. IEEE Int. Conf. Image Process. 3, 585–588 (2002)
27. Romberg, J., Wakin, M., Baraniuk, R.: Approximation and compression of piecewise smoothimages using a Wavelet/Wedgelet geometric model. IEEE Int. Conf. Image Process. 1, 49–52(2003)
28. Wakin, M., Romberg, J., Choi, H., Baraniuk, R.: Rate-distortion optimized image compressionusing Wedgelets. IEEE Int. Conf. Image Process. 3, 237–244 (2002)
29. Willet, R.M., Nowak, R.D.: Platelets: a multiscale approach for recovering edges and surfacesin photon limited medical imaging. IEEE Trans. Med. Imaging 22, 332–350 (2003)
30. Bruckstein, A.M., Donoho, D.L., Elad, M.: From sparse solutions of systems of equations tosparse modeling of signals and images. SIAM Rev. 51(1), 34–81 (2009)
31. Elad, M.: Sparse and Redundant Representations: From Theory to Applications in Signal andImage Processing. Springer, New York, USA (2010)
32. Elad, M., Figueiredo, M.A.T., Ma, Y.: On the role of sparse and redundant representations inimage processing. Proc. IEEE 98(6), 972–982 (2010)
33. Rubinstein, R., Bruckstein, A.M., Elad, M.: Dictionaries for sparse representation modeling.Proc. IEEE 98(6), 1045–1057 (2010)
34. Wright, J., Ma, Y., Mairal, J., Sapiro, G., Huang, T.S., Yan, S.: Sparse representation forcomputer vision and patternrecognition. Proc. IEEE 8(6), 1031–1044 (2010)
35. Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification andRegression Trees. Chapmanand Hall/CRC, Boca raton (1984)
36. Candès, E.: Ridgelets: theory and applications. PhD Thesis, Department of Statistics, StanfordUniversity, Stanford, USA (1998)
37. Candès, E., Donoho, D.L.: Curvelets–A surprisingly effective nonadaptive representation forobjects with edges. In: Cohen, A., Rabut, C., Schumaker, L.L. (eds.) Curves and Surface Fitting,Vanderbilt University Press, pp. 105–120 (1999)
38. Do, M.N., Vetterli, M.: Contourlets. In: Stoeckler, J., Welland, G.V. (eds.) Beyond Wavelets,pp. 83–105. Academic Press, San Diego (2003)
39. Kutyniok, G., Labate, D. (eds.): Shearlets: Multiscale Analysis for Multivariate Data. Springer,New York, USA (2012)
40. Labate, D., Lim, W., Kutyniok, G., Weiss, G.: Sparse multidimensional representation usingshearlets. Proc. SPIE 5914, 254–262 (2005)
41. Velisavljevic, V., Beferull-Lozano, B., Vetterli, M., Dragotti, P.L.: Directionlets: anisotropicmultidirectional representation with separable filtering. IEEE Trans. Image Process. 15(7),1916–1933 (2006)
42. Donoho, D.L., Huo, X.: Beamlet pyramids: a new form of multiresolution analysis, suited forextracting lines, curves and objects from very noisy image data. In. Proceedings of SPIE, vol.4119 (2000)
43. Lisowska, A.: Effective coding of images with the use of geometrical wavelets. In: Proceedingsof Decision Support Systems Conference, Zakopane, Poland (in Polish) (2003)
44. Meyer, F.G., Coifman, R.R.: Brushlets: a tool for directional image analysis and image com-pression. Appl. Comput. Harmonic Anal. 4, 147–187 (1997)
12 1 Introduction
45. Pennec, E., Mallat, S.: Sparse geometric image representations with bandelets. IEEE Trans.Image Process. 14(4), 423–438 (2005)
46. Mallat, S.: Geometrical grouplets. Appl. Comput. Harmonic Anal. 26(2), 161–180 (2009)47. Krommweh, J.: Image approximation by adaptive tetrolet transform. In: International Confer-
ence on Sampling Theory and Applications, Marseille, France (2009)48. Starck, J.-L., Murtagh, F., Fadili, J.M.: Sparse Image and Signal Processing: Wavelets.
Curvelets. Cambridge University Press, USA, Morphological Diversity (2010)49. Alzubi, S., Islam, N., Abbod, M.: Multiresolution analysis using wavelet, ridgelet, and curvelet
transforms for medical image segmentation. Int. J. Biomed. Imaging 2011, 136034 (2011)50. Gómez, F., Romero, E.: Texture characterization using curvelet based descriptor. PatternRecog-
nit. Lett. 32(16), 2178–2186 (2011)51. Yi, S., Labate, D., Easley, G., Krim, H.: A shearlet approach to edge analysis and detection.
IEEE Trans. Image Process. 18(5), 929–941 (2009)52. Shukla, R., Dragotti, P.L., Do, M.N., Vetterli, M.: Rate-distortion optimized tree structured
compression algorithms for piecewise polynomial images. IEEE Trans. Image Process. 14(3),343–359 (2005)
53. Peyré, G., Mallat, S.: Discrete bandelets with geometric orthogonal filters. In: Proceedingsfrom ICIP’05, vol. 1, pp. 65–68 (2005)
54. Peyré, G., Mallat, S.: Surface compression with geometric bandelets. Proc. ACM SIGGRAPH24(3), 601–608 (2005)
55. Maalouf, A., Carré, P., Augereau, B., Fernandez-Maloigne, C.: Inpainting using geometricalgrouplets. EUSIPCO’08, Lausanne, Switzerland (2008)
56. Plonka, G.: The easy path wavelet transform: a new adaptive wavelet transform for sparserepresentation of two-dimensional data. SIAM Multiscale Model. Simul. 7(9), 1474–1496(2009)
57. Wang, D.M., Zhang, L., Vincent, A., Speranza, F.: Curved wavelet transform for image coding.IEEE Trans. Image Process. 15(8), 2413–2421 (2006)
58. Huo, X., Chen, J., Donoho, D.L.: JBEAM: Coding Lines and Curves via Digital Beamlets. In:IEEE Proceedings of the Data Compression Conference, Snowbird, USA (2004)
59. Lisowska, A., Kaczmarzyk, T.: JCURVE–Multiscale curve coding via second order beamlets.Mach. Graph. Vis. 19(3), 265–281 (2010)
60. Candès,E.,Romberg, J., Tao,T.:Robust uncertainty principles: exact signal reconstruction fromhighly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)
61. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)
Part IMultismoothlet Transform
Chapter 2Smoothlets
Abstract In this chapter the family of functions, called smoothlets, was presented.A smoothlet is defined as a generalization of a wedgelet and a second order wedgelet.It is based on any curve beamlet, named as a curvilinear beamlet. Smoothlets, unlikethe other adaptive functions, are continuous functions. Thanks to that they can adapt toedges of different blur. In more details, the smoothlet can adapt to location, scale, ori-entation, curvature and blur. Additionally, a sliding smoothletwas introduced. It is thesmoothlet with location and size defined freely within an image. The Rate-Distortiondependency and the M-term approximation of smoothlets were also discussed.
Recent research in image processing is concentrated on finding efficient, sparse,representations of images. There has been defined plenty of methods that are usedin image approximation. The nonadaptive methods (like ridgelets [1], curvelets [2],contourlets [3], shearlets [4], etc.), usually based on frames, are known to be fastand efficient. The overcompletness of these methods is not a problem, since the bestcoefficients are only used in a representation. The adaptive methods (like wedgelets[5], beamlets [6], platelets [7], surflets [8], smoothlets [9], multiwedgelets [10], etc.),based on dictionaries, are known to bemore efficient than the nonadaptive ones, sincea dictionary can be defined more accurate than a frame. But, on the other hand, theyare much slower due to the fact that the additional decision has to be made “how tochose the best functions” for image representation.
All adaptive methods based on dictionaries have been defined on discontinuousfunctions [5, 7, 8]. Only the well-defined edges could be therefore represented bysuch functions. In reality, an edge presented on an image can be of different levelof blur. There are many reasons of that fact, for instance, it can be a motion blur,it can be caused by a scanning method inaccuracy or a light shadow falling intothe scene. Some of the blurred edges are undesirable and should be sharpened inthe preprocessing step, but some of them are correct and should be represented asblurred ones. To represent such blurred edges “as they are” smoothlets were defined[9]. Smoothlets are defined as continuous functions, which can adapt not only tolocation, scale, orientation and curvature, like second order wedgelets [11], but alsoto blur.
A. Lisowska, Geometrical Multiresolution Adaptive Transforms, 15Studies in Computational Intelligence 545, DOI: 10.1007/978-3-319-05011-9_2,© Springer International Publishing Switzerland 2014
16 2 Smoothlets
Let us note that such an approach led to the definition of a quite newmodel that canbe used in image approximation [9]. So far, the horizonmodel has been considered forgeometrical multiresolution adaptive image approximations. It is a simple black andwhite model with smooth horizon discriminating two constant areas. Smoothlets aredefined to give optimal approximations of a blurred horizon model. In this model, alinear transition between two constant areas is assumed, in other words, it is a blurredversion of the horizon model. Because it is a generalization of the commonly usedapproach, it enhances the possibilities of the approximation theory.
2.1 Preliminaries
Consider an image F : D → C where D = [0, 1] × [0, 1] and C ⊂ N. In practicalapplications C = {0, . . . , 255} for grayscale images and C = {0, 1} for binaryimages. DomainD can be discretized on different levels of multiresolution. It meansthat one obtains 2j · 2j elements of size 2−j × 2−j for j ∈ {0, . . . ,J}, J ∈ N. Letus assume that N = 2J. In that way one can consider an image of size N ×N pixelsin a natural way.
Let us define subdomain
Di1,i2,j = [i1/2j, (i1 + 1)/2j] × [i2/2j, (i2 + 1)/2j] (2.1)
for i1, i2 ∈ {0, . . . , 2j − 1}, j ∈ {0, . . . ,J}, J ∈ N. To simplify the considerationsthe renumerated subscripts i, j are used instead of i1, i2, j where i = i1 + i22j,
i ∈ {0, . . . , 4j − 1}. Subdomain Di,j is thus parametrized by location i and scale j.Let us note that D0,0 denotes the whole domain D and Di,J for i ∈ {0, . . . , 4J − 1}denote pixels from an N × N image.
Let us define next, a horizon as a smooth function h : [0, 1] → [0, 1] andlet us assume that h ∈ Cα, α > 0. Further, consider the characteristic functionH : D → {0, 1},
H(x, y) ={1, for y ≤ h(x),
0, for y > h(x),x, y ∈ [0, 1]. (2.2)
Then, functionH is called a horizon function if h is a horizon. FunctionH modelsthe black andwhite imagewith a horizon.Let us define then ablurred horizon functionas the horizon function HB : D → [0, 1] with a linear smooth transition betweenblack and white areas, more precisely, between h and its translation hr, hr(x) =h(x)+ r, r ∈ [0, 1]. Examples of a horizon function and a blurred horizon functionare presented in Fig. 2.1. In this book a blurred horizon function is considered, unlikein the literature, where a horizon function is used. Let us note, however, that the latterfunction is a special case of the former one. So, in this book, awider class of functionsthan in the literature is taken into consideration.
2.1 Preliminaries 17
Fig. 2.1 a A horizon function, b a blurred horizon function
(a) (b)
Fig. 2.2 Sample subdomains with denoted a beamlets, b curvilinear beamlets
Consider a subdomain Di,j for any i ∈ {0, . . . , 4j − 1}, j ∈ {0, . . . ,J}, J ∈ N.
A line segment bi,j,p, p ∈ R2, connecting two different borders of the subdomain
is called a beamlet [5]. A curvilinear segment bi,j,p, p ∈ Rn, n ∈ N, connecting
two borders of the subdomain is called a curvilinear beamlet [9]. In Fig. 2.2, samplesubdomains with denoted sample beamlets and curvilinear beamlets are presented.
Consider an image of size N × N pixels. The set of curvilinear beamlets canbe parametrized by location, scale, and curvature. So, the dictionary of curvilinearbeamlets is defined as [9]
B = {bi,j,p : i ∈ {0, . . . , 4j − 1}, j ∈ {0, . . . , log2 N}, p ∈ Rn,n ∈ N}. (2.3)
The most commonly used curvilinear beamlets are paraboidal or elliptical ones.They are usually parametrized by p = (θ, t, d), where θ, t are the polar coordinatesof the straight segment connecting the two ends of the curvilinear beamlet and dis the distance between the segment’s center and the curvilinear beamlet. Let usnote that, by setting d = 0, one obtains linear beamlets, which are parametrized byp = (θ, t). Any other classes of functions and any other parametrizations are alsopossible, depending on the applications.
18 2 Smoothlets
2.2 Image Approximation by Curvilinear Beamlets
Curvilinear beamlets can be used in binary image approximation [12]. In such a casethe image must consist of edges, any kind of an image with contours is allowed. Thealgorithm of image approximation consists of two steps.
In the first step, for each square segmentDi,j , i ∈ {0, . . . , 4j −1}, j ∈ {0, . . . ,J},J ∈ N, of the quadtree partition, the curvilinear beamlet that best approximates imageF : Di,j → {0, 1} has to be found. In the case of binary images with edges the errormetric thatmeasures the accurateness of edge approximation by a curvilinear beamlethas to be applied. The most convenient metric is the Closest Distance Metric [13],which is used in this book in the simplest form
CDM0(F ,FB) = |F ∩ FB ||F ∪ FB | , (2.4)
where F denotes the original image and FB is the curvilinear image representa-tion. CDM0 measures the quotient between the number of properly detected pixels(F ∩ FB) and the number of all pixels belonging either to the edge or to the curvilin-ear beamlet (F ∪ FB). The measure is normalized and for identical images is equalto 1, whereas for quite different images it is equal to 0.
In the second step of the image approximation algorithm, a tree pruning has to beapplied. The best choice is the bottom-up tree pruning algorithm due to the fact thatthe approximation given by that algorithm is optimal in the Rate-Distortion (R-D)sense [5] (see Appendix B for detailed explanation). Indeed, the algorithmminimizesthe following R-D problem
Rλ = minP∈PQ
{1 − CDM(F ,FB) + λ2K}, (2.5)
where the minimum is taken within all possible image partitionsP from the quadtreepartition QP , K denotes the number of bits needed to code curvilinear beamlets andλ is the penalization factor. In the case of the exact image representation λ = 0.In general, the larger the value of λ, the lesser the accurateness of approximation.Sample image representations by curvilinear beamlets for different values of λ arepresented in Fig. 2.3.
2.3 Smoothlet Definition
Consider a smooth function b : [0, 1] → [0, 1]. The translation of b is defined asbr(x) = b(x) + r, for r,x ∈ [0, 1]. Given these two functions, an extruded surfacecan be defined, represented by the following function
2.3 Smoothlet Definition 19
Fig. 2.3 Image approximation by curvilinear beamlets: a image consists of 392 curvilinear beam-lets, b image consists of 241 curvilinear beamlets
E(b,r)(x, y) = 1
rbr(x) − 1
ry, x, y ∈ [0, 1], r ∈ (0, 1]. (2.6)
In otherwords, this function represents the surface that is obtained as the trace createdby translating function b in R
3. It is obvious that equation (2.6) can be rewritten inthe following way:
r · E(b,r)(x, y) = br(x) − y, x, y ∈ [0, 1], r ∈ [0, 1]. (2.7)
Let us note that for r = 0 one obtains br = b and y = b(x). In that case theextruded surface is degenerate, this is function b, and is called a degenerated extrudedsurface [9].
Having extruded surface E(b,r), let us define a smoothlet as [9]
S(b,r)(x, y) =
⎧⎪⎨⎪⎩1, for y ≤ b(x),
E(b,r)(x, y), for b(x) < y ≤ br(x),
0, for y > br(x),
(2.8)
for x, y, r ∈ [0, 1]. Sample smoothlets for different functions b and different valuesof r, together with their projections on R
2, are presented in Fig. 2.4.Let us note that some special cases of smoothlets are well-known functions. Let
us examine some of them [9].
Example 2.1. Assume that r = 0 and b is a linear function. One then obtains
S(b,r)(x, y) ={1, for y ≤ b(x),
0, for y > b(x),(2.9)
for x, y ∈ [0, 1]. This is the well-known function called wedgelet [5].
20 2 Smoothlets
0
(a) (b) (c)
(d) (e) (f)
Fig. 2.4 Smoothlet examples (a)–(c) and their projections (d)–(f), respectively, gray areasdenote linear part; a y = 0.75x2 − x + 0.6, r = 0.4, b y = 0.2 sin (12x)+ 0.5, r = 0.2,c y = −0.8x + 0.7, r = 0.1
Example 2.2. Assume that r = 0 and b is a segment of a parabola, ellipse or hyper-bola. One then obtains S(b,r)(x, y) given by (2.9). This is the function called secondorder wedgelet [11].
Example 2.3. Assume that r = 0 and b is a segment of a polynomial. One thenobtains S(b,r)(x, y) given by (2.9). This is the function called two-dimensionalsurflet [8].
Example 2.4. Assume that r > 0, br is a linear function and b is fixed accordingly.One then obtains
S(b,r)(x, y) ={
E(b,r)(x, y), for y ≤ br(x),
0, for y > br(x),(2.10)
for x, y, r ∈ [0, 1]. In this way one obtains the special case of a platelet [7]. In fact,in the definition of the platelet any linear surface is possible instead of E(b,r).
Consider a subdomain Di,j for any i ∈ {0, . . . , 4j − 1}, j ∈ {0, . . . ,J}, J ∈ N.Let us denoteSi,j,b,r as the smoothletS(b,r) defined on that subdomain. Consider thenan image of size N × N pixels. In order to use smoothlets in image representationa dictionary of them has to be defined. Let us note that a smoothlet is parametrized
2.3 Smoothlet Definition 21
by location, scale, curvature and blur (in practical applications the discrete values ofblur r are used). So, the dictionary of smoothlets is defined as
S = {Si,j,b,r : i ∈ {0, . . . , 4j − 1}, j ∈ {0, . . . , log2 N}, b ∈ B, r ∈ [0, 1]}. (2.11)
2.4 Image Approximation by Smoothlets
Smoothlets are used in image approximation by applying the following grayscaleversion of a smoothlet [9]
S(u,v)(b,r) (x, y) =
⎧⎪⎨⎪⎩
u, for y ≤ b(x),
E(u,v)(b,r) (x, y), for b(x) < y ≤ br(x),
v, for y > br(x),
(2.12)
for x, y, r ∈ [0, 1], where
E(u,v)(b,r) (x, y) = (u − v) · E(b,r)(x, y) + v. (2.13)
In the case of grayscale images u, v ∈ {0, . . . , 255}. Let us note that the grayscaleversion of the smoothlet is obtained as S
(u,v)(b,r) = (u − v) · S(b,r) + v.
Image approximation by smoothlets consists of two steps [9]. In the first one, thefull smoothlet decomposition of an image with the help of the smoothlet dictionary isperformed. This means that for each squareDi,j , i ∈ {0, . . . , 4j −1}, j ∈ {0, . . . ,J},the best approximation in the MSE sense by a smoothlet is found. After the fulldecomposition, on all levels, the smoothlets’ coefficients are stored in the nodesof a quadtree. Then, in the second step, the bottom-up tree pruning algorithm [5] isapplied to get a possiblyminimal number of atoms in the approximation, ensuring thebest image quality (see Appendix B for detailed explanation). Indeed, the followingLagrangian cost function is minimized:
Rλ = minP∈QP
{||F − FS ||22 + λ2K}, (2.14)
where P is a homogenous quadtree partition of the image (elements of which arestored in the quadtree from the first step), F denotes the original image, FS denotesits smoothlet representation, K is the number of smoothlets used in the image rep-resentation or the number of bits used to code it, depending on the application, andλ is the distortion rate parameter known as the Lagrangian multiplier. In the case ofexact image approximation, the quality is determined and the reconstructed image isexactly like the original one. Two examples of image representation by smoothletsare presented in Fig. 2.5 with the use of different values of parameter λ.
22 2 Smoothlets
Fig. 2.5 Examples of image representation by smoothlets for different values ofλ; a image consistsof 400 smoothlets, b image consists of 1,000 smoothlets
2.5 Sliding Smoothlets
All geometrical multiresolution adaptive methods that are based on dictionariesdefined so far are related to a quadtree partition. The appropriate transform canbe therefore fast and is multiresolution. But it is not shift invariant. So, it cannot beused, for instance, in object recognition because any shift of the object leads to aquite different set of coefficients. To overcome that problem, a notion of a slidingwedgelet was introduced [14]. In this section a sliding smoothlet is described, whichis defined in a similar way.
A sliding smoothlet is the smoothlet with location and size fixed freely within animage. So, it is not stored in any quadtree. It rather cannot thus be used in imageapproximation but gives good results in edge detection. In this situation, the smooth-let transform-based algorithm can be not efficient enough because the positions ofsmoothlets are determined by the quadtree partition. In fact, some edges can be betterapproximated by smoothlets lying freely within the image domain. Such an exam-ple is presented in Fig. 2.6. As one can see, the appropriately fixed location of thesmoothlet caused that the edge ismore likely than the one from the quadtree partition.
2.6 Smoothlets Sparsity
In general, images obtained from different image capture devices are correlated. Itmeans that they are represented bymany coefficients, which are rather large. Geomet-rical multiresolution methods lead to, usually overcomplete, sparse representations.Sparse representation of an imagemeans that themain image content (in other words,geometry of an image) is represented by a few nonzero coefficients. The rest of themrepresent image details. They are, usually, sufficiently small to be neglected withouta noticeable quality degradation.
2.6 Smoothlets Sparsity 23
Fig. 2.6 a The edge detected by the smoothlet from the quadtree partition, location= (192, 64),size=64; b the edge detected by the sliding smoothlet, location= (172, 40), size=64
Fig. 2.7 An example ofapproximation of blurredhorizon function by smooth-lets
Sparsity is expressed by theM-term approximation. It is a number of significant,large in magnitude, coefficients for a given image representation. From an efficientimage coding point of view another measure is commonly used—the R-D depen-dency. It is used to relate the minimal number of bits, denoted as rate R, used to codea given image with a distortion not exceeding D, to the distortion D. In this section,both these measures are applied to smoothlets’ sparsity evaluation.
Consider an image domain D = [0, 1] × [0, 1]. Consider then a blurred horizonfunction defined on D. It can be approximated by a number of smoothlets on agiven level of multiresolution, as presented in Fig. 2.7. In more details, an edgepresented in that image can be approximated by nearly 2j elements of size 2−j ×2−j ,j ∈ {0, . . . ,J}. In this section, the use of smoothlets based on second-order beamletsis assumed, because they were used in all practical applications throughout this book.The R-D dependency of smoothlet approximation can be computed as follows.
Rate
In order to code a smoothlet the following number of bits is needed [9] (seeSection5.2.1 for more details on image coding by smoothlets):
24 2 Smoothlets
• 2 bits for a node type coding and• the following number of bits for smoothlet parameters coding:
– 8 bits for degenerate smoothlet or– (2j + 3) + 16 + 1 bits for smoothlet with d = 0 and r = 0 or– (2j + 3) + 16 + j + 1 bits for smoothlet with d > 0 and r = 0 or– (2j + 3) + 16 + j bits for smoothlet with d = 0 and r > 0 or– (2j + 3) + 16 + j + j bits for smoothlet with d > 0 and r > 0.
The number R of bits needed to code a blurred horizon function at scale j istherefore evaluated as follows:
R ≤ 2j · 2 + 2j((2j + 3) + 16 + 2j) ≤ kR2jj, kR ∈ R. (2.15)
Distortion
Consider a square of size 2−j ×2−j containing an edge. Let us assume that this edgeis a Cα function for α > 0. From the mean value theorem, it follows that the edgeis totally included between two linear beamlets with distance 2−2j (see Fig. 2.8a)[5]. Similarly, the edge is totally included between two second order beamlets withdistance 2−3j (see Fig. 2.8b) [9]. So, the approximation distortion of edgeh by secondorder beamlet b is evaluated as
∫ 2−j
0(b(x) − h(x))dx ≤ k12
−j2−3j, k1 ∈ R. (2.16)
Consider then a blurred horizon function HB . The approximation distortion ofthis function by smoothlet S(b,r) is computed as follows [9]:
∫ 2−j
0
∫ 2−j
0(S(b,r)(x, y) − HB(x, y))dydx = I1 + I2 + I3, (2.17)
where
I1 =∫ 2−j
0
∫ b(x)
0(S(b,r)(x, y) − HB(x, y))dydx, (2.18)
I2 =∫ 2−j
0
∫ br(x)
b(x)(S(b,r)(x, y) − HB(x, y))dydx, (2.19)
I3 =∫ 2−j
0
∫ 2−j
br(x)(S(b,r)(x, y) − HB(x, y))dydx. (2.20)
2.6 Smoothlets Sparsity 25
(a) (b)
Fig. 2.8 a The distortion evaluation for linear beamlets, b the distortion evaluation for second-orderbeamlets
From the definition of functions S(b,r) and HB , evaluation (2.16), and the directcomputations, one obtains that
I1 ≤ 2−3j, I2 ≤ 2−j2−3j, I3 ≤ 2−3j . (2.21)
Then, the distortion of approximation of blurred horizon function by a smoothletis evaluated as follows [9]:
∫ 2−j
0
∫ 2−j
0(S(b,r)(x, y) − HB(x, y))dydx ≤ k22
−j2−3j, k2 ∈ R. (2.22)
Let us take into account the whole blurred edge defined on [0, 1] × [0, 1], approx-imated by nearly 2j smoothlets. One then obtains that the overall distortion D onlevel j is
D ≤ kD2−3j, kD ∈ R. (2.23)
Rate-Distortion
To compute theR-D dependency for smoothlets, let us summarize that the parametersR and D were evaluated by (2.15) and (2.23), respectively. So, let us recall that
R ∼ 2jj, D ∼ 2−3j . (2.24)
Then, let us compute j from R and substitute it in D. In that way one obtains thefollowing R-D dependency for smoothlet coding:
D(R) = kSlogR
R3 , kS ∈ R. (2.25)
26 2 Smoothlets
For comparison purposes, let us recall that for waveletsD(R) = kVlogR
R , kV ∈ R
[15] and for wedgelets D(R) = kWlogRR2 , kW ∈ R [5]. However, let us note that the
R-D dependencies for wavelets and wedgelets were evaluated for the horizon model.In the case of the blurred horizon model they can be even worse, especially in thecase of wedgelets, which cannot cope with this model efficiently (see Fig. 1.3d).
M-term approximation
The M-term approximation is used in the case in that there is no need to code animage efficiently (e.g., image denoising). From the above considerations, it followsthat each of 2j elements of size 2−j × 2−j generates distortion kD2−j2−3j. So,a blurred horizon function, consisting of M ∼ 2j elements, generates distortionD ∼ 2−3j. Therefore, D ∼ M−3.
References
1. Candès, E.: Ridgelets: theory and applications. PhD thesis, Department of Statistics, StanfordUniversity, Stanford, USA (1998)
2. Candès, E., Donoho, D.L.: Curvelets—a surprisingly effective nonadaptive representation forobjects with edges. In: Cohen A., Rabut C., Schumaker L.L. (eds.) Curves and Surface Fitting,pp. 105–120. Vanderbilt University Press, Nashville (1999)
3. Do, M.N., Vetterli, M.: Contourlets. In: Stoeckler J., Welland G.V. (eds.) BeyondWavelets, pp.83–105. Academic Press, San Diego (2003)
4. Labate, D., Lim, W., Kutyniok, G., Weiss, G.: Sparse multidimensional representation usingshearlets. Proc. SPIE 5914, 254–262 (2005)
5. Donoho, D.L.: Wedgelets: nearly-minimax estimation of edges. Ann. Stat. 27, 859–897 (1999)6. Donoho, D.L., Huo X.: Beamlet pyramids: a new form of multiresolution analysis, suited for
extracting lines, curves and objects from very noisy image data. In: Proceedings of SPIE, vol.4119. San Diego, California (2000)
7. Willet, R.M., Nowak, R.D.: Platelets: a multiscale approach for recovering edges and surfacesin photon limited medical imaging. IEEE Trans. Med. Imaging 22, 332–350 (2003)
8. Chandrasekaran, V., Wakin, M.B., Baron, D., Baraniuk, R.: Surflets: A Sparse Representa-tion for Multidimensional Functions Containing Smooth Discontinuities. IEEE InternationalSymposium on Information Theory, Chicago, USA (2004)
9. Lisowska, A.: Smoothlets—multiscale functions for adaptive representations of images. IEEETrans. Image Process. 20(7), 1777–1787 (2011)
10. Lisowska, A.: Multiwedgelets in image denoising. Lect. Notes. Electr. Eng. 240, 3–11 (2013)11. Lisowska, A.: Geometrical wavelets and their generalizations in digital image coding and
processing. PhD Thesis, University of Silesia, Poland (2005)12. Lisowska, A., Kaczmarzyk, T.: JCURVE—multiscale curve coding via second order beamlets.
Mach. Graphics Vision 19(3), 265–281 (2010)13. Prieto, M.S., Allen, A.R.: A similarity metric of edge images. IEEE Trans. Pattern Anal. Mach.
Intell. 25(10), 1265–1273 (2003)14. Lisowska, A.: Edge detection by sliding wedgelets. Lect. Notes Comput. Sci. 6753(1), 50–57
(2011)15. Mallat, S.: AWavelet Tour of Signal Processing: The SparseWay.Academic Press, USA (2008)
Chapter 3Multismoothlets
Abstract In this chapter, the theory of multismoothlets was introduced.A multismoothlet is defined as a vector of smoothlets. Such a vector can adaptefficiently to multiple edges. So, the multismoothlet can adapt to edges of differentmultiplicity, location, scale, orientation, curvature and blur. Additionally, a notionof sliding multismoothlet was introduced. It is the multismoothlet with location andsize defined freely within an image. Based on that, the shift invariant multismoothlettransform was proposed as well. The Rate-Distortion dependency and the M-termapproximation of multismoothlets were also discussed.
Geometrical multiresolution methods are concentrated on an efficient representationof edges present on images. This approach is very important, since human eye per-ceives edges much better than textures [1]. Nonadaptive geometrical multiresolutionmethods cope well with all kinds of edges [2–5]. But adaptive methods are not effi-cient in the case ofmultiple edges due to the fact that they are quadtree-based. Indeed,a single edge can be represented efficiently by these methods [6–11]. But multipleedge needs more and more quadtree partition in order to represent each single edgeindependently by a quadtree segment. Indeed, quadtree-based methods do not allowfor more than one edge per quadtree segment.
To overcome that problem multiwedgelets were defined [12]. In this chapter, thesimilar approach was proposed, named multismoothlets. Since the multiwedgelet isa vector of wedgelets [7], the multismoothlet is a vector of smoothlets [10]. It meansthat the multismoothlet can represent a multiple edge within a quadtree partitionsegment. Depending on the application, one can assume the maximal multiplicity ofan edge to be represented. In that waymultismoothlets can adapt to edges of differentmultiplicity, location, scale, orientation, curvature, and blur.
Let us note that, similarly as in the case of smoothlets, a new image modelhas to be introduced. So far, the horizon and blurred horizon models have beenconsidered for geometrical multiresolution adaptive image approximations. In thischapter, the blurred multihorizon model was introduced. It is defined as a vector ofblurred horizons. Such amodel represents a blurmultiple edge (called shortly blurredmultiedge). Multismoothlets are defined to give optimal approximations of blurred
A. Lisowska, Geometrical Multiresolution Adaptive Transforms, 27Studies in Computational Intelligence 545, DOI: 10.1007/978-3-319-05011-9_3,© Springer International Publishing Switzerland 2014
28 3 Multismoothlets
multihorizon model. Let us note that, as a generalization of the commonly usedmodel, this approach enhances further the possibilities of the approximation theory.
Additionally, the shift invariant multismoothlet transform was proposed in thischapter. It is based on sliding multismoothlets, which are defined similarly as slidingsmoothlets presented in Chap.2. Let us note that in the case of a quadtree-basedtransform any shift of an object causes that one obtains quite different set of functionsthan in the case of the original image. In the case of the proposed transform, the shift ofthe object causes that some functions still remain the same (in fact, they do remain thesame, but are situated in shifted places). The proposed transform can have numerousapplications in image analysis.
3.1 Multismoothlet Definition
Consider an image F : D → C where D = [0, 1]× [0, 1] and C ⊂ N. Consider thensubdomains Di,j for i ∈ {0, . . . , 4j − 1}, j ∈ {0, . . . , J} as defined in Chap.2. Let usdenote B(Di,j) as a set of nondegenerated curvilinear beamlets within Di,j for anyi ∈ {0, . . . , 4j − 1}, j ∈ {0, . . . , J}. Consider then a vector of curvilinear beamletsbM
i,j = [b1i,j, . . . , bMi,j], i ∈ {0, . . . , 4j −1}, j ∈ {0, . . . , J}, M ∈ N. Vector bM
i,j is called
a multibeamlet if for all m ∈ {1, . . . ,M} bmi,j ∈ B(Di,j) for fixed i ∈ {0, . . . , 4j − 1},
j ∈ {0, . . . , J}. Some examples of multibeamlets are presented in Fig. 3.1.Let us define S(Di,j) as a dictionary of smoothlets defined on Di,j for any
i ∈ {0, . . . , 4j − 1}, j ∈ {0, . . . , J}. Consider then a vector of smoothlets SMi,j =
[S1i,j, . . . , SMi,j], i ∈ {0, . . . , 4j − 1}, j ∈ {0, . . . , J}, M ∈ N. Vector SM
i,j is called a
multismoothlet if for all m ∈ {1, . . . ,M} Smi,j ∈ S(Di,j) for fixed i ∈ {0, . . . , 4j − 1},
j ∈ {0, . . . , J}. Finally, a multismoothlets’ dictionary is defined as
SM = {SMi,j : i ∈ {0, . . . , 4j − 1}, j ∈ {0, . . . , J}}. (3.1)
Some examples of multismoothlets for M = 3 are presented in Fig. 3.2. However,let us note that, unlike in the smoothlets case, there can be many ways of multi-smoothlet visualization.
3.2 Multismoothlet Visualization
Formally, a multismoothlet is visualized as a vector of smoothlets. It means thatM smoothlets are drawn in order to present a given multismoothlet. This method,though very good for theoretical considerations, is not applicable to real applications.So, the application of multismoothlets to image processing assumes the use of themethod that allows drawing all smoothlets from a given vector within one domain.
3.2 Multismoothlet Visualization 29
(a) (b) (c) (d)
Fig. 3.1 Sample multibeamlets: a for M = 1 one obtains a curvilinear beamlet, b nonoverlapping,M = 2, c overlapping, M = 2, d overlapping, M = 3
Fig. 3.2 Sample multismoothlets: a based on nonoverlapping beamlets, b based on overlappingbeamlets
Twomethods ofmultismoothlet visualizationwere proposed in this section, the so-called serial and parallel visualization. In both cases, the multismoothlet coefficientsare computed in different ways. The first method is defined in order to obtain thebest quality and the second one is defined to obtain the result relatively fast. SeeAppendix C for explanation on how to visualize a given smoothlet, if needed.
Serial Visualization
In the serial visualization of a multismoothlet the multibeamlet must benonoverlapping. The use of such a method is very good in images with many linesor edges that are more or less parallel.
To visualize a multismoothlet in the serial way let us consider a sample mul-tismoothlet S = [S1, S2, S3], in which smoothlets Si, i ∈ {1, 2, 3}, are based oncurvilinear beamlets bi, i ∈ {1, 2, 3}, defined on domain D. Assume that the smooth-lets are defined as follows:
30 3 Multismoothlets
Fig. 3.3 The method of serial multismoothlet visualization
S1(x, y) ={
h1, y ≤ b1(x),0, y > b1(x),
S2(x, y) ={
h2, y ≤ b2(x),0, y > b2(x),
S3(x, y) ={
h3, y ≤ b3(x),h4, y > b3(x),
where (x, y) ∈ D, hi ∈ Z for i ∈ {1, 2, 3, 4}. Then, the image colors are definedaccordingly: c1 = h1 + h2 + h3, c2 = h2 + h3, c3 = h3 and c4 = h4 (see Fig. 3.3 forvisual explanation).
In general, the following definition is made for a multismoothlet of size M. Letus define smoothlets as follows:
Si(x, y) ={
hi, y ≤ bi(x),0, y > bi(x),
for i ∈ {1, . . . ,M − 1}, (3.2)
SM(x, y) ={
hM , y ≤ bM(x),hM+1, y > bM(x),
where (x, y) ∈ D, hi ∈ Z for i ∈ {1, . . . ,M + 1}. Then, the image colors are definedas follows
ck =M∑
i=k
hi for k ∈ {1, . . . ,M}, cM+1 = hM+1. (3.3)
In practical applications there is a need to proceed inversely. First, the colors ci, i ∈{1, . . . ,M+1}, are found and then the smoothlets’ coefficients hi , i ∈ {1, . . . ,M+1},defining the multismoothlet are computed according to the formulas hi = ci − ci+1for i ∈ {1, . . . ,M} and hM+1 = cM+1. Any color ci, i ∈ {1, . . . ,M + 1}, can beeasily computed as a mean of all pixel values for pixels belonging to the same areabounded by any beamlet, beamlets or a segment border. Additionally, the color hasto be updated depending on the value of parameter r denoting blur. Finally, let usnote that coefficients hi, i ∈ {1, . . . ,M + 1}, can be lesser than zero.
3.2 Multismoothlet Visualization 31
Fig. 3.4 The method of parallel multismoothlet visualization
Parallel Visualization
In the parallel visualization of amultismoothlet themultibeamletmaybe overlapping.Let us note that the multibeamlet may be also nonoverlapping. The use of such amethod is good in images with many lines or edges that intersect.
In order to visualize the multismoothlet in the parallel way, let us consider mul-tismoothlet S = [S1, . . . , SM ], where smoothlets Si, i ∈ {1, . . . ,M}, are based oncurvilinear beamlets bi, i ∈ {1, . . . ,M}, defined on domain D. Assume that thesmoothlets are defined as follows:
Si(x, y) ={
h1i , y ≤ bi(x),
h2i , y > bi(x),for i ∈ {1, . . . ,M}, (3.4)
where (x, y) ∈ D, h1i , h2i ∈ Z for i ∈ {1, . . . ,M}. Then, the appropriate colors aredefined as
ca = 1
M
M∑k=1
huk for a ∈ {1, . . . ,A}, u ∈ {1, 2}, (3.5)
where A denotes the number of areas defined by the multismoothlet (see Fig. 3.4for visual explanation). In other words, the colors are the means of all smoothlets’colors.
3.3 Image Approximation by Multismoothlets
Image approximation by multismoothlets is similar to approximation by smoothlets[10]. In the first order, full quadtree decomposition is performed. It means that foreach subdomain Di,j, i ∈ {0, . . . , 4j − 1}, j ∈ {0, . . . , J}, the best approximation inthe MSE sense by a multismoothlet is found. In each node of the quadtree the para-meters of the optimal multismoothlet are stored. In the second step, the bottom–uptree pruning algorithm is applied (see Appendix B for detailed explanation). Depend-ing on the penalization factor λ from formula (2.14), approximations with differentqualities are obtained. Two examples of image approximation by multismoothletsare presented in Fig. 3.5 with the use of different values of parameter λ.
32 3 Multismoothlets
Fig. 3.5 Examples of image approxiamtion by multismoothlets for different values of λ, M = 2;a λ = 140, b λ = 70
Fig. 3.6 Image “Peppers” approximation by multismoothlets, M = 2, dmax = 0, rmax = 0: a serialvisualization, 460 atoms, PSNR= 24.46 dB, b parallel visualization, 457 atoms, PSNR= 24.91 dB
Let us note that the presented methods of visualization (and the multismoothletcomputation as well) lead to different results. The computation time of the serialmethod is unacceptable, the parallel method is rather fast. But the results of theformer method are slightly better than the results of the latter one. So, dependingon the application, one has to decide which method to use. The sample results ofapproximation are presented in Fig. 3.6 for these twomethods (for M = 2; dmax = 0,it means that the maximal curvature of a curvilinear beamlet equals zero, so allbeamlets are linear; rmax = 0, it means that the maximal blur of an edge equals zero,so only sharp edges are used).
Both methods of visualization are characterized by different mean distancebetween edges. In the parallel method, the beamlets are far closer than in the ser-ial one. The examples of image approximation by these two methods are presentedin Fig. 3.7. Unlike in the previous example, the parallel visualization assured betterrepresentation than the serial one. Additionally, one can see the tendency for edgesoccurrence in both methods.
3.4 Sliding Multismoothlets 33
Fig. 3.7 Image “Objects” approximation by multismoothlets with denoted multibeamlets, M = 2,dmax = 0, rmax = 0: a serial visualization, 103 atoms, PSNR= 22.24 dB, b parallel visualization,103 atoms, PSNR= 23.01 dB
3.4 Sliding Multismoothlets
Similarly as in Chap.2 and work [13] a sliding multismoothlet is defined. It is themultismoothlet that is not related to any quadtree but its position and size can befreely chosen within an image.
Image representation by sliding multismoothlets of size M is performed in thefollowing way. Consider an image of size N × N pixels, F : D → C, whereD = [0, 1] × [0, 1] and C ⊂ N. Let us fix j ∈ {0, . . . , J}. Consider then over-lapping subdomains DO
i,j of size 2−j × 2−j for i ∈ {0, . . . , (2J(1 − 2−j) + 1)2 − 1}.
Then, for each subdomain DOi,j compute the best smoothlet in the MSE sense. Such
obtained coefficients can be stored in a tree, a list or in a matrix. Let us note thatthis representation cannot be used for image visualization in a simple way, sincethe subdomains are overlapping. But, by choosing some coefficients (for the mul-tismoothlets that are related with a quadtree) one can represent and visualize animage in the commonly used way. The pseudocode of the coefficients computationalgorithm is presented in Algorithm 3.1.
Algorithm 3.1 Sliding Multismoothlet Representation
Input: F, M, size, shift;Output: multismoothlets’ coefficients;1. for (x=0; x+size<ImageSize; x+=shift)2. for (y=0; y+size<ImageSize; y+=shift)3. compute multismoothlet(F,x,y,size,M);
The algorithm works in the way that the size of a subdomain and its shift arefixed by a user. The shift can be made pixel by pixel or sparser, depending on theapplication. Then, for each such subdomain of a given size and with its upper leftcorner fixed at point (x, y), the multismoothlet coefficients are computed.
34 3 Multismoothlets
Fig. 3.8 Image “Balloons” (a, b) and its copy with the shifted balloon (the small one that is abovethe other ones) by four pixels to the right (b, d). a The effect of the multismoothlet transform onthe original image; b the effect of the multismoothlet transform on the shifted copy; c the effectof the shift invariant multismoothlet transform on the original image, the first and the last framesare denoted on the image; d the effect of the shift invariant multismoothlet transform on the shiftedcopy, the first and the last frames are denoted on the image. The multismoothlets were degeneratedto wedgelets (it means that M = 1, dmax = 0, rmax = 0) for better visualization of the differences
The proposed image representation can be used, for instance, in edge detection. Itcan be also used in object recognition. Let us note that the multismoothlet transformis nonshift invariant. It means that a slightly shift of an object leads to another setof coefficients. In the case of sliding multismoothlets the shift of an object causesthat some coefficients still remain the same. Thanks to that the transform based onsliding multismoothlets was named as shift invariant multismoothlet transform.
The shift invariance of the transform can be observed in Fig. 3.8. Two images aretaken into considerations: the original one and its copy with the small balloon shiftedby four pixels to the right. In the case of the quadtree-basedmultismoothlet transform
3.4 Sliding Multismoothlets 35
quite different sets of multismoothlets from these both images were obtained (seeimages Fig. 3.8a and b, in each image only three multismoothlets are shown from thetransform for the best clarity of the explanation). In the case of the transform based onsliding multismoothlets the sets of multismoothlets, nearly the same for both images,were obtained. The only difference is that the same multismoothlets are on differentpositions but in the same order. In the presented example, the shift of a subdomainwas set as four pixels. So, since the balloon was also shifted by four pixels, thedifference between these two sets of multismoothlets is by one multismoothlet. Itmeans that the second multismoothlet from Fig. 3.8d is the same as the first one fromimage Fig. 3.8c, the third one from Fig. 3.8d is the same as the second one fromFig. 3.8c, etc. All the same considerations apply to vertical shift and, what follows,to any direction shift.
To summarize, as one can see from the presented example, the shift invariantmultismoothlet transform can be used in probably all applications in which directvisualization is not an issue. So, such an image representation can be efficiently usedin an advanced image analysis and recognition. Indeed, let us consider the transformwith the shift parameter equal to one. Consider then the object that was shifted by xpixels to the right and y pixels to the down. Let us assume then that all the coefficientsare stored in the set of matrices, where each matrix is related to each scale and eachentry of the matrix is related to one multismoothlet. In such a situation, for a fixedscale the entries of the matrix of the shifted image are also shifted x pixels to theright and y pixels to the down in comparison to the matrix of the original image. Allthe above explanation is enough to apply the proposed theory in practice. However,because it is out of the scope of this book, it is not presented here.
3.5 Multismoothlets Sparsity
So far, the sparsity of adaptive geometrical multiresolution methods have been com-puted on the base of a horizon function or a blurred horizon function. Since multi-smoothlets are defined to represent multiple edges, they should be tested on blurredmultiple horizon functions. So, the R–D dependency and the M-term approxima-tion presented in this section are computed for a model called blurred multihorizondefined as a vector of blurred horizon functions.
Consider an image domain D = [0, 1] × [0, 1]. Consider then a blurredmultihorizon HM = [H1, . . . ,HM ] defined on D. Each blurred horizon functionfrom this vector can be approximated by a number of smoothlets, different for eachfunction, in the sameway as presented in Fig. 2.7. In more details, each edge from themultiedge canbe approximated bynearly 2j elements of size 2−j×2−j, j ∈ {0, . . . , J}.In the same way, a multiedge can be approximated by nearly M · 2j vector elementsof size 2−j × 2−j, j ∈ {0, . . . , J}.
36 3 Multismoothlets
Rate
From estimation (2.15) it follows that mth smoothlet from a multismoothlet of sizeM is coded with the following number of bits
Rm ≤ kRm2jj, kRm ∈ R, m ∈ {1, . . . ,M}. (3.6)
So, the number of bits needed to code this multismoothlet is estimated as follows:
R = R1 + . . . + RM ≤ M · kR2jj, (3.7)
where kR = max{kR1 , . . . , kRM }.
Distortion
From estimation (2.23) it follows that the distortion of mth smoothlet from a multi-smoothlet of size M is bounded as
Dm ≤ kDm2−3j, kDm ∈ R, m ∈ {1, . . . ,M}. (3.8)
So, the distortion of a multismoothlet, defined as l1 norm, is estimated as follows:
D = |D1| + . . . + |DM | ≤ M · kD2−3j, (3.9)
where kD = max{kD1 , . . . , kDM }. Let us note that the use of l1 norm is the mostintuitive one. Indeed, the edges from an multiedge generate distortions usually inde-pendently (even if these edges intersect). So, the distortion of the multiedge imageshould be considered as a sum of distortions of respective edges.
Rate-Distortion
From estimations (3.7) and (3.9), and by computing j from R and substituting it in D,it follows that the R–D dependency for multismoothlets of size M can be computedas follows:
D(R) = M4 logR − logM
R3 . (3.10)
But, taking into account the fact that R is usually far larger than M, the R–D depen-dency can be shortened, without the loss of generality, to the following formula
3.5 Multismoothlets Sparsity 37
Table 3.1 The M-term approximations for different image models and for different kinds ofsmoothlets and multismoothlets
Wedgelets WedgeletsII Smoothlets Multismoothlets
Horizon function O(M−2) O(M−3) O(M−3) O(M−3)
Blurred horizon function O(M−2 + r2 ) O(M−3 + r
2 ) O(M−3) O(M−3)
Blurred multihorizon >O(M4M−3) >O(M4M−3) >O(M4M−3) O(M4M−3)
D(R) = M4 logR
R3 . (3.11)
For comparison purposes let us note that for approximation of a blurred multi-horizon by smoothlets the R–D dependency is far worse than for multismoothlets,because the smoothlet can adapt to only one blurred horizon function a time from ablurred multihorizon. It cannot adapt to all of them, like the multismoothlet. Indeed,a vector of functions cannot be approximated by one function optimally.
M-term Approximation
Similarly as for smoothlets, the M-term approximation is computed for multi-smoothlets. From the above distortion estimation, it follows that a blurred hori-zon function of nearly 2j elements from a blurred multihorizon generates distortionD ∼ 2−3j. So, a blurred multihorizon of nearly M · 2j elements generates distortionD ∼ M · 2−3j. Therefore, D ∼ M4M−3. The M-term approximations for differenthorizon functions and multihorizons and for different smoothlets and multismooth-lets are gathered in Table3.1 for comparison purposes.
As one can see, the R–D dependency and the M-term approximation of multi-smoothlets are both the same as of smoothlets in the case of blurred horizon functionapproximation. In the case of blurred multihorizon approximation multismoothletsare the best.
References
1. Humphreys, G.W.: Case Studies in the Neuropsychology of Vision. Psychology Press, UK(1999)
2. Candès, E.: Ridgelets: theory and applications. Ph.D. thesis, Department of Statistics, StanfordUniversity, Stanford, USA (1998)
3. Candès E., Donoho, D.L.: Curvelets—a surprisingly effective nonadaptive representation forobjects with edges. In: Cohen, A., Rabut, C., Schumaker, L.L. (eds.) Curves and Surface Fitting,pp. 105–120. Vanderbilt University Press, Nashville (1999)
4. Do, M.N., Vetterli, M.: Contourlets. In: Stoeckler, J., Welland, G.V. (eds.) Beyond Wavelets,pp. 83–105. Academic Press, San Diego (2003)
38 3 Multismoothlets
5. Labate, D., Lim, W., Kutyniok, G., Weiss, G.: Sparse multidimensional representation usingshearlets. Proc. SPIE 5914, 254–262 (2005)
6. Chandrasekaran, V.,Wakin, M.B., Baron, D., Baraniuk, R.: Surflets: a sparse representation formultidimensional functions containing smooth discontinuities. In: IEEE International Sympo-sium on Information Theory, Chicago, USA (2004)
7. Donoho, D.L.: Wedgelets: nearly-minimax estimation of edges. Ann. Stat. 27, 859–897 (1999)8. Donoho, D.L., Huo, X.: Beamlet pyramids: a new form of multiresolution analysis, suited for
extracting lines, curves and objects from very noisy image data. In: Proceedings of SPIE, vol.4119 (2000)
9. Lisowska, A.: Geometrical wavelets and their generalizations in digital image coding andprocessing. Ph.D. thesis, University of Silesia, Poland (2005)
10. Lisowska, A.: Smoothlets—multiscale functions for adaptive representations of images. IEEETrans. Image Process. 20(7), 1777–1787 (2011)
11. Willet, R.M., Nowak, R.D.: Platelets: a multiscale approach for recovering edges and surfacesin photon limited medical imaging. IEEE Trans. Med. Imaging 22, 332–350 (2003)
12. Lisowska, A.: Multiwedgelets in Image Denoising. Lecture Notes in Electrical Engineering,Springer 240, 3–11 (2013)
13. Lisowska, A.: Edge Detection by Sliding Wedgelets. Lecture Notes in Computer Science,Springer, Heidelberg 6753(1), 50–57 (2011)
Chapter 4Moments-Based Multismoothlet Transform
Abstract In this chapter, the moments-based multismoothlet transform wasproposed. It is based on Custom-Built moments used to compute multiedge para-meters. The transform was presented in the consecutive steps, starting from a linearbeamlet computation. Further, smoothlet parameters are computed. And finally, mul-tismoothlet parameters are determined. At the end of this chapter, the computationalcomplexity of the presented transform was discussed followed by some numericalresults.
In the case of sparse image representation, two issues are considered. The first oneis related to the concept how to define a method that can approximate an image ina sparse way? The second one is the question: how to do it fast? In the previoustwo chapters of this book, the answer was given to the first question, whereas in thischapter the answer is given to the second one. The proposed solution is based on thefastwedgelet transform (FWT) and the idea ofmoments computation [1]. The compu-tational complexity of the presented multismoothlet transform is O(M · N 2 log2 N ),for an image of size N × N pixels and multismoothlets of size M . Asymptotically,the computational complexity of this transform is the best possible one.
Let us note, however, that the presented transform is not optimal. But it worksin the way that one can improve the result of image approximation by adjustingthe parameters, what leads to enlarging of computation time. The idea that standsbehind this approach is the following. First, a linear edge approximation is foundwithin a given image subdomain, basing onmoments computation. Then, its locationis improved in a given neighborhood. It is further adapted to the best curvilinear edgeand the best blur. Finally, the best multismoothlet is computed. The better adaptationone assumes, the more time the computations take. Depending on the applications,a user can choose between a high approximation quality and a low computationtime. By adjusting the parameters as maximal, one can obtain even the optimalapproximation, what leads to the use of the naive algorithm of the multismoothlettransform.
For comparison purposes, consider the dictionary ofmultismoothlets. It is parame-trized by location, scale, curvature, blur, andmultiplicity of an edge. It causes that the
A. Lisowska, Geometrical Multiresolution Adaptive Transforms, 39Studies in Computational Intelligence 545, DOI: 10.1007/978-3-319-05011-9_4,© Springer International Publishing Switzerland 2014
40 4 Moments-Based Multismoothlet Transform
dictionary is quite substantial. Let us note that the asymptotic number ofmultismooth-lets of size M for an image of size N × N pixels is O(M · dmax · rmax · N 2 log2 N )
(since the the asymptotic number of wedgelets is O(N 2 log2 N ) [2]), where dmaxdenotes the maximal curvature and rmax denotes the maximal blur of edges. It fol-lows that the computational complexity of the naive algorithm of the multismoothlettransform is O(M ·dmax·rmax·N 4 log2 N ). By assuming that dmax = N and rmax = None obtains O(M · N 6 log2 N ). This is rather unacceptable in practical applications.
4.1 Fast Wedgelet Transform
The theory presented in this section follows the work [1] because, as so far, themethod of wedgelet computation presented there is the fastest one. Indeed, so far,the method based on top–down prediction was proposed, with the computationalcomplexity O(N 4) [3]. Also the method based on Green’s theorem was introduced,with computational complexity O(N 3) [4]. The FWT described in this section hastime complexity O(N 2 log2 N ).
Moments are commonly used in image processing, especially in function approx-imation [5]. They are usually defined for one dimensional wavelet functions. Amoment for a wavelet function ψ is defined as∫
[0,1]m(x)ψ(x) dx, (4.1)
where m is often a power function. Such moments are used to catch point disconti-nuities of a function.
However, in a two-dimensional image there are line discontinuities instead ofpoint ones. So, to catch them properly two-dimensional moments should be appliedfor two-dimensional functions [6, 7]. A two-dimensional moment is defined as:∫∫
D
M(x, y)S(x, y) dxdy. (4.2)
Depending on the definition of function M , different kinds of moments can be used.For example, the most commonly used moments are Tchebichef [8], Zernike [9] orpower moments. Additionally, a method of Custom-Built moments construction waspresented in [10].
Consider an image F : D → C with an edge discontinuity. This image can berepresented by wedgelet W that differentiates two constant areas of colors h1 andh2. In order to determine parameters α,β, γ of linear beamlet b given by equationαx + βy = γ, approximating this edge, one can use the following theorem [10].
4.1 Fast Wedgelet Transform 41
Theorem 4.1 (Popovici and Withers, 2006). Let K be a continuously differentiablefunction of two variables, identically zero outside a bounded set D. Define
A = ∂K
∂x, B = ∂K
∂y, C = ∂
∂x(x K ) + ∂
∂y(yK )
and
α =∫∫D
A(h1 + (h2 − h1)W ) dxdy,
β =∫∫D
B(h1 + (h2 − h1)W ) dxdy,
γ =∫∫D
C(h1 + (h2 − h1)W ) dxdy.
Then, all (x, y) belonging to the plot of b satisfy equation
αx + βy = γ.
The best choice for K is the following function, according to [10]:
K (x, y) ={(1 − x2)(1 − y2), for (x, y) ⊂ [−1, 1]2,0, otherwise.
(4.3)
This function assures the best approximation properties among different methods,such as power or Zernike moments [10].
Consider, once more, the image F : D → C with the edge discontinuity. Let usnote that wedgelet function parameters h1, h2 are not known a priori. The followingformulas are thus used to determine α,β, γ instead of the ones from Theorem4.1:
α =∫∫D
AF dxdy , β =∫∫D
B F dxdy , γ =∫∫D
C F dxdy. (4.4)
Such computed parameters represent exactly only thewell-defined edges that maybe represented by linear beamlets. In other cases the computed edges are not accurate.Let us recall that wedgelet W is a characteristic function of subdomain DW ∈ D.Let us define then W ≤ as a characteristic function of subdomain DW ≤ = D \ DW .Then, to approximate an image by wedgelets, the following wedgelet coefficients(denoted as h1, h2) have to be computed [1]:
42 4 Moments-Based Multismoothlet Transform
h1 =
∫∫D
W ≤F dxdy∫∫D
W ≤ dxdy, h2 =
∫∫D
W F dxdy∫∫D
W dxdy. (4.5)
To summarize, the FWT of an image of size N × N pixels works in the followingway [1].
1. For each scale j = 0 to log2 N and for each location i = 0 to 4 j −1 do steps2–3:2. Compute parameters α,β, γ of the edge from subdomain Di, j according to for-
mula (4.4).3. Compute parameters h1, h2 according to formula (4.5).
Such computed parameters of wedgelets are further used to find the best smoothlets.The process is described in Sect. 4.2.
4.2 Smoothlet Transform
The smoothlet transform is performed in the way that for each subdomain Di, j ,i ⊂ {0, . . . , 4 j − 1}, j ⊂ {0, . . . , log2 N } of an image of size N × N pixels the bestapproximation by a smoothlet in the MSE sense has to be found. As a result, oneobtains the quadtree with all coefficients of the best smoothlets stored in the appro-priate nodes. Then, to approximate an image the bottom–up tree pruning algorithmhas to be performed (see Appendix B for detailed explanation). The most time-consuming operation in the presented method is search of the optimal smoothletapproximation.
The smoothlet transform used in this book is performed in the following steps[11]:
1. For each scale j = 0 to log2 N and for each location i = 0 to 4 j −1 do steps2–5:2. Perform the FWT [1] based on moments computation—as a result parameters
α,β, γ of the edge from subdomain Di, j are given.3. Improve the parameters by searching better wedgelets in proximity of the one
found, like proposed in [1]—the search is performed in the following way: let usdenote the ends of the beamlet that defines a given wedgelet as b1 and b2. Then,compute the wedgelet parameters of all wedgelets lying in the R-neighborhoodof the beamlet (b1, b2), that is (b1 + k, b2 + l) for k, l ⊂ {−R, . . . , 0, . . . , R}.The wedgelet with the best MSE is taken as the optimal one.
4. Improve the curvature and blur by trying different values of curvature andblur—given the optimal wedgelet from the previous step, try different valuesof curvature parameter d ⊂ {0, . . . , dmax}. For each value of the curvature trydifferent values of blur parameter r ⊂ {0, . . . , rmax}. The smoothlet with the bestMSE is taken as the optimal one. Let us note, that in this step all combinations ofparameters d and r are considered.
4.2 Smoothlet Transform 43
5. Improve the contrast—given the optimal smoothlet from the previous step trydifferent parameters (h1, h2). Let us assume that h1 > h2. Then, try different(h1 + c, h2 − c) for c ⊂ {0, . . . , cmax}. Strictly speaking, try to brighten thebrighter color and try to darken the darker color. In more details, perform thecomputations in the following way. Start from c = 0, then increment c. If MSE ofthe new smoothlet is smaller thanMSE of the previous one, follow incrementationof c, otherwise break. The smoothlet with the best MSE is taken as the optimalone. It is sufficient to fix cmax = 8.
As one can easily notice, the presented transform does not support the optimalresult, but it assures quite good approximation quality versus reasonable computationtime. Additionally, one can influence the values of parameters R, dmax and rmax inorder to improve the quality of approximation or to shorten the computation time.This transform works in the way that the longer the computations take, the better theapproximation result is obtained. Parameter cmax is predefined, because it is rathersmall, what follows from the performed computations.
4.3 Multismoothlet Transform
The main question to answer in the multismoothlet transform case is how to computedifferent smoothlets for a given domain? The simplest solution is to try all combi-nations of beamlets, to compute appropriate smoothlet coefficients independentlyfor each beamlet and to choose the best combination of M smoothlets, the one thatassures the best MSE. Although such approach assures the optimal solution it is notacceptable from the practical point of view due to the substantial computation time.
So, two methods of multismoothlet computation were proposed in this section.They lead to two versions of the multismoothlet transform. The first one assumesthat multibeamlets have to be nonoverlapping (what is related to serial visualization).In the second one, the domain is slightly modified in order to compute differentsmoothlets (what is related to parallel visualization).
Serial Multismoothlet Transform
In the case of serial visualization the optimal multismoothlet is computed in thefollowing way. For a given domain, set a beamlet from the dictionary of beamlets.Next, set any beamlet that is laying below the first one. Set another one that islying below the previous one. And finally, set the M th beamlet that is laying belowthe M − 1-th one. For such a multibeamlet configuration compute the coefficientsof smoothlets. Try different configurations of beamlets and choose as the optimalmultismoothlet the one with the best MSE. In other words, this is a naive algorithmapplied for the dictionary of all nonoverlapping multibeamlets.
44 4 Moments-Based Multismoothlet Transform
Fig. 4.1 The method ofmultismoothlet computationfor M = 3 by shifting thesupport
The multismoothlet transform is then defined in the similar way as the smoothletone. It means that for each subdomain that is determined by a quadtree partition theoptimal multismoothlet has to be computed. The coefficients of all multismoothletsare stored in a quadtree. Then, to visualize the image, the bottom–up tree pruningalgorithm is applied.
Parallel Multismoothlet Transform
In the case of parallel visualization the optimal multismoothlet is computed in thefollowing way. Compute the optimal smoothlet for the given domain, obtainingthe first entry of the multismoothlet. Next, slightly shift the domain (see Fig. 4.1)by one pixel in any direction and compute the optimal smoothlet for this domain,obtaining the second entry of the multismoothlet. Obtain the remaining entries of themultismoothlet in the same way, by shifting the support in different directions. Sincethe number of directions is bounded, shift by two pixels is possible, etc. However,the optimal value is M = 3, what follows from the numerical computations. Theuse of more than three smoothlets in a multismoothlet does not lead to spectacularresults in image approximation but only lengthens the computation time.
The multismoothlet transform is then defined in the typical way. The main advan-tage of the method is that the parallel multismoothlet transform (PMT) is, in someway, shift invariant. Indeed, for a slightly shifted image (by one pixel) the multi-smoothlet transform leads to identical coefficients for some entries of the transformwith the coefficients from the transform applied to the original image. This can bevery useful in some applications.
4.4 Computational Complexity 45
4.4 Computational Complexity
In this section, the computational complexities of the presented transforms arediscussed. The success of the multismoothlet transform is based on the use of theFWT. So, its computational complexity is presented in detail, following the work [1].
Fast Wedgelet Transform
FWT is really fast. It works in O(N 2 log2 N ) time for an image of size N × Npixels. Moreover, since the number of coefficients obtained as a result is equal toN 2 log2 N , the transform cannot have better asymptotic complexity. In more details,the following theorem is true [1].
Theorem 4.2 Consider an image of size N×N pixels. The computational complexityof the FWT is O(N 2 log2 N ).
Proof. Consider any subdomain Di, j , i ⊂ {0, . . . , 4 j − 1}, j ⊂ {0, . . . , log2 N }from the quadtree partition of an image of size N × N pixels. The size of thesquare is N · 2− j × N · 2− j pixels. In order to compute the beamlet parametersα,β, γ, three integration operations are needed for each such subdomain, accordingto formula (4.4). Similarly, to compute wedgelet coefficients h1, h2 two integrationoperations are needed, according to formula (4.5). Integration denotes addition in thediscrete domain. So, this process is linear according to the number of pixels from agiven domain. Let us note that, from the definition of a quadtree partition it followsthat there are 2 j ·2 j squares of size N ·2− j ×N ·2− j pixels. So, to integrate all squaresfrom one level of decomposition (2 j · 2 j ) · (N · 2− j · N · 2− j ) dominant operationsare needed. Because the integration is performed on all levels of decomposition, thetotal number of integration operations is computed as follows [1]
log2 N∑j=0
(2 j · 2 j ) · (N · 2− j · N · 2− j ) =log2 N∑
j=0
N · N
= N 2(1 + log2 N ).
Because integration is the dominant process in the algorithm, it follows that its timecomplexity is O(N 2 log2 N ).
As a direct result from the above considerations, one obtains the following propo-sition [1].
Proposition 4.3 Consider an image of size N × N pixels. Since the full quadtreedecomposition of the image consists of U = N 2(1 + log2 N ) pixels, the computa-tional complexity of the FWT is O(U ).
46 4 Moments-Based Multismoothlet Transform
Fig. 4.2 Edge detection with the use of a FWT, b FWT with additional search for R = 6
Let us note that the result obtained from theFWT is usually not optimal because themodel assumes a well defined edge, which in a real image is not perfect. The detectededge can thus be slightly misplaced. An example of such a situation is presented inFig. 4.2a. In order to improve the final result, search a of better wedgelet in proximityof the one just found is performed, like described in step3 of the smoothlet transform.The result of the additional search for R = 6 is presented in Fig. 4.2b. As one cansee, in such a case the edge was detected ideally.
Let us note that, though the computation takes more time for the additional search,the computational complexity still remains the same due to the fact that the additionalcomputations are performed a constant number of times. However, the computationtime can be lengthened so much that the classical wedgelet transform (WT) is com-puted (that is, the naive algorithm in which the really optimal solution is found).Indeed, the following theorem is true [1].
Theorem 4.4 Consider an image of size N ×N pixels and computational complexityof FWT as O(N 2 log2 N ). And let denote R as the range of best wedgelet search forFWT, denoted as FWT+R. Then, by tending with range R to 3N − 5, one obtainsWT with computational complexity O(N 4 log2 N ).
Proof. Let us note that for a subdomain of size N ×N pixels and for arbitrary beamlet(b1, b2) from that subdomain the maximal R equals 3N −5 for bi not situated at anycorner of the square and it equals 2N −1 for bi lying at any corner, i ⊂ {1, 2}. Finally,the maximal R = 3N − 5. For the values of R larger than 3N − 5 the computationsbegin to repeat for the same beamlets. Let us note that the dominant operation for asubdomain Di, j , i ⊂ {0, . . . , 4 j − 1}, j ⊂ {0, . . . , log2 N }, is integration. Considerthen range R. For this range (2R + 1)2 integrations are performed with the help offormula (4.5), since the range length is 2R + 1 and beamlet connections for eachpair (b1 + k, b2 + l), where k, l ⊂ {−R, . . . , 0, . . . , R}, are checked. Therefore, ifR tends to 3N − 5, the number of integrations for any domain tends to
(2R + 1)2 = (2(3N − 5) + 1)2 = (6N − 9)2.
4.4 Computational Complexity 47
50 100 150 200 2500
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Image size (pixels)
Tim
e of
com
puta
tions
(se
c.)
Time complexity for FWT+R
computed FWT timecomputed FWT+1 timecomputed FWT+2 timecomputed FWT+3 timecomputed FWT+4 timecomputed FWT+5 timeestimated FWT timeestimated FWT timeestimated FWT timeestimated FWT timeestimated FWT timeestimated FWT time
Fig. 4.3 Computed and estimated computational complexity for images of different sizes fordifferent ranges of additional search of FWT+R. The computations were performed on a Pen-tium IV 3GHz processor
0.5 1 1.5 2 2.5 3 3.5 4 4.5 5440
460
480
500
520
540
560
580
Time (sec.)
MS
E
Time of computations versus MSE
FWT
FWT+1
FWT+2
FWT+3
FWT+4 FWT+5
"Monarch", level 5
Fig. 4.4 Computational complexity versus MSE for different searching ranges for fifthdecomposition level of image “Monarch.” The computations were performed on a Pentium IV3GHz processor
48 4 Moments-Based Multismoothlet Transform
So, for all subdomains from the quadtree decomposition the number of dominantoperations is computed as follows [1]
log2 N∑j=0
(2 j · 2 j ) · (N · 2− j · N · 2− j ) · (6N − 9)2 =log2 N∑
j=0
N 2(36N 2 − 108N + 81)
= (36N 4 − 108N 3 + 81N 2)
· (1 + log2 N ).
From that and from the fact that integration is the dominant operation it followsthat the time complexity is O(N 4 log2 N ). Additionally, note that when R is set asthe maximal, R = 3N − 5, all possible beamlet connections are checked and thebest wedgelets are determined. This is a naive algorithm of the WT.
The following proposition is the direct result of the above theorem and the defin-ition of FWT+R [1].
Proposition 4.5 Consider an image F of size N × N pixels. Denote FFW T +R andFW T as the approximation of F by FWT+R and WT, respectively, for a fixed λ.If R → 3N − 5 then ||FFW T +R − FW T || → 0.
From the above considerations, it follows that by enlarging the range of the bestwedgelet computation one lengthens the time of computations. On the other hand,from the construction of the algorithm it follows that in the same time one improvesthe quality of approximation. Both tendencies are confirmed in practice. Indeed, theset of plots for the FWT with different values of R are presented in Fig. 4.3. Notethat the estimated and computed times fit well. The computational complexity versusMSE is presented in Fig. 4.4 for an arbitrarily chosen image. The tendency is wellvisible—the better result one wants to obtain, the more time is needed to performthe computations.
Smoothlet Transform
Because the smoothlet transform is based on the FWT it is also fast in the asymptoticmeaning. Indeed, the following theorem is true.
Theorem 4.6 Consider an image of size N × N pixels. The time complexity of theFast Smoothlet Transform (FST) is O(N 2 log2 N ).
Proof. Consider the algorithm of the smoothlet transform presented in Sect. 4.2.From the Theorem4.2 it follows that the computational complexity of step2 ofthe transform is O(N 2 log2 N ). By assuming that R from step3 is constant, thecomputational complexity remains the same. The number of operations in step4 isdmax · rmax · O(N 2 log2 N ), since for each combination of parameters d and r one
4.4 Computational Complexity 49
Table 4.1 Computation times (s) of the third level of image “Monarch” decomposition (subdomainsof size 32 × 32 pixels) for different parameters configurations
dmax rmax M = 1 M = 2 M = 3R = 0 R = 5 R = 10 R = 0 R = 5 R = 10 R = 0 R = 5 R = 10
0 0 0.59 6.21 21.44 0.81 12.17 42.46 1.00 17.98 63.784 1.06 8.60 27.86 1.73 16.84 55.14 2.41 24.85 82.698 1.38 9.07 28.83 2.34 17.56 56.30 3.34 26.15 84.61
4 0 0.84 7.56 24.72 1.79 15.45 51.61 2.45 23.06 77.944 4.70 14.09 40.42 8.99 27.41 77.08 13.16 40.54 115.128 4.81 17.68 43.59 15.76 34.42 83.00 23.77 51.34 123.16
8 0 1.50 8.26 26.45 2.70 16.69 53.09 3.83 24.83 80.064 8.39 17.97 43.31 16.05 34.72 82.64 24.11 52.4 124.418 14.77 24.28 49.14 29.40 48.61 96.17 44.31 72.95 142.52
M = 1 denotes the smoothlet transform, M > 1 stands for the multismoothlet transform
needs to compute the FWT. Because parameters dmax and rmax are constant, the com-putational complexity still remains O(N 2 log2 N ). In step5 the new coefficients areonly considered, without performing the FWT. So, the computational complexity ofstep5 is O(N 2). Finally, the computational complexity of the smoothlet transformis O(N 2 log2 N ).
Multismoothlet Transform
Let us recall that in the case of the multismoothlet transform two versions are con-sidered. The one based on serial visualization and the second one based on parallelvisualization. The first one is more time-consuming but usually assures better visualresults than the second one. On the other hand, the second one is faster but the resultsare nonoptimal.
In the case of the serial visualization-based multismoothlet transform, one has toto consider all combinations of curvilinear beamlets that are non overlapping. Sincethe asymptotic number of linear beamlets is O(N 2 log2 N ) [2] the following theoremis true.
Theorem 4.7 Consider an image of size N × N pixels and the dictionary of mul-tismoothlets of size M. The time complexity of the Serial Multismoothlet Transform(SMT) is O(M · N 4 log2 N ).
In the case of the parallel visualization-based multismoothlet transform, oneobtains the following theorem as a straightforward result of the above considera-tions.
Theorem 4.8 Consider an image of size N × N pixels and the dictionary of multi-smoothlets of size M. The time complexity of the PMT is O(M · N 2 log2 N ).
50 4 Moments-Based Multismoothlet Transform
In order to test the computation times in practice, the algorithms of the smoothletand multismoothlet transforms were run for an image of size 256 × 256 pixelsfor different values of the parameters: dmax reflecting the maximal curvature, rmaxdenoting the maximal blur and R responsible for additional search. The tests wereperformed on an Intel Core2 Duo 2GHz processor.
The computation times are gathered in Table4.1. In order to test the multismooth-let transform computation time, the bottom–up tree pruning algorithm was not per-formed. In more details, the data presented in this table are the times of the thirdlevel of image decomposition. This is the level for that the computations are the mosttime-consuming. Let us note that by increasing the value of M the computation timesbecome longer by the factor equal nearly to M .
The presented times seem to be unacceptable in real-time applications. But, letus note, that the algorithm was not optimally coded. The software can be speed updrastically by using a coding optimization. Additionally, the use of a multithreadingcoding can further shorten the computation time.
References
1. Lisowska, A.: Moments-based fast wedgelet transform. J. Math Imaging Vis. 39(2), 180–192(2011). (Springer)
2. Donoho, D.L.: Wedgelets: nearly-minimax estimation of edges. Ann Stat 27, 859–897 (1999)3. Romberg, J.,Wakin,M.,Baraniuk,R.:Multiscalewedgelet image analysis: fast decompositions
and modeling. In: IEEE International Conference on Image Processing vol. 3, pp. 585–588(2002)
4. Friedrich, F., Demaret, L., Führ, H., Wicker, K.: Efficient moment computation over polygonaldomains with an application to rapid wedgelet approximation. SIAM J. Sci. Comput. 29(2),842–863 (2007)
5. Walker, J.S.: Fourier analysis andwavelet analysis. Not. Am.Math. Soc. 44(6), 658–670 (1997)6. Liao, S.X., Pawlak, M.: On image analysis by moments. IEEE Trans. Pattern Anal. Mach.
Intell. 18(3), 254–266 (1996)7. Teh, C.H., Chin, R.T.: On image analysis by the methods of moments. IEEE Trans. Pattern
Anal. Mach. Intell. 10(4), 496–513 (1988)8. Mukundan, R., Ong, S.H., Lee, P.A.: Image analysis by Tchebichef moments. IEEE Trans.
Image Process. 10(9), 1357–1364 (2001)9. Chong, C.W., Mukundan, R., Raveendran, P.: A comparative analysis of algorithms for fast
computation of Zernike moments. Pattern Recogn. 36, 731–742 (2003)10. Popovici, I., Withers, W.D.: Custom-Built moments for edge location. IEEE Trans. Pattern
Anal. Mach. Intell. 28(4), 637–642 (2006)11. Lisowska, A.: Smoothlet transform: theory and applications. Adv. Imaging Electr. Phys. 178,
97–145 (2013). (Elsevier)
Part IIApplications
Chapter 5Image Compression
Abstract In this chapter, the compression methods of binary and grayscale stillimages were presented. They are based on curvilinear beamlets and smoothlets,respectively. Both methods are based on quadtree decomposition of images. Eachdescription of the compression method was followed by the results of numericalexperiments. These results were further compared to the known state-of-the-artmethods.
Image compression plays a very important role in these days. Taking into accountthat there is more and more data to be stored, either a huge disc space should beused or an efficient method of data coding should exist. The construction of such amethod is not simple, since an algorithm of lossless compression cannot support acompression ratio larger than the entropy of a coded image. On the other hand, lossycompression degrades the quality of the coded image.
Transform-basedmethods of image compression are used in these days. The well-known example is JPEG [1] and JPEG2000 [2]. The latter algorithm is known asthe best standard, what follows from the fact that it is multiresolution (as humanvisual perception is), it works in a progressive way, it allows for a region of interestcoding and it is fast. Many attempts were undertaken in order to construct an imagecompression algorithm that can be better than JPEG2000. Many of them are basedon multiresolution geometrical methods of image representation [3–11].
Twomethods of image compression are presented in this chapter. Both of them arebased on a quadtree partition. The first one is dedicated for binary images with edgesand it is based on the curvilinear beamlet transform. The second one is applied for stillimages and it is based on the smoothlet transform. In general, the scheme of these twocompressionmethods is presented in Fig. 5.1. Image compression is performed in thefollowing way. In the first order, the appropriate transform is performed, followedby the bottom-up tree pruning algorithm. The information from the nodes of suchobtained quadtree is then written to a bitstream. Then, a compression algorithmcan be used (like the arithmetic coding) to shorten this bitstream. However, in bothpresented algorithms this step is omitted, since the obtained bitstream is optimal andits further compression is not necessary.
A. Lisowska, Geometrical Multiresolution Adaptive Transforms, 53Studies in Computational Intelligence 545, DOI: 10.1007/978-3-319-05011-9_5,© Springer International Publishing Switzerland 2014
54 5 Image Compression
Fig. 5.1 The scheme of transform-based image compression
(a) (b)
Fig. 5.2 a An example of an image and b its curvilinear beamlet representation applied in thezig-zag mode
5.1 Binary Images
The most commonly used algorithm for binary image compression is JBIG2 [12].However, this algorithm is rather old and some attempts were undertaken in order topropose a better solution, like for instance, JBEAM [4] or JCURVE [7]. The formerone is based on linear beamlets representation. The latter one is based on curvilinearbeamlets, which are described in Chap. 2. JCURVE algorithm is presented in thissection, since it outperforms JBEAM both in lossless and lossy compression.
5.1.1 Image Coding by Curvilinear Beamlets
Consider an image of size N × N pixels. Let us assume that a quadtree is relatedwith this image. Let us define then three kinds of quadtree nodes: Q—split node,N—no beamlet in a node, B—a curvilinear beamlet in a node. The nodes markedas N and B are leaves and are visible in an image. An example of an image and itsquadtree representation are presented in Fig. 5.2. This image consists of two reallycurvilinear beamlets and two linear beamlets. The symbolic representation of thisimage on a quadtree is presented in Fig. 5.2b.
The information of a coded image is stored in a quadtree in a binary way. So,all symbols have to be converted into a bit representation. Let us assume that thefollowing conversion of node symbols into bits is applied [7]:
5.1 Binary Images 55
N → “0”, Q → “10” B → “11”.
In that way, to code a node symbol, one or two bits per node are used. Additionally,any curvilinear beamlet can be represented by:
• 2 j + 3 bits for a square of size N · 2− j × N · 2− j pixels, j ⊂ {0, . . . , log2 N }, tocode linear beamlet parameters [4] and
• 3 bits to code nonzero parameter d (this is the optimal value, what follows fromthe performed simulations) or 1 bit for d = 0.
So, to code the image from Fig. 5.2 the following number of bits is used:
level 0 : Q → 2,
level 1 : B → 2 + (2 · 4 + 3) + 1 = 14, B → 2 + (2 · 4 + 3) + 3 = 16,
Q → 2, N → 1,
level 2 : B → 2 + (2 · 3 + 3) + 1 = 12, B → 2 + (2 · 3 + 3) + 3 = 14,
N → 1, N → 1,
what gives totally 63 bits.The above example is a little bit general. In practical applications, the represen-
tation of any image supports two special cases [7]:
• Let us note that a curvilinear beamlet is difficult to draw in a small square. So,such a beamlet is used only in squares of size larger or equal to 8 × 8 pixels. Insmaller squares, linear beamlets are used (parameter d is not considered in such arepresentation).
• For squares of size 2 × 2 pixels the representation kind is a little bit different—instead of considering beamlets in such small squares and dividing them into foursquares of one pixel size, quadruples of pixels are used. Smaller number of bits isthus used to code such small squares than in the case of beamlets coding.
The algorithm of image coding by curvilinear beamlets consists of two steps. Inthe first step, the data from a quadtree is converted to a bitstream in a progressiveway. In the second step, a compression of this bitstream is performed, if needed.Below, the first step of this algorithm is described.
Consider the image from Fig. 5.2a. In the first step, two numbers are coded toa bitstream—the size of the image (that is 32) and the number of bits needed tocode parameter d (the best choice is 3 [13]). Then, the binary symbols are codedto this bitstream from the top to the bottom of the tree. In more details, in the firstorder all node symbols are coded from a given level. Additionally, after each symbolB, denoting a curvilinear beamlet, appropriate symbol d, denoting the curvature ofthis beamlet, is coded. Let us denote by Bk(l) the l-th bit of the k-th curvilinearbeamlet. After the node symbols from a given level, two bits of every curvilinear
56 5 Image Compression
beamlet from the previous levels are coded followed by three bits of every curvilinearbeamlet from the present level. Consider the sample image from Fig. 5.2. It is codedby this algorithm as follows:
32, 3; Q; B, d, B, d, Q, N , B1(1), B1(2), B1(3), B2(1), B2(2), B2(3);
B, d, B, d, N , N , B1(4), B1(5), B2(4), B2(5);
B3(1), B3(2), B3(3), B4(1), B4(2), B4(3);
B1(6), B1(7), B2(6), B2(7), B3(4), B3(5), B4(4), B4(5);
B1(8), B1(9), B2(8), B2(9), B3(6), B3(7), B4(6), B4(7);
B1(10), B1(11), B2(10), B2(11), B3(8), B3(9), B4(8), B4(9), . . .
The second step of the algorithm can be omitted because the obtained bitstreamis nearly optimal. Further, compression of typical size images is not useful becauseit even lengthens the bitstream. However, it can be considered in the case of reallyhuge images.
5.1.2 Numerical Results
To present the effectiveness of JCURVE coding algorithm the numerical results arepresented. The experiments were performed on the benchmark images presented inFig. 5.3. The algorithms used in these experiments (and elsewhere in this book) werecoded in Borland’s C++ Builder environment.
First, the results of lossless coding are discussed. In Table5.1 the results of codingare presented for the following algorithms: JPEG2000 [2], JBEAM [4], JCURVE [7]and JBIG2 [12]. As one can see JPEG2000 gives the worst results because it isdeveloped, generally, for still images. JBIG2 is the old algorithm, so, the results arenot optimistic. JBEAM, as the new kind algorithm, assures quite good results. But,since edges on images are usually of different curvature, JCURVE assures the bestresults.
Lossy compression in JCURVE algorithm is performed in the bottom-up treepruning step. Depending on the value of parameter λ from formula (2.5) an imagemay be more or less compressed. By setting λ = 0 one obtains lossless compression.The results of images “Switzerland” and “Denmark” compression by JBEAM andJCURVE algorithms are presented in Figs. 5.4 and 5.5. As one can see, the results ofthese two algorithms are similar and they use nearly the same number of bits.
5.1 Binary Images 57
Fig. 5.3 The contours of the following countries: Belgium, Canada, China, Denmark, Germany,Switzerland
To summarize, one can conclude that the use of multiresolution geometricalalgorithms is quite optimistic. As one can see, JCURVE algorithm, based on thecurvilinear beamlet transform, can assure the best results of image compression.The improvement over JBEAM is about 12% and JBEAM is far better than theknown standard JBIG2.
5.2 Grayscale Images
Undoubtedly, the most commonly used algorithm for still image compression isJPEG2000 [2]. However, since its invention many attempts have been undertakento propose better solutions [5, 6, 8, 10, 14–16]. In this section, the compressionalgorithm based on the smoothlet transform is presented [6]. It is dedicated to smoothimages with smooth geometry.
5.2.1 Image Coding by Smoothlets
Consider an image of size N × N pixels. Let us assume that a quadtree is relatedwith this image. Information of the coded image is stored in this quadtree in the
58 5 Image Compression
Table 5.1 The numerical results of the losless compression (bits) [7]
Image JPEG2000 JBIG2 JBEAM JCURVE
Belgium 9,672 7,888 6,114 5,513Canada 18,904 15,224 14,555 12,710China 7,832 5,792 4,609 3,961Denmark 10,376 7,992 6,780 6,079Germany 7,968 5,792 4,916 4,101Switzerland 7,624 6,088 4,171 3,826
Fig. 5.4 An example of image “Switzerland”, coded lossy by a JBEAM (2,768 bits), b JCURVE(2,746 bits)
Fig. 5.5 An example of image “Denmark”, coded lossy by a JBEAM (4,812 bits), b JCURVE(4,795 bits)
following way. In each node of the quadtree, a node symbol is stored: Q—splitnode, N—degenerate smoothlet (without any edge), W—a smoothlet with curvatured = 0, and S—a smoothlet with curvature d ∈= 0. Depending on the symbol, furtherinformation is stored (or not) in the appropriate node in the following way [6]:
• Q : no information,• N : (N) (color),• W : there are two cases:– when r = 0:
(W) (number of beamlet) (color) (color) (0),
5.2 Grayscale Images 59
Fig. 5.6 a An example of aquadtree partition and b therelated quadtree applied in thezig-zag mode
(a) (b)
– when r > 0:
(W) (number of beamlet) (color) (color) (1) (r),
• S : there are two cases:– when r = 0:
(S) (number of beamlet) (color) (color) (d) (0),
– when r > 0:(S) (number of beamlet) (color) (color) (d) (1) (r).
A sample quadtree partition with smoothlets and the appropriate quadtree withmarked node symbols are presented in Fig. 5.6. To obtain a streamof data the quadtreeis traversed in the preorder mode. An example of such an image coding is presentedbelow (the additional marks like commas, fullstops, etc., are used only for the clarityof the code):
Q, Q, S:18066:128.22:2:1.2, W:16235:19.152:1.2, S:4298:17.143:16:1.5,S:14008:5.162:6:1.2, Q, W:13875:22.154:1.3, S:22503:141.24:8:1.2,W:11203:137.5:1.1, W:7617:156.74:1.1, W:45015:118.160:1.3, Q,
S:15223:151.78:4:1.1, W:12495:156.88:1.1, Q, W:1892:165.24:1.1, W:4659:111.170:0,W:4385:166.62:1.1, W:4326:161.50:1.1, S:21565:154.76:26:1.7.
In order to efficiently code the data from the quadtree to a bitstream it is necessaryto convert the decimal data to the binary representation. The only issue to addressis to fix how many bits are needed to code a given parameter. The number of bits isadditionally dependent on a square size. Indeed, for sufficiently small squares somesimplifications can be applied. The following convention is used [6]:
• For a square size larger than 2 × 2 pixels, the node symbols are translated as:Q—“00”, N—“11”, W—“01” and S—“10”. When the square size equals 2 × 2pixels one can choose between degenerated wedgelet and wedgelet, so N—“1”and W—“0”.
• The number of bits needed to code a beamlet is evaluated as 2 j +3 for a square ofsize N · 2− j × N · 2− j pixels [4]. So, for a square size larger than 3× 3 pixels, thenumber of bits needed to code a beamlet is 2 j + 3. When the square size equals3 × 3 pixels only 6 bits are used. And when the square size equals 2 × 2 pixels,
60 5 Image Compression
3 bits are enough (only six possible beamlets are considered: two horizontal ones,two vertical ones and two diagonal ones).
• Color is stored using 8 bits.• Parameter d is stored using j bits for a square of size N · 2− j × N · 2− j pixels(this means j − 1 bits for the curvature and 1 bit for the sign). But it is possibleonly for squares larger than 4 × 4 pixels.
• Parameter r is stored using j − 1 bits (applicable for squares larger than 2 × 2pixels).
Theoretically, such a bitstream should be further compressed by an arithmeticcoder in order to shorten the output. However, the proposed method of the bitstreamgeneration produces quite compact code. So, further stream processing is not nec-essary. On the other hand, an arithmetic coder does not give good results for shortstreams. From both these facts, it follows that the compression of the bitstream byan arithmetic coder produces a longer output than the input in the case of a shortbitstream and insignificantly shorter output than the input in the case of longer bit-streams.
5.2.2 Numerical Results
To present the effectiveness of the coding algorithm the numerical results are pre-sented. The experiments were performed on the benchmark images presented inAppendix A. The results of image coding by smoothlets for different configurationsof parameters are presented in Table5.2. Namely, in the first column smoothlets withd = 0 and r = 0 (known also as wedgelets [17]) are presented, in the second columnsmoothlets with d ∈= 0 and r = 0 (known also as second order wedgelets [13],shortly wedgeletsII) are presented and in the last column smoothlets with d ∈= 0 andr ∈= 0 are presented. As one can see, real smoothlets (not degenerated to wedgelets)assure the best compression results.
Additionally, in Figs. 5.7, 5.8, 5.9, 5.10, 5.11 and 5.12 the plots of theRate-Qualitydependency for the tested images for wedgeletsII and smoothlets are presented. Asone can see, for images with much blurred edges the differences between two plotsare substantial (like for “Chromosome”), whereas for images with rather sharp edges(like “Balloons”) they are not so large.
In Fig. 5.13 sample compression results are presented, namely images coded bywedgelets, smoothlets and JPEG2000. As one can see, for the same file size, smooth-lets assure the best visual quality for image “Objects”. However, JPEG2000 assuresbetter results than smoothlets for large bits per pixel (bpp) rates, as in the case ofimage “Monarch.”
5.2 Grayscale Images 61
Table 5.2 Numerical results of image coding for different bits per pixel (bpp) rates (PSNR)
Image bpp Wedgelets WedgeletsII Smoothlets
Balloons 0.1 20.48 20.57 20.680.2 22.09 22.20 22.330.4 24.16 24.25 24.450.7 26.37 26.41 26.671.0 28.00 28.04 28.38
Bird 0.1 28.95 28.98 29.820.2 31.25 31.28 32.360.4 33.85 33.87 34.900.7 36.36 36.38 37.171.0 38.11 38.12 38.65
Chromosome 0.1 33.14 33.11 38.160.2 36.05 35.97 40.490.4 39.23 39.19 42.590.7 41.44 41.45 44.241.0 43.15 43.18 45.35
Monarch 0.1 19.20 19.28 19.240.2 20.91 21.07 21.050.4 23.66 23.79 23.890.7 26.08 26.22 26.471.0 27.77 27.86 28.14
Objects 0.1 26.11 26.31 27.820.2 28.06 28.09 30.050.4 30.33 30.35 32.240.7 32.96 32.96 34.331.0 35.09 35.09 35.96
Peppers 0.1 22.90 23.04 23.470.2 25.28 25.37 26.130.4 27.81 27.85 28.730.7 30.08 30.11 31.001.0 31.80 31.82 32.65
To further improve the quality of compressed images, the blocking artifactsmaybeslightly suppress. This can be done by smoothing domain blocks by a filter. However,it is a known technique and the use of it one can find in [6].
To summarize, smoothlets assure better coding performance than the otheradaptive quadtree-based methods. However, these functions cannot compete withJPEG2000 at high rates. As one can expect, the more spectacular results are obtainedfor smooth images with smooth geometry. In the case of other images, smoothletscan be used to code smooth parts of an image, whereas the rest can be coded withthe use of another method.
62 5 Image Compression
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 1000014
16
18
20
22
24
26
28
30
number of bytes
PS
NR
Rate−Quality dependency for image "Balloons"
wedgeletsIIsmoothlets
Fig. 5.7 The Rate-Quality dependency for image “Balloons”
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 1000015
20
25
30
35
40
number of bytes
PS
NR
Rate−Quality dependency for image "Bird"
wedgeletsIIsmoothlets
Fig. 5.8 The Rate-Quality dependency for image “Bird”
5.2 Grayscale Images 63
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 1000015
20
25
30
35
40
45
50
number of bytes
PS
NR
Rate−Quality dependency for image "Chromosome"
wedgeletsIIsmoothlets
Fig. 5.9 The Rate-Quality dependency for image “Chromosome”
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 1000015
20
25
30
number of bytes
PS
NR
Rate−Quality dependency for image "Monarch"
wedgeletsIIsmoothlets
Fig. 5.10 The Rate-Quality dependency for image “Monarch”
64 5 Image Compression
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 1000010
15
20
25
30
35
40
number of bytes
PS
NR
Rate−Quality dependency for image "Objects"
wedgeletsIIsmoothlets
Fig. 5.11 The Rate-Quality dependency for image “Objects”
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 1000014
16
18
20
22
24
26
28
30
32
34
number of bytes
PS
NR
Rate−Quality dependency for image "Peppers"
wedgeletsIIsmoothlets
Fig. 5.12 The Rate-Quality dependency for image “Peppers”
5.2 Grayscale Images 65
Fig. 5.13 Left image “Monarch” compressed by awedgelets, size = 1, 504B, PSNR = 20.63dB, csmoothlets, size = 1, 495B, PSNR = 20.75dB, e JPEG2000, size = 1, 498B, PSNR = 22.67dB.Right image “Objects” compressed by b wedgelets, size = 1, 004B, PSNR = 26.67dB, d smooth-lets, size = 1, 001B, PSNR = 28.42dB, f JPEG2000, size = 1, 001B, PSNR = 29.18dB
66 5 Image Compression
References
1. Sayood, K.: Introduction to Data Compression. Morgan Kaufman, San Francisco (2006)2. Christopoulos, C., Skodras, A., Ebrahimi, T.: The JPEG2000 still image coding system: an
overview. IEEE Trans. Consum. Electron. 46(4), 1103–1127 (2000)3. Demaret, L., Dyn, N., Iske, A.: Image compression by linear splines over adaptive triangula-
tions. Signal Process. J. 86(7), 1604–1616 (2006)4. Huo, X., Chen, J., Donoho, D.L.: JBEAM: coding lines and curves via digital beamlets. In:
IEEE Proceedings of the Data Compression Conference, Snowbird, USA (2004)5. Lisowska, A.: Second order wedgelets in image coding. In: Proceedings of EUROCON ’07
Conference, pp. 237–244. Warsaw, Poland (2007)6. Lisowska, A.: Smoothlets—multiscale functions for adaptive representations of images. IEEE
Trans. Image Process. 20(7), 1777–1787 (2011)7. Lisowska, A., Kaczmarzyk, T.: JCURVE—multiscale curve coding via second order beamlets.
Mach. Graph. Vis. 19(3), 265–281 (2010)8. Romberg, J., Wakin, M., Baraniuk, R.: Approximation and compression of piecewise smooth
images using a wavelet/wedgelet geometric model. IEEE Int. Conf. Image Process. 1, 49–52(2003)
9. Shukla, K.K., Prasad, M.V.: Lossy image compression: domain decomposition-based algo-rithms. In: Zdonik S., Ning P., Shekhar S., Katz J., Wu X., Jain L. C., Padua D., Shen X., FurhtB. (eds.), Springer Briefs in Computer Science. Springer, New York (2011)
10. Wakin, M., Romberg, J., Choi, H., Baraniuk, R.: Rate-distortion optimized image compressionusing wedgelets. IEEE Int. Conf. Image Process. 3, 237–240 (2002)
11. Wang, D.M., Zhang, L., Vincent, A., Speranza, F.: Curved wavelet transform for image coding.IEEE Trans. Image Process. 15(8), 2413–2421 (2006)
12. Howard, P.G., Kossentini, F., Martins, B., Forchhammer, S., Rucklidge, W.J.: The emergingJbig2 standard. IEEE Trans. Circuits Syst. Video Technol. 8(7), 838–848 (1998)
13. Lisowska, A.: Geometrical wavelets and their generalizations in digital image coding andprocessing. Ph.D. thesis, University of Silesia, Poland (2005)
14. Alani, D., Averbuch, A., Dekel, S.: Image coding with geometric wavelets. IEEE Trans. ImageProcess. 16(1), 69–77 (2007)
15. Kassim, A.A., Lee, W.S., Zonoobi, D.: Hierarchical segmentation-based image coding usinghybrid quad-binary trees. IEEE Trans. Image Process. 18(6), 1284–1291 (2009)
16. Meyer, F.G., Coifman, R.R.: Brushlets: a tool for directional image analysis and image com-pression. Appl. Comput. Harmon. Anal. 4, 147–187 (1997)
17. Donoho, D.L.: Wedgelets: nearly-minimax estimation of edges. Ann. Stat. 27, 859–897 (1999)
Chapter 6Image Denoising
Abstract In this chapter, the image denoising algorithmbased on themultismoothlettransform was presented. The algorithm works in the way that image representationsare computed for different values of the penalization factor and the optimal approx-imation is taken as the result. The algorithm description was followed by the resultsof numerical experiments. These results were further compared to the known state-of-the-art methods. The proposed algorithm assures the best denoising results in themost cases.
Nearly all digital images, which are used in different applications, are imperfect.It follows from the fact that images are obtained from electronic devices, whichare not perfect. Indeed, depending on the sensor used, different kinds of noise areintroduced. For example, astronomical images are characterized by Gaussian andPoisson noise, medical images by Gaussian noise and SAR images by speckle noise.The knowledge of the noise kind is important in order to apply the proper denoisingmethod. This chapter is devoted to image denoising of Gaussian noise, since it is themost cumbersome one.
There are many denoising techniques in use. The most commonly used ones arebased on wavelets [1, 2], since they can efficiently suppress high resolution signalthat is noise. However, many new techniques have been developed recently. Theyare mainly geometrical multiresolution methods [3–7]. They lead to better resultsof denoising than wavelets because they can better adapt to edges. Indeed, waveletstend to smooth edges, whereas geometrical methods preserve them well.
Let us note that many denoising methods exist other than geometrical multires-olution ones. Some of them assure the results that are hard to compete with. Theyare based, for instance, on dictionary learning or patches [8–10]. However, they areoutside the scope of this book, because they are either not geometrical or notmultires-olution and the aim of this chapter is to show denoising possibilities of geometricalmultiresolution methods.
A. Lisowska, Geometrical Multiresolution Adaptive Transforms, 67Studies in Computational Intelligence 545, DOI: 10.1007/978-3-319-05011-9_6,© Springer International Publishing Switzerland 2014
68 6 Image Denoising
In this chapter, the new method of image denoising was presented. This methodis based on the multismoothlet transform, introduced in this book. The algorithmworks in the way that, for different values of the penalization factor, different imagerepresentations are computed and the optimal representation is taken as the result.The algorithm description is followed by the numerical results of performed experi-ments. The proposed algorithm assures the best denoising results in the most casescomparing to the state-of-the-art-methods.
6.1 Image Denoising by Multismoothlets
Image denoising is a so-called inverse problem. It means that, instead of havingoriginal image F and transforming it, one has the contaminated image and wants torecover the original. This is not a simple task because the original is not known, onlyits noised version I is available
I (x, y) = F(x, y) + σ Z(x, y), x, y ∈ [0, 1], (6.1)
where Z is an additive zero-mean Gaussian noise with standard deviation σ . Addi-tionally, the intensity of noise is usually also unknown. Successfully, there is theefficient method that allows to obtain the noise intensity level fully automatically[11]. This method is based on computation of the wavelet transform of the imagewith simulated Gaussian noise. Knowing the noise level, some noise can be removedby a multiresolution method, since it is a high resolution signal added to an image.
Imagedenoisingbymultismoothlets is performed in the samewayas bywedgelets,smoothlets, and multiwedgelets [3–5, 12]. The algorithm of denoising is similar inconstruction to the one of image approximation. The only difference is that it has tobe repeated many times. In more detail, the process is performed as follows:
1. find the best multismoothlet for each node of the quadtree partition in the MSEsense,
2. apply the bottom-up tree pruning algorithm to find the optimal approximation fora fixed value of λ,
3. repeat step 2 for different values of λ and choose, as the final result, the one thatgives the best result of denoising.
The method described above is simple in construction. However, the main draw-back is that the bottom-up tree pruning algorithm has to be repeated many times. Asfollows from the performed computations, depending on the intensity of noise, theappropriate values of λ can be used. It means that when one knows the intensity ofnoise, one can reduce the range of lambda search (see Table6.1) and, what follows,
6.1 Image Denoising by Multismoothlets 69
Table 6.1 The ranges for optimal values of lambda for different noise variancesNoise variance 0.001 0.010 0.022 0.030 0.050 0.070Lambda 12–16 44–60 68–100 82–114 110–136 120–160
0 50 100 150 20024
25
26
27
28
29
30
lambda
PS
NR
PSNR for "Bird"
lambda=56
0 50 100 150 2000
2000
4000
6000
8000
10000
12000
14000
16000
18000Number of multismoothlets for "Bird"
lambda
num
ber
of m
ultis
moo
thle
tslambda=56
(a) (b)
Fig. 6.1 The plots of dependency between λ and a PSNR, b the number of multismoothlets. Thepeak in the left plot is situated in the same place as the saddle point in the right plot
the computation time. However, in order to choose the best result, a method shouldexist that allows to do it in an automatic way. Such a method is described below.
Let us note that, during simulations, one can easily judge which denoised image isthe best one. Indeed, this can be done by computing PSNR of each denoised image,since one knows the original one. The plot of dependency between PSNR and λ ispresented in Fig. 6.1a. As one can see, the best result in this example was obtained forλ = 56. In practice, however, one does not deal with the original image, so the bestresult has to be found somehow in an automatic way. Such a method was proposedfor wedgelets denoising [3]. Similar approach was also presented in Shukla [13]. Themethod is based on the observation that the peak on the quality versus lambda plot issituated nearly in the same place as the saddle point of the number of atoms versuslambda plot (see Fig. 6.1). Since the latter dependency is known, one can easily findthe optimal value of λ. Additionally, by using the reduced range, it can be done quitequickly. Thismethodwas also applied tomultismoothlets andworks in the sameway.
6.2 Numerical Results
The experiments were performed on images presented inAppendixA, noised by zeromean Gaussian noise with different variances. The noise was added with the use ofImage Processing Toolbox in Matlab. The numerical results of image denoising by
70 6 Image Denoising
Table 6.2 The numerical results of image denoising (PSNR) by different methods for differentvalues of noise variance V
Image Method 0.001 0.010 0.022 0.030 0.050 0.070
Balloons Wavelets 30.01 20.14 16.94 15.77 13.85 12.64Curvelets 30.47 23.62 22.92 22.49 21.55 21.53wedgelets 30.50 24.03 22.29 21.72 20.60 19.94Smoothlets 29.99 24.45 22.49 21.93 20.75 20.05Multiwedgelets 29.23 24.59 22.91 22.33 21.24 20.45Multismoothlets 27.56 24.52 23.06 22.40 21.39 20.68
Bird Wavelets 29.98 20.06 16.72 15.51 13.63 12.43Curvelets 24.24 27.31 25.00 27.32 23.38 23.27Wedgelets 34.24 28.76 27.35 26.82 25.71 25.21Smoothlets 34.61 29.25 27.74 27.24 26.01 25.38Multiwedgelets 34.55 29.70 27.99 27.50 26.47 25.72Multismoothlets 33.76 29.91 28.43 27.90 26.80 26.07
Chromosome Wavelets 29.99 19.99 16.68 15.41 13.48 12.27Curvelets 23.69 24.98 28.66 26.65 22.47 21.53Wedgelets 36.45 31.48 29.56 29.07 28.31 27.15Smoothlets 38.00 33.24 31.30 30.71 29.52 28.71Multiwedgelets 37.34 32.46 30.57 29.95 29.04 28.28Multismoothlets 37.95 33.49 31.67 30.85 30.03 29.04
Monarch Wavelets 30.01 20.11 16.81 15.65 13.72 12.54Curvelets 31.92 24.09 22.46 22.10 21.25 20.61Wedgelets 30.47 24.34 22.33 21.63 20.50 19.70Smoothlets 29.15 24.37 22.50 21.80 20.59 19.81Multiwedgelets 28.32 24.54 22.83 21.91 21.01 20.41Multismoothlets 26.74 24.29 22.81 22.33 21.25 20.72
Objects Wavelets 30.13 20.26 16.94 15.69 13.72 12.52Curvelets 31.97 26.42 25.83 23.68 24.52 21.99Wedgelets 33.02 26.90 25.16 24.43 23.51 22.73Smoothlets 33.36 27.85 25.96 25.26 24.13 23.24Multiwedgelets 31.69 27.64 25.56 24.87 23.81 23.35Multismoothlets 30.81 27.38 25.95 25.47 24.35 23.56
Peppers Wavelets 30.03 20.18 16.86 15.72 13.72 12.53Curvelets 25.74 25.59 24.57 24.09 24.02 22.54Wedgelets 31.71 25.82 24.10 23.41 22.43 21.75Smoothlets 31.82 26.21 24.47 23.72 22.63 21.95Multiwedgelets 31.63 26.58 24.83 24.21 22.99 22.32Multismoothlets 30.03 26.60 25.14 24.34 23.27 22.62
differentmethods are presented in Table6.2.Wavelets and curvelets were taken as thereference methods. The method of wavelets denoising is based on the soft threshold-ing with “sym4” wavelet [1]. The method of curvelets denoising was performed withthe CurveLab software [14]. The optimal value of σ , used by curvelets, was chosen
6.2 Numerical Results 71
0 20 40 60 80 100 120 140 160 180 20020
21
22
23
24
25
26
27
28
29
30
lambda
PS
RN
"Balloons" denoising by multismoothlets (V=0.001)
M=1M=2M=3
Fig. 6.2 Denoising by multismoothlets for different values of parameter M of image “Balloons”contaminated by Gaussian noise with variance V = 0.001
as the one that assures the optimal denoising results instead of the real level of noise.This means that for noise with standard deviation σ = 10, for instance, the optimalσ equal to 12 was used, since it assured better denoising results. Additionally, theresults of denoising by wedgelets, smoothlets, multiwedgelets, and multismoothletsare presented. It is important to know that the numerical results for multiwedgeletsand multismoothlets that are presented in this chapter were performed for R = 5.By enlarging the parameter R one can obtain better results of denoising. Addition-ally, the following parameters were used: M = 3, dmax = 5 and rmax = 5 formultismoothlets.
By analysing Table6.2 one can conclude that multismoothlets assure best denois-ing results, especially for imageswith blurred edges like “Chromosome”or “Objects”.In the other cases, the special case of multismoothlets, named multiwedgelets, giveoptimal denoising results. Additionally, the more smoothlets in multismoothlet, thebetter the denoising results. This can be observed in Figs. 6.2, 6.3, 6.4, 6.5, 6.6and 6.7. In these images, the plots of denoising quality are presented, given fordifferent values of λ and for different values of M (of course, M = 1 denotessmoothlets).
72 6 Image Denoising
0 20 40 60 80 100 120 140 160 180 20021
22
23
24
25
26
27
28
29
30
lambda
PS
NR
"Bird" denoising by multismoothlets (V=0.010)
M=1M=2M=3
Fig. 6.3 Denoising by multismoothlets for different values of parameter M of image “Bird” con-taminated by Gaussian noise with variance V = 0.010
0 20 40 60 80 100 120 140 160 180 20016
18
20
22
24
26
28
30
32
lambda
PS
NR
"Chromosome" denoising by multismoothlets (V=0.022)
M=1M=2M=3
Fig. 6.4 Denoising by multismoothlets for different values of parameter M of image “Chromo-some” contaminated by Gaussian noise with variance V = 0.022
6.2 Numerical Results 73
0 20 40 60 80 100 120 140 160 180 20016
17
18
19
20
21
22
23"Monarch" denoising by multismoothlets (V=0.030)
lambda
PS
NR
M=1M=2M=3
Fig. 6.5 Denoising by multismoothlets for different values of parameter M of image “Monarch”contaminated by Gaussian noise with variance V = 0.030
0 20 40 60 80 100 120 140 160 180 20015
16
17
18
19
20
21
22
23
24
25
lambda
PS
NR
"Objects" denoising by multismoothlets (V=0.050)
M=1M=2M=3
Fig. 6.6 Denoising by multismoothlets for different values of parameter M of image “Objects”contaminated by Gaussian noise with variance V = 0.050
74 6 Image Denoising
0 20 40 60 80 100 120 140 160 180 20013
14
15
16
17
18
19
20
21
22
23
lambda
PS
NR
"Peppers" denoising by multismoothlets (V=0.070)
M=1M=2M=3
Fig. 6.7 Denoising by multismoothlets for different values of parameter M of image “Peppers”contaminated by Gaussian noise with variance V = 0.070
The results of denoising by all methods for different values of noise variance arepresented in Figs. 6.8, 6.9, 6.10, 6.11, 6.12 and 6.13. As one can see, wavelets tendto blur images. They do not preserve edges properly. Curvelets cope quite well withnoise, edges are quite sharp. However, multismoothlets assure also good results. Incomparison to smoothlets and multiwedgelets, the results are definitively better. Theonly drawback—blocking artifacts—can be drastically reduced by a smoothing post-processing [5]. It can improve PSNR even up to 0.5 dB. But, since the postprocessingwas not used for the reference methods (wedgelets, smoothlets and multiwedgelets),it was also not used to multismoothlets. Anyway, the power of blocking artifactsreduction can be seen in Fig. 6.14.
Finally, observation of the difference image between a noised image and itsdenoised copy is a quite good method used in evaluation of denoising efficiency [8].To examine such an example, two such difference images are presented in Fig. 6.15.These images represent the noise that was removed by curvelets (the left image)and multismoothlets (the right image). In both cases, one can see that the noise isa nearly pure Gaussian noise, what means that the methods are quite good. What is
6.2 Numerical Results 75
Fig. 6.8 Image denoising of image “Balloons” contaminated by zero-mean Gaussian noise withvariance V = 0.001 by the methods: a wavelets, b curvelets, c wedgelets, d smoothlets, e multi-wedgelets and f multismoothlets
76 6 Image Denoising
Fig. 6.9 Imagedenoising of image “Bird” contaminated by zero-meanGaussian noisewith varianceV = 0.010 by the methods: a wavelets, b curvelets, c wedgelets, d smoothlets, e multiwedgeletsand f multismoothlets
6.2 Numerical Results 77
Fig. 6.10 Image denoising of image “Chromosome” contaminated by zero-mean Gaussian noisewith variance V = 0.022 by the methods: a wavelets, b curvelets, c wedgelets, d smoothlets, emultiwedgelets and f multismoothlets
78 6 Image Denoising
Fig. 6.11 Image denoising of image “Monarch” contaminated by zero-mean Gaussian noise withvariance V = 0.030 by the methods: a wavelets, b curvelets, c wedgelets, d smoothlets, e multi-wedgelets and f multismoothlets
6.2 Numerical Results 79
Fig. 6.12 Image denoising of image “Objects” contaminated by zero-mean Gaussian noise withvariance V = 0.050 by the methods: a wavelets, b curvelets, c wedgelets, d smoothlets, e multi-wedgelets and f multismoothlets
80 6 Image Denoising
Fig. 6.13 Image denoising of image “Peppers” contaminated by zero-mean Gaussian noise withvariance V = 0.070 by the methods: a wavelets, b curvelets, c wedgelets, d smoothlets, e multi-wedgelets and f multismoothlets
6.2 Numerical Results 81
Fig. 6.14 A segment of “Bird” approximated by a second order wedgelets, b smoothlets, c smooth-lets with reducing blocking artifacts postprocessing
Fig. 6.15 Difference images representing removed noise by the following methods: a curvelets,b multismoothlets. The noise was removed from image “Balloons” contaminated by zero meanGaussian noise with variance V = 0.022
interesting, in the case of multismoothlets-based denoising there are no blockingartifacts, which are usually seen in denoised images. Similar results were obtainedfor the other tested images.
References
1. Daubechies, I.: Ten Lectures on Wavelets. SIAM, Philadelphia (1992)2. Mallat, S.: A Wavelet Tour of Signal Processing: The Sparse Way. Academic Press, New York
(2008)3. Demaret, L., Friedrich, F., Führ, H., Szygowski, T.: Multiscale wedgelet denoising algorithms.
Proc. SPIE (Wavelets XI, San Diego) 5914, 1–12 (2005)4. Lisowska, A.: Image denoising with second order wedgelets. Int. J. Signal Imaging Syst. Eng.
1(2), 90–98 (2008)
82 6 Image Denoising
5. Lisowska, A.: Efficient Denoising of Images with Smooth Geometry. Lecture Notes in Com-puter Science, vol. 5575, pp. 617–625. Springer, Heidelberg (2009)
6. Starck, J.L., Candès, E., Donoho, D.L.: The curvelet transform for image denoising. IEEETrans. Image Process. 11(6), 670–684 (2002)
7. Welland, G.V. (ed.): Beyond Wavelets. Academic Press, San Diego (2003)8. Buades, A., Coll, B., Morel, J.M.: A review of image denoising algorithms, with a new one.
Multiscale Model. Simul. 4(2), 490–530 (2005)9. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3D transform-
domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)10. Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned
dictionaries. IEEE Trans. Image Process. 15(12), 3736–3745 (2006)11. Starck, J.-L., Murtagh, F., Bijaoui, A.: Image Processing and Data Analysis: The Multiscale
Approach. Cambridge University Press, Cambridge (1998)12. Lisowska, A.: Multiwedgelets in Image Denoising. Lecture Notes in Electrical Engineering,
vol. 240, pp. 3–11. Springer, Dordrecht (2013)13. Shukla R.: Rate-distortion optimized geometrical image processing. PhD thesis, Swiss Federal
Institute of Technology, Lausanne, Switzerland (2004)14. CurveLab.: http://www.curvelet.org/software.html (2012)
Chapter 7Edge Detection
Abstract In this chapter, the edge detectionmethods based onmultismoothlets wereproposed. The first one is based on the multismoothlet transform. The second oneis based on sliding multismoothlets. Both methods were compared to the state-of-the-art methods. As follows from the performed experiments, the method based onsliding multismoothlets leads to the best results of edge detection.
Edge detection is used in many applications. However, it is a rather difficult task,since the definition of an edge is not simple. It follows from many reasons. The firstone is that an image, taken by a camera, may not be of a good quality. Some noisecan be present on such an image. Also, a light can give shadows that are not part ofthe scene. Finally, the image can be blurred. From all that follows that any edge canbe noised, affected by a light or blurred. A perfect method of edge detection, in thefirst order, should overcome all these inconveniences.
Besides all these difficulties, edge detection is widely used in image processingtasks like image segmentation, object analysis, or recognition. There is a wide spec-trum of methods, which are fast and efficient, such as Canny, Sobel, or Prewitt filters[1, 2]. À Trous transformmethod should be also mentioned, since it can detect edgesof different strength of blur [3]. However, all the mentioned methods are pointwise,what causes that they are very fast but they are not noise resistant. So, their practicalapplication in advanced image processing tasks is rather poor.
In these days, geometrical methods of edge detection are used. The most knownmethods are the ones based on the Radon transform [4], moments [5, 6] or wedgelets[7]. The methods based on second-order wedgelets [8] and sliding wedgelets [9]were also introduced recently. All these methods work in a geometrical way. Themethods thus are rather time consuming, but, on the other hand, they support ageometrical description of detected edges (that is location, length and orientation ofline segments).
In this chapter, two methods of edge detection were presented. The first one isbased on the multismoothlet transform, which is related to an image quadtree parti-tion. The second one is based on sliding multismoothlets. As performed experimentsshow, the method based on sliding multismoothlets gives the best results among the
A. Lisowska, Geometrical Multiresolution Adaptive Transforms, 83Studies in Computational Intelligence 545, DOI: 10.1007/978-3-319-05011-9_7,© Springer International Publishing Switzerland 2014
84 7 Edge Detection
tested state-of-the-art methods. Additionally, an obtained edge is parametrized bylocation, scale, orientation, curvature and thickness. This is, probably, the largestset of coefficients used in an edge parametrization within all geometrical methodsproposed so far.
7.1 Edge Detection by Multismoothlets
Two kinds of edge detection methods were introduced in this chapter. The first oneis based strictly on the multismoothlet transform. The second one is based on slidingmultismoothlets, that means multismoothlets with position and scale defined freelywithin an image.
7.1.1 Edge Detection by Multismoothlet Transform
The algorithm of edge detection by the multismoothlet transform works in thefollowing way. First, the transform is performed (in the mean of the full quadtreedecomposition, followed by the bottom-up tree pruning algorithm). But instead ofdrawing multismoothlets, their multibeamlets are drawn. Such an approach causesthat detected edges are geometrical and multiresolution. A similar method of edgedetection, with the use of the second-order wedgelet transform, was presented in [8].The method described in this chapter is defined in the way that different parame-ters can be used to obtain different results, depending on the application. The usedparameters are described further in this chapter.
7.1.2 Edge Detection by Sliding Multismoothlets
The method of edge detection, presented in this chapter, is based on the use of theshift invariant multismoothlet transform. This transform is based on sliding multi-smoothlets of size M .
Consider image F : D → C . Consider then any subdomain Di, j ⊂ D, i, j ∈ N,of sidelength equal to si ze. In the presented algorithm the subdomains are squarebut they may be rectangular as well. Let us denote the shifting step of subdomainsas shi f t . For the multismoothlet transform shi f t is always equal to si ze. On theother hand, shi f t = 1 means that subdomains may be located freely within a givenimage.
The algorithmof edge detectionworks in theway that optimalmultismoothlets arecomputed for different subdomains. For each suchmultismoothlet, all smoothlets areconsidered. For each such smoothlet its beamlet is drawn if the difference betweenits colors c1m and c2m is larger than threshold T . The parameters si ze and shi f t can
7.1 Edge Detection by Multismoothlets 85
be fixed freely, depending on the application. The pseudo-code of the edge detectionalgorithm is presented in Algorithm7.1.
Algorithm 7.1 Edge detection by sliding multismoothlets
Input: F, M, size, shift, T;Output: an image with detected edges;1. for (x=0; x+size<ImageSize; x+=shift)2. for (y=0; y+size<ImageSize; y+=shift)3. compute multismoothlet(F, x, y, size, M);4. for each m-th smoothlet from the
multismoothlet (with colors cm1, cm2)5. if (abs(cm1-cm2)>T)6. draw m-th beamlet from
multibeamlet(F, x, y, size, M);
7.1.3 Edge Detection Parameters
In both algorithms of edge detection, presented above, a few parameters have tobe used. These parameters influence different aspects of presented results like thethickness of edges or the level of multiresolution. All these aspects are discussedbelow.
Multiplicity of Edges
Multismoothlets used in edge detection can be defined by any number of smoothlets.Since there is no need to efficiently code an image the number of smoothlets M canbe as large as one needs, depending on the application.
Geometry
Multismoothlets adapt to an image geometry, so, theoretically, all edges are repre-sented by multibeamlets. But, as the result of the image approximation algorithm,used in edge detection, also the so-called false edges are detected because for smoothareas some multismoothlets are also assigned. In order to remove false edges fromthe image, threshold T is used. It is applied in the way that a beamlet from a mul-tismoothlet is drawn only if the difference between two colors of the appropriatesmoothlet from this multismoothlet is larger than a value of T . Examples of detectededges with different values of threshold T are presented in Fig. 7.1.
86 7 Edge Detection
Fig. 7.1 Detected edges, M = 1, a T = 0, b T = 30
Fig. 7.2 Detected edges, M = 1, T = 10, a λ = 20, b λ = 60
Multiscale
Edge detection by multismoothlets is performed in a multiresolution way. Indeed,depending on the application, objects that are present on an image can be detected ondifferent levels of multiresolution. Depending on the fact whether one is interestedeither in small details or in silhouettes of objects, parameter λ from formula (2.14)can be adjusted accordingly. Sample images with detected edges on different levelsof details are presented in Fig. 7.2.
7.1 Edge Detection by Multismoothlets 87
Fig. 7.3 Detected edges, M = 1, T = 20, λ = 60, a rmax = 0, b rmax = 5
Multibeamlet Thickness
Since multismoothlets adapt to a degree of blur of an edge, detected edges are ofdifferent thickness. A beamlet from a multibeamlet can be thus drawn in two ways inthe case of edge detection. It can be drawn with thickness r (r ≤ rmax) according tothe edge blur or its thickness can be fixed as one pixel no matter how much blurredthe edge is (in that case rmax = 0). Some examples of detected edges by these twomethods are presented in Fig. 7.3.
Noise Resistance
The method of edge detection by multismoothlets is also noise-resistant becausethe multismoothlet transform preserves the geometry of an image. Moreover, themultismoothlet transform is also used in image denoising. Let us note that, dependingon the strength of noise, different values of parameters T and λ should be fixed toobtain satisfactory results. In general, the more substantial noise, the larger valuesof T and λ should be assigned.
7.2 Numerical Results
In this section, some examples of edge detection by multismoothlets and slidingmultismoothlets are presented for the test set of images presented in Appendix A.Edges were detected by the following methods:
(a) second-order wedgelets [8], in other words smoothlets for d ≥ 0, r = 0;(b) sliding wedgelets [9], in other words sliding smoothlets for d = 0, r = 0;(c) multismoothlets visualized in a serial way for M = 2, d = 0, r = 0;
88 7 Edge Detection
Fig. 7.4 Edge detection from image “Balloons” by the following methods: a second-orderwedgelets, b sliding wedgelets, c multismoothlets visualized in a serial way, d sliding multismooth-lets, e multismoothlets visualized in a parallel way, f sliding multismoothlets for noised image
7.2 Numerical Results 89
Fig. 7.5 Edge detection from image “Bird” by the following methods: a second-order wedgelets,b sliding wedgelets, c multismoothlets visualized in a serial way, d sliding multismoothlets, e mul-tismoothlets visualized in a parallel way, f sliding multismoothlets for noised image
90 7 Edge Detection
Fig. 7.6 Edge detection from image “Chromosome” by the following methods: a second-orderwedgelets, b sliding wedgelets, c multismoothlets visualized in a serial way, d sliding multismooth-lets, e multismoothlets visualized in a parallel way, f sliding multismoothlets for noised image
7.2 Numerical Results 91
Fig. 7.7 Edge detection from image “Monarch” by the following methods: a second-orderwedgelets, b sliding wedgelets, c multismoothlets visualized in a serial way, d sliding multismooth-lets, e multismoothlets visualized in a parallel way, f sliding multismoothlets for noised image
92 7 Edge Detection
Fig. 7.8 Edge detection from image “Objects” by the followingmethods: a second-orderwedgelets,b sliding wedgelets, c multismoothlets visualized in a serial way, d sliding multismoothlets,e multismoothlets visualized in a parallel way, f sliding multismoothlets for noised image
7.2 Numerical Results 93
Fig. 7.9 Edgedetection from image “Peppers” by the followingmethods:a second-orderwedgelets,b sliding wedgelets, c multismoothlets visualized in a serial way, d sliding multismoothlets,e multismoothlets visualized in a parallel way, f sliding multismoothlets for noised image
94 7 Edge Detection
(d) sliding multismoothlets for M = 2, d = 0, r = 0;(e) multismoothlets visualized in a parallel way for M = 3, d = 0, r = 0;(f) sliding multismoothlets for M = 2, d = 0, r = 0 applied for images contami-
nated by zero-mean Gaussian noise with variance V = 0.005.
All detected edges of all tested images are presented in Figs. 7.4, 7.5, 7.6, 7.7,7.8 and 7.9. The reference methods, based on second-order wedgelets and slidingwedgelets, were treated as special cases of smoothlets or sliding smoothlets-basedmethods. So, all edges were detected by the (multi)smoothlet software.
The reference methods, presented in this chapter, are known to be competitiveto the state-of-the-art ones, like the Hough or Radon transform, or moments [8, 9].So, there is no need to compare the multismoothlets-based methods to all commonlyused state-of-the-art methods. Moreover, the most commonly used Hough or Radontransform-based edge detectors work well only for a small class of images (the oneswith well defined straight or ellipsoidal edges). In the case of the benchmark imagesused in this book, they do not give satisfactory results.
From the presented results, one can see that the method based on sliding multi-smoothlets assures better results than the one based on the multismoothlet transform.Indeed, freely placed multismoothlets can better adapt to edges than multismoothletsplaced according to a quadtree partition.
By observing the method based on the (multi)smoothlet transform one can con-clude that multismoothlets visualized in a serial way assure better results than multi-smoothlets visualized in a parallel way. Additionally, smoothlets support best results.In the case of sliding (multi)smoothlets, one can observe that the use of M = 2 insteadof M = 1 improve the results of edge detection.
To test noise resistance of the proposed method (the one based on sliding multi-smoothlets), the images contaminated by zero-mean Gaussian noise with varianceV = 0.005 were tested. By comparing edges detected by sliding multismoothletsfrom a given image and from its noised copy, one can see that the edge detectionresults of noised images are satisfactory.
References
1. Canny, J.: Computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell.8, 679–714 (1986)
2. Gonzales, R.C., Woods, R.E.: Digital Image Processing. Prentice Hall, New Jersey (2008)3. Starck, J.-L., Murtagh, F., Bijaoui, A.: Image Processing and Data Analysis: The Multiscale
Approach. Cambridge University Press, Cambridge (1998)4. Deans, S.R.: The Radon Transform and Some of Its Applications. Wiley, New York (1983)5. Lisowska, A.: Multiscale moments-based edge detection. In: Proceedings of EUROCON ’09
Conference, pp. 1414–1419. St.Petersburg, Russia (2009)
References 95
6. Popovici, I., Withers, W.D.: Custom-built moments for edge location. IEEE Trans. Pattern Anal.Mach. Intell. 28(4), 637–642 (2006)
7. Donoho, D.L.: Wedgelets: nearly-minimax estimation of edges. Ann. Stat. 27, 859–897 (1999)8. Lisowska, A.: Geometrical multiscale noise resistant method of edge detection. Lect. Notes
Comput. Sci. 5112, 182–191 (2008). (Springer)9. Lisowska, A.: Edge detection by sliding wedgelets. Lect. Notes Comput. Sci. 6753(1), 50–57
(2011). (Springer, Heidelberg)
Chapter 8Summary
Abstract In this chapter, the concluding remarks and the further directions in thearea of multismoothlets, which were introduced in this book, were presented. Inthis book, the theory of multismoothlets was presented. This theory is the general-ization of the concepts of geometrical multiresolution adaptive methods of imageapproximation proposed so far. Such generalization leads to new possibilities andapplications to image processing and analysis. There is still plenty of work that canbe done in this field. Some open problems are described in this section.
8.1 Concluding Remarks
In this book, the notion of a multismoothlet was introduced as a vector of smoothlets.Themethods ofmultismoothlet computation and visualizationwere proposed aswell.Two kinds of transforms were described: the one based on quadtree decompositionand the second one based on slidingmultismoothlets. The transforms were applied toimage denoising and edge detection. In both cases, they assured satisfactory results,usually better than the well known state-of-the-art methods.
Smooth horizon functions were taken into considerations in this book as well.This class of functions was first proposed in [1]. It is a wider class of functionsthan the commonly used horizon class. In fact, the horizon functions class is a littlebit more theoretical because edges in images are usually of different sharpness. So,the approximation of them by discontinuous functions is rather not efficient. Toomany functions are used to approximate a blurred edge. And the result is not visuallypleasant as well. The class of smooth horizon functions better fits to real applications.
Additionally, consideration of a multiedge model instead of a single edge oneseems to be better, because in the case of edges that are very close, toomany functionshave to be used to represent these edges properly by the single edgemodel. In the caseof multifunctions far lesser number of them must be used. A well-defined dictionaryallows to fit even only one multismoothlet to a given multiedge. The only drawback
A. Lisowska, Geometrical Multiresolution Adaptive Transforms, 97Studies in Computational Intelligence 545, DOI: 10.1007/978-3-319-05011-9_8,© Springer International Publishing Switzerland 2014
98 8 Summary
is to find a really fast method of approximation. The computational complexity ofthe method proposed in this book is asymptotically optimal, but probably much workcan be yet done to make it faster.
The class of geometrical multiresolution adaptive methods described in this bookseems to be very useful in obtaining sparse representations for a well defined classof images. As was shown throughout this book, some applications of this theory,like edge detection, lead to the best results among the state-of-the-art methods. Onthe other hand, some of them, like image denoising, cannot compete with the well-known efficient methods. Some applications seem to be even beyond the ability ofthis approach, like inpainting or texture recognition. However, all these methodswere applied to an image as a whole, but they can be also used in hybrid approachesto model efficiently multiple edges, whereas other parts of an image can be modeledby other approaches.
8.2 Future Directions
The introduction of multismoothlets seems to summarize the theory of geometricalmultiresolution adaptive methods based on a quadtree. In fact, it opens new possi-bilities of their use. A few open problems are presented in this section.
8.2.1 Fast Optimal Multismoothlet Transform
The multismoothlet transform described in this book is rather fast but does not leadto optimal approximation. The fast wedgelet transform does not lead to the optimalresult too, but there is a method that allows to lengthen the computation time toobtain a better result. In fact, it works very well. In the same way, the multismoothlettransform is defined. The open problem is, however, to define the multismoothlettransform that will be fast and optimal.
8.2.2 Texture Generation
Multismoothlets are characterized by presence of many curvilinear edges within agiven support. So, the dictionary of multismoothlets can be enormous. Let us notethat many multismoothlets give very regular patterns after visualization. In otherwords, they define useful textures. A few examples of such multismoothlets arepresented in Fig. 8.1. Many multismoothlets can be thus used like textures that canbe implemented in future image processing software (in fact, a few of them havebeen used already).
8.2 Future Directions 99
Fig. 8.1 Examples of textures generated by multismoothlets
8.2.3 Image Compressor Based on Multismoothlets
Todays’ image compressors are constructed based on the fact that an image is char-acterized by a wide morphological diversity [2]. It means that there are differentkinds of areas within an image: smooth areas, edges, and textures. An intelligentimage compressor can differentiate these areas and code them in different ways. Theimage compressor based on smoothlets (presented in Sect. 5.2.1) can be thus furtherimproved. It could code (multi)edges by (multi)smoothlets and textures by otherdifferent methods.
8.2.4 Hybrid Image Denoising Method
As stated in Chap. 6, image denoising based on geometrical multiresolution methodscan be overcome by the ones based, for instance, on sparse dictionary learning [3]or nonlocal means [4]. However, a hybrid denoising method may be defined that canremove noise in different ways from an image, depending on the considered region ofinterest, which means edges, textures and smooth areas. In such a case, a geometricalmultiresolution method can be used for edges denoising, whereas textures or smoothareas can be denoised by other different methods.
8.2.5 Object Recognition Based on Shift Invariant MultismoothletTransform
Quadtree-based transforms are known from the fact that they are not shift invari-ant. This completely excludes them from the area of object recognition, becausea small shift of an object leads to quite different coefficients. The shift invariant(multi)smoothlet transform can be used in object recognition. It will be probablytime-consuming but can lead to interesting results.
100 8 Summary
References
1. Lisowska, A.: Smoothlets—multiscale functions for adaptive representations of images. IEEETrans. Image Process. 20(7), 1777–1787 (2011)
2. Starck, J.-L., Murtagh, F., Fadili, J.M.: Sparse Image and Signal Processing:Wavelets, CurveletsMorphological Diversity. Cambridge University Press, Cambridge (2010)
3. Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learneddictionaries. IEEE Trans. Image Process. 15(12), 3736–3745 (2006)
4. Buades, A., Coll, B., Morel, J.M.: A review of image denoising algorithms, with a new one.Multiscale Model. Simul. 4(2), 490–530 (2005)
Appendix A
A.1 Benchmark Images
In Fig. A.1 all tested images are presented, namely: “Balloons”, “Bird”, “Chromo-some”, “Monarch”, “Objects”, “Peppers”.
A. Lisowska, Geometrical Multiresolution Adaptive Transforms, 101Studies in Computational Intelligence 545, DOI: 10.1007/978-3-319-05011-9,© Springer International Publishing Switzerland 2014
102 Appendix A
Fig. A.1 Tested images: a Balloons, b Bird, c Chromosome, d Monarch, e Objects, f Peppers
Appendix B
B.1 The Bottom-Up Tree Pruning Algorithm
The bottom-up tree pruning algorithm used in this book is a CART-like algorithmapplied for a quadtree. Initially, Classification and Regression Trees (CART) algo-rithm was defined for binary decision trees by Breiman et al. The algorithm givesthe optimal solution of a Rate-Distortion minimization problem. To make easier tofollow this book, the algorithm is described in this section in details. In fact, it wasused in nearly all works related to geometrical multiresolution adaptive methods.
Consider image F : D → C and its approximation FS : D → C . Consider thena set D of all subdomains Di, j ⊂ D and dictionary S of functions Si, j,p : Di, j → Cfor i ∈ {0, . . . , 4 j − 1}, j ∈ {0, . . . , J }, p ∈ R
n , J, n ∈ N. Let Fi, j = F |Di, j fori ∈ {0, . . . , 4 j − 1}, j ∈ {0, . . . , J }, J ∈ N. Consider then a quadtree of heightJ + 1 related to that image. Each node of this quadtree is related to appropriatesubdomain Di, j from D. The full quadtree decomposition is the process, in whichfor each subdomain Di, j from D the best approximation in the Mean Square Errorsense of Fi, j by a function Si, j,p from dictionary S is found. As the result, the setsof coefficients p, defining the best functions over Di, j for i ∈ {0, . . . , 4 j − 1},j ∈ {0, . . . , J }, J ∈ N, are stored in the nodes of this quadtree. To perform thebottom-up tree pruning, the following cost has to be computed and stored in eachnode
Rλi, j = min
Si, j,p∈S{||Fi, j − Si, j,p||22 + λ2Ki, j }, (B.1)
where the minimum is taken within functions from a given dictionary, λ ∈ R is thepenalization factor and Ki, j is the rate discussed further in this section.
Let us define image partition P as a homogenous partition of domain D. It meansthat it consists of subdomains Di, j from D that are nonoverlapping and cover thewhole domain D. Let us define Q P as the set of all possible homogenous partitions ofD. Each homogenous partition is related to a quadtree image partition. The questionarises—how to chose the optimal quadtree image partition? That is, the one that gives
A. Lisowska, Geometrical Multiresolution Adaptive Transforms, 103Studies in Computational Intelligence 545, DOI: 10.1007/978-3-319-05011-9,© Springer International Publishing Switzerland 2014
104 Appendix B
the best approximation quality with the use of the possible smallest number of ap-proximating functions? In other words, the following Rate-Distortion minimizationproblem has to be solved
Rλ = minP∈Q P
{||F − FS||22 + λ2K }, (B.2)
where λ is the penalization factor and K denotes the number of functions or thenumber of bits, depending on the application, used in this approximation. In moredetails, in the case of image approximation, K is defined as the number of functions,whereas in the case of image compression more sophisticated model is considered,in which K denotes the number of bits used in approximation.
The answer to the posed question is given by the bottom-up tree pruning algorithm.This algorithm finds the optimal solution of the Rate-Distortion problem. Althoughthis is a well known algorithm, its pseudocode is presented in Algorithm B.1 to makethis book easier to follow.
Algorithm B.1 Bottom-Up Tree Pruning
Input: a full quadtree decomposition;Output: the optimal homogenous image partition;1. set all nodes of the quadtree as nonvisible;2. BottomUp(RootOfTheQuadtree)3. if (size of node > 1x1 pixel)4. BottomUp(UpperLeftNode);5. BottomUp(UpperRightNode);6. BottomUp(LowerLeftNode);7. BottomUp(LowerRightNode);8. if (cost of node > sum of its four
children costs)9. cost of node = sum of its four
children costs;10. else11. set node as visible;
This is a recursive implementation of the algorithm. It works in the way that,starting from the bottom of the tree, the cost of a given node, computed according toformula (B.1), is compared to the sum of costs of its four children. If the cost is largerthan the children’s cost it is updated as the children’s cost else the node is marked asvisible and the subtree lying below this node is pruned. After the tree pruning, all thevisible nodes (all leaves of the quadtree) can be easy visualized because they form ahomogenous partition.
Appendix C
C.1 Pseudocode for Smoothlet Visualization
To visualize a smoothlet a distance function has to be applied. This function is usedto obtain a smooth transition between two colors of the smoothlet. This transition isobtained by brightening or darkening of these two colors, depending on the distancefrom the edge. Then, depending on the kind of the edge—linear or curvilinear one(ellipsoidal in this case)—different distance functions have to be used. For a linearbeamlet the distance function is based on the equation of a line in two-point form,going through points (x1, y1) and (x2, y2),
wl(x, y) = (x − x1)(y1 − y2) − (y − y1)(x1 − x2), (C.1)
and, for instance, for an ellipsoidal beamlet the distance function is based on theequation of an ellipse with center point (p, q) and major and minor axes a and d,respectively,
we(x, y) = ±d
√1 −
( x − p
a
)2 − (y − q). (C.2)
Let us consider a sample ellipsoidal edge, as presented in Fig. C.1. Suppose thatthe following data of a smoothlet are known: s = (s.x, s.y)—the start point of thebeamlet, e = (e.x, e.y)—the end point of the beamlet, d—the minor radius of theellipse, color1 and color2—the colors of the smoothlet. Then, the following datacan be easily computed: a—the major radius of the ellipse, and c = (c.x, c.y)—thecenter point of the ellipse. Having all these data, one can compute the distance ofa considered pixel to a given edge. Depending on the distance, the pixel’s color iscomputed. Let us note that real computation is based on a little bit different useof parameter r than in the mathematical model from Sect. 2.3. In the former caseparameter r reflects the half size of a blur, whereas in the latter case r reflects theexact size of a blur (see Fig. C.1 for details, theoretical r is denoted as T r ). Such an
A. Lisowska, Geometrical Multiresolution Adaptive Transforms, 105Studies in Computational Intelligence 545, DOI: 10.1007/978-3-319-05011-9,© Springer International Publishing Switzerland 2014
106 Appendix C
Fig. C.1 A sample ellipsoidaledge
improved approach makes the computations easier. Since ellipsoidal beamlets wereused in the experiments reported in this book, they are described here in details. Butthe smoothlet may be defined based on any kind of function. Its visualization is alsovery simple.
The pseudocode of the smoothlet visualization algorithm is presented in Algo-rithm C.1. As the input all coefficients that are used for visualization of a smoothletare taken. As the result one obtains a color of a given pixel. If r = 0 the standardprocedure of obtaining color is used, the same as for a wedgelet (lines 1–2). For asmoothlet, two kinds of distance functions are used, depending on the curvature ofthe beamlet (lines 4–17). Let us note that in line 17 one should use plus or minussign, depending on the value of y and a convexity of the beamlet. Having the distancecomputed, one can determine a color of a given pixel (lines 18–23). The center coloris the color exactly between the two colors of the wedgelet, the maximal color is thebrightest one. Once more, in line 23 the plus or minus sign has to be used, dependingon the fact whether one has to brighten or darken the base color. It depends on thefact which color is maximal.
Algorithm C.1 Smoothlet Visualization
Input: s, e, d, color1, color2, size, (x, y);Output: color of pixel (x, y);1. if (r==0)2. pixelColor=appropriate wedgelet color;3. else4. if (d==0)5. w=(x-s.x)*(s.y-e.y)-(y-s.y)*(s.x-e.x);6. dist=abs(w/(size-1));7. else8. a=0.5*sqrt(sqr(e.x-s.x)+sqr(e.y-s.y));9. c.x=(s.x+e.x)*0.5;10. c.y=(s.y+e.y)*0.5;11. sin=(e.y-s.y)/(2*a);12. cos=(e.x-s.x)/(2*a);13. tX=x-c.x; //translation14. tY=y-c.y; //translation
Appendix C 107
15. rX=tX*cos-tY*sin; //rotation16. rY=tY*cos+tX*sin; //rotation17. dist=abs((+/-)d*sqrt(1-sqr(rX/a))-rY);18. if (dist>abs(r))19. pixelColor=appropriate wedgelet color;20. else21. centerColor=0.5*(color1+color2);22. maxColor=max{color1,color2};23. pixelColor=centerColor+
(+/-)dist*(maxColor-centerColor)/r;
Let us note, that the presented algorithm is simplified. Indeed, in lines 17 and 23different cases have to be considered to determine the use of plus or minus signs. But,because they are easy in implementation, they were omitted to keep the algorithmpresentation clear.
Additionally, this algorithm can be adapted to any class of functions in an easyway. Indeed, let us consider curvilinear beamlet b expressed by equation y = b(x)(it can be e.g. paraboidal, polynomial or trigonometrical beamlet). Then, the distancefunction is defined as
wb(x, y) = b(x) − y. (C.3)
To visualize the smoothlet defined by this beamlet, line 17 of Algorithm C.1 has tobe exchanged with the following instruction
17. dist=abs(b(rX,rY)-rY);
Additionally, lines 8–12 may be updated accordingly, if needed.
![FACTORING WAVELET TRANSFORMS INTO LIFTING STEPSjskang/files/factor.pdf · represent general functions [13, 18, 22, 34, 21]. In the mid eighties the introduction of multiresolution](https://img.dokumen.tips/doc/110x75/5f7eb90eaaaa8966a861c7b5/factoring-wavelet-transforms-into-lifting-steps-jskangfiles-represent-general.jpg)
