17
Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers. Manufactured in The Netherlands. Geometry and Color in Natural Images VICENT CASELLES Department of Technology, University Pompeu Fabra, La Rambla, 30-32, 08002 Barcelona, Spain [email protected] BARTOMEU COLL Department of Mathematics and Informatics, University of Illes Balears, 07071 Palma de Mallorca, Spain [email protected] JEAN-MICHEL MOREL Ecole Normale Sup´ erieure de Cachan, 61 Avenue du Pdt Wilson, 94235 Cachan Cedex, France [email protected] Abstract. Most image analysis algorithms are defined for the grey level channel, particularly when geometric information is looked for in the digital image. We propose an experimental procedure in order to decide whether this attitude is sound or not. We test the hypothesis that the essential geometric contents of an image is contained in its level lines. The set of all level lines, or topographic map, is a complete contrast invariant image description: it yields a line structure by far more complete than any edge description, since we can fully reconstruct the image from it, up to a local contrast change. We then design an algorithm constraining the color channels of a given image to have the same geometry (i.e. the same level lines) as the grey level. If the assumption that the essential geometrical information is contained in the grey level is sound, then this algorithm should not alter the colors of the image or its visual aspect. We display several experiments confirming this hypothesis. Conversely, we also show the effect of imposing the color of an image to the topographic map of another one: it results, in a striking way, in the dominance of grey level and the fading of a color deprived of its geometry. We finally give a mathematical proof that the algorithmic procedure is intrinsic, i.e. does not depend asymptotically upon the quantization mesh used for the topographic map. We also prove its contrast invariance. Keywords: color images, level sets, morphological filtering, luminance constraint 1. Introduction: Color from Different Angles In this paper, we shall first review briefly some of the main attitudes adopted towards color in art and science (Section 1) We then focus on image analysis algorithms and define some of the needed terminology (Section 2), in particular the topographic map: we support therein the view that the geometrical information of a grey level image is fully contained in the set of its level lines. Section 3 is devoted to the description of an To whom correspondence should be addressed. experimental procedure to check whether the color in- formation contents can be considered as a mere non geometrical complement to the geometry given by the topographic map or not. In continuation, a description of several experiments on color digital images is per- formed. In Section 4, which is mainly mathematical, we check from several point of views the soundness of the proposed algorithm, in particular its independence of the quantization procedure, its consistency with the assumption that the grey level image has bounded variation and the contrast invariance of the proposed algorithm.

Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

Journal of Mathematical Imaging and Vision 16: 89–105, 2002c© 2002 Kluwer Academic Publishers. Manufactured in The Netherlands.

Geometry and Color in Natural Images

VICENT CASELLESDepartment of Technology, University Pompeu Fabra, La Rambla, 30-32, 08002 Barcelona, Spain

[email protected]

BARTOMEU COLL∗

Department of Mathematics and Informatics, University of Illes Balears, 07071 Palma de Mallorca, [email protected]

JEAN-MICHEL MORELEcole Normale Superieure de Cachan, 61 Avenue du Pdt Wilson, 94235 Cachan Cedex, France

[email protected]

Abstract. Most image analysis algorithms are defined for the grey level channel, particularly when geometricinformation is looked for in the digital image. We propose an experimental procedure in order to decide whetherthis attitude is sound or not. We test the hypothesis that the essential geometric contents of an image is containedin its level lines. The set of all level lines, or topographic map, is a complete contrast invariant image description:it yields a line structure by far more complete than any edge description, since we can fully reconstruct the imagefrom it, up to a local contrast change. We then design an algorithm constraining the color channels of a givenimage to have the same geometry (i.e. the same level lines) as the grey level. If the assumption that the essentialgeometrical information is contained in the grey level is sound, then this algorithm should not alter the colors of theimage or its visual aspect. We display several experiments confirming this hypothesis. Conversely, we also showthe effect of imposing the color of an image to the topographic map of another one: it results, in a striking way, in thedominance of grey level and the fading of a color deprived of its geometry. We finally give a mathematical proofthat the algorithmic procedure is intrinsic, i.e. does not depend asymptotically upon the quantization mesh used forthe topographic map. We also prove its contrast invariance.

Keywords: color images, level sets, morphological filtering, luminance constraint

1. Introduction: Color from Different Angles

In this paper, we shall first review briefly some of themain attitudes adopted towards color in art and science(Section 1) We then focus on image analysis algorithmsand define some of the needed terminology (Section 2),in particular the topographic map: we support thereinthe view that the geometrical information of a greylevel image is fully contained in the set of its levellines. Section 3 is devoted to the description of an

∗To whom correspondence should be addressed.

experimental procedure to check whether the color in-formation contents can be considered as a mere nongeometrical complement to the geometry given by thetopographic map or not. In continuation, a descriptionof several experiments on color digital images is per-formed. In Section 4, which is mainly mathematical,we check from several point of views the soundness ofthe proposed algorithm, in particular its independenceof the quantization procedure, its consistency with theassumption that the grey level image has boundedvariation and the contrast invariance of the proposedalgorithm.

Page 2: Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

90 Caselles, Coll and Morel

1.1. Painting, Linguistics

The color-geometry debate in the theory of paintinghas never been closed, each school of painters mak-ing a manifesto of its preference. Delacroix claimed“L’ennemi de toute peinture est le gris!” [29]. Toimpressionnists, “la couleur est tout”(Monet), whilethe role of contours and geometry is prominent inthe Renaissance painting, but also for the surrealisticschool or for cubists : this last school is mostly con-cerned with the deconstruction of perspective and shape[23]. To Kandinsky, founder and theoretician of ab-stract painting, color and drawing are treated in a totallyseparate way, color being associated with emotions andspirituality, but the building up of a painting being es-sentially a question of drawing and the (abstract) shapecontent relying on drawing, that is on black strokes[14]. In this discussion, we do not forget that, whilean accurate definition of color is given in quantummechanics by photon wavelength, the human or ani-mal perception of it is extremely blurry and variable.Red, green and blue color captors on the retina have astrong and variable overlap and give a very poor wave-length resolution, not at all comparable to our auditivefrequency receptivity to sounds. Different civilisationshave even quite different color systems. For instance,the linguist Louis Hjelmslev, [12], notes that the se-mantic division of colors is simply different in French(or English) and Welsh and there is no easy translation,four colors in French being covered by three differentones in Welsh.

1.2. Perception Theory

Perception theory does not support the absolutenessof color information. In his monumental work on vi-sual perception, Wolfgang Metzger [20], dedicates onlyone tenth of his treatise (2 chapters over 19) to theperception of colors. Those chapters are mainly con-cerned with the impossibility to define absolute colorsystems, the variability of the definition of color underdifferent lighting conditions, and the consequent visualillusions.1 Noticeably, in this treatise, 100 percent ofthe experiments not directly concerned with color aremade with black and white drawings and pictures. Infact, the gestaltists not only question the existence of“color information”, but go as far as to deny any phys-ical reality to any grey level scale: the grey levels arenot measurable physical quantities in the same levelas, say, temperature, pressure or velocity. A main rea-

son invoked is that most images are generated under nocontrol or even knowledge of the illumination condi-tions or of the physical reflectance of objects. This mayalso explain the failure of several interesting attempts touse shape from shading information in shape analysis[13]. The contribution of black and white photographsand movies has been to demonstrate that the essentialshape content of images can be encoded in a gray scaleand this attitude seems to be corroborated by the imageanalysis research. Indeed, and although we are not ableto deliver faithful statistics, we can state that an over-whelming majority of image analysis algorithms arebeing designed, tested and validated on grey level im-ages. Satellite multispectral images (SPOT images forinstance) attribute a double resolution to panchromatic(i.e. grey level images) and a simple one to color chan-nels: somehow, color is assumed to only give a seman-tic information, (like e.g. presence of vegetation) andnot a geometric one: this engineering decision meetsKandinsky’s claim!

1.3. Image Analysis Algorithms

Let us now consider the practice of image analy-sis. When an algorithm has to be applied to colorimages, it is generally first designed and tested ongrey level images and then applied independently toeach channel. It is experimentally difficult to demon-strate the improvements due to a joint use of thethree color channels instead of this independent pro-cessing. Antonin Chambolle [8] tried several strate-gies to generalize mean curvature algorithms to colorimages. His overall conclusion (private communica-tion) was that no perceptible improvement was madeby defining a color gradient: diffusing each channelindependently led to essentially equal results, froma perception viewpoint. A more recent, and equiva-lent, attempt to define a “color gradient” in order toperform anisotropic diffusion in a more sophisticatedway than just diffusing each channel independently isgiven by [26]. Following these ideas, let � : IR2 →IRm be a multi-valued image, where for color im-ages we have m = 3 components. The first fundamen-tal form associated to � is given by the quadratic formd�2 = ∑2

i=1

∑2j=1

∂�∂ui

∂�∂u j

dui du j . Using the standardnotation of Riemannian geometry, gi j := ∂�

∂ui· ∂�

∂u j, then

the extrema of the quadratic form defined above areobtained in the directions of the eigenvectors of themetric tensor [gi j ] and the values attained there are thecorresponding eigenvalues called λ+, λ−. Therefore,

Page 3: Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

Geometry and Color in Natural Images 91

a first approximation to the gradient for color imagesshould be a function f = f (λ+, λ−) The authors donot provide a comparison with an algorithm makingan independent diffusion on each channel, however, sothat their study does not contradict the above mentionedconclusion. Pietro Perona [24] performed experimentson color images where he successfully applied sep-arately to each channel the Perona-Malik anisotropicdiffusion [25] with no visible colour alteration.

In this work, we only focus on image analysis andshape analysis, where the mainstream algorithms arealways designed on grey level, or even binarized images(see [3, 15, 16, 18, 27]). We propose a numerical exper-imental procedure to check that color information doesnot contribute to our geometric understanding of natu-ral images. This statement has to be made precise. Wehave first considered it a common sense consequenceof the arbitrariness of lighting conditions and total in-accuracy of our color wavelength perception, whichmakes the definition of color channels very context-dependent. This explains why the literature on coloris mainly devoted to a “restoration” of universal colorcharacteristics such as saturation, hue and luminance.Now, of these three characteristics, only luminance, de-fined as a sum of color channels has a (relative) physicalmeaning. Indeed, luminance, or “grey level” is definedas a photon count over a period of time (exposure time).Thus, we can relate a linear combination of R, G, B,channels to this photon count. Now, we mentionnedthat from the perception theory viewpoint, (see e.g.Wertheimer [31]), even this grey level information issubject to so much variability due to unknown illumina-tion and reflectance condition that we cannot considerit as a physical information.

2. Mathematical Morphologyand the Topographic Map

This explains why Matheron and Serra [28] developeda theory of image analysis, focused on grey level andwhere, for most operators of the so called “flat mor-phology”, contrast invariance is the rule. Flat morpho-logical operators (e.g. erosions, dilations, openings,closings, connected operators, etc.) commute with con-trast changes and therefore process independently thelevel sets of a grey level image.

In the following, let us denote by u(x) the grey levelof an image u at point x . In digital images, the onlyaccessible information is a quantized and sampled ver-sion of u, u(i, j), where (i, j) is a set of discrete points

(in general on a grid) and u(i, j) belongs in fact to a dis-crete set of values, 0, 1, . . . , 255 in many cases. Since,by Shannon theory, we can assume that u(x) is recov-erable at any point from the samples u(i, j), we canin a first approximation assume that the image u(x) isknown, up to the quantization noise. Now, since the il-lumination and reflectance conditions are arbitrary, wecould as well have observed an image g(u(x)) where gis any increasing contrast change. Thus, what we reallyknow are in fact the level sets

Xλu = {x, u(x) ≥ λ},

where we somehow forget about the actual value of λ.According to the Mathematical Morphology doctrine,the reliable information in the image is contained in thelevel sets, independently of their actual levels. Thus,we are led to consider that the geometric information,the shape information, is contained in those level sets.This is how we define the geometry of the image. Inthis paragraph, we are simply summarizing some ar-guments contained explicitly or implicitly in the Math-ematical Morphology theory, which were further de-veloped in [7]. We can further describe the level setsby their boundaries, ∂ Xλu, which are, under suitablevery general assumptions, Jordan curves. Jordan curvesare continuous maps from the circle into the plane IR2

without crossing points. To take an instance which wewill invoke further on, if we assume that the image uhas bounded variation, then for almost all levels of u,Xλu is a set with bounded perimeter and its boundary acountable family of Jordan curves with finite length [2].In the mentioned work, it is demonstrated that the levelline structure whose existence was assumed in [6] in-deed is mathematically consistent if the image belongsto the space BV of functions with bounded variation. Itis also proved that the connected components of levelsets of the image give a locally contrast invariant de-scription.

In the digital framework, the assumption that thelevel lines are Jordan curves is straightforward if weadopt the naive but useful view that u is constant oneach pixel. Then level lines are concatenations of ver-tical and horizontal segments and this is how we shallvisualize them in practice. As explained in [7], levellines have a series of structure properties which makethem most suitable as building blocks for image anal-ysis algorithms. We call the set of all level lines of animage topographic map. The topographic map is invari-ant under a wide class of local contrast changes ([7]),

Page 4: Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

92 Caselles, Coll and Morel

so that it is a useful tool for comparing images of thesame object with different illuminations. This applica-tion was developed in [4] who proposed an algorithmto create from two images a third whose topographicmap is, roughly speaking, an intersection of the twodifferent input images. Level lines never meet, so thatthey build an inclusion tree. A data structure giving fastaccess to each one of them is therefore possible and wasdeveloped in [22], who proved that the inclusion treesof upper and lower level sets merge. We can conceivethe topographic map as a tool giving a complete de-scription of the global geometry for grey level images.A further application to shape recognition and registra-tion is developed in [21], who proposes a topographicmap based contrast invariant registration algorithm and[17], who propose a registration algorithm based on therecognition of pieces of level lines. Such an algorithm isnot only contrast invariant, but occlusion stable. Severalmorphological filters are easy to formalize and imple-ment in the topographic map formalism. For instance,the Vincent-Serra connected operators [19, 30]. Suchfilters, as well as local quantization algorithms are eas-ily defined [7]. Our overall assumption is first that all ofthe reliable geometric information is contained in thetopographic map and second that this level line struc-ture is under many aspects (completeness, inclusionstructure) more stable than, say, the edges or regionsobtained by edge detection or segmentation. In partic-ular, the advantage of level lines over edges is strikingfrom the reconstruction viewpoint: we can reconstructexactly the original image from its topographic map,while any edge structure implies a loss of informationand many artifacts. It is therefore appealing to extendthe topographic map structure to color images and thisis the aim we shall try to attain here. Now, we will attainit in the most trivial way. We intend to show by an ex-perimental and mathematically founded procedure thatthe geometric structure of a color image is essentiallycontained in the topographic map of its grey level. Inother terms, we propose the topographic map of colorimage to be simply the topographic map of a linearcombination of its three channels. If that is true, thenwe can claim that tasks like shape recognition, and ingeneral, anything related to the image geometry, shouldbe performed on the subjacent grey level image. As weshall see in the next section where we review from sev-eral points of view the attitude of scientists and artiststoward color, this claim is nothing new and implicit inthe way we usually proceed in image processing. Ourwish, is, however, to make it explicit and get rid of this

bad consciousness we feel by ever working and think-ing and teaching with grey level images. And “aboutcolor images”, we propose to prove that we do not needthem for geometric analysis.

2.1. Definition of the Geometry

We are led to define the geometry of a digital imageas defined by its topographic map. What about color?Our aim here is to prove that we can consider the colorinformation as a subsidiary information, which maywell be added to the geometric information but doesnot yield much more to it and never contradicts it. Hereis how we proceed:

In general words, we shall define an experimentalprocedure to prove that Replacing the colors in an im-age by their conditional expectation with respect to thegrey level does not alter the color image.

Of course, this statement needs some mathematicalformalization before being translated into an algorithm.We prefer, for a sake of clarity, to start with a descrip-tion of the algorithm (in the next section). Then, weshall display some experiments since, as far as color isconcerned, visual inspection here is the ultimate deci-sion tool used to check whether a colored image hasbeen altered or not. Section 4 is devoted to the completemathematical formalization, i.e. a proof that the definedprocedure converges and in no way depends upon thechoice of special quantization parameters. Althoughthis was not our aim here, let it be mentionned thatextensions of this work for application to the compres-sion of multi-channel images are in course [9]. Theidea is to compress the grey level channel by encod-ing its topographic map. Then, instead of encoding thecolor channels separately, an encoding of the condi-tional expectation with respect to the topographic mapis computed. This definition will become clear in thenext two sections.

3. Algorithm and Experiments

3.1. The Algorithm: Morphological Filteringof Color Images

The algorithm we present is based on the idea that thecolor component of the image does not give contra-dictory geometric information to the grey level and,in any case, is complementary. Following the ideas ofprevious works (see [6, 7]), we describe the geometric

Page 5: Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

Geometry and Color in Natural Images 93

contents of the image by its topographic map, a con-trast invariant complete representation of the image.We discuss in Section 4 an extension to color channelsof this contrast invariance property. The algorithm weshall define now imposes to the chromatic components,saturation and hue, to have the same geometry, or thesame topographic map, as the luminance component.We shall experimentally check that by doing so, we donot create new colors and the overall color aspect ofthe image does not change with this operation. In somesense, and although this is not our aim, the algorithm isan alternative to color anisotropic diffusions algorithmsalready mentioned [5, 8, 26].

To define the algorithm, we take a partition P ={a0 = a < a1 < · · · < aN = b} of the grey levelrange of the luminance component. In practice a0 = 0and aN = 255 and this partition is defined by a grey levelquantization step. The resulting set of level lines is arestriction to the chosen levels of the topographic map.We then consider the connected components of the setcomplementary to the level lines. In the discrete case,these connected components are sets of finite perime-ter given by a finite number of pixels, and constitute apartition of the image. In the continuous case, we mustimpose some restriction about the space of functionswe take (see Section 4).

The algorithm we propose is the following. LetU : → IR3, U = (U1, U2, U3), where the three chan-nels U1, U2, U3 are the intensity of red, green and blue.

(i) From these channels, we compute the L , S andH values of the color signal, i.e., the luminance,saturation and hue, defined by

L(U )(x) = 〈U (x), σ 〉 = red + green + blue√3

with σ = 1√3

1

1

1

,

S(U )(x) = 1 −√

3mini=1,2,3Ui (x)

L(x), (1)

or the perceptually less correct but simpler,

S(U )(x) = ‖U (x) − 〈U (x), σ 〉σ‖= ‖σ×U (x)‖

H(U )(x) = angle

σ×U (x), σ×

1

0

0

.

(ii) Compute the topographic map associated to Lwith a fixed quantization step. In other terms, wecompute all level lines with levels multiple of afixed factor, for instance 10. This quantization pro-cess yields the topographic representation of a par-tial, coherent view of the image structure (see [7]for more details about visualization of the topo-graphic map). This computation is obvious: wefirst compute the level sets, as union of pixels, forlevels λ which are multiples of the entire quanti-zation step. We then simply compute their bound-ary as concatenations of vertical and horizontallines.

(iii) In each connected component A of the topogra-phic map of L , we take the average value of S, H .More precisely, let {x1, . . . , xn} be the pixels ofA. We compute the value

vA =∑n

i=1 v(xi )

n,

where v ∈ {S, H} and vA will be the new con-stant value in the connected component A, for thecomponents S and H . In other terms, we trans-form the channels S, H into S, H , where S, Hhave a constant value, in fact the average value, in-side each connected component of the topographicmap of L , the luminance component. As a con-sequence we obtain a new representation of theimage, piecewise constant on the connected com-ponents of the topographic map and we thereforeconstrain the color channels S and H to have thesame topographic map as the grey level.

(iv) Finally, in order to visualize the color image, wecompute (U1, U2, U3), defined respectively as thenew red, green and blue channels of the image byperforming the inverse color coordinates changeon (L , S, H).

Remark 1. Note that in order to perform a visualiza-tion, each channel of the (U1, U2, U3)-space must havea range of values in a fixed interval [a, b]. In practice,in the discrete case, a = 0 and b = 255. After applyingthe preceding algorithm, this range can be altered andwe can’t recover the same range of values for the finalcomponents. We therefore threshold or normalize thesecomponents so that their final range be [0, 255].

Remark 2. A slight change of this algorithm can beobtained if we take, instead of the average on thechromatic components of the image in each connectedcomponents, the average of these components weighted

Page 6: Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

94 Caselles, Coll and Morel

by the modulus of the gradient of the luminance compo-nent (see the remark after Theorem 3). This means thatwe replace the step (iii) of the algorithm given above by(iii)’. That is, (iii)’. In each connected component A ofthe topographic map of L let {x1, . . . , xn} be the pixelsof this region. Compute the valuevA =

∑ni=1 v(xi )|∇L(xi )|∑n

i=1 |∇L(xi )| ,

where v ∈ {S, H} and vA will be the new constant valuein the connected component A, for the components Sand H . We shall explain in Section 4 the measure ge-ometric meaning of this variant.

3.2. Experiments and Discussion

In Experiment 1, Fig. 1(a)–(c) are the original images,which we shall call the “tree” image, the “peppers”image and the “baboon” image, respectively. In the L-component of the (L , S, H) color space, we take thetopographic map of the level lines of Fig. 1(a) for alllevels which are multiples of p = 5. We can see thisresult in Fig. 1(d), where large enough regions are to benoticed, on which the grey level is therefore constant.We then apply the algorithm, by taking the average ofS, H components on the connected components of thetopographic map, i.e. the flat regions of the grey levelimage quantized with a mesh equal to 5. Equivalently,this results in averaging the color of “tree” on eachwhite region of Fig. 1(d). We then obtain Fig. 1(e). InFig. 1(f), we display, above, a detail of the original im-age “tree” and below the same detail after the color av-eraging process has been applied. This is the only partof the image where the algorithm brings some geomet-ric alteration. Indeed, on this part, the grey level con-trast between the sea’s blue and the tree’s light brownvanishes and a tiny part of the tree becomes blue. Thisdetail is hardly perceptible in the full size image be-cause the color coherence is preserved: no new coloris created and, as a surprising matter of fact, we neverobserved the creation of new colors by the averagingprocess. This fact can be checked by way of the asso-ciated histogram. In fact, Fig. 2(a) is the histogram ofthe original image Fig. 1(a) and Fig. 2(b) correspondsto the histogram of the processed image Fig. 1(e).

In Experiment 2, Fig. 3(a) is the topographic map of“peppers” for levels which are multiples of 10. By a wayof practical rule, quantizations of an image by a mesh of5 are seldom visually noticeable; in that case, we can goup to a mesh of 10 without visible alteration. Figure 3(b)displays the outcome of averaging the colors on the flatzones of Fig. 1(b), which is equivalent to averaging

the colors on the white regions of Fig. 3(a). If colordoes not bring any relevant geometrical informationcomplementary to the grey level, then it is to be ex-pected that Fig. 3(b) will not have lost geometric de-tails with respect to Fig. 1(b). This is the case and, inall natural images where we applied the procedure, theoutcome was the same: the conditional expectation ofcolor with respect to grey level yields, perceptually,the same image as the original. We are aware that onecan numerically create color images where the differ-ent colors have exactly the same grey level, so that thegrey level image has no level lines at all ! If we ap-plied the above procedure to such images, we would ofcourse see a strong difference between the processedone, which would become uniform, and the original.Now, the generic situation for natural images is that“color contrast always implies (some) grey level con-trast”. This empiric law is easily checked on naturalimages. We have seen in Experiment 1 the only casewhere we noticed that it was slightly violated, two dif-ferent colors happening to have (locally) grey levelswhich differed by less than 5 grey levels. In Fig. 3(c),we explore further the dominance of geometry on colorby giving to “tree” the colors of “peppers”. We obtainedthat image by averaging the colors of peppers on thewhite regions of “tree” and then reconstructing an im-age whose grey level was that of “tree” and the colorsthose of “peppers”.

We further explore the striking results of mixing thecolor of an image with the grey level of another one.Our conclusion will be in all cases that, like in Fig. 3(c),the dominance of grey level above color, as far as ge-ometric information is concerned, is close to absolute.In Fig. 3(d), we do the converse procedure to the one inFig. 3(c): we average the colors of tree conditionally tothe topographic map of peppers. The amazing result isa new pepper image, where no geometric content from“tree” is anymore visible. Of course, those two exper-iments are not totally fair, since we force the color ofthe second image to have the topographic map of thefirst one. Thus, in Fig. 3(e), we simply constructed animage having the grey level of “peppers” and the colorsof “tree”. Notice again the dominance of “peppers” andthe fading of the shapes of “tree”. In Fig. 1(c) we displayan original “baboon” image and in Fig. 3(f) the resultof imposing the colors of “tree” to “baboon”. Again,we mainly see “baboon”. As in Experiment 1 and in or-der to check the creation of new colors, Fig. 4(a) is thehistogram of the original image Fig. 1(b) and Fig. 4(b)is the histogram of Fig. 3(b).

Page 7: Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

Geometry and Color in Natural Images 95

Figure 1. Images of Experiment 1.

Page 8: Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

96 Caselles, Coll and Morel

Figure 2. Histograms of Experiment 1.

In all experiments, we used the first version of themean value algorithm and not the one described inRemark 2. Our experimental records show no visibledifference between both algorithms.

3.3. Conclusions

We summarize our conclusions in the following points:

(i) In Experiment 1, despite some geometric alter-ation of the image, the color coherence is pre-served and, experimentally, no new color arecreated by the averaging process.

(ii) We check the dominance of the geometry given bythe grey level above color component. This dom-inance, as far as geometric information is con-cerned, is close to absolute. Experimentally, if weaverage the colors of a given image to the topo-graphic map of the other one, we can recognizethe image which keeps the geometric informationfrom the topographic map. This is due to the factthat color does not bring any relevant geometricalinformation complementary to the grey level.

(iii) Experiment 2 is made to the effect of proving thatthe same amount of color gains a lot of efficiencywhen it coincides with the topographic map and

Page 9: Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

Geometry and Color in Natural Images 97

Figure 3. Images of Experiment 2.

Page 10: Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

98 Caselles, Coll and Morel

Figure 4. Histograms of Experiment 2.

loosed totally effect, up to the point that the imagelooks essentially grey or lightly water-coloured,when the color does not fit the topographic map.In fact, very few color boundaries of Fig. 1(a) arekept when we superpose the color of Fig. 1(a) ontothe grey level of Fig. 1(b). Actually, in order to“demonstrate” the claimed prominence of color,the pointillists (Seurat) and late impressionnists(Pissaro, Monet) were led to break all contoursand to divide the paintings surface into color spots.

(iv) Possible applications in color image superreso-lution. Although the presented algorithm has nomore application than a visual inspection and acheck of the independence of geometry and color,we can deduce from this inquiry some possible

new ways to process color images. First of all, wecan consider that the compression of multichan-nels images should be led back to the compres-sion of the topographic map given by the panchro-matic (grey level) image. This kind of attempt isin course [9]. Also, one may ask whether colorimages, when given at a resolution smaller than thepanchromatic image (this is e.g. the case for SPOTimages) should not be deconvolved and led backto the grey level resolution. This seems possible,if this deconvolution is made under the (strong)constraint that the topographic map of each colorcoincides with the grey level topographic map. Ina forthcoming paper [1], a color superresolutionalgorithm has been designed which follows the

Page 11: Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

Geometry and Color in Natural Images 99

ideas developed in the paper, with some succes.It turns out that SOPT satellite images have threecolor channels and one panchromatic (grey level)channel. This last channel has twice the resolu-tion of the color channels. The question was re-cently raised by Bernard Rouge’s team at CNESof whether one could take advantage of the bet-ter resolution of the grey level channel in orderto increase by numerical means the resolution ofthe other channels. This is the inverse problem ofwhat we considered here. In fact, we use the ideathat the color channel can be deconvolved with theadditional constraint that its topographic map fitsto the topographic map of the grey level.

Experiment 1. Figure 1(a)–(c) are the original im-ages. Figure 1(d) is the topographic map of the L-component of the (L, S, H) color space of Fig. 1(a)for all level lines which are multiples of 5. The appli-cation of the algorithm by taking the average of S, Hcomponents on the connected components of Fig. 1(d)gives us Fig. 1(e). In Fig. 1(f) we display, above, a detailof the original image “tree” and below the same detailafter the algorithm has been applied. Figure 2(a) is thehistogram of Fig. 1(a) and Fig. 2(b) is the histogram ofFig. 1(e). We can compare the histogram between theoriginal image and the resultant image after applyingthe algorithm and as a consequence, experimentally nonew color are created by the averaging process.

Experiment 2. Figure 3(a) is the topographic mapof the L-component of the (L, S, H) color space ofFig. 1(b) for all level lines which are multiples of 10.The application of the algorithm to Fig. 1(b) over thewhite regions of Fig. 3(a) give us Fig. 3(b). Figure 3(c)is obtained by averaging the colors of Fig. 1(b) on thewhite regions of Fig. 1(a). In Fig. 3(d), we average thecolors of tree conditionally to the topographic map ofFig. 1(b). Figure 3(e) shows the grey level of Fig. 1(b)and the colors of Fig. 1(a). Figure 3(f) is the result ofimposing the colors of Fig. 1(a) to Fig. 1(c). Figure 4(a)is the histogram of Fig. 1(b) and Fig. 4(b) is the his-togram of Fig. 3(b). As in the previous experiment, wecan compare the histogram of the original image withthe one of the processed image.

4. Formalization of the Algorithm

Let (Y,B, µ) be a measure space and F ⊂ B be a fam-ily of measurable subsets of Y . A connected component

analysis of (Y,F) is a map which assigns to each setX ∈ F a family of subsets Cn(X) ∈ B of X such that

(i) µ(Cn(X)) > 0, µ(Cn(X) ∩ Cm(X)) = 0 for alln, m ∈ IN , n �= m and

µ

( ∞⋃n=1

Cn(X)

)= meas(X),

(ii) If X ⊆ X ′ ⊆ E , then each Cn(X) is contained insome Cm(X ′), n, m ∈ IN .

Notice that this definition asks more than the usualdefinition [28], since we request that sets ofF be essen-tially decomposable into connected components withpositive measure. If, e.g., F is the set of open sets ofIR2, then the usual definition of connectedness applies,i.e. satisfies requirements i) and ii).

Let (Y,B, µ) and u : Y → [a, b] be a measurablefunction. Let F(u) be a family of sets contained in theσ -algebra generated by the level sets {u−1([α, β)) :α < β}. Assume that a connected component analysisis given on (,F(u)). In other words, F(u) is a fam-ily of subsets where the connected components can becomputed and satisfy i) and ii).

Let P = {a0 = a < a1 < · · · < aN = b} be a parti-tion of [a, b] such that [ai ≤ u < ai+1] ∈ F(u),i = 0, . . . , N − 1. Such partitions will be called ad-missible. Let us denote by CC(P) the set of con-nected components of the sections [ai ≤ u < ai+1],i = 0, 1, . . . , N − 1. Let v ∈ L1(Y,B, µ). For eachconnected component A ∈ CC(P) we define the aver-age value of v on A by

vA = 1

µ(A)

∫Av(x) dµ.

Then we define the function

E(v | u,P) =∑

A∈CC(P)

vAχA.

This function is nothing but the conditional expectationof v ∈ L1(Y,B, µ) with respect to the σ -algebra APof subsets of Y generated by CC(P).

In the next proposition we summarize some basicproperties of conditional expectations, [32], Ch. 9.

Proposition 1. Let (X,B, µ) be a measure space andlet A be a sub-σ -algebra of B. Let L1(X,B, µ) bethe space of measurable (with respect to B) functionswhich are Lebesgue integrable with respect to µ. Let

Page 12: Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

100 Caselles, Coll and Morel

L1(A) be the subspace of L1(X,B, µ) of functionswhich are measurable with respect to A. Then E(.|A)

is a bounded linear operator from L1(X,B, µ) ontoL1(A).

(i) If v ∈ L1(), 0 ≤ v ≤ M, then 0 ≤ E(v |A) ≤M. We have

E(1 |A) = 1.

(ii) If v ∈ L1(X,B, µ), z ∈ L1(A) then

E(zv |A) = zE(v |A). (2)

In particular, if v is measurable with respect to Athen E(v |A) = v.

(iii) If A′ is a sub-σ -algebra of A then

E(E(v |A) |A′) = E(v |A′). (3)

In particular,

E(E(v |A) |A) = E(v |A).

Lemma 1. Let P = {a0 = a < a1 < · · · < aN = b} bean admissible partition of [a, b]. Let P ′ = {b0 = a <

b1 < · · · < bM = b} be a refinement of P, i.e., a parti-tion of [a, b] such that each ai coincides with one of theb j , and such that each section [b j ≤ u < b j+1] ∈F(u),

j = 0, . . . , M − 1. ThenAP is a sub-σ -algebra ofAP ′ .

Proof: Let A ∈ CC(P). Then for some i ∈{0, . . . , N − 1}, A is a connected component of [ai ≤u < ai+1]. Let j, k be such that ai = b j , ai+1 = b j+k .According to our Axiom (i i), each connected compo-nent of [bl ≤ u < bl+1] is contained in a connectedcomponent of [ai ≤ u < ai+1], l ∈ { j, . . . , j + k − 1}.Then, by Axiom (i), each connected component of[bl ≤ u < bl+1], l ∈ { j, . . . , j + k − 1}, is either con-tained in A or disjoint to A (modulo a null set).Let (An)n be the set of connected components of[bl ≤ u < bl+1], l ∈ { j, . . . , j + k − 1} containedin A. We have that

A =∞⋃

n=1

An (mod a null set).

Indeed, if this was not true, then we would find byAxiom (i i) another connected component of [bl ≤ u <

bl+1], l ∈ { j, . . . , j + k − 1}, contained in A and wewould obtain a contradiction. ✷

Let L1(Y,AP , µ) be the space of functions whichare measurable with respect to the σ -algebra AP andµ-integrable. Proposition 1 can be translated as a state-ment about the operator E(.|u,P) = E(.|AP).

Proposition 2. Let P be an admissible partition of[a, b]. Then E(.|u,P) is a bounded linear operatorfrom L1(Y,B, µ) onto L1(Y,AP , µ) satisfying prop-erties (i) and (ii) of Proposition 1. If P ′ is admissibleand is a refinement of P then (iii′)

E(E(v | u,P ′) | u,P) = E(v | u,P). (4)

Proof: We have shown that AP is a sub-σ -algebra ofAP ′ . Let A ∈ CC(P) and An ∈ CC(P ′) be such thatA = ⋃∞

n=1 An (mod a null set). Then

vA = 1

µ(A)

∫Av(x) dx =

∞∑n=1

1

µ(A)

∫An

v(x) dx

=∞∑

n=1

µ(An)

µ(A)vAn ,

which is equivalent to (4).For any partition P = {a0 = a < a1 < · · · < aN = b}

of [a, b] we define

‖P‖ := supi∈{0,1,...,N−1}

ai+1 − ai .

Let Pn be a sequence of partitions of [a, b] such that

(a) Pn are admissible, i.e., the sections of u associatedto levels of Pn , are in F(u).

(b) Pn+1 is a refinement of Pn , n = 1, 2, . . . .

(c) ‖Pn‖ → 0 as n → ∞.

Let v ∈ L1(Y,B, µ). We define vn = E(v|u,Pn). Notethat the σ -algebras APn form a filtration, i.e., an in-creasing sequence of σ -algebras contained in B and,by Proposition 2, we have vn ∈ L1(Y,APn , µ) and

vn = E(vn+1 | u,Pn).

Thus, vn is a martingale relative to ({APn }n, µ) ([10],Ch. VII, Section 8). ✷

According to the martingale convergence theoremwe have

Theorem 1. Let A∞ be the σ -algebra generatedby the sequence of σ -algebras APn , n ≥ 1, i.e.,

Page 13: Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

Geometry and Color in Natural Images 101

the smallest σ -algebra containing all of them. Ifv ∈ L p(Y,B, µ), p ≥ 1, then vn converges in L1(Y,

A∞, µ) and a.e. to a function v∞ = E(v |A∞), whichmay be considered as the projection of v into thespace L p(Y,A∞, µ). If v ∈ L1(A∞), then v∞ = v. Inparticular, u∞ = u.

Proof: Bounded martingales in L p converge in L p

and a.e. if p > 1 and a.e. if p = 1 ([32]; [10], Ch. VII,p. 319). Martingales generated as conditional expecta-tions of a function v ∈ L1() with respect to a filtra-tion are equiintegrable and, thus, converge in L1 ([32];[10], Ch. VII, p. 319). Now, by Levy’s Upward The-orem v∞ = E(v |A∞) ([32], Ch. 14; [10], Ch. VII,p. 331). The final statement is a consequence of theproperties of conditional expectations as described inProposition 2. ✷

4.1. The BV Model

Let be an open subset of IRN . A function u ∈ L1()

whose partial derivatives in the sense of distributionsare measures with finite total variation in is called afunction of bounded variation. The class of such func-tions will be denoted by BV(). Thus u ∈ BV() if andonly if there are Radon measures µ1, . . . , µN definedin with finite total mass in and∫

u Diϕ dx = −∫

ϕ dµi (5)

for all ϕ ∈ C∞0 (), i = 1, . . . , N . Thus the gradient of

u is a vector valued measure whose finite total variationin an open set ′ ⊆ is given by

‖ Du ‖ (′)

= sup

{∫′

u div ϕ dx : ϕ ∈ C∞0 (′, IRN ), |ϕ(x)| ≤ 1

for x ∈ ′}. (6)

This defines a positive measure called the variationmeasure of u. For further information concerning func-tions of bounded variation we refer to [11] and [33].

For a Lebesgue measurable subset E ⊆ IRN and apoint x ∈ RN , the following notation will be used:

D(x, E) = lim supr→0

|E ∩ B(x, r)||B(x, r)| (7)

and

D(x, E) = lim infr→0

|E ∩ B(x, r)||B(x, r)| . (8)

D(x, E) and D(x, E) will be called the upper andlower densities of x in E . If the upper and lower densi-ties are equal, then their common value will be calledthe density of x in E and will be denoted by D(x, E).

The measure theoretic boundary of E is defined by

∂M E = {x ∈ IRN : D(x, E) > 0, D(x, IRN \E) > 0}.(9)

Here and in what follows we shall denote by Hα theHausdorff measure of dimension α ∈ [0, N ]. In partic-ular, H N−1 denotes the (N −1)-dimensional Hausdorffmeasure and H N , the N -dimensional Hausdorff mea-sure, coincides with the Lebesgue measure in IRN .

Let E be a subset of IRN wih finite perimeter.This amounts to say that χE ∈ BV(IRN ), the spaceof functions of bounded variation. Then ∂M E is rec-tifiable, i.e., ∂M E ⊆ ⋃∞

i=0 Ei where each Ei is a(N − 1)-dimensional embedded C1-submanifold ofIRN and H N−1(E0) = 0 [11, 33]. We also have thatH N−1(∂∗E) =‖ DχE ‖. We shall denote by P(E) theperimeter of E , i. e., P(E) = H N−1(∂∗E).

As shown in [2], we can define the connected com-ponents of a set of finite perimeter E so that they aresets of finite perimeter and constitute a partition of E(mod H N ). Let us describe those results.

Definition 1. Let E ⊆ IRN be a set with finite perime-ter. We say that E is decomposable if there exists apartition (A, B) of E such that P(E) = P(A)+ P(B)

and both, A and B, have strictly positive measure. Wesay that E is indecomposable if it is not decomposable.

Theorem 2 ([2]). Let E be a set of finite perimeter inIRN . Then there exists a unique finite or countable fam-ily of pairwise disjoint indecomposable sets {Yn}n∈I

such that

(i) meas(Yn) > 0 for all n ∈ I and

meas

( ⋃n∈I

Yn

)= meas(E).

(ii) Yn are sets of finite perimeter and

P(E) =∑n∈I

P(Yn). (10)

Page 14: Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

102 Caselles, Coll and Morel

(iii) The sets Yn are maximal indecomposable, i.e. anyindecomposable set F ⊆ E is contained, moduloa null set, in some set Yn .The sets Yn will be called the M-components ofE . Moreover, we have

If F is a set of finite perimeter contained in E,

then each M-component of F is contained in a M-component of E . In other words, if Fper denotesthe family of subsets of IRN with finite perimeterthen the above statement gives a connected com-ponent analysis of (IRN ,Fper ).

Let u ∈ BV() ∩ L∞() and v ∈ L1(). Withoutloss of generality we may assume that u takes valuesin [a, b]. We know that for almost all levels λ ∈ [a, b],the level set [u ≥ λ] is a set of finite perimeter. LetG(u) = {u−1([α, β)) : α < β} such that u−1([α, β)) isof finite perimeter. Then Theorem 2 describes a con-nected component analysis of G(u).

Let Pn be a sequence of partitions of [a, b] such that

(a) Pn+1 is a refinement of Pn , n = 1, 2, . . ..(b) For each n ∈ IN , the sections of u associated to

levels of Pn , are sets of finite perimeter.(c) meas([u = α]) = 0 for each α ∈ Pn .(d) ‖Pn‖ → 0 as n → ∞.

Given a σ -algebra A and a measure µ we denote byA the completion of A with respect to µ, i.e., A ={A ∪ B : A ∈ A, B a µ-null set}.Lemma 2. Let Pn,Qn be sequences of partitions of[a, b] satisfying (a), (b), (c), (d). Let A∞, resp. B∞,

be the σ -algebra generated by the sequence of σ -algebras APn , resp. AQn . Then A∞ =B∞.

Proof: It suffices to prove that given n ∈ IN andX an M-component of [c ≤ u < d], [c, d) being aninterval of Pn , then there is a set Z ∈B∞ such thatµ(X�Z) = 0. Let ε > 0. Since µ([u = c]) = µ([u =d]) = 0, let m ∈ IN be large enough so that [c, d) ⊆∪p−1

i=1 [bi , bi+1), [bi , bi+1) being intervals of Qm , withc ∈ [b1, b2), d ∈ [bp−1, bp), and µ([b1 ≤ u < b2]) < ε,µ([bp−1 ≤ u < bp]) < ε. Now, since any M-componentof a set [bi ≤ u < bi+1], i = 2, . . . , p − 2, is eithercontained in X or disjoint to it, we have

Z∗ ⊆ X ⊆ Z∗ ∪ [b1 ≤ u < b2] ∪ [bp−1 ≤ u < bp],

where Z∗ is the union of M-components of the sets[bi ≤ u < bi+1], i = 2, . . . , p−2, which are contained

in X . Obviously, Z∗ ∈ B∞ and µ(X \ Z∗) < 2ε. Thisimplies our statement. ✷

The above Lemma proves that the σ -algebra gen-erated by the sequence of σ -algebras APn , n ≥ 1, isindependent of the sequence of partitions satisfying(a), (b), (c), (d). Let us denote by Au the σ -algebragiven by the last Lemma. Observe that E(u|Au) = u.

Theorem 2. Let v ∈ L p(), p ≥ 1. Let Pn bea sequence of partitions satisfying (a), (b), (c), (d).Let vn = E(v|Pn). Then vn is a martingale relative to({APn })n. Moreover, vn converges in L1(Au) and a.e.to a function v∞ = E(v |Au), which may be consid-ered as the projection of v into the space L p(Au). Thelimit is independent of the sequence of partitions sat-isfying (a), (b), (c), (d). If v ∈ L1(Au), then v∞ = v.In particular, u∞ = u.

Remark. Let u : → [a, b] is in BV() and v ∈L1(|Du|dx). Then we may also define

E ′(v | u,P) =∑

A∈CC(P)

vAχA

where

vA =

∫Av|Du|dx∫

A|Du|dx

.

Then results similar to Proposition 2 and Theorem 3hold for E ′ as an operator from L1(, |Du|dx) intoL1(Au, |Du|dx). Formally, we have

vA =∫

A v|Du|dx∫A |Du|dx

=∫

[c≤uχA<d] v|Du|dx∫[c≤uχA<d] |Du|dx

=∫ d

cddt

∫[uχA>t] v|Du|dx∫ d

cddt

∫[uχA>t] |Du|dx

=∫ d

c

∫∂M [uχA≥t] vd H 1dt∫ d

c

∫∂M [uχA≥t] d H 1dt

for any connected component A of [c ≤ u < d]. Now,if we let c, d → λ then

vA →∫

cc[u=λ] vd H 1∫cc[u=λ] d H 1

Page 15: Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

Geometry and Color in Natural Images 103

where cc[u = λ] denotes the connected component of[u = λ] obtained from A when letting c, d → λ. Inthe case under consideration, this amounts to interpretour algorithm as the computation of the average of v

along the connected components of the level curves ofu. Note that, if we take v = u the above formula gives

vA → λ,

when letting c, d → λ.

4.2. Contrast Invariance for Color

We can state the contrast invariance axiom for color asContrast Invariance of Operations on Color [8].We say that an operator T is morphological if for anyU and any

h(U ) = h(〈U, σ 〉)〈U, σ 〉 U, (11)

where h is a continuous increasing real function wehave

T (h(U )) = h(T (U )).

We refer to such functions h as contrast changes forcolor vectors.

Proposition 3. Let U : → IR3 be a color imagesuch that L ∈ BV() ∩ L∞() and S, H ∈ L1().Let AL be the σ -algebra associated to the luminancechannel as described in the previous section. Let usdefine the filter

F(U ) =

L(U )

E(S(U ) |AL)

E(H(U ) |AL)

(in L , S, H system).

Then F is a morphological operator.

Proof: Let h be a contrast change for color vec-tors. Let V be any color image. Then the Luminance,Saturation and Hue of the color vector h(V ) are givenby

L(h(V )) = 〈h(V ), σ 〉 = h(L(V ))

S(h(V )) = ‖σ × h(V )‖ = h(L(V ))

L(V )S(V ),

H(h(V )) = angle

σ × h(V ), σ ×

1

0

0

= H(V ).

(12)

Then the Luminance, Saturation and Hue of the colorvector F(h(U )) are

F(h(U )) =

h(L(U ))

E(

h(L(U ))

L(U )S(U )|AL

)E(H(U )|AL)

(in L , S, H system). (13)

Now, using Proposition 1, (i i), we may write (13) as

F(h(U )) =

h(L(U ))

h(L(U ))

L(U )E(S(U )|AL)

E(H(U )|AL)

(in L , S, H system). (14)

By definition of F(U ), it is immediate that

h(F(U )) =

h(L(U ))

h(L(U ))

L(U )S(F(U ))

H(F(U ))

(in L , S, H system),

(15)

which is equal to

h(L(U ))

h(L(U ))

L(U )E(S(U ) |AL)

E(H(U ) |AL)

(in L , S, H system). (16)

Thus F(h(U )) = h(F(U )), i.e., the operator F is con-trast invariant.

Acknowledgment

We gratefully acknowledge partial support by CYCITProject, reference TIC99-0266 and by the TMR

Page 16: Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

104 Caselles, Coll and Morel

European Project Viscosity Solutions and their Appli-cations, FMRX-CT98-0234.

Note

1. “Gibt es eine physikalisch festliegende “normale” Beleuchtungund eine “eigentliche” Farbe?” (Is there any physically wellfounded “normal” illumination and a proper color)?

References

1. A. Almansa, B. Coll, and J. Froment, “A geometric applicationto color images,” Private Communication, 2000.

2. L. Ambrosio, V. Caselles, S. Masnou, and J.M. Morel, “Con-nected components of sets of finite perimeter and applicationsto image processing,” Preprint, 1999.

3. H. Asada and M. Brady, “The curvature primal sketch,” IEEEPAMI, Vol. 8, No. 1, pp. 2–14, 1986.

4. C. Ballester, E. Cubero-Castan, M. Gonzalez, and J.M. Morel,“Contrast invariant image intersection,” Cahiers de l’ENSCachan, 9817, 1998.

5. P. Blomgren and T.F. Chan, “Color TV: Total variation methodsfor restoration of vector-valued images,” IEEE Transactions onImage Processing, Vol. 7, pp. 304–309, 1998.

6. V. Caselles, B. Coll, and J.M. Morel, “A Kanizsa programme,”Preprint, CEREMADE, Univ. Paris-Dauphine, 1995.

7. V. Caselles, B. Coll, and J.M. Morel, “Topographic maps andlocal contrast changes in natural images,” International Journalof Computer Vision, Vol. 33, No. 1, pp. 5–27, 1999.

8. A. Chambolle, “Partial differential equations and imageprocessing,” in Proceedings of the Fourth IEEE InternatinalConference on Image Processing, ICIP-94, Austin, Texas, Nov.1994, pp. 16–20.

9. B. Coll and J. Froment, “Topographics maps of color images,”in Proceedings of the 15th International Conference on PatternRecognition, Barcelona, 2000, Vol. 3, pp. 613–616.

10. J.L. Doob, Stochastic Processes, Wiley Classics Library,1953.

11. L.C. Evans and R.F. Gariepy, Measure Theory and Fine Proper-ties of Functions, Studies in Advanced Math., CRC Press: BocaRaton, FL, 1992.

12. L. Hjelmslev, Prolegomenes a une theorie du langage, Minuit,1971.

13. B.K. Horn, Robot Vision, MIT Press: Cambridge, MA,1986.

14. W. Kandinsky, Point et ligne sur plan, Folio essais, Gallimard,1991.

15. J.J. Koenderink, “The structure of images,” Biol. Cybern.,Vol. 50, pp. 363–370, 1984.

16. J.J. Koenderink and A.J. van Doorn, “Dynamic shape,” Biolog-ical Cybernetics, Vol. 53, pp. 383–396, 1986.

17. J.L. Lisani, P. Monasse, L. Moisan, and J.M. Morel, “Affineinvariant mathematical morphology applied to generic shaperecognition algorithm,” in Proceedings of the International Sym-posium on Mathematical Morphology 2000, Palo Alto, KluwerAcademic Publishers, 2000, pp. 91–98.

18. A. Mackworth and F. Mokhtarian, “A theory of multiscale,curvature-based shape representation for planar curves,” IEEEPAMI, Vol. 14, pp. 789–805, 1992.

19. S. Masnou, “Filtrage et desocclusion d’images par methodesd’ensembles de niveau,” Ceremade, Universite Paris-Dauphine,Ph.D. Thesis, 1998.

20. W. Metzger, Gesetze des Sehens, Waldemar Kramer, 1975.21. P. Monasse, Image Registration, Cahiers de l’ENS Cachan,

1999.22. P. Monasse and F. Guichard, “Fast Computation of a Contrast In-

variant Image Representation,” Cahiers de l’ENS Cachan, 9815,1998.

23. J. Paulhan, La peinture cubiste, Gallimard, 1990.24. P. Perona, Private communication, 1989.25. P. Perona and J. Malik, “Scale-space and edge detection using

anisotropic diffusion,” IEEE Transactions on Pattern Anal. Ma-chine Intell, Vol. 12, pp. 629–639, 1990.

26. G. Sapiro and D.L. Ringach, “Anisotropic diffusion of mul-tivalued images with applications to color filtering,” IEEETransactions on Image Processing, Vol. 5, pp. 1582–1586,1996.

27. G. Sapiro and A. Tannenbaum, “On affine plane curve evolu-tion,” Journal of Functional Analysis, Vol. 119, No. 1, pp. 79–120, 1994.

28. J. Serra, Image Analysis and Mathematical Morphology, Aca-demic Press: New York, 1982.

29. L. Signac, D’Eugene Delacroix au neoimpressionnisme, Collec-tion Savoir, Hermann, 1987.

30. L. Vincent, “Morphological area openings and closings for gray-scale images,” in Proceedings of the Worshop “Shape in Picture”,1992, Driebergen, The Netherlands, Springer, Berlin, pp. 197–208, 1994.

31. M. Wertheimer, “Untersuchungen zur Lehre der Gestalt, II.” Psy-chologische Forschung, Vol. 4, pp. 301–350, 1923.

32. D. Williams, Probability with Martingales, Cambridge Mathe-matical Textbooks, 1991.

33. W.P. Ziemer, Weakly Differentiable Functions, GTM 120,Springer Verlag: Berlin, 1989.

Vicent Caselles received the Licenciatura and Ph.D. degrees in math-ematics from Valencia University, Spain, in 1982 and 1985, respec-tively. Currently, he is an associate professor at the Pompeu-FabraUniversity in Barcelona (Spain). He is an associate member of IEEE.His research interests include image processing, computer vision,and the applications of geometry and partial differential equations toboth previous fields.

Page 17: Geometry and Color in Natural Images - BGUben-shahar/ftp/papers/Color... · 2005. 7. 24. · Journal of Mathematical Imaging and Vision 16: 89–105, 2002 c 2002 Kluwer Academic Publishers

Geometry and Color in Natural Images 105

Bartomeu Coll received the Ph.D. degree from the AutonomaUniversity of Barcelona in Math Applied in 1987. In 1989, hehas received a postdoc fellowship to start the research in thefield of image processing in the laboratory of Ceramade, Paris.

Since 1992 he has been a senior research fellow at the Univer-sity of Balearic Islands. His scientific interests include the studyof mathematical models for image processing, namely non-linearsmoothing, morphological representation of the images based onthe level set methods, color images, application to medical images,etc.

Jean-Michel Morel is currently professor of applied mathematics atEcole Normale Superieure de Cachan, France. His main interest is themathematical formalization of visual perception. He is author of onebook (Variational Methods in Image Segmentation) in collaborationwith S. Solimini. He has written several papers on the applicationsof P. D. E.’s, and, more recently, of geometric probability to imageanalysis problems.