Image Fusion based on Overcomplete Rational Wavelet transform with an improved IHS method.

Mallika K Computational Engineering and Networking

Amrita Vishwa Vidyapeetham, Coimbatore, India. mallika.k@dcemail.com

Soman K P Computational Engineering and Networking

Amrita Vishwa Vidyapeetham , Coimbatore, India. kp_soman@ettimadai.amrita.edu

Abstract: This paper introduces a new hybrid fusion method which is comprised of fast intensity–hue-saturation (FIHS) and over complete discrete wavelet transform with rational dilation factors. Apart from speed, this hybrid fusion method can help to overcome the problems inherent in IHS and wavelet based methods. This method can also be used to extract the spatial information from the difference image of the panchromatic (PAN) and intensity images. Keywords: Pan-sharpening, Intensity-Hue-Saturation transform, Wavelet transforms, frames, filter bank, rational dilation factors.

I. INTRODUCTION

There are number of applications in remote sensing that require images with both high spatial resolution and high spectral resolution. The fusion of multispectral and panchromatic images or pan sharpening provides a solution by combining the clear geometric features of the panchromatic image and the color or spectral information from the multispectral image. Of the hundreds of image fusion techniques, the best known are the intensity-hue-saturation (IHS) method, principal component analysis, arithmetic combinations and wavelet based methods. Wavelet based fusion methods, are based on multi resolution analysis. Generally, Wavelet based image fusion produces the high quality spectral content in fused images. That means it preserves the spectral characteristics of multispectral (MS) image better than the IHS based method. However, the spatial resolution obtained by most wavelet based methods is less than that obtained by the intensity-hue-saturation (IHS) method. On the other hand, the disadvantages of IHS and wavelet based methods is that, IHS destroys the spectral characteristics of the MS image, and wavelet based fusion methods are not efficient to quickly merge massive volumes of data from satellite images.

To reduce the computational cost, a Fast IHS (FIHS) transform introduced by Tu et al. [11] is employed on a new hybrid fusion method. Aside from its fast computing capability for fusing images, this method can extend traditional three-order transformations to any arbitrary order. In this paper we are using three-order transformations. In addition we employed an over complete DWT with rational dilation factors. This hybrid fusion method is able to eliminate drawbacks of the IHS and wavelet based methods, while keeping the advantages.

II. OVER COMPLETE DISCRETE WAVELET

TRANSFORM WITH RATIONAL DILATION FACTORS Over complete discrete wavelet transform or frames, have become a well recognized tool in signal processing. The over complete DWT can be implemented using self inverting FIR filter banks, which are approximately shift invariant and can provide a dense sampling of the frequency plane. Most of the frames are dyadic wavelet transforms; the resolution is doubled from each scale to the next scale. In [4] the wavelet frames with rational dilation factor (3/2) for discrete-time signals was described, where the resolution is increased more gradually from scale to scale. If the dilation factor is a rational number between one and two, we have a ‘rational’ wavelet transform. The over complete rational wavelet transform or frames has a fixed redundancy rate independent on the number of levels, so it can be used to analyze large data sets with out incurring the high memories and computing time costs. One motivation for developing rational wavelet transform is the higher frequency resolution that can be achieved, compared to the other wavelet transforms. Before we proceed further, the basic concepts of frame theory is discussed as in [6], [7], [8]. A. A Rational Tight frame The design of a rational (2/3) wavelet tight frame based on over-sampled filter bank is shown in Fig1 (b).Like some other wavelet tight frames, the over complete transform is approximately shift invariant and sample the time frequency plane more densely than a critically-sampled wavelet transform [4]. The tight frame introduced here is based on the filter bank in Fig1 (b), which possesses two additional high pass channels (G1, G2, in addition to G0).

Figure 1. (a) Low pass, high pass branches of Rational filter bank.

(b) An over-sampled filter bank for implementing a rational wavelet tight frame.

B. Orthonormal rational filter banks We consider orthonormal rational filter banks and their design using Grobner basis methods [5]. The perfect reconstruction conditions for the filter bank in Fig1 (a), which provides minimal length solutions for some special cases given in [3].Referring to Fig1 (a), we have

2009 International Conference on Advances in Recent Technologies in Communication and Computing

978-0-7695-3845-7/09 $25.00 © 2009 IEEE

DOI 10.1109/ARTCom.2009.199

330

2009 International Conference on Advances in Recent Technologies in Communication and Computing

978-0-7695-3845-7/09 $26.00 © 2009 IEEE

DOI 10.1109/ARTCom.2009.199

330

2009 International Conference on Advances in Recent Technologies in Communication and Computing

978-0-7695-3845-7/09 $26.00 © 2009 IEEE

DOI 10.1109/ARTCom.2009.199

330

2ˆ ( ) ( ) ( )X z X z H z= (1) where -( ) ( ) n

n

X z x n z= ∑ . (2)

Then, defining W = 2 /3ie π 21 ˆ ˆ ˆ( ) ( ( ) ( ) ( ) )

3X z X z X z W X z W= + + (3)

and therefore

]1 / 2 1 / 2 1 / 2 1 / 21

1( ) ( ) ( ) ( ) ( )2

Y z X z H z X z H z− −⎡= + − −⎣ (4)

From Eq (1)

]

]

]

1/ 2 1/ 2 1/ 2 1/ 21

1/ 2 1/ 2 1/ 2 1/ 2 2

1/ 2 2 1/ 2 1/ 2 2 1/ 2

1( ) ( ) ( ) ( ) ( ) ( )6

1 ( ) ( ) ( ) ( ) ( )61 ( ) ( ) ( ) ( ) ( )6

Y z H z H z H z H z X z

H z W H z H z W H z X zW

H z W H z H z W H z X zW

− −

−

− −

⎡= + − −⎣

⎡+ + − −⎣

⎡+ + − −⎣

(5)

These conditions can be written in matrix form as 21 1 1 2

1 1 1 2 2 2 2

2 2 2 2 2 2

( ) ( ) 2 ( )( ) ( ) ( )( ) ( ) ( ) ( ) ( ) 2 ( ) 6

2 ( ) 2 ( ) 2 ( ) ( ) ( ) 2 ( )

H z H z G zH z H z W H z WH z H z W H z W H zW H zW G z W I

G z G z W G z W H zW H zW G z W

− − −

− − −

− − −

⎛ ⎞−⎛ ⎞⎜ ⎟⎜ ⎟

− − − − =⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟−⎝ ⎠⎝ ⎠

(6)

These perfect reconstruction conditions are more suitable for Grobner basis computation [5], [3]. The design of an ortho normal rational filter bank [3] (with FIR filters) with vanishing moments requires the solution of a number of nonlinear equations. In contrast with the M-band case, the design of the low pass filter can not be simplified by expressing the design problem in terms of autocorrelation sequences. However for the tight frame, the low pass filter must satisfy the constraints are significantly relaxed [5]. The high pass filters that complement the low pass filter (so as to ‘complete’ the filter bank) can be obtained via polynomial matrix spectral factorization [4]. Consider the system in Fig1 (b), where the filters 0 ( )H z and

1( )H z are related to H (z) by 3 2

0 1( ) ( ) ( )H z H z z H z−= + (7)

Given 0 ( )H z and 1( )H z , our aim is to find ( )iG z (using spectral factorization) so that the resulting filter bank forms a tight frame. Consider the polyphase decompositions of the filters. For H (z), this is: 3 1 3 2 3

0 00 01 02( ) ( ) ( ) ( )H z H z z H z z H z− −= + + (8) Suppose similar notation is used to denote the polyphase components of the remaining filters ( 0G ( )z , 1G ( )z , 2G ( )z ). The Filter Bank (FB) is a tight frame iff

( )1

1

( )( ) ( )

( )

T

T

H zH z G z

G z

−

−

⎛ ⎞⎜ ⎟⎜ ⎟⎝ ⎠

=I. (9)

where 0 0 1 0

0 1 1 1

0 2 1 2

( ) ( )( ) ( )( )=( ) ( )

H z H zH z H zH zH z H z

⎛ ⎞⎜ ⎟⎜ ⎟⎜ ⎟⎝ ⎠

(10)

0 0 1 0 2 0

0 1 1 1 2 1

0 2 1 2 1 3

( ) ( ) ( )G (z )= ( ) ( ) ( ) .

( ) ( ) ( )

G z G z G zG z G z G zG z G z G z

⎛ ⎞⎜ ⎟⎜ ⎟⎜ ⎟⎝ ⎠

. (11)

Thus we can complete the FB to a tight frame iff the matrix 1( ) ( )TI H z H z−− is a non-negative definite on the unit

circle. It turns out that a given a low pass filter can be completed to a tight frame as shown in Fig 1 (b).

(D) Two- Dimensional Extension The 2D extension of the filter bank is illustrated in Fig.2. Here we are iterating on the low pass branch. For getting wav let planes (of images) we have to use four filters

0, 0 1 2, ,h g g g with rational (dilation) sampling factors (3/2).

The values and frequency response of 0, 0 1 2, ,h g g g was

given in [3]. Here h0 is low pass filter, 0g , 1g and 2g are three high pass filters.

Figure 2. An over sampled rational filter bank for 2-D image.

III. THE PROPOSED FUSION METHOD

For applying any of the image fusion methods to the MS image and the PAN image these two images must be accurately superimposed. Thus the both images must be co registered, and the MS image resampled to make its pixel size the same as the PAN image. In order to achieve this, a robust registration technique and a bicubic interpolation is used.

A. FIHS Fusion Method The FIHS fusion for each pixel can by formulated by the

following procedure. ( ) ( )

( ) ( )( ) ( )

F R R P A N IF G G PA N IF B B P A N I

+ −⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥= + −⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥+ −⎣ ⎦ ⎣ ⎦

. (12)

where F(X) is the fused image of the X band, for X = R, G, and B, respectively.

Multi resolution wavelet decomposition is used to execute the detailed extraction phase, and the IHS procedure is followed to inject the spatial details of the PAN image into the MS image. In other words, instead of using the PAN image in Eq. (12), the results of the PAN image and the intensity image fused by the wavelet method is used. The fusion results of the PAN image and the intensity image are expressed as follows:

( )1

k I

n

n e w r p a nk

I I W−

=

= + ∑ (13)

Where, Ir is the low-frequency version of the wavelet transformed intensity image and

1k

n

p a nk

W=∑

is the sum of high

frequency versions of the wavelet-transformed PAN image.

B. The Proposed Hybrid Method Assume that with out loss of generality, the hybrid fusion method is based on the FIHS fusion method instead of the

331331331

traditional IHS method. This is because Eq. (12) holds .The hybrid fusion method can be simplified by the following procedure.

( )

( )

( )

1

1

1

( )( )( ) ( )( ) ( )

k I

k I

k I

n

p a nk

n e w n

n e w p a nk

n e w n

p a nk

R WR I IF R

F G G I I G WF B B I I

B W

−

−

−

=

=

=

⎡ ⎤+⎢ ⎥⎢ ⎥+ −⎡ ⎤⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥ = + − = +⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥ + −⎣ ⎦ ⎣ ⎦ ⎢ ⎥

+⎢ ⎥⎣ ⎦

∑

∑

∑

(14)

where ( )

1k I

n

p ank

W−

=∑ is the sum of the high-frequency versions

of the wavelet-transformed difference image of the PAN image and the Ir image. The proposed hybrid scheme involves the simple procedure based on the FIHS fusion method [9]. Therefore, the proposed hybrid fusion method is much simpler and faster than the other hybrid methods. In summary, we employed the FIHS method in order to reduce the computational cost of the original hybrid method along with the simplification of the mathematical model, and suggested that over complete discrete wavelet transform with rational dilation factors is used to enhance the overall performance of the fused images.

IV. EXPERIMENTAL ANALYSIS For the experimental studies, an image of the Korean city

of Daejeon [9], which was acquired on 9th march 2002, is taken. The imagery contains a 1m resolution pan image and 4m resolution MS image. Table. I shows the quality analysis of the proposed method with five quality metrics: Correlation coefficients (CC), spectral angle mapper (SAM), extended SAM (ESAM), QNR (Quality with no Reference), QEC (Quality with Edge information) [12], [13], [14].

TABLE I.QUALITY ANALYSIS

This approach performs significantly better than other wavelet based methods. A. Visual analysis Fig.3 shows the results of full scale visual fusion. The images fused by the FIHS method are easily explained by Eq. (12). The aliasing patterns, present in MS images, and also in the intensity image are cancelled when the intensity image is subtracted from each of the spectral band. This aliasing artifacts also will disappear when the images are fused by the over complete DWT methods. Consequently the hybrid fusion method is more acceptable for the visual analysis.

(a) (b)

(c) (d) Figure3. (a) Resampled Multi spectral Image (b) PAN image (c)Fused by the Fast IHS transform (some spectral distortion is appearing) (d) Fused by the Over complete discrete wavelet transform (DWT) based on rational dilation

factors. V. CONCLUSION

Orthonormal over complete wavelet transform with rational sampling factors for image fusion were presented. To validate this new approach PAN and MS images were merged. To analyze the spatial and spectral quality of the resulting image, we used five quality metrics: correlation coefficients (CC), spectral angle mapper (SAM), extended spectral angle mapper (ESAM), QNR (Quality with No Reference), QEC (Quality with Edge information).

REFERENCES [1] C.K.Munechika, J.S.Warnick, C.Salvaggi and J.R.Schott, “Resolution Enhancement of Multispectral Image Data to Improve

Classification Accuracy”, Photogrammetric Engineering and Remote sensing, Vol.59, no.1, 1993, pp.67-72

[2] Y.Zhang, “Understanding Image Fusion,” Photogram metric Engineering and Remote Sensing, vol. 70, no. 6, 2004, pp. 653-760.

[3] I.Bayram and I. Selesnickb.“Design of orthonormal and over complete wavelet transforms based on rational sampling factors”. Fifth SPIE Conference on Wavelet Applications in Industrial Processing, 2007.

[4] I.Bayram and I.W. Selesnickb.“Over complete discrete wavelet transform with rational dilation factors” 57(1):131-145, January 2009.

[5] J.Lebrun and I. Selesnick.”Grobner bases and wavelet design. Journal of Symbolic Computing”, 37(2):227–259, February 2004.

[6] Martin Vetterli, Jelena Kovacevic Vivek K Goya l ” The World of Fourier and Wavelets: Theory, Algorithms and Applications” 1April 3, 2007.

[7] Selesnickb, I. W., 2004. “Symmetric wavelet tight frames with two generators”. ACHA, 17, pp.211-225.

[8] A. F. Abdelnour and I. W. Selesnickb, “Symmetric Nearly Shift-Invariant Tight Frame Wavelets,” IEEE Transactions on Signal Processing, vol. 53, no. 1, 2005, pp. 231-239.

[9] Myungjin Choi “Introduction of a Symmetric Tight Wavelet Frame to Image Fusion Methods Based on Substitutive Wavelet Decomposition”..... Elementary Introduction to Algebras,” New York: Springer-Verlag, 1989

[10] González-Audícana, M., Saleta, J. L., Catalan, R. G.,and Garcia, R., 2004. “Fusion of Multispectral and Panchromatic images Using Improved IHS and PAC mergers based on Wavelet

Decomposition”. IEEE TGRS, 42(6), pp. 1291-1299. [11] T.-M. Tu, P. S. Huang, C.-L. Hung and C.-P. Chang, “A Fast

Intensity- Hue-Saturation Fusion Technique with Spectral Adjustment for IKONOS Imagery,” IEEE Geoscience and Remote sensing letters, vol. 1, no. 4, 2004, pp. 309-312.

[12] Gemma Piella and Henk Heijmans, CWI,Kruislaan 413, 1098 SJ Amsterdam, The Netherlands. “QEC - a new quality metric for image fusion”.

[13] Luciano Alparone(1), Bruno Aiazzi(2), Stefano Baronti(2), Andrea Garzelli(3), Filippo Nencini(3) (1)DET-UniFI: “QNR -information-theoretic image fusion assessment without reference”. Department of Electronics and Telecommunications, University of Firenze, (2) IFAC-CNR: Institute of Applied Physics “Nello Carrara”, Italian National Research Council, (3) DII-UniSI: Department of Information Engineering, University of Siena.

[14] Tania Stathaki, “SAM, ESAM ,CC - Image Fusion: Algorithms and Applications”, Academic Press, 2008 edition.

Quality metrics

CC

SAM

ESAM

QNR

QEC

Values

0.00000088

100.98

100.97

0.9968

0.0565

332332332