_-

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 39, NO. 1, JANUARY 1993 7

Wavelet Decomposition of Harmonizable Random Processes

Ping Wah Wong, Member, IEEE

Abstruct-The discrete wavelet decomposition of second-order harmonizable random processes is considered. The deterministic wavelet decomposition of a complex exponential is examined, where its pointwise and bounded convergence to the function is proved. This result is then used for establishing the stochastic wavelet decomposition of harmonizable processes. The similari- ties and differences between the wavelet decompositions of gen- eral harmonizable processes and a subclass of processes having no spectral mass at zero frequency, e.g., those that are wide-sense stationary and have continuous power spectral densities, are also investigated. The relationships between the harmonizability of a process and that of its wavelet decomposition are examined. Finally, certain linear operations such as addition, differentiation, and linear filtering on stochastic wavelet decompositions are considered. It is shown that certain linear operations can be performed term by term with the decomposition.

Index Terms-Wavelet transform, wavelet decomposition, mul- tiresolution analysis, second-order random processes, harmoniz- able random processes, spectral representations.

I. INTRODUCTION ONTINUOUS wavelet transform is a technique for de- C composing a signal into components that have good

localization properties both in time and frequency [ 11-[3]. Because of these time and frequency localization properties, wavelet transform can provide local information on a signal that cannot be obtained using traditional methods such as the Fourier transform. Mallat considered the wavelet transform on a grid in the transform domain (discrete wavelet decomposi- tion), and introduced a multiresolution analysis technique for square integrable signals, i.e., those that are in L2 [4], [5]. The discrete wavelet decomposition is remarkable not only because it provides a tool for multiresolution analysis, but it also provides an orthonormal decomposition of L2. Mathe- matically, a discrete wavelet decomposition is an orthogonal decomposition of functions on a Hilbert space where the basis functions are formed by translations and dilations of a single wavelet basis function [SI. Many wavelet basis functions have already been found, including those with compact support [3],

There are two different but equivalent forms for representing the wavelet decompositions of functions in Lz. In the first one,

~ 1 , [71.

Manuscript received May 16, 1991; revised January 22, 1992. This work was supported in part by the National Science Foundation under Grant MIP- 9108410. This work was presented in part at the Allerton Conference on Communication, Control, and Computing, Allerton, IL, September, 1991.

The author was with the Department of Electrical and Computer Engi- neering, Clarkson University, Potsdam, NY 13699. He is now with Hewlett- Packard Laboratories, 1501 Page Mill Road, Palo Alto, CA 94304.

IEEE Log Number 9203011.

a square integrable function ~ ( t ) is represented as a weighted sum of the dilated and translated versions of a wavelet function $J(.). This can be thought of as a superposition of bandpass components because the Fourier transform of $(.) has a bandpass characteristic. In the second form, ~ ( t ) is represented as a superposition of the dilated and translated versions of $J(.) at high frequencies, plus a weighted sum of shifted versions of a scaling function q5(.). Since the Fourier transform of +(+) has a low-pass characteristic, this representation can be interpreted as a decomposition using both low-pass and bandpass components.

The discrete wavelet decomposition can also be used for analyzing nonsquare integrable deterministic functions [8]. In such case, it is necessary to impose certain additional condi- tions on the scaling and wavelet functions so that the wavelet coefficients are well defined. Furthermore, the b o different forms, that are equivalent for representing L2 functions, are no longer equivalent. Such functions outside of L2 must be represented by the second form, viz., as a superposition of low-pass and bandpass components. The reason is roughly that non-square integrable functions can have discrete spectral components at zero frequency, and hence cannot be completely described by scaled and translated versions of $J( e ) .

Since many useful and interesting signals encountered in practice are best modeled as random processes, a theory of wavelet decomposition for general random processes is needed. In [9], the wavelet decomposition for nearly l/f processes are considered. Certain assumptions are made on the correlation of the wavelet coefficients to derive the wavelet decomposition of these processes. A related work is the investigation of fractional Brownian motion using wavelet decompositions [lo], [ l l ] . In [12], the wavelet decomposition of finite energy and periodically correlated processes are considered. In addition, truncated finite power processes, i.e., on a finite subset of the real line, are also considered.

In many engineering problems, one frequently encounters random processes that do not have square integrable sample functions. For example, any process that has periodic com- ponents falls under this category. The concept of frequency is very important in many practical problems involving fil- tering, spectral analysis, estimation, prediction, etc. Random processes with an emphasis on frequency can be studied under the theory of harmonizable processes introduced by L o h e [13, Section 37.41. Loosely speaking, a harmonizable process is a second-order process that can be represented as a superposition of complex exponentials (A precise definition will follow in a later section). In general, a harmonizable

0018-9448/93$03.00 0 1993 IEEE

8 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 39, NO. 1, JANUARY 1993

process can either be stationary, wide-sense stationary, or nonstationary. In the special case where it is wide-sense stationary, the harmonic decomposition is identical to the well known spectral representation theorem of Bochner [14, Sections 20-211. Since the spectral measure may or may not be absolutely continuous, the power spectral density need not exist even if the process is wide-sense stationary. In particu- lar, the spectrum might consist of discrete components, i.e., distinct frequency components with a finite amount of power. Hence, one suspects that a theory of the wavelet decomposition for general harmonizable processes would require a close examination of the two different, forms, viz., the representation using bandpass components versus the representation using both low-pass and bandpass components

It is interesting to note that a spectral representation of cyclostationary processes has been studied [ 151 also using the harmonizability concept of Lobve [13]. A cyclostationary process is decomposed in [15] so that the frequency supports of the component processes are of a uniform width. The wavelet decomposition, on the other hand, partitions the fre- quency scale dyadically [3], [16], [17] so that high-frequency components have good time resolution, while low-frequency components have good frequency resolution.

In this paper, we consider and establish a theory for the wavelet decompositions of general harmonizable processes. We shall see that as in the deterministic case [8], the wavelet decompositions of harmonizable processes can be represented in two different forms. These two forms are equivalent for processes where there is no spectral mass at zero frequency. An important example of these processes are those having continu- ous power spectral densities. Only one of these forms, how- ever, is valid for the decomposition of general harmonizable processes. This is analogous to the deterministic wavelet decomposition of functions outside the space L2 [8].

In Section 11, we briefly review the two forms of discrete wavelet decompositions for deterministic functions. Section 111 establishes the bounded convergence of the wavelet decompo- sition of a complex exponential function. This result is then applied in Section IV to establish the wavelet decompositions of harmonizable processes. The harmonizability of wavelet decompositions will also be considered. Section V considers certain linear operations on wavelet decompositions of har- monizable processes. Finally, the results of this paper are summarized in Section VI.

dilated and translated version of a waveletfunction $(t) E L2,

viz.,

y!Jm,n(t) = 2”2$(2”t - n).

Associated with each $(t) , there is a scaling function +(t), also in L2, that satisfies the relationship

00 L-1 00

n=-m m=-m n=-m

(3) where c,&(t) is defined similarly as $m,n(t), and

SL,n rz Ju z(t)$L,n(t) d t . (4) -CO

Let $(f) and &(f) be the Fourier transforms of $(t) and +(it), respectively. They satisfy [5]

1, n = 0, 0, n = & l , f 2 , . . . , integer n (5)

and

$(0) = 0. (6)

Many scaling and wavelet functions, including those with finite support, have already been found [3], [6], [7]. In the language of multiresolution analysis [4], S L ( t ) can be interpreted as the approximation of z ( t ) up to the resolution level L. For any z ( t ) E Lz, s ~ ( t ) satisfies

lim s L ( t ) = 0 (7) L--00

and

lim s L ( t ) = z ( t ) , L--00

where the limits are in the L2 sense. Because of (3), z ( t ) can also be written as

0 3 - 0 0

m=K n=--oo 00

+ SK,n$K,n( t ) any finite K . (8) n=--00

Furthermore, a Parseval equality

11. DETERMINISTIC WAVELET DECOMPOSITIONS

integrable function z ( t ) , denoted by z ( t ) E L2, is [4], [5], [7] The discrete wavelet decomposition of a deterministic square

M M

where

-W

the overbar denotes complex conjugate, and $m,n(t) is a

0 0 0 0 00

n=--00

holds for all finite integer K . Mallat [4] developed a recursive algorithm for computing

the wavelet coefficients Xm,n and Sm,n based on a digital filtering approach. This f m s the basis of a multiresolution signal processing framework.’ Here we only write down those

‘The wavelet and scaling functions must be regular (to be defined later) to guarantee that they form a multiresolution framework [5 ] .

-

9 WONG: WAVELET DECOMPOSITION OF HARMONIZABLE RANDOM PROCESSES

results that will be useful in our proofs later. In particular, we have

M oc

k = - w

where

k=--03

(9)

and

-w

The discrete-time Fourier transforms of h k and g k , denoted by H ( f ) and G(f), respectively, are periodic functions of f with period one, and satisfy

J z 6 ( 2 f ) = H(f)&f) (10)

and

For any z(t) E Lz , the representations (1) and (8) are completely equivalent in the sense that the right hand sides of both equations converge to z( t ) . As noted by Meyer [8], this is not the case for functions outside of Lz. To see this, we first describe the wavelet decomposition of functions that are not in Lz.

In order to define the wavelet decomposition for nonsquare integrable functions, we need to impose more restrictive conditions on both $(.) and 4(.) so that the coefficients Xm+’s and S K , ~ ’ S in (2) and (4) are well defined, i.e., so that the integrals converge. This is commonly referred to as a regularity condition, which is defined by Meyer as follows.

Definition: A wavelet function $(t) is said to be regular of order T [8] if

for some constants Bm’s. The regularity of 4(.) is defined in the same manner.

Under such condition, it can be shown [8] using the theory of generalized functions [ 181 that the discrete wavelet decom- position of the form (8) holds for any generalized function of order less than T . The form of (l), however, fails to hold for these functions. For example, if z ( t ) = 1 for all t , then Xm,n = 0 for all m and n, and hence (1) is false. The reason is that (7) no longer holds for functions outside of LZ [8].

The regularity condition of (12) can be relaxed so that the discrete wavelet decomposition of the form (8) still holds for many interesting functions. A less restrictive regularity condition used by Mallat is the following.

Definition: A wavelet function $(.) is said to be regular [4], [5], if and only if it is continuously differentiable and also satisfies

where DO and D1 are finite constants. The regularity of 4(.) is defined in the same way.

In addition to their usefulness in proving theorems, the regularity conditions are also very important in practice even if we only consider discrete wavelet decompositions of functions in L 2 . For example, they guarantee that the multiresolution components of smooth functions will not exhibit fractal like characteristics [7]. They also control the localization properties of wavelet decompositions because of the specified rate of decay. For these reasons, most wavelet functions used in practice are regular [3], [7IL[8].

The smoothness of both $(f) and q(f) can be controlled from the filters H(f) and G ( f ) [7]. In particular, one can choose the filters so that

&(f> = O(f-l-€) (14)

and

$<f) = O(f+€) (15)

as f goes to inffinity, wh9e E is strictly positive. Note that (13) implies both $(f) and $(f) are O(f-’) as f --t 00. Hence, (14) and (15) are slightly stronger than (13). In practice, most wavelet basis functions, such as the Lemarie wavelet basis [4] and the Daubechies compactly supported wavelet basis [7], satisfy both (14) and (15). The constant E can in fact be made arbitrarily large by suitably choosing H ( f ) and G(f) [7]. In the rest of this paper, we assume that both the wavelet and scaling functions satisfy the regularity conditions (13), (14) and (15).

111. BOUNDEDNESS OF A DETERMINISTIC WAVELET DECOMPOSITION

The main objective of this section is to show that the wavelet decomposition of the complex exponential ej2.1rfot converges to the function boundedly. This will be needed in Section IV to prove the wavelet decomposition of harmonizable processes. Since this convergence result does not appear to have been reported, we provide a detailed proof. -

We first establish upper bounds for $(.) and &.). Lemma 1: Let $(.) be a wavelet function that satisfies the

regularity conditions (13) and (15). Then its Fourier transform is bounded by

where CO, C1 and CZ are finite constants, and 6 > 0. Comment: It is evident that the bound Cl/lfll+‘ is most

useful for large f (in magnitude), while Czlfl is particularly useful for small f .

10 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 39, NO. 1, JANUARY 1993

Proofi The first bound l $ ( f ) l 5 CO for all f follows from the absolute integrability of +(t) which is a result of (13). The second bound follows directly from (15). Finally, since $( f ) is uniformly continuous, bounded, a_"d equals zero at f = 0, there exist a constant CZ such that l + ( f ) l 5 Czlfl,

0 Lemma 2: Let 4 ( . ) be a scaling function that satisfies the

regularity conditions (13) and (14). Then its Fourier transform can be bounded by

and the lemma is proved.

where C3, Cq, and C5 are finite constants, and E > 0.

Proof: The proof is almost identical to that of Lemma 1. The only difference is in the bound 1 +C,lfl, which is a result of F(0) = 1. 0

Next, we define a function IK,M,N(~, t ) as

A IK,M,N(~, t ) =

m=K n = - N N

+ ~ ( 2 - K f ) e j 2 * f 2 - K n ~ ( 2 K t - n) . (16)

The next lemma concerns the uniform boundedness of IK,M,N(~, t). Its convergence to a complex exponential function is addressed in the theorem thereafter. A similar theorem concerning the pointwise convergence of the continuous wavelet transform of a bounded function which oscillates around zero is proved in [19]. Here we use the properties of the scaling and wavelet functions, viz., (9)-(ll), to show in Theorem 1 the pointwise convergence of (16) to the complex exponential.

Lemma 3: Let + ( e ) be a regular wavelet function and 4( .) be its corresponding regular scaling function. Given any fixed finite integer K , the function IK,M,N( f, t ) is uniformly bounded for all M > K , and all N , f, t.

n = - N

The proof of Lemma 3 is given in Appendix A.

Theorem 1: Let +(.) be a regular wavelet function and 4( .) be its corresponding regular scaling function. The wavelet decomposition of e j a x f o t is given by

Furthermore, the convergence is uniformly bounded for all fo and t , i.e., IK,M,N(~o, t ) is uniformly bounded for all M > K and all N , fo, t.

Comment: It can be verified easily that for s(t) = ej2*fot, (16) is a truncated version of the right-hand side of (8). If we were to assume both +(.) and +(-) satisfy (12), then it is well known [8] that I K M , N , (fo, t ) would converge pointwise to ejaafot. Since we use a weaker regularity condition, viz., (13), (14) and (15), we give a proof of the pointwise convergence.

Proof: In view of Lemma 3 and the foregoing com- ments, it remains to show that I K M , N , ( ~ o , ~ ) converges to ej2*fot for any t , f o , and any finite integer K . To this end, we have from (9) that

n = - N

n = - N . r w

k = - w J

holds for any finite integer L. We emphasize that (9) is a basic property of any valid pair of wavelet and scaling functions, regardless of the function being decomposed (in this case, the complex exponential). Since the summations over IC are absolutely convergent, we can interchange the order of summations, and apply (10) and (11) to give

N - E J( 2 - L - l f 0 ) ej2a fo2 -L-1 n 4 ( 2 L + l t - n) = n = - N

N

n = - N

N

n = - N

Letting N go to infinity on both sides, we have for any L that

k=-oo 0 3 -

+ ~ ( 2 - ~ f o ) e 3 2 * f 0 2 - ~ ~ + ( 2 ~ t - IC) . (18)

Hence, we can conclude from (16) and (18) that limN-a IK,M,N(~o,~) is invariant for all finite integer K satisfying K < M , i.e.,

N-02 lim I K ~ , M , N ( ~ o , ~ ) = J$mI~Z,~,~(f~rt)l

This implies the right-hand side of (17) is invariant of K. Because of (18), it remains to show

k = - w

any K 1 , K2 < M .

WONG: WAVELET DECOMPOSITION OF HARMONIZABLE RANDOM PROCESSES 11

any fo and t , (19)

and w

any f o and t. (20) Since the summation over n in (19) is bounded independently of M , fo and t , we can apply the dominated convergence theorem to interchange the limit and the summation. Hence, (6 ) implies (19) is true. To show the validity of (20), we use the Poisson sum formula to write

M

. e j 2 ~ ( f o - 2 ' n ) t .

Because of the second bound in Lemma 2, we can apply the dominated convergence theorem to interchange the limit and summation of the right-hand side. We can then conclude from

0 (5) that (20) is true, proving the theorem.

IV. STOCHASTIC WAVELET DECOMPOSITION

We consider in this section the stochastic wavelet decompo- sition of second-order harmonizable random processes. Since we can write a random process as the sum of a mean function and a corresponding zero-mean random process, we can con- sider the wavelet decompositions of the deterministic and the random parts separately. We, therefore, assume without loss of generality that all the random processes are of zero mean.

A process z ( t ) is said to be of second-order if E[x2( t ) ] < 00 for all t [13], [20]. By virtue of the Cauchy-Schwartz inequality, all second-order moments and cross moments of z ( t ) exist. Note that a second-order random process can either be stationary, wide-sense stationary, or nonstationary. A large class of second-order processes are harmonizable, which, roughly speaking, are those that can be represented as a superposition of complex sinusoids. A precise definition of harmonizable processes due to Lobve is given as follows.

Definition (13, Section 37.41: A covariance function R(t, t') is said to be hurmonizuble if there exists a covariance ~ ( f , f') of bounded variation on IR x IR such that

-CC -CC

The symbol R denotes the set of real numbers. Definition [13, Section 37.41: A second-order random

process z ( t ) is said to be hurmonizuble if there exists a second- order process [(f) with a covariance function E [[(f)f(f')] = ~ ( f , f') of bounded variation on IR x R such that

m

A necessary condition for a process to be harmonizable is that it is second-order continuous. Lobve also showed that a random process is harmonizable, if and only if its covariance function is harmonizable. Observe that the func- tion ~ ( f , f') determines, with suitable normalization at the points of discontinuity, a distribution of mass over the two- dimensional frequency plane. Hence, it can be considered as a two-dimensional spectral distribution of z( t ) . (See, for example, [21, Section 8.31.)

In the special case where z ( t ) is also wide-sense stationary, the process ( ( f ) , usually called the spectral process, has orthogonal increments, i.e.,

E[dE(t) d T ( s ) ] = d F ( s ) L ,

where St is the Kronecker delta, and I?(.), usually called the spectral measure, is a finite measure on the real line. Equation (21) for wide-sense stationary processes then be- comes

E[z(t)Z(t ')] = R,(t - t') = ej2*f(t-t ' ) d F ( f ) . -CC

This is the well-known theorem of Bochner for the spectral de- composition of mean-square continuous wide-sense stationary random processes [14, Sections 20-211.

With these notations established, we can then proceed to develop results concerning the wavelet decomposition of harmonizable processes.

z ( t ) can be decomposed as Theorem 2: A second-order harmonizable random process

1.i.m. x ( t ) = M,N+0O m=K n=-N

N

+ S l ( , n d K , n ( t ) ] 3 any finite KI any t , n=-N

(23)

where 1.i.m. denotes limit in the mean-square, m

-m M

and 00

-m M

-m

Furthermore, the convergence in (23) is uniformly bounded in the mean-square sense. with probability one.

~

12

E

Comment: Equation (23) is said to be the discrete wavelet decomposition of second-order harmonizable processes. It is formally identical to (8) except that the equalities in the two cases must be interpreted in different senses.

Proof: First note that if

M N

~ ( t ) - Xm,n+m,n(t) m=K n=-N

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 39, NO. 1, JANUARY 1993

and m m

n=-N

l o o 1’

0 0 0 0 F P

IK,M,N( . f r t ) ) , = I I ( e 3 2 * f t -

-m -m -

. ( e - ~ ~ n f ’ ~ - I K , M , N v , t ) ) d ~ f , f ’ ) . (28)

Since the integrand in (28) is uniformly bounded (Lemma 3), and y(f, f’) is of bounded variation, the interchange of inte- gration and taking expectation is valid because of the Fubini’s theorem. Futhermore, we can apply the dominated conver- gence theorem and Theorem 1 to conclude that (28) converges to zero when M and N goes to infinity, which proves (23). The fact that the convergence is uniformly bounded follows because (28) is uniformly bounded for any N , M , and t. 0

Note that although the equality (23) is in the mean- square sense over the probability space, the equalities (24) and (25) hold with probability one because of the regularity condition and Elz(t)l < CO [20, p. S O ] . Theorem 2 says that given a harmonizable process ~ ( t ) , we can represent it without loss of inforhation by the coefficients {Xm,n, S K , ~ : m = K , K + 1,. . . ; finite K } . For any fixed m, Xm,n can be considered a discrete-time random process. Similarly, SK,, also constitutes a random process. In the actual computation of the coefficients, one can apply as in the deterministic case the pyramidal algorithm of Mallat [4]. That is, given S L , ~ ’ S for a certain L, we can compute

SL-l,n’s and XL- l ,n ’ s by passing through two digital filters followed by down-samplers. The procedure can be repeated recursively to compute all the wavelet coefficients Xm,n for m < L. The reconstruction of S L , ~ from S K , ~ and Xm,n (m = K , K + 1 , . . . , L - 1) can also be proceeded as described in [4].

Corollary 1: Let z ( t ) and its wavelet decomposition be as in Theorem 2. Then, the equality

M

n=-m

n=--03 n=-m

holds in the mean-square sense for any finite integer L and any t .

Proof: Letting K = 1; - 1 in (23), we have

m m

n=-w M c c

n=--00

o=

n=-m

Since (23) holds for any finite integer K , the second and third sums combined must equal in the mean-square sense to C n S L , n 4 L , n ( t ) . 0

Theorem 3: Let z ( t ) be a second-order harmonizable process. Assume that its spectral distribution y(., .) does not have a point mass at zero frequency, then the equality

M L-1 03

holds in the mean-square sense for any finite integer L and any t.

any t that Proof: In view of Corollary 1, it suffices to show for

WONG: WAVELET DECOMPOSITION OF HARMONIZABLE RANDOM PROCESSES

lim lim E K - - w N - w

~

13

S K , n 4 K , n ( t ) n=-N

To do so, consider

where 0 3 0 0

-w -w

. e j2s2-K( fn- f ’1> d d r ( f , f ’ ) . (31)

The foregoing exchange of integration and taking expectation is justified by the Fubini’s theorem. Then, we can write

I N 12

N N

= lim a K , n , 1 q h ( 2 K t - n ) $ ( 2 K t - 1 ) . (32) K+--m

Note that aK,n,l is bounded independently of K , n, and 1 because $(.) is bounded and -y(., .) is of bounded variation. We can then apply the dominated convergence theorem to in- terchange the limit and the infinite sums in (32). consequently, it remains to show aK,n,l + 0 as K + --oo for any n and 1. To this end, we use (31) and the dominated convergence theorem to write

-cc -w

-w -w

= IY(O+, ()+)I- h ( O + , 0-)I

-l7(0-, 0+) l+ l7(0-, ()-)I. Hence, if y(., .) has no point mass at the origin, the limit

0

Theorems 2 and 3 imply an alternative form of wavelet decomposition for a subclass of harmonizable processes as follows.

Theorem 4: If z ( t ) is a second-order harmonizable process with no spectral point mass at the origin, its wavelet decom- position can be represented as

vanishes and the theorem is proved.

w w

where Xmn, is defined in (24). The equality is in the mean- square sense, and the convergence in (33) is also uniformly bounded in the mean-square sense.

0

It is evident that the class of processes in Theorem 4 includes all wide-sense stationary processes having absolutely integrable autocorrelation functions, i.e., those. where power spectral densities exist. This is an important class of random processes frequently seen in engineering literatures. It is easy to give examples where (33) fails to be true if the autocorrela- tion function is not absolutely integrable. To this end, consider a random dc process z ( t ) = U where U is a binary random variable with probability distribution

Proof: Substitute (29) into (23).

PT{U = 1) = PT{U = -1} = 0.5.

We have for this process E z ( t ) = 0 for all t and E[z(t )z( t + T ) ] = 1 for all t and T . It can be easily verified that Xm,n = 0 for all m and n, while S K , ~ = U for all n and for any fixed K , confirming that (33) fails to hold for such a process. In general, if the limit in (30) is nonzero, then Theorem 3 is not valid and hence Theorem 4 also does not hold. If we trace through the proof of Theorem 3, we can observe from (31) that for any process where the spectral distribution r ( f , f ’ ) has a point mass at the origin, the sequence aK,n,l will not go to zero as K goes to negative infinity. Roughly speaking, this is because +)m,n( f ) equals zero at f = 0 for any m and n. As a result, no linear combination of q&(t) can represent a process that has a point mass at zero frequency. In such case, we must use a wavelet decomposition of the form (23).

Next, we consider the wavelet decompositions of the co- variance functions of harmonizable processes. The following theorem relates (23) to the wavelet decomposition of a covari- ance function.

Theorem 5: The wavelet decomposition of a second-order process z ( t ) exists, if and only if its covariance function has the decomposition

h

When they exist, the coefficients are related by -

am,m‘,n,n’ = E [ X m , n X m ’ , n ’ ] >

Vm,K,n,n’ = E [ X m , n T K , n ’ ] >

and

@K,n,n‘ = E [ S K , n s K , n ’ ]

14 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 39, NO. 1, JANUARY 1993

This theorem is a direct const, .ice of the convergence in quadratic mean (mean-square) L 1 1-rion [13, pp. 135-1361, and hence its proof is omitted.

In the context of signal processiiig, (41, [17], the wavelet decomposition can be interpreted a i i multistep linear filtering

end, we write

7 7 1 g g $ ( 2 - " f + k ) -W -W k = - m k ' = - m

and down-sampling procedure on the original process. Hence, if a process is harmonizable, i.e., if it can be represented as a superposition of sinusoids, the outpui of such a process through

. 4 ( 2 - m f ) 4 ( 2 - m f ' + k ' )

. G ( 2 - y ' ) ddy(f + 2"k, f' + 277%') -

linear filters and down-samplers should also be a superposition of sinusoids. The next theorem asserts this precisely.

Theorem 6: Let x ( t ) be a second-order continuous random process, and let its wavelet decomposition be formally given by (23). Let

xL( t ) = XL.n$tL,n(t)

= 7 7 I @ " f ) ( 2 4 ( 2 - " f - 1)

4 ( 2 - - f - k 9 $ ( 2 - " f ' ) 1 k = - m -m -cc

W ' ( k f I W

. Iddr(f, f". n=--00

and

n=-m

Then, the following statements are true.

Lemma 1 implies that the integrand in the double integral is bounded. Since y(., .) is of bounded variation, we can conclude that the spectral measure is of bounded variation, and hence x ~ ( t ) is harmonizable. Similarly, s ~ ( t ) is also harmonizable. Statement 2 follows from similar steps as before and the fact that

00

xm,n = X(t)$m,n(t) d t s 1) If x ( t ) is harmonizable, then i's wavelet decomposition 2. Furthermore, x ~ ( t )

+aonizable, then Z L ( ~ )

is valid in the sense of The1 and s ~ ( t ) are also harmon? 1 'or any L. -W

2) If for any fixed K , s ~ ( t ) and s ~ ( t ) for all L < K ai monizable. = s ̂SK(t)&&)dt, m < K ,

3) If for any fixed K , s ~ ( t ) is .tmonizable, and x ~ ( t ) ' s -m

are harmonizable for all L nizable.

K , then x ( t ) is harmo-

Proof: First consider statement 1. If x ( t ) is harmoniz- able, then Theorem 2 implies that its wavelet decomposition is valid. We can use (24) to write

Z L ( t ) = 2 2 " f t d < L ( f ) , L = K , K + 1,. . . , 7 Note that the equality holds with probability one because both (24) and (25) hold with probability one. It can be verified using the regularity of $(.) and the Fubini's theorem that we can interchange the order of integration and summation. We can then apply the Poisson sum formula to write

-m

and

-W

where < ~ ( f ) ' s and p ~ ( f ) are the spectral processes. Because of Corollary 1, and the fact that the sum of two harmonizable processes are harmonizable, we can write

4q-y - I )e--32-t

k=-cc

-W

. 4(2T"f) d<(f + 2' where

To conclude that x ~ ( t ) is harmoni we need to show that aded variation. To this the induced spectral measure is oi P L + l ( f ) = P L ( f ) +

WONG: WAVELET DECOMPOSITION OF HARMONIZABLE RANDOM PROCESSES 15

Letting E [ ~ L ( f ) p L ( f’)] = y~( f , f’), we want to show that m m

E [ s ~ ( t ) g ~ ( t ’ ) ] = 1 1 ej2r(f t - - f ’ t ’ ) d d y L ( f , f’) (34)

converges to a harmonizable covariance function. From Theorem 2, s ~ ( t ) converges boundedly to z ( t ) in mean- square. Hence, (34) is uniformly bounded for all L and t. Then it follows from a convergence theorem of Stieltjes integrals [14, pp. 3221 that

-m -m

lim E[sL( t )gL( t ’ ) ] = 7 7 ej2r( f t - - f ’ t ’ ) d d y ( f , f’), L-+m

-w -m

for some measure y( f , f’). Using the harmonizability theo- rem of Lobve [13, p. 1421, we can conclude that z(t) is harmonizable. 0

v. LINEAR OPERATIONS ON STOCHASTIC WAVELET DECOMPOSITIONS

In this section, we consider linear operations on the wavelet decomposition of harmonizable processes. The first result concerns the sum of two harmonizable processes.

Theorem 7: Let ~ ( t ) and y(t) be harmonizable processes with wavelet decompositions

M

n=-w m=K n=-w

and w m

m=K n=--03 n=--03

Then, the wavelet decomposition of their sum is

m=K n=-m W

n=-w

The proof of Theorem 7 is given in Appendix B. Theorem 7 says that the wavelet decompositions of two

harmonizable processes can be summed term by term. One can see by induction that the same conclusion holds true for a finite sum of harmonizable processes.

Next, we consider the derivative of harmonizable processes. Using the fact that y( f , f’) is of bounded variation, it can be shown that the derivative of z ( t ) exists in the mean-square sense, if and only if

m m

-m -m

Its wavelet decomposition is then given by the following theorem.

Theorem 8: Let z ( t ) be a harmonizable process with a spectral measure satisfying (35). Then its derivative X ( t ) exits in the mean-square sense, and its wavelet decomposition is given by

. . r M N

N

+ B K , n $ K , n ( t ) ] , any finite K , n=-N

(36)

where

-m

= 7 j2= f2-”/2$(2-” f ) e j 2 r f 2 - m d E ( f 1, -W

m = K , K + l , . . . ,

and

B K , n 2 7 k ( t ) q K , n ( t ) d t -W

= 7 j2=f2-K/2;(2-“ f ) e j 2 n f 2 - K n d t(.f ) * -m

Futhermore, the convergence in (36) is bounded in the mean- square sense.

The proof of this theorem is almost identical to that of Theorem 2, except that we need to use (35) to guarantee the existence of integrals. This theorem can be generalized in an obvious way for the wavelet decomposition of higher order derivatives of z ( t ) provided they exist and that (35) is suitably modified.

Finally, we consider the integration of harmonizable pro- cesses in the form of linear filtering. In particular, we consider the convolution integral

00

J h(t - T)Z(T) dT. (37) -W

We assume here that h(.) is absolutely integrable, which is both necessary and sufficient to ensure that the filter is bounded input bounded output stable [22, Section 2.61. Then, it can be shown [13, pp. 138-1401 that (37) exists in the mean-square sense, and the convolution integral can also be written as

7 e i 2 “ f t x ( f ) d [ ( f ) , (38) -m

where ;(.) is the Fourier transform of h(.). The next theorem concerns the convolution of a harmonizable process using the discrete wavelet decomposition.

16 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 39, NO. 1, JANUARY 1993

Theorem 9: Let h(t) be an absolutely integrable function, and z ( t ) be a harmonizable process with the wavelet decom- position given by (23). Then, the following equality holds in the mean-square sense:

W - M Jm h(t - r ) z ( T ) d T 1 xm,nAm,n( t ) m=K n=--M --M

W

(39) n=-w

where

and

That is, the convolution can be performed term by term using the wavelet decomposition.

The proof of Theorem 9 is given in Appendix C.

Notice that the convolutions of h(.) with $m,n(.) and +m,n(.) are completely determined once a wavelet function is chosen. In terms of system theory, Theorem 9 says that if a filter is bounded input bounded output stable, a method to filter a harmonizable process is as follows: First decompose the input process using the wavelet decomposition, and then simply form a linear combination according to (39) using the functions A,,,(t) and p ~ , ~ ( t ) . This result can be interpreted as term by term convolution of the wavelet decomposition. We should remark that the right-hand side of (39) is no longer a wavelet decomposition of the left-hand side because A,,, (t) and p ~ , ~ ( t ) are no longer dilations and translations of a wavelet function and a scaling function, respectively.

One special case of Theorem 9 worth mentioning is when h(.) in (37) is an indicator function, viz., h(t - T ) = l { a s T 5 b ) . In such case, the convolution integral becomes ssz(r) dT, i.e., the integral of z(t) over a finite interval, and the term by term integration of the wavelet decomposition is valid. It can be easily seen that the same result holds true for the integral Jsz ( t ) dt where S is a set of finite Lebesgue measure over the real line. On the other hand, one can also generalize Theorem 9 to a superposition integral of the form

Jm h( t ,r)x(r) dr , -W

where h ( t , ~ ) is the time varying impulse response of a linear system. In such case, we require h ( t , ~ ) be absolutely integrable over T for any fixed t. Although (38) is not valid and has no meaning in such case, Theorem 9 still holds when X,,,(t) and p ~ , ~ ( t ) are replaced by the corresponding superposition integrals.

VI. CONCLUSION We have considered in this paper the wavelet decomposition

of complex exponential functions, and showed in particular that the convergence is uniformly bounded. This result is then applied to establish the wavelet decomposition of second- order harmonizable processes. This class of processes are those that can be represented as a superposition of complex exponentials. These processes can be stationary, wide-sense stationary, or nonstationary ; and their spectral measures can either be continuous, discrete, or a mixture of both. It is shown that there are two different forms for representing the discrete wavelet decompositions of harmonizable processes that do not have point masses at zero frequency. An important example is the class of wide-sense stationary processes having power spectral densities. Only one of the two forms, however, is valid in providing a multiresolution decomposition of general harmonizable processes that has a discrete component at zero frequency.

We have also examined the addition, differentiation, and integration of stochastic wavelet decompositions. In particular, term by term addition of wavelet decompositions of harmo- nizable processes is justified. In the case of convolution, it is shown that if the filter impulse response is bounded input bounded output stable, then the output process can be directly obtained from the wavelet decomposition of the input process through an interpolation formula, which is a result of term by term convolution.

APPENDIX A PROOF OF LEMMA 3

We write M N

I I K , M , N U , ~ ) I 5 1 $ ( 2 - ~ f ) 1 1 1 ~ 1 ( 2 ~ t - n ) ~ m=K n=-N

N

n=-N W

m=K n=-w

The bounds for $J(.) and 4(.) follow from the regularity condi- tion. Since the infinite sums over n are bounded independently of m, K and t, it remains to show that

M

so = 14(2-mf)I (A.1) m=K

is bounded independently okf. First if f = 0, then the sum is identically zero because $(O) = 0. Next suppose f > 0.

17 WONG: WAVELET DECOMPOSITION OF HARMONIZABLE RANDOM PROCESSES

Pick and fix an arbitrary positive and finite number fl. For 0 < f 5 f 1 , we have from (A.l) and Lemma 1 that

so L C22-“f 5 C,2-”f1 = c , f 1 2 - K + 1 , m = K m = K

O < f I f l , which is bounded. For f > f l , let M I = \logz (f / f l ) ] where la] denotes the largest integer no greater than a. Then, we can apply Lemma 1 to give

5 C 1 2 ” ( l + ‘ ) 00

f 1 f E + C,2-mf m = K m=M1+1

1 + E cc

k=O

Note that M I = [log, (f / f l ) ] implies f / 2 f l < 2 M 1 I f / f 1 . Therefore,

f > f l , (A.3)

which is again bounded. Equations (A.2) and (A.3) show that SO is uniformly bounded for all f > 0. A similar argument shows that the same conclusion holds true for all .f < 0, and the lemma is proved. 0

APPENDIX B PROOF OF THEOREM 7

For any M and N , we write

M N

I m = K n=-N

2 l I 2 1

r I 2 1 l I2

Letting M and N go to infinity, we obtain the desired result. 0

APPENDIX C PROOF OF THEOREM 9

Let

N M N

Then, for any M and N , we write

M N 00

m m

-w -m

Since h(.) E L1 and J K , M , N ( ~ ) converges boundedly to ~ ( t ) , the Fubini’s theorem implies that the interchange of integration and taking expectation in the first equality is valid. Furthermore, the dominated convergence theorem implies that (C.l) converges to zero as M and N goes to infinity. U

ACKNOWLEDGMENT

The author is indebted to M. Genossar for his insightful comments, and for his suggestion in strengthening Theorem 3.

REFERENCES

[l] A. Grossmann and J. Morlet, “Decomposition of Hardy functions into square integrable wavelets of constant shape,” SIAM Math. Anal., vol. 15, pp. 723-736, July 1984.

[2] A. Grossmann, R. Kronland-Martinet, and J. Morlet, “Reading and understanding continuous wavelet transforms,” in Wavelets: Time- Frequency Methods and Phase Space, Proceedings of the International Conference, J. M. Combes, A. Grossmann, and P. Tchamitchian, Eds. Marseille, France.: Springer-Verlag, Dec. 1987, pp. 2-20.

[3] I. Daubechies, “The wavelet transform, time-frequency localization and signal analysis,” IEEE Trans; Inform. Theory, vol. 36, pp. 961-1005, Sept. 1990.

[4] S. G. Mallat, “A theory for multiresolution signal decomposition: The wavelet representation,’’ IEEE Trans. Pattern Anal. Machine Intell., vol. 11, pp. 674-693, July 1989.

[5] -, “Multiresolution approximations and wavelet orthonormal bases of L2(R),” Trans. Amer. Math. Soc., vol. 315, pp. 69-87, Sept. 1989.

[6] G. Battle, “A block spin construction of ondelettes, Part I: Lemarie functions,” Commun. Math. Phys., vol. 110, pp. 601-615, 1987.

[7] I. Daubechies, “Orthonormal bases of compactly supported wavelets,” Commun. Pure Appl. Math., vol. 41, pp. 909-996, Nov. 1988.

18 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 39, NO. 1, JANUARY 1993

181 Y. Meyer, “Orthonormal wavelets,” in Wavelets: Time-Frequency Meth- ods and Phase Space, Proceedings of the International Conference, J. M. Combes, A. Grossmann, and P. Tchamitchian, Eds. Marseille, France: Springer-Verlag, Dec. 1987, pp. 21-37.

[9] G. W. Womell, “A Karhunen-LoBve-like expansion for l/f processes via wavelets,” IEEE Trans. Inform. Theory, vol. 36, pp. 859-861, July 1990.

[IO] A. H. Tewfik and M. Kim, “Correlation structure of the discrete wavelet coefficients of fractional Brownian motion,” IEEE Trans. Inform. The- ory, vol. 38, pp. 904-909, Mar. 1992.

[ l l ] P. Flandrin, “Wavelet analysis and synthesis of fractional Brownian motion,” IEEE Trans. Inform. Theory, vol. 38, pp. 910-917, Mar. 1992.

[12] M. J. Genossar, M. Goldberg, H. Lev-Ari, and T. Kailath, “Extending wavelet decompositions to random processes with applications to pe- riodically correlated processes,” Tech. Rep. 91GGLK-l, Inform. Syst. Lab., Stanford Univ., Stanford, CA, Aug. 1991.

New York: Springer-Verlag, 1977.

Princeton, NJ: Princeton Univ. Press, 1959. (Translated from the original by M. Tenenbaum and H. Pollard).

[13] M. Lokve, Probability Theory II, 4th ed.

[14] S. Bochner, Lectures on Fourier Integrals.

[ 151 H. Ogura, “Spectral representation of a periodic nonstationary random process,” IEEE Trans. Inform. Theory, vol. IT-17, pp. 143-149, Mar. 1971.

[16] S. G. Mallat, “Multifrequency channel decompositions of images and wavelet models,” IEEE Trans. Acoust., Speech, Signal Processing, vol. 37, pp. 2091-2110, Dec. 1989.

[I71 0. Rioul and M. Vetterli, “Wavelets and signal processing,” IEEE Signal Processing Mag., pp. 14-38, Oct. 1991.

[18] M. J. Lightbill, Introduction to Fourier Analysis and Generalised Func- tions, Cambridge Monographs on Mechanics and Applied Mathematics. London, U K Cambridge Univ. Press, 1958.

[I91 M. Holschneider and P. Tchamitchian, “Regularite locale de la fonction ‘nondifferentiable’ de Riemann,” in Les Ondelettes en 1989: Seminaire d’Analyse Harmonique, P. G. Lemarie, Ed. Berlin: Springer-Verlag, 1990, pp. 102-124.

[20] E. Wong, Stochastic Processes in Information and Dynamical Systems. New York: McGraw-Hill, 1971.

[21] H. Cram& and M. R. Leadbetter, Stationary and Related Stochastic Processes.

[22] T. Kailath, Linear Systems. Englewood Cliffs, NJ: Prentice-Hall, Inc., 1980.

New York: John Wiley, 1967.