ON WAVELETS AND MIT OCW 18.327 WAVELETS, FILTERS, AND APPLICATIONS

ERNEST YEUNG

Abstract. I wanted to expound here, from an introductory and pedagogical starting point, on wavelets. Note,

my goal here is not to reinvent the wheel. This is meant to be the “missing companion manual” to the MIT

OCW 18.327 http://ocw.mit.edu/courses/mathematics/18-327-wavelets-filter-banks-and-applications-spring-2003/

material and to any book or course on or involving Wavelets, with (one of) the value being the Python code

closely linked with each concept.

One day on 18 juin 2015, I find myself needing to learn about wavelets, fast. Here are notes, solutions to

various exercises and “textbook” problems, and associated code and code snippets that I found useful in

learning about and applying wavelets.

The first thing I now do when I want to pick up a new skill is to see if the MIT (Massachusetts Institute of

Technology) OCW (Open-Course Ware) has a course on it. So far, I’ve seen 18.327 Wavelets, Filter Banks

and Applications, taught by Strang and Amaratunga in Spring 2003 [1] (http://ocw.mit.edu/courses/

mathematics/18-327-wavelets-filter-banks-and-applications-spring-2003/). I haven’t seen it up-

dated recently. I’ll try to give some updates from my perspective. (Note, this past week, 20150709, I saw that

coursera has Computational Methods for Data Analysis; I’ll try to look into that).

Here are some features I want to mention about this exposition on Wavelets:

• EY : 20150720 I wanted to raise the issue about the wavelet and scaling function coefficients at each jth

resolution. I wanted to make resolve the distinction between doing the Fourier transform or essentially

z-transform on the time t-domain (keep in mind that this could also be the spatial domain) and what

it means when parts of the filter bank “acts” on the coefficients themselves.

My original thought was this (and please give any and all (negative) feedback on where this might

fail); with filtration being a collection of nested subspaces, i.e. V 0 ⊂ V 1 ⊂ V 2 ⊂ . . . (I was also

curious if we can think of multiresolution analysis in terms of topological spaces):

Multiresolution analysis as a filtration of L2(R)-algebras, with each subspace V j equipped

with a k-algebra, kj, that is an algebra for the coefficients of the j-level of resolution

scaling function and wavelets. I explicitly make clear what spaces we are living on and the

mapping involved and make explicit the mapping itself in the so-called filter bank representation with

commutative diagrams 5.2, 7.1.1. The operations of convolution and downsampling and upsampling

on L2(R) functions are clearly defined as an algebra and the corresponding operations in the so-called

z-domain, which is really going to L2(C), is also clearly defined and the correspondence or “dictionary”

between L2(R) and L2(C) clearly shown.

Date: 4 juillet 2015.

1991 Mathematics Subject Classification. Wavelets.

Key words and phrases. Wavelets.

If you like this pdf and LaTeX file and/or find it useful, please consider making a financial contribution to the crowdfunding

campaign I am running at Tilt/Open. Funds will go into new computer equipment so I could push a number of numerical

simulations I want to try out in computational fluid dynamics, first, and quantum super-A-polynomials, which I had written

my Masters thesis on, pay myself, and in general, go into a general fund for supporting basic research in the future: one of my

ultimate goals is to directly fund theoretical and mathematical physics research with the biggest private and crowdsourced funds

seen.

1

Then, with scaling functions and wavelets in a multiresolution analysis, I define a k-algebra, kj ,

associated with each subspace V j , for the j-level resolution scaling function and wavelet coefficients.

This way, we know exactly which (sub)space (V j or V j+1, or kj or kj+1) we are living on at each

step of a filter bank and again, the operations of convolution and downsampling and upsampling are

reduced to an algebra (which is easy).

• Explicit association of where concept and theory meets application and code. Learning

and using wavelets should be as fun as playing video games. The original MIT OCW 18.327 material

does not tell you which piece of code corresponds to directly with which Handout or Lecture. In this

pdf/LaTeX file, I do tell you, and being able to directly do the Wavelet transforms and plot them

immediately and interactively helps out a lot in learning and using wavelets. Also, all the code is on

github, making it more amenable to editing and playing around with, and won’t get dated like the

MIT OCW 18.327 materials.

• Python. The Matlab Wavelet Toolkit is a black box, as exemplified by the documentation 1. We

can do everything we’d want to and more in Python, using the open-source PyWavelets library. With

Homebrew and pip, installing packages is easy; I’ve found that installing packages in GNU Octave

isn’t as easy.

• Again, I want to emphasize that we are not trying to reinvent the wheel. There are plenty of reading

material (I should say that the original lectures by Ingrid Daubechies [5] are excellent and lucid, and

she founded the field) and plenty of disparate course lecture notes online. This is meant to be the

“missing companion manual” to the MIT OCW 18.327 material and to any book or course on or

involving Wavelets, building on top of the MIT OCW 18.327 material. (One of) The value of these

notes is the closely linked Python code, and I want to encourage and foster its use, interactively.

For more exposition (that’d usually go into an “Introduction” section) and the rationale behind all this notes

and code, and how you can help, go to the end part for the Colophon 3.

Part 1. MIT OCW 18.327 Spring 2003 3

Lecture Notes 3

1. Discrete-time Filters: Convolution; Fourier Transform; Lowpass and Highpass Filters: Handout 1 3

2. Sampling Rate Change Operations: Upsampling and Downsampling; Fractional Sampling;

Interpolation: Handout 2 6

3. Filter Banks: Time Domain (Haar example) and Frequency Domain; Conditions for Alias

Cancellation and no Distortion; Handout 3 9

4. Filter Banks (contd.): Perfect Reconstruction; Halfband Filters and Possible Factorizations 14

5. Modulation and Polyphase Representations: Noble Identities; Block Toeplitz Matrices and Block

z-transforms; Polyphase Examples 14

6. MATLAB Wavelet Toolbox 17

7. Orthogonal Filter Banks: Paraunitary Matrices; Orthogonality Condition (Condition O) in the

Time Domain, Modulation Domain and Polyphase Domain 17

8. Maxflat Filters: Daubechies and Meyer Formulas. Spectral Factorization 18

9. Multiresolution Analysis (MRA): Requirements for MRA; Nested Spaces and Complementary

Spaces; Scaling Functions and Wavelets: Handout 9 22

10. Refinement Equation: Interative and Recursive Solution Techniques; Infinite Product Formula;

Filter Bank Approach for Computing Scaling Functions and Wavelets: Handout 10 26

11. Project Brief 29

12. Orthogonal Wavelet Bases: Connection to Orthogonal Filters; Orthogonality in the Frequency

Domain. Biorthogonal Wavelet Bases 33

1http://www.mathworks.com/help/wavelet/ref/upcoef.html

2

13. Mallat Pyramid Algorithm; Handout 12 33

14. Accuracy of Wavelet Approximations (Condition A); Vanishing Moments; Polynomial Cancellation

in Filter Banks 35

15. Smoothness of Wavelet Bases: Convergence of the Cascade Algorithm (Condition E); Splines.

Bases vs. Frames 35

16. Signal and Image Processing: Finite Length Signals; Boundary Filters and Boundary Wavelets;

Wavelet Compression Algorithms 35

17. Guest Lecture. Physical Wavelets and their Sources: Real Physics in Complex Spacetime 36

18. Lifting: Ladder Structure for Filter Banks; Factorization of Polyphase Matrix into Lifting Steps;

Lifting Form of Refinement Equation; Handout 16 36

19. Wavelets and Subdivision: Nonuniform Grids; Multiresolution for Triangular Meshes;

Representation and Compression of Surfaces; Slides and Handouts 17,18 38

20. Numerical Solution of PDEs: Galerkin Approximation; Wavelet Integrals (Projection Coefficients,

Moments and Connection Coefficients); Convergence. Subdivision Wavelets for Integral

Equations. Compression and Convergence Estimates 39

21. M-band Wavelets: DFT Filter Banks and Cosine Modulated Filter Banks. Multiwavelets 39

Part 2. Extra fun 40

22. Fun with Quandl 40

Part 3. Colophon, Rationale, PitchDeck, and Help Me and Help Others 41

23. Why Wavelets (for me)? 41

24. The Problem (and the Solution) 42

References 46

Contents

Part 1. MIT OCW 18.327 Spring 2003

Lecture Notes

Here are my notes (and my personal perspectives) on the Slides and Handouts for the Lecture Notes in

[1].

READINGS refer to the Text: Strang and Nguyen. Wavelets and Filter Banks. Wellesley-Cambridge

Press, 1997.[2]

1. Discrete-time Filters: Convolution; Fourier Transform; Lowpass and Highpass Filters:

Handout 1

Readings: Sec 1.1-1.4, 2.1 [2]

Let x : R→ V

x(t) ∈ V

, where V vector space, or R-module.

Let y : R→ V

y(t) ∈ V

.

x represents input, as a function of time t, and y represents output, as a function of time.3

Let linear L : V → V , L ∈ L(V ;V ) be a linear transform (automorphism?) s.t. Lx = y.

Note for a “time delay” δ ∈ R,

Lx(t− δ) = y(t− δ)

Let x(t) = δ(t), with δ being the Dirac Delta function (in this context). Then

Lx(t) = Lδ(t) = h(t)

where h, as defined as such immediately above, is the “response.”

Then for x(t) =∑∞τ=−∞ x(τ)δ(t− τ)

Lx(t) =

∞∑τ=−∞

x(τ)h(t− τ)

Recall the L1 Fourier transform, such that for x ∈ L1(Rn), with n = 1 for time, in this case [3],

(1.1) x(ω) =

∫ ∞−∞

x(t)e−iωtdt =

∞∑n=−∞

x(n)e−iωn

and the “inverse”, (the inverse is well defined for L2; EY : 20150619, correct me if I’m wrong)

(1.2) x(t) =1

2π

∫ π

−πx(ω)eiωtdω

Convolution. define convolution of f, g on Rn

Definition 1.

(1.3) f ∗ g(x) =

∫Rnf(x− y)g(y)dy

Note f ∗ g = g ∗ f by change of variables.

Now

y(t) =

∫ ∞−∞

x(τ)h(t− τ)dτ =

∫ ∞−∞

dτ1

2π

∫ π

−πx(ω)eiωτdωh(t− τ)e−iωte−ωt =

1

2π

∫ π

−πx(ω)h(ω)eiωtdω

=⇒ y(ω) = x(ω)h(ω)

Discrete case: if f, g are discrete, i.e. f = f(i)

g = g(j)

, i, j ∈ Z, then

∫ ∞−∞

dyf(x− y)g(y) =

∞∑k=−∞

f(n− k)g(k) = (f ∗ g)(n)

Toeplitz Matrix. The so-called Toeplitz Matrix representation is thus (I believe): now

(1.4) y(t) =

∞∑τ=−∞

x(τ)h(t− τ) =

∞∑τ=−∞

h(t− τ)x(τ)

Let τ become matrix index j.

Let t become matrix index i.

Let htoep = (htoep)ij be the so-called Toeplitz matrix. Now from Eq. 1.4,

(htoep)ijxj = h(t− τ)xj .

So (htoep)ij = h(i− j) are the matrix entries.4

One should consider causality : causality means that t − τ ≥ 0 (otherwise, for t = 0, for say τ = 1, x(τ = 1)

causes a response for y(t = 0).

Examples.

(a) Low-pass Filter:

(1.5) y(t) =1

2x(t) +

1

2x(t− 1) =

∞∑τ=−∞

h(t− τ)x(τ)

with h(0) =1

2

h(1) =1

2

and 0 otherwise.

pp. 8 of Sec. 1.2 Lowpass Filter = Moving Average of Strang and Nguyen [2] defines the acronym

FIR, finite impulse response. Again, consider this low-pass filter example immediately above:

∞∑τ=−∞

h(t− τ)x(τ) =

∞∑n=−∞

h(n)x(t− n) = y(t) =1

2x(t) +

1

2x(t− 1)

for n = t− τ

τ = t− nSo h(0) = h(1) = 1

2 . Else, h(n) = 0.

So note that so-called “H” can be rewritten in many ways.

For H : L2(R)→ L2(R), defined as

∞∑τ=−∞

h(t− τ)x(τ) =

∞∑n=−∞

h(n)x(t− n) = y(t)

if h(n) 6= 0 only for finite number of n’s, then H = h(n) is a finite impulse response (FIR) filter.

Then

h(ω) =

∫ ∞−∞

h(t)e−iωtdt =1

2+

1

2e−iω

(imagine doing Fourier transforms of Diract delta functions; they “pick off” values, -Prof. Alan J.

Weinstein, Caltech Ph 12a)

(1.6) =⇒ h(ω) =1

2(1 + e−iω) = cos

(ω2

)e−iω/2

(b) High pass filter:

y(t) =1

2x(t)− 1

2x(t− 1)

h(t) =1

2δ(t)− 1

2δ(t− 1)

h(ω) =1

2− 1

2e−iω = i sin

(ω2

)e−i

ω2 = sin

(ω2

)e−i

(ω−π)2

Why they’re called low-pass and high-pass filters, I think, is because of this: consider |h(ω)|. Then for the

low-pass filter, |h(ω)| = cosω/2 so that low frequencies near 0 have a large magnitude of h relative to high

frequencies near ω = π. Vice-versa for the high-pass filter with |h| = sinω/2.5

Python examples. At this point, I would run example1.py and type into the Python prompt plt.show()

and look at Figure 1 for the Haar lowpass filter and Figure 2 for the Haar lowpass filter.

Notice the use of numpy’s fft in fft module. All we’re doing is 1.1.

If you’re wondering what the other “square” shaped graphs in the Handout 1 was (as I did), run Handout_examples.py

(doing python -i Handout_examples.py in the directory of tools) and do plt.show() and look at Figure 3

and Figure 5. It’s the phase or “angle” of the complex valued f as a function of ω.

Readings: Sec 1.1-1.4, 2.1 [2]

2. Sampling Rate Change Operations: Upsampling and Downsampling; Fractional Sampling;

Interpolation: Handout 2

READING: Sec 3.1-3.3 [2]

Downsampling by M .

(2.1) y(t) = x(Mt)

Apply the Fourier transform, Eq. 1.1, to both sides:

y(ω) =

∞∑n=−∞

x(Mn)e−iωm =∑

m=nN

x(m)e−iωmM

At this point, we need an identity, namely,

(2.2)1

M

M−1∑k=0

(e−i2πM m)k =

1 if m = nM

0 otherwise

For the first few cases:

M = 1,

(e−i2πm)0 = 1 m = n

M = 2

1

2

1∑k=0

(e−iπm)k =1

2(1 + (−1)m) =

1 if m = 2n

0 otherwise

M = 3

1

3

2∑k=0

(e−i2πm3 )k =

1

3(1 + e−i

2πm3 + e−i2πm

(2)3 ) =

1 if m = 3n

0 otherwise

However, I think the = 0 case is nontrivial. The m = nM case, leading to a sum of 1 is clear (multiples of 2π

in the argument yields 1 for e−i2πM m = e−i2πn = 1). But 0 otherwise?

I suspect that this should have something to do with, in complex analysis, when integrating around a circle,

if there are no poles, then the integral (or summation) is 0. I’d like to hear someone show this to me, (very)

explicitly.

Otherwise, inserting in Eq. 2.2, by the clever trick of multiplying by 1,

(2.3)

y(ω) =∑

m=nM

1

M

M−1∑k=0

(e−i2πM m)kx(m)e−iω

mM =

∑m=nM

1

M

M−1∑k=0

e−i2πkmM

1

2π

∫ π

−πdωx(ω′)eiω

′me−iωmM =

=1

M

M−1∑k=0

∫ π

−πdω′x(ω′)

1

2π

∑m=nM

eim(ω′− ωM−

2πkM ) =

1

M

M−1∑k=0

x(ω + 2πk

M)

6

Upscaling by L. Apply the Fourier transform, Eq. 1.1, to both sides:

y(t) = x

(t

L

)

(2.4)

y(ω) =

∞∑t=−∞

y(t)e−iωt =

∞∑m=−∞

x(m)e−iωmL =1

2π

∫ π

−πdω

∞∑m=−∞

x(ω′)eiω′me−iωmL =

=1

2π

∫ π

−πdωx(ω′)

∞∑m=−∞

exp (im(ω′ − ωL)) =

∫ π

−πdωx(ω′)δ(ω′ − ωL) = x(ωL)

EY : 20150620, in Handout 2, I find the asymmetry in the downsampling and upscaling scaling (M vs. L)

disconcerting (Eq. 2.3, Eq. 2.4). Why y(ω) = 1M x( ωM ) for downsampling, but y(ω) = x(ωL) for upscaling (no

constants in front)?

Interpolation. The interpolation filter is applying a low pass filter, after a downsampling by M (cf. Eq.

(2.1), Eq. (2.3)). I don’t know if there’s a general form of a low pass filter, so I’ll use the basic example in

Eq. 1.5, Eq. 1.6.

(1) Apply downsampling by M (↓M):

y(t) = x(Mt)

y(ω) =1

M

M−1∑k=0

x

(ω + 2πk

M

)

(2) Apply low pass filter

h(t) =1

2(x(t) + x(t− 1))

h(ω) = cos(ω

2

)e−iω/2 =

1

2x(ω)(1 + e−iω)

For the very last equation, think of the superposition property of the Fourier transform.

Then for interpolation filter

h ◦ y(t) =1

2(x(Mt) + x(M(t− 1))

h ◦ y(ω) =1

M

M−1∑k=0

1

2x

(ω + 2πk

M

)(1 + e−iω)

Fractional Sampling. I’ll employ this notation for downsampling by M (↓M) and upsampling by L (↑ L),

so that

(2.5)

(↓M) : (↓M)x(t) = x(Mt)

(↓M)x(ω) =1

M

M−1∑k=0

x

(ω + 2πk

M

)

(↑ L) : (↑ L)x(t) = x

(t

L

)(↑ L)x(ω) = x(ωL)

7

Consider

↓M◦ ↑ Lx(t) = x

(Mt

L

)↓M ◦ ↑ Lx(ω) =

1

M

M−1∑k=0

x

(ωL

M+

2πk

M

)=

1

M

M−1∑k=0

x

(L

M(ω +

2πk

L)

)I think this is “synthesis”, “decompressing”, “fractional synthesis.”

↑ L◦ ↓Mx(t) = x

(Mt

L

)↑ L ◦ ↑Mx(ω) =

1

M

M−1∑k=0

x

(L

M(ω + 2πk)

)I think this is “analysis”, “compressing”, or “fractional sampling.”

Look at how the immediately above two blocks of equations are different in the frequency ω domain. The

question to ask, in light of Theorem 3.6 of Strang and Nguyen (1996) [2] is when does ↓ M and ↑ L com-

mute?

Look at the phases in the frequency domain.

We know that x doesn’t change because of the periodicity in ω by 2π. Then, examine the phase of ↓M ◦ ↑ Land ↑ L ◦ ↓M , respectively:

L

M(ω +

2πk

L) =

L

M(ω +

2πk

L+ 2πl)

L

M(ω + 2πk) =

L

M(ω + 2πk + 2πm)

with k = 0 . . .M − 1 and l,m ∈ Z. We see that if M ,L are relatively prime, i.e. M/L, M divided by L doesn’t

yield an integer, the greatest common factor is 1. k < M , so k/L is an integer, L being a multiple of one of

k = 0 . . .M − 1.

At this point, run example1.py by typing in python -i example1.py (where -i is the “flag” to get to the

interactive prompt). To look at that the plots, type in the prompt plt.show().

Open up example1.py in your favorite text editor (I use emacs; you can use TextEdit on a Mac OSX) and

compare the plots with the comments (with a hashtag # in Python) and axis titles and you’ll see what’s going

on. One should note that in the frequency domain, we’re effectively changing the L or M factor multiplying ω

(cf. Eq. 2.5), and that translates to in Python indexing or “slicing.” (take a look at the code in example1.py

for Downsampling)

2.1. On the associated Reading for Handout 2, Sec.3.1-3.3 [2]. READING: Sec 3.1-3.3 [2]

cf. Sec. 3.3. “Sampling Operations in the z-Domain.

On pp. 97, Theorem 3.5, here’s some clarification with some algebra [2].

This is true

(↓M ◦ x)(ω) =1

M

M−1∑k=0

x

(ω + 2πk

M

)

=⇒ (↑M ◦ ↓M ◦ x)(ω) = (↓M ◦ x)(Mω) =1

M

M−1∑k=0

x(ω +2πk

M)

In the z-transform representation, one obtain ∀ k ∈ Z

x(ω +2πk

M) =

∫ ∞−∞

dt exp (−i(ω +2πk

M)t)x(t) =

∫ ∞−∞

(zei2πk/M )−tx(t) = x(zei2πk/M )

8

with z = eiω and for k ∈ Z.

3. Filter Banks: Time Domain (Haar example) and Frequency Domain; Conditions for Alias

Cancellation and no Distortion; Handout 3

READING: Sec 4.1 [2]

Consider e.g. M = 2

y(t) = x(2t)

htj = δ(2t− j) = δ(t− j2 )

(3.1)

(↓ 2)h0 = δ(t− k

2)

1√2

(δ(k − τ) + δ(k − τ − 1)) =1√2

(δ(2t− τ) + δ(2t− (τ + 1))

(↓ 2)h1 =1√2

(δ(2t− τ)− δ(2t− (τ + 1))

Now

((↓ 2)h0)tτxτ = ((↓ 2)h0)tτ

...

x(−2)

x(−1)

x(0)

x(1)

x(2)...

=

1√2

...

x(−2) + x(−3)

x(0) + x(−1)

x(2) + x(1)...

Then

((↓ 2)h0)tτ =1√2

. . .

1 1

1 1

1 1. . .

So also

((↓ 2)h1)tτ =1√2

. . .

−1 1

−1 1

−1 1. . .

(↑ L)

y(t) = x(tL

)= htjxj if j =

t

L, then htj = 1

else, 0

So

htj = δ(t

L− j) = δ(t− jL)

9

so for example, e.g. L = 2,

(↑ 2)tjxj = (↑ 2)

...

x(−2)

x(−1)

x(0)

x(1)

x(2)...

=

...

x(−1)

0

x(0)

0

x(1)...

Now consider

h0(↑ L) =1√2

(δ(t− k) + δ(t− k − 1))δ(k − jL) =1√2

(δ(t− jL) + δ(t− jL− 1))

=⇒ h0(↑ L) =1√2

(δ(t− jL) + δ(t− jL− 1))

e.g.

(h0(↑ 2))tjxj = (h0(↑ 2))tj

...

x(−1)

x(0)

x(1)...

=

1√2

...

x(−1)

x(−1)

x(0)

x(0)

x(1)...

Back to the reading of the 3rd Handout and Slides. I didn’t understand it the first time around reading it:

let’s gather up what we know so far.

The handout begins with the Haar Filter Bank slide. I’m assuming that the example presented is called the

Haar Filter Bank (does it have anything to do with the Haar wavelet? is my question).

2-channel FIR - FIR is Finite Impulse Response; 2 channel (I think it’s called 2-channel because of the low

pass and high pass filter for low and high frequencies, respectively).

Define a linear mapping H : L2(R)→ L2(R such that ∀x = x(t) ∈ L2(R),

Hx(t) = y(t) =

∞∑τ=−∞

h(t− τ)x(τ)

It takes a number of forms or representations:

Hx(t) =

∞∑τ=−∞

h(t− τ)x(τ) =

∞∑n=−∞

h(n)x(t− n) =

∞∑τ=−∞

htτxτ

where htτ is the so-called Toeplitz matrix,

with t− τ ≥ 0 for causality, or n ≥ 0.

Because of the convolution property, that y(ω) = x(ω)h(ω) for y = h ∗ x = x ∗ h, then

y = hx10

Let’s collect the facts that we know about downsampling (↓M) and upsampling (↑ L).

(↓M)x(t) = x(Mt) =

∞∑τ=−∞

δ(Mt− τ)x(τ) =

∞∑n=−∞

δ(n− (1−M)t)x(t− n)

(↑ L)x(t) = x

(t

L

)=

∞∑τ=−∞

δ

(t

L− τ)x(τ) =

∞∑n=−∞

δ(n− (1− 1

L)t)x(t− n) =

∞∑τ=−∞

δ(t− Lτ)x(τ)

Take the simple example for a low pass filter (EY: 20150630 are there other kinds of low pass filters of note

to consider?)

h0x(t) =1√2x(t) +

1√2x(t− 1) =

1√2

∞∑τ=−∞

(δ(t− τ) + δ(t− 1− τ))x(τ) =1√2

∞∑n=−∞

(δ(n) + δ(n− 1))x(t− n)

Take the simple example for a high pass filter

h1x(t) =1√2x(t)− 1√

2x(t− 1) =

1√2

∞∑τ=−∞

(δ(t− τ)− δ(t− 1− τ))x(τ) =1√2

∞∑n=−∞

(δ(n)− δ(n− 1))x(t− n)

So the so-called Haar Filter Bank is just

f0 ◦ (↑ 2) ◦ (↓ 2) ◦ h0 + f1 ◦ (↑ 2) ◦ (↓ 2) ◦ h1 : L2(R)→ L2(R)

with (↓ 2) ◦ h0(↓ 2) ◦ h1

being “analysis” and

f0 ◦ (↑ 2)

f1 ◦ (↑ 2)

being “synthesis”

Instead of doing operations from right to left, one can write the operations going from left to right by “filter

bank” diagram. I’ll write it as a commutative diagram instead. For M = 2

x(t) r0(t) y0(t) t0(t) v0(t)h0 (↓ 2) (↑ 2) f0

x(t) r1(t) y1(t) t1(t) v1(t)h0 (↓ 2) (↑ 2) f0

and then x′(t) = v0(t) + v1(t).

Now work it out: from Eq. 3.1,

(↓ 2)h0x(t) =1√2

(x(2t) + x(2t− 1))

(↓ 2)h1x(t) =1√2

(x(2t)− x(2t− 1))

(↑ 2)(↓ 2)h0x(t) =1√2

(x(2t))

(↑ 2)(↓ 2)h1x(t) =1√2

(x(2t))

f0(↑ 2)(↓ 2)h0x(t) =1

2(x(t) + x(t− 1)) = v0(t)

f1(↑ 2)(↓ 2)h1x(t) =1

2(x(t)− x(t− 1)) = v1(t)

with f0 = h0, f1 = h1. So then x′(t) = v0(t) + v1(t) = x(t). “Perfect reconstruction.”

For x(t) ∈ L2(R), suppose

x(t) =

∞∑τ=−∞

x(τ)δ(t− τ)

11

The z-transform is simply the Fourier transform of x(t):

(3.2) x(ω) =

∫ ∞−∞

dte−iωt∞∑

τ=−∞x(τ)δ(t− τ) =

∞∑τ=−∞

x(τ)e−iωτ =

∞∑τ=−∞

x(τ)z−τ = x(z)

for z = eiω.

Then the z-transform of downsampling ↓ M and upsampling ↑ L is the following (cf. Sec. 3.3 Sampling

Operations in the z-Domain [1]):

(3.3)

(↓M)x(z) =1

M

M−1∑k=0

x

(ω + 2πk

M

)=

1

M

M−1∑k=0

∞∑τ=−∞

x(τ)e−i(ω+2πkM )τ =

1

M

∞∑τ=−∞

x(τ)

M−1∑k=0

(ei2πkM z1/M )−τ =

=1

M

M−1∑k=0

x(ei2πkM z

1M )

(3.4) (↑ L)x(z) = x(Lω) =

∞∑τ=−∞

x(τ)e−iLωτ =

∞∑τ=−∞

x(τ)(zL)−τ = x(zL)

For the simple example of a low pass filter h0, recall that h0x(t) = 1√2(x(t) + x(t− 1)) and

x(t) =∑∞τ=−∞ x(τ)δ(t− τ)

Now

H0x(ω) :=

∫ ∞−∞

dte−iωth0x(t) =1√2

∫ ∞−∞

dte−iωt

( ∞∑τ=−∞

x(τ)δ(t− τ) + x(τ)δ(t− 1− τ)

)=

=1√2

∞∑τ=−∞

x(τ)e−iωτ + x(τ)e−iω(1+τ) =

=1√2x(z)

(1 +

1

z

):= H0x(z)

Then for for the simple example of a high pass filter, h1,

H1x(z) =1√2x(z)

(1− 1

z

)

DTFT - discrete-time Fourier Transform.

The perfect reconstruction condition is

x′(t) = x(t− δ)

where δ is the time delay.

∫ ∞−∞

dte−iωtx(t− δ) =

∫ ∞−∞

dte−iωt∞∑

τ=−∞x(τ)δ(t− δ − τ) =

∞∑τ=−∞

x(t− δ)e−iω(δ+τ) = z−δx(ω)

So

x′(z) = z−δx(z)

12

3.1. Aliasing, alias. Let’s consider this example. Suppose

constant x1(t) =∑n∈Z

δ(t− n)

x1(ω) =∑n∈Z

e−iωn

alternating x2(t) =∑n∈Z

(−1)nδ(t− n)

x2(ω) =∑n∈Z

(−1)ne−iωn =∑n∈Z

e−i(ω−π)n

Apply downsampling (↓ 2).

(↓ 2)x1(t) = x1(2t) =∑n∈Z

δ(2t− n) =∑j∈Z

δ(2(t− j))

(↓ 2)x2(t) = x2(2t) =∑n∈Z

(−1)nδ(2t− n) =∑j∈Z

(−1)2jδ(2(t− j)) =∑j∈Z

δ(2(t− j)) = (↓ 2)x1(t)

So x2(t) is the “alias” of x1(t) with respect to (↓ 2).

So, I want to try to generalize this definition of “aliasing”:

∀x1(t), x2(t) ∈ L2(R), (↓M) : L2(R)→ L2(R),

if (↓M)x1 = (↓M)x2 then x1, x2 are aliases with respect to ↓M

It was interesting to me to consider the frequency domain:

(↓ 2)x1(ω) =1

2

2−1∑k=0

∑n∈Z

e−i(ω+2πk

2 )n =1

2

1∑k=0

∑n∈Z

exp(−i(ω

2+ πk

)n)

=1

2

∑n∈Z

e−iωn2 (1 + e−iπn)

(↓ 2)x2(ω) =1

2

2−1∑k=0

∑n∈Z

e−i(ω+2πk

2 −π)n =1

2

1∑k=0

∑n∈Z

exp(−i(ω

2+ πk − π

)n)

=1

2

∑n∈Z

e−iωn2 (1 + eiπn)

Since(1 + e−iπn)

2

eiπn/2

eiπn/2=

cos(πn2

)eiπn/2

= (−1)m/(−1)m = 1

1 + eiπn

2

e−iπ2 n

e−iπn/2=

cos(πn2

)e−iπn/2

= (−1)m/(−1)m = 1

where n = 2m and m ∈ Z. So in the frequency domain, they’re equal too in this example.

3.2. wkeep. We’re going to try to mimic the behavior of Matlab’s wkeep 2

Let length of list by N0 ∈ Z

N0 ∈ Z so either N0 =

2K0 even

2K0 − 1 oddK0 ∈ Z

Suppose we want l ordered elements of the list l ∈ Z, so l =

2p even

2p− 1 oddp ∈ Z

Suppose N0 = 2K0 − 1

l = 2p− 1

(odd, odd case)

N0 = 2(K0 − 1) + 1

(K0 − 1 is what modulus arithmetic would give you, i.e. [N0/2] = K0 − 1, N0%2 = 1)

2MathWorks Documentation, wkeep Keep part of vector or matrix http://www.mathworks.com/help/wavelet/ref/wkeep.html

13

l = 2(p− 1) + 1

On the “left” side of the center, there are K0−1 “ordered elements and we want the last p−1 elements

K0 − 1− (p− 1) = K0 − p

So start counting the elements to choose from position K0 − p+ 1, counting from 1.

So if we start counting from 0, we want the K0 − p position. K0 − p = (K0 − 1)− (p− 1)

on the “right” side of center,

center’s position is at K0 − 1 + 1 = K0 (counting from 1). Counting from 0, K0 − 1 is the position.

K0 + (p− 1) is position of last element to pick off (counting from 1). K0 + (p− 1)− 1 is the position of the

last element to pick off (counting from 0).

if l = 2p,

let’s try to mimic what wkeep does in Matlab. Looking at the MathWorks Documentation, it seems that if we

pretend that we eliminate the last element of our list, so that the length is now N0−1 = 2K0−2 = 2(K0−1),

then we’d now want the center piece of length l.

start at (K0 − 1)− p+ 1 and last element’s position at K0 + p− 1. (counting from 1)

If N0 = 2K0

l = 2p

, start at K0 − p+ 1; last element’s position at K0 + p (counting from 1)

if l = 2p− 1 = 2(p− 1) + 1, let’s mimic what wkeep does in Matlab. Pretend we eliminate the last element of

our list, so that the length is now N0 − 1 = 2K0 − 1 = 2(K0 − 1) + 1

start at (K0 − 1)− (p− 1) + 1 = K0 − p + 1; last element’s position at (K0 − p) + (2p− 1) = K0 + p− 1

(counting from 1).

4. Filter Banks (contd.): Perfect Reconstruction; Halfband Filters and Possible

Factorizations

READING: Sec 4.1 [2]

5. Modulation and Polyphase Representations: Noble Identities; Block Toeplitz Matrices

and Block z-transforms; Polyphase Examples

READING: Sec 3.4, 4.1-4.4 [2]

5.1. On the Readings.

5.1.1. Noble Identities. cf. Sec. 3.4 “Filters Interchanged with Samplers” [2]

This is all about the so-called Noble Identities.

For clarification, I will work out many steps explicitly, and repeat many previous derivations.

Given x(t) ∈ L2(R), recall these various representations:

(5.1) x(t)F7→ F [x(t)](ω) ≡ x(ω) =

∫ ∞−∞

dtx(t)e−iωt =

∫ ∞−∞

dtx(t)z−t ≡ x(z)

14

Now note how the so-called z-transform is related to the usual Fourier transform. These objects are formally

equivalent and is only a substitution via eiω = z. However, the functions live on L2(C) (see Subsection

2.1).

Thusx

(ω + 2πk

M

)= x

(ei2πkM z1/M

)since exp

(−i(ω + 2πk

M

)t

)=(ei2πkM z1/M

)−t

and x(ωL) = x(zL)

since exp (−iωLt) = (zL)−t

.

Also, note that the algebra on L2(R) in the so-called time domain is different from the algebra in the frequency

domain. Explicitly, recall the convolution and its Fourier transform:

f ∗ g(t) =

∫ ∞−∞

dτf(t− τ)g(τ)

F [f ∗ g(t)](ω) =

∫ ∞−∞

dte−iωt∫ ∞−∞

dτf(t− τ)g(τ)u=t−τ

=

∫ ∞−∞

dτg(τ)

∫ ∞−∞

due−iω(u+τ)f(u) = f(ω) · g(ω)

For L2(R) (which I’ll refer to being in the time domain as convention), the multiplication operation for its

algebra is the convolution ∗, but in the frequency domain (which I’ll denote as F [L2(R)], multiplication · is

by pointwise multiplication! Thus,

L2(R) F [L2(R)]

f ∗ g(t) (f · g)(ω) = f(ω) · g(ω)

F

F

Thus

(5.2)

x(t)↓M7→ (↓M ◦ x)(t) = x(Mt)

x(ω)↓M7→ (↓M ◦ x)(ω) =

1

M

M−1∑k=0

x

(ω + 2πk

M

)=

1

M

M−1∑k=0

x(ei2πkM z1/M )

and

x(t)↑L7→ (↑ L ◦ x)(t) = x

(t

L

)t∈Z7→

x(m) if t = Lm, m ∈ Z

0 otherwise

x(ω)↑L7→ (↑ L ◦ x)(ω) = x(ωL) = x(zL)

witht∈Z7→ denoting the discretization of t ∈ Z.

Notice that the operation of ↓M◦, ↑ L◦, the ◦ composition, is distinguished from the multiplication operator

in the respective algebras. On L2(R), ◦ is distinguished from convolution ∗. On F [L2(R)], ◦ is distinguished

from pointwise multiplication ·.

It’s unclear to me if ◦ can be made to “fit into” the respective algebras with multiplication operators ∗ or ·.Please tell me otherwise. Otherwise, operations involving ↓M , ↑ L are distinguished as such.

15

Theorem 1 (Noble identities). The Noble identities are the following:

↓M ◦ (G(zM )x(z))(z) = G · ( ↓M ◦ x)(z)(5.3)

↑ L ◦ (G · x)(z) = G(zL) · (↑ L ◦ x)(z)(5.4)

while noting that its form in L2(R) is, respectively,

(G∗ ↓M ◦ x)(t) =

∫ ∞−∞

dτG(t− τ)x(Mτ)τ∈Z7→

∑m∈Z

G(t−m)x(Mm)(5.5)

(↑ L) ◦G ∗ x(t) =

∫ ∞−∞

dτG

(t

L− τ)x(τ)

t∈Z7→

∑τ∈ZG(m− τ)x(τ) if t = Lm, m ∈ Z

0 otherwise(5.6)

noting that (z) notation simply denotes that these are functions of z ∈ C, and thatt∈Z7→ denotes discretization

of t.

Proof. Distinguishing between operations and understanding that algebra over a field is just algebra makes

this easy.

Let G(zM )x(z) := t(z). Then

↓M ◦G(zM )x(z) = ↓M ◦ t(z) =1

M

M−1∑k=0

t(ei2πkM z1/M ) =

1

M

M−1∑k=0

G(ei2πkz)x(ei2πkM z1/M ) = G(z)

1

M

M−1∑k=0

x(ei2πkM z1/M ) =

= G(z) · (↓M ◦ x)(z) = G · ( ↓M ◦ x)(z)

For all of this in L2(R) “time” domain,

(G ∗ (↓M ◦ x))(t) =

∫ ∞−∞

dτg(t− τ)(↓M ◦ x)(τ) =

∫ ∞−∞

dτg(t− τ)x(Mτ)τ∈Z7→

∑τ∈Z

g(t− τ)x(Mτ)

Likewise,

↑ L ◦ (G · x)(z) = ↑ L ◦G(z) · x(z) = G(zL)x(zK) = G(zL) · (↑ L ◦ x)(z)

and

(↑ L) ◦G ∗ x(t) =↑ L ◦∫ ∞−∞

dτG(t− τ)x(τ) =

∫ ∞−∞

dτG

(t

L− τ)x(τ)

t∈Z7→

∑τ∈ZG(m− τ)x(τ) if t = Lm, m ∈ Z

0 otherwise

�

5.2. On Perfect Reconstruction. Consider this filter bank:

r0(z) 1M

∑M−1k=0 r0

(ei

2πkM z1/M

)t0(z) F0 · t0(z)

x(z) (F0 · t0 + F1 · t1)(z) = x′(z)

r1(z) 1M

∑M−1k=0 r1

(ei

2πkM z1/M

)t1(z) F1 · t1(z)

H0

H1

↓M ↑M F0

⊕

↓M ↑M F1

⊕

withr0(z) = H0 · x(z)

t0(z) :=1

M

M−1∑k=0

r0(ei2πkM z)

F0 · t0(z) = F0(z) · t0(z) = F0(z) · 1

M

M−1∑k=0

H0(ei2πkM z) · x(ei

2πkM z)

16

and likewise for r1, t1, F1.

Theorem 2. The so-called perfect reconstruction condition is

1

M

M−1∑m=0

Fm(z)Hm(z) = z−δ(5.7)

1

M

M−1∑m=0

Fm(z)Hm(ei2πkM z) = 0 ∀ k = 1 . . .M − 1(5.8)

Note that for M = 2, this becomes

1

2(F0(z)H0(z) + F1(z)H1(z)) = z−δ

1

2(F0(z)H0(−z) + F1(z)H1(−z)) = 0

Proof.

M−1∑m=0

Fm(z) · 1

M

M−1∑k=0

Hm(ei2πkM z) · x(ei

2πkM z) =

1

M

M−1∑k=0

M−1∑m=0

Fm(z)Hm(ei2πkM z) · x(ei

2πkM z) = z−δx(z)

Then match different x(ei2πk/Mz) on both sides. �

6. MATLAB Wavelet Toolbox

SOFTWARE: PyWavelets 3

7. Orthogonal Filter Banks: Paraunitary Matrices; Orthogonality Condition (Condition

O) in the Time Domain, Modulation Domain and Polyphase Domain

READING: Sec 5.1-5.2 [2]

7.1. Reading.

7.1.1. Notes on Sec. 5.2. Orthonormal Filter Banks. Consider filter coefficients c(k), d(k), with

c(k) representing the low pass filter

d(k) representing the low pass filter

For input x = x(t) ∈ L2(R),

recall

C : x(t) 7→∫ ∞−∞

dτc(t− τ)x(τ) =∑τ

c(t− τ)x(τ)k=t−τ

=∑k

c(k)x(t− k) t, k ∈ Z

D : x(t) 7→∫ ∞−∞

dτd(t− τ)x(τ) =∑τ

d(t− τ)x(τ)k=t−τ

=∑k

d(k)x(t− k) t, k ∈ Z

with t−k ≥ 0 or equivalently t−τ ≥ 0 for a nonzero c or d coefficient being the (time) causality condition.

The commutative diagram (equivalent to filter bank diagram) is

3PyWavelets, Python.org, https://pypi.python.org/pypi/PyWavelets/

17

Cx(t) Cx(2t) = y0(t) t0(t) Ct0(t)

x(t) Ct0(t) + Dt1(t)

Dx(t) Dx(2t) = y1(t) t1(t) Dt1(t)

C

D

↓ 2 ↑ 2 C

⊕

↓ 2 ↑ 2 D

⊕

where

t0(t) =

y0(t/2) if t even

0 otherwise

t1(t) =

y1(t/2) if t even

0 otherwise

Claim: The claim by Strang and Nguyen[2] is that in order to achieve x(t) = Ct0(t) + Ct1(t), then

C, D are synthesis filters that are time reversals of the analysis filters:

(7.1)C = CT and c(n) = c(−n)

D = DT and d(n) = d(−n)

Indeed this is true since

∑τ

c(t− τ)t0(τ) =∑τ

ctτ t0(τ)ctτ=(c)Ttτ=

∑τ

cτtt0(τ) =∑τ

c(τ − t)t0(τ)

k=t−τ∑kc(−k)t0(t− k)

8. Maxflat Filters: Daubechies and Meyer Formulas. Spectral Factorization

SOFTWARE: daub.py

READING: Sec 5.3-5.5 [2]

Notes on the Handout:

Then Halfband condition is

P (ω) + P (ω + π) = 2

for P (ω) ∈ R[ω], the set of polynomials of ω. Look at the first slide of Handout 8. Notice how ∀ω ∈ [0, 1], the

two curves representing P (ω) + P (ω + π) add up to 2.

The slide says we want P (ω) to be lowpass, which I think means that P (ω) should “let low frequencies pass”

i.e. P (ω) is bigger when frequency ω is low and smaller (in magnitude) when ω is “high” or big.

The slide also says we want p(n) to be symmetric: I think that means that P (ω) should be symmetric about

ω = 0 so that P (−ω) = P (ω) is a symmetry (Z/2, mod-2, symmetry?).

EY : 20150701 I noticed that

P (ω) =

∫ ∞−∞

dtp(t)e−iωt

P (ω) + P (ω + π) =

∫ ∞−∞

dtp(t)(e−iωt + e−i(ω+π)t)eiπ2 t

eiπt/2=

∫ ∞−∞

p(t)e−iωtcos(π2 t)

eiπt/2

=⇒ F(P (ω) + P (ω + π)) = p(t) cos

(πt

2

)exp (−i(ω +

π

2)t) = 2δ(t)

18

Daubechies’ Approach. Let

(8.1) P (ω) = 2(1− y)pp−1∑k=0

(p+ k + 1

k

)yk

with

y =1− cosω

2=

(1− eiω

2

)(1− e−iω

2

)where y symmetric about ω = 0

Note that 1− y = 1+cosω2 .

Substituting z = eiω, so that y =(1−z2

) ( 1−1/z2

)= 1

4 (1 − 1z − z + 1) = 1

4 (2 − z − 1z ), and so (1 − y) =

14 (2 + z + 1

z ).

Note some properties of P (ω):

• P (y = 1) = 0 for y = 1. Note that y = 1 when ω = π2 .

• P (ω) of degree 2p− 1 (highest order term is 2yp(

2pp−1)yp−1 = 2y2p−1

(2pp−1))

• Let

(8.2) Bp(y) := 1 + py +p(p+ 1)

2y2 + · · ·+

(2p− 2

p− 1

)yp−1 =

p−1∑k=0

(p+ k − 1

k

)yk

which is Eq. (5.64) Sec. 5.5 in Strang and Nguyen (1996) [2]. There’s a typo in Slide 5 of Handout 8

[1].

Now note the binomial expansion formula 4

1

(1− y)p=

∞∑k=0

(p+ k − 1

k

)yk =

p−1∑k=0

(p+ k − 1

k

)yk +O(yp)

Let P (y) = 2(1− y)pBp(y) (EY: 20150701 P (y) = P (ω), they’re formally equivalent).

Then

P (y) = 2(1− y)p(

1

(1− y)p+O(yp)

)= 2 +O(yp)

Then clearly,

P (y = 0) = 2(1− 0)pBp(0) = 2

P (y = 1) = 0

and the derivatives of order l = 1 . . . p− 1 of P is 0

P (l)(y = 0) ≡ dl

dylP (y = 0) = 0

since, the idea is that

P (y) = 2(1− y)p(

1

(1− y)p+O(yp)

)= 2 + 2(1− y)pO(yp) = 2 +O(yp)

and because of the (1− y)p factor,

P (l)(y = 1) ≡ dl

dylP (y = 1) = 0

By the Z/2-symmetry, i.e. P (ω) = P (−ω), then

=⇒ P (z) = P

(1

z

)So if z0 is a root of P (z), i.e. P (z0) = 0, then P

(1z0

)= 0.

4Binomial theorem, wikipedia https://en.wikipedia.org/wiki/Binomial_theorem

19

Cepstral Method. Factor out zeros which lie on unit circle:

P (z) =

((1 +

1

z

)(1 + z)

)pQ(z)

then factor Q(z) into Q(z) = R(z)R( 1z ) s.t.

R(z) has all its zeros inside the unit circle, i.e. R(z0) = 0 for z0 s.t. |z0| < 1. R(z) causal

Now

lnQ(z) = lnR(z) + lnR(z−1)

Apply inverse z-transform (i.e. inverse Fourier transform F−1)

(8.3) F−1[lnQ(z)](t) = F−1[lnR(z)](t) + F−1[lnR(z−1)](t) ≡ q(t) = r(t) + r(−t)

where r(t) ≡ F−1[lnR(z)](t) is Strang and Amaratunga’s notation; my notation isq(t) = F−1[lnQ(z)](t)

r(t) = F−1[lnR(z)](t).

Note that r(−t) = F−1[lnR(z−1)](t) because remember how inverse Fourier transforms are defined: 12π

∫ π−π dωr(ω)z−t =

12π

∫ π−π dωr(ω)e−iωt = r(−t).

So r(t) = 0 for t < 0 so that r(t) causal. (Strang and Amaratunga’s notation).

With r(t) causal, then clearly

q(t) =

0 if t < 0

2r(0) if t = 0

r(t) if t > 0

(using my notation)

F−1[lnQ(z)](t) ≡ q(t) is the “complex cepstrum” of q(t). (Strang and Amaratunga’s notation)

Explanation of causal. Consider this very simple example (because I also forgot a lot of complex analysis):

Let

x(z) = exp

(1

1− zkz

)=

∞∑j=0

(1

1− zkz

)jj!

x(z = zk) = 0

(so z = zk is a pole of infinite order)

Then for y(z) = lnx(z) = 11− zkz

,

|y(zk)| =∞ is a singularity at z = zk.

Now y(ω) = 11−zke−iω , so that since |zk| < 1,

=⇒ y(ω) =

∞∑n=0

(zk)ne−iωn

Taking the inverse Fourier transform,

F−1[y(ω)](t) =1

2π

∫ π

−πdωeiωt

∞∑n=0

(zk)ne−iωn =

∞∑n=0

δ(t− n)(zk)n = ztk

So y(t) = F−1[y(ω)](t) is causal (I think this causal condition is that |y(t)| = |zk|t < 1, that the singularities

lie inside the unit circle.

So y(t) = 0 for t < 0

So analogously, r(t) = 0 for t < 0, so that r(t) causal. (Strang and Amaratunga’s notation)20

Algorithm. The problem is this: for

P (ω) = 2(1− y)pp−1∑k=0

(p+ k + 1

k

)yk = 2

(2 + z + 1

z

4

)p p−1∑k=0

(p+ k + 1

k

)(2− z − 1

z

4

)k= ((1 + z)(1 + 1/z))pQ(z)

Given P (ω) (and thus Q(z)),

we want to find H0(z) (i.e. the spectral factorization problem, cf. Slide 8 of Handout 8 [1]) of

P (z) = H0(z)H0(z−1)

The product filter for the orthogonal case, P (z), is the polynomial P (ω) (they’re the same thing).

With Q(z) = R(z)R(1z

), then

R(z) is a piece of H0(z)

R

(1

z

)is a piece of H0(z−1)

So we want to get H0(z) from R(z).

Notice that Q(z) is a piece of the binomial expansion of (1/(1 − y)p) and so that can be easily obtained by

formula.

EY : 20150701 At this point you should open up daub.py and see the algorithm implemented in Python

alongside, in parallel, with the following explanation.

We’re given Q(z), which can be calculated from the formula for Bp(z) (Eq. 8.2).

So first do

Q(z) Q(k) lnQ(k) q(t) = F−1[lnQ(k)](t)z = eiω = ei

2πkM ln F−1

where z = eiω = ei2πkM ∀ k = 0, 1, . . .M − 1, and M is chosen to be sufficiently large, and

q(t) =1

M

M−1∑k=0

lnQ(k)ei2πkM t Inverse Discrete Fourier transform

Then, find r(t) using the fact (causality) that

r(t) =

0 if t < 0

12q(0) if t = 0

q(t) if t > 0

Next,

r(t) exp ◦F [F−1 lnR(k)(t)] = R(k) H0(k) = R(k)(1− e−i 2πkM )p h0(t)exp ◦F ×(1 + e−i

2πkM )p F−1

where exp ◦F [F−1 lnR(k)(t)] = exp (lnR(k)) = R(k),

with the discrete Fourier transform of r(t) given by

F [r(t)](k) =

∞∑t=0

r(t)e−i(2πkM )t k = 0, 1, . . .M − 1

(EY: 20150701 I completely disagree with step v of Slide 17 Handout 8 [1])

and the multiplication ×(1 + e−i2πkM )p = ×(1 + z−1)p gets us H0(z).

Finally the inverse discrete Fourier transform F−1 is given by

h0(t) =1

M

M−1∑k=0

H0(k)ei2πkM t

21

At this point, one should open up and run daub.py and play with the daub function. I think Nh, the argument

for the function daub is 2p, so try only even values. Remember, daub outputs h0(t).

9. Multiresolution Analysis (MRA): Requirements for MRA; Nested Spaces and

Complementary Spaces; Scaling Functions and Wavelets: Handout 9

READING: Sec 1.5, 6.1 [2]

Looking at Handout 10, instead, I will following closely Wojtaszczyk (1997)[4]. Begin with the definition of a

wavelet :

Definition 2. A wavelet is a function ψ(t) ∈ L2(R s.t. if, for

ψjk := 2j/2ψ(2jx− k) j, k ∈ Z,

{ψjk}j,k∈Z is an orthonormal basis in L2(R), then ψ(t) is a wavelet.

(cf. Def. 2.1. of Wojtaszczyk (1997) [4])

9.1. Multiresolution Analysis. Consider a multiresolution analysis starting here, with these 2 defini-

tions.

Definition 3. ∀ k ∈ R, define translation operator Tk : L(R,R)→ L(R,R) where L(R,R) is the set of all

functions f defined on R (EY: 20150701 Would this set of functions necessarily be qualified to be all linear

functions (which is implied by the notation L)? That’s my question.) s.t.

Tk(f)(x) = f(x− k)

(cf. Def. 2.3. of Wojtaszczyk (1997) [4])

In frequency or wavenumber space, so that for f(t− k) ∈ L2(R),

F [f(t− k)](ω) =

∫ ∞−∞

dte−iωtf(t− k)u=t−k

=

∫ ∞−∞

due−iω(u+k)f(u) = e−iωkf(ω)

so that

Tkf(ω) = e−iωkf(ω)

Definition 4. ∀ j ∈ Z, define dyadic dilation operator Jj : L(R,R)→ L(R,R) s.t.

Jj(f)(x) = f(2jx)

(cf. Def. 2.4. of Wojtaszczyk (1997) [4])

In frequency or wavenumber space, so that for f(jt) ∈ L2(R),

F [f(jt)](ω) =

∫ ∞−∞

dte−iωtf(jt) =u=jt=

∫ ∞−∞

du

je−iω

uj f(u) = f(

ω

j)

so

Jj f(ω) = f(ω

j)

Definition 5. The multiresolution analysis is a sequence (V j)j∈Z of subspaces of L2(R, V j ⊆ L2(R),

∀ j ∈ Z s.t.

(1)

· · · ⊂ V −1 ⊂ V 0 ⊂ V 1 ⊂ . . .

(2)

span{⋃j∈Z

V j} = L2(R)

22

(3) ⋂j∈Z

V j = {0}

(4)

f(x) ∈ V j iff f(2−jx) ∈ V 0 i.e. equivalently V j = Jj(V 0) ∀ j ∈ Z

(5)

f ∈ V 0 iff f(x−m) ∈ V 0 ∀m ∈ Z i.e., equivalently V 0 = Tn(V 0) ∀n ∈ Z

(6) ∃φ ∈ V 0 called scaling function s.t.

{φ(t−m)}m∈Z is an orthonormal basis in V 0

i.e. equivalently, ∀ j ∈ Z, {2j/2φ(2jx− k)}k∈Z is an orthonormal basis in V j

(cf. Def. 2.2. of Wojtaszczyk (1997) [4])

EY : 20150701 Note on notation of t, ω’s and x, k’s and indices n, k,m.n; I really don’t know what’s the

best notation to remain consistent. I’ll trust that from the context, the meaning is clear. In the spirit of

open-source software (OSS), feel free to edit, copy, delete and make your own versions (and remember to share

it!). I’m interested in wavelet collocation on the physical spatial space x ∈ R3, so that’s why I’ll be writing in

terms of position x and wavenumber k.

In this easy lemma, that translation and dyadic dilation Tk, 2j/2Jj is an isometry is a good property to have,

ensuring we stay on L2(R).

Lemma 1. (1) ∀ k ∈ R, Tk is an isometry on L2(R)

(2) ∀ j ∈ Z, 2j/2Jj is an isometry on L2(R)

Proof. Main idea: change of variable

Recall ‖f‖22 =∫∞−∞ dx|f(x)|2.

‖Tkf‖22 =

∫ ∞−∞

dx|f(x− k)|2 =

∫ ∞−∞

dx|f(x)|2 = ‖f‖22

‖2j/2Jjf‖22 =

∫ ∞−∞

dx|2j/2f(2j(x))|2 =

∫ ∞−∞

dx|f(x)|2

�

9.2. W j. We’re going to follow Sec. 2.4 of Wojtaszczyk (1997) [4].

Let subspace W j ⊆ L2(R) s.t. V j ⊕W j = V j+1.

Now Jj(V 1) = V j+1, so then

V j+1 = Jj(V 1) = Jj(V 0 ⊕W 0) = Jj(V 0)⊕ Jj(W 0) = V j ⊕ Jj(W 0).

Then W j = Jj(W 0).

Since we’re given a multiresolution analysis (V j)j∈Z, by def., (EY: 20150701 I’m not sure about this following

step; if you can do this explicitly, let me know)

L2(R) = span(⋃j∈Z

V j) = span(⋃j∈Z

J−j+1(V 0)⊕W j) = ⊕∑j∈Z

W j

=⇒ L2(R) = ⊕∑j∈Z

W j

i.e. W j ’s give an orthogonal decomposition of L2(R). So if we get the orthonormal basis of each of the W j ’s,

then we get an orthogonal decomposition of L2(R).23

In application of this orthogonal decomposition,

L2(R) = V 0 ⊕W 0︸ ︷︷ ︸V 1

⊕W 1 ⊕W 2 ⊕ . . .

where V 0 is a coarse approximation and W j is the jth level of detail.

From Theorem 2.20 and Eq. (2.47) of Wojtaszczyk (1997) [4], we get this fact:

Theorem 3. Given a multiresolution analysis,

(9.1) ψ(x) =∑n∈Z

an(−1)nφ(2x+ n+ 1)

where ψ ∈W 0 = V 1 V 0 is a wavelet and

an =

∫ ∞−∞

φ(x

2)φ(x− n)dx

9.3. On the Question of the construction of the scaling function φ. Start with

(9.2) V 0 ⊂ V 1

which is the so-called scaling equation or refinement equation aka two-scale difference equation or dilation

equation.

Using the facts from the definition of a multiresolution analysis, Def. 5, and in particular fact 6,

for scaling function φ = φ(x) ∈ V 0,

since V 0 ⊂ V 1 and {21/2φ(2x−n)}n∈Z is an orthonormal basis in V 1, then φ ∈ V 0 ⊂ V 1 is of the form

φ(x) =∑n∈Z

an21/2φ(2x− n)

Take the Fourier transform of φ(x):

φ(k) =

∫ ∞−∞

dxφ(x)e−ikx =

∫ ∞−∞

dx∑n∈Z

an21/2φ(2x− n)e−ikx =∑n∈Z

21/2an

∫−∞

duφ(u)e−ik(u+n

2 ) =

=∑n∈Z

21/2ane−ik n2 φ

(k

2

)= mφ

(k

2

)φ

(k

2

)

using the u-substitution

u = 2x− n

du = 2dx

x =u+ n

2

and where m(k2

)≡ mφ

(k2

):=∑n∈Z 21/2ane

−i k2n.

Notice that φ(k) = mφ(k/2)φ(k/2). We have a recursion relation. By induction, for N ∈ Z (with the feeling

that N is large), then

φ(k) =

N∏j=1

mφ

(k

2j

)φ

(k

2N

)So given a sequence of numbers (ak)k∈Z (which gives us mφ), ak ∈ R, one can obtain

θ(k) :=

∞∏j=1

m

(k

2j

)and construct φ(k).

Indeed, for the construction of an appropriate multiresolution analysis,

it’s sufficient to construct a multiresolution analysis with a compactly supported scaling function

φ (and then we’ll get wavelets), from Theorem 4.1 of Wojtaszczyk (1997)[4], which I’ll recap here:24

Theorem 4. Suppose

m(k) =

S∑n=T

ane−ikn

where m(k), a trigonometric polynomial, satisfies the following properties:

|m(k)|2 + |m(k + π)|2 = 1 ∀ k ∈ R

m(0) = 1

m(k) 6= 0 ∀ k ∈[−π2,π

2

]

Then the infinite product

θ(k) =

∞∏j=1

m

(k

2j

)converges almost uniformly.

Thus, θ(k) continuous.

Moreover, θ(k) ∈ L2(R).

Function φ, given by φ = 1√2πθ, has compact support on [T, S] and is a scaling function of a multiresolution

analysis.

Formula

ψ(x) = 2

S∑n=T

an(−1)nφ(2x+ n+ 1)

gives a compactly supported wavelet with compact support suppψ ⊂[T−S−1

2 , S−T−12

].

cf. Theorem 4.1, Wojtaszczyk (1997)[4] Notice that this theorem gives us desirable properties and a wavelet.

EY : 20150701 the properties for m(k) look very familiar to what was discussed about the Daubechies filter

coefficients: are they related?

EY : 20150701 Note, I’m not trying to cuff the question of the construction of a compactly supported scaling

function here by referencing and copying directly from Wojtaszczyk (1997)[4]. Wojtaszczyk has the answer

given by Theorem 4.1 and it is what it is. We’re not trying to reinvent the wheel here in these notes or i.e.

suffer from “Not Invented Here”-syndrome.

Otherwise, if you think that including the full proof here is valuable, let me know; I’ll type it up though it’s

essentially a copy of the proof in Wojtaszczyk (1997)[4] with my notation, or I’d invite you to copy, paste,

and edit the LaTeX and add it yourself (and remember to share!).

Ingrid Daubachies original 10 lectures on wavelets is also a great resource I’m trying to look into; she neatly

summarizes in a theorem the take away facts from this entire section in Theorem 6.3.6. [5]:

Theorem 5. Suppose m0 is a trigonometric polynomial s.t.

|m0(k)|2 + |m0(k + π)|2 = 1

m0(0) = 1

Define φ, ψ by

φ(k) =1

(2π)1/2

∞∏j=1

m0

(k

2j

)

ψ(k) = e−ik2 m0

(k

2+ π

)φ

(k

2

)25

Then φ, ψ compactly supported L2-functions s.t.

φ(x) =√

2∑n

hnφ(2x− n)

ψ(x) =√

2∑n

(−1)nh−n+1φ(2x− n)

where hn determined by m0, via

m0(k) =1√2

∑n

hne−ink

ψjk(x) =1

2j/2ψ(

x

2j− k), j, k ∈ Z

ψjk(x) tight frame for L2(R) with frame constant 1.

This tight frame is an orthonormal basis iff for m0, ∃ compact set K, congruent to [−π, π] modulo 2π,

containing a neighborhood of 0, so that

infn>0

infk∈K|m0(2−nk)| > 0

I think Daubachies’ convention is for j to actually be −j in our understanding of V j ⊂ V j+1 cf. Chapter

6, Sec. 6.1 of Strang and Nguyen (1996) [2]. So the (important) orthonormal bases for the W j subspaces

is

(9.3) ψjk(x) = 2j/2ψ(2jx− k) j, k ∈ Z

10. Refinement Equation: Interative and Recursive Solution Techniques; Infinite Product

Formula; Filter Bank Approach for Computing Scaling Functions and Wavelets:

Handout 10

SOFTWARE: cython for pyWavelets package, pyWavelets package, 5

READING: Sec 6.2-6.4 [2]

Yes, the previous section was mostly theory, but it will payoff dividends here.

Slides 5-8 of Handout 10 [1] are essentially an application of the multiresolution analysis definition, Def. 5,

but on a discrete grid x ∈ Z.

For φ0 ∈ V 0, with orthogonal basis {φ0(x− n)}n∈Z for V 0, then

φ0(x) =∑n∈Z

a0nφ0(x− n)

It’s unclear to me from Handout 10 [1] which quantities do we know and don’t know, but I think we want to

find a0n given the values of scaling function φ at various points x, and vice versa.

Discretize x ∈ Z: x ∈ Z→ x ∈ Z. So

φ0(x) =∑n∈Z

a0nφ0(x− n)

u=x−n=

∑u∈Z

a0x−uφ0(u)

Solve for this matrix equation now.

If x is treated as time t, then (time) causality condition is clear: for

φ(t) =∑u∈Z a

0t−uφ(u)

then the causality condition is a0t−u = 0 if t− u < 0.

EY : 20150701 My question is what happens when x is treated as position in physical space? What is the

physical analogue to time causality? Locality?

5Discrete Wavelet Transform in Python http://www.pybytes.com/pywavelets/

26

Now for Jj(V 0) = V j , i.e. orthonormal basis for V j is {2j/2φ(2jx− n)}, then forφj(x) ∈ V j

φj(x) =∑n∈Z

ajn2j/2φ0(2jx− n)

Suppose j = 1. Suppose x = 2m+12 , x ∈ Z/2 (half integers).

φ1(x =2m+ 1

2) = 21/2

∑n∈Z

a1nφ0(2m+ 1− n)

u=2m+1−n= 21/2

∑u∈Z

a12m+1−uφ0(u)

Treating x as time t, the causality condition is 2m+12 ≥ u or m+ 1

2 ≥ u.

Scaling Relation and Wavelet Equation in Frequency Domain. cf. Slides 9-17. [1]

I think this is an application of Theorem 6.3.6. from Ingrid Daubachies [5]. It’d be nice to see that proof from

[5] in comparison to the presentation in Slides 9-17 [1]

Software. I installed cython and PyWavelets packages. For the Mac OSX, at the command prompt of

Terminal on an administrator account, I installed both with pip (it’s an improvement on easy_install 6. By

the way, I find that pip makes installation very easy and robust (easier than easy_install, heh), so I highly

recommend that if you don’t have pip, install pip.

The website for PyWavelets http://www.pybytes.com/pywavelets/#install says that PyWavelets is writ-

ten in Python, Cython, and C and features over 70 built-in wavelet filters, and that results are compatible

with Matlab’s Wavelet Toolbox. And PyWavelets is open source (MIT licenese)! So it looks very promising.

We’re going to use it as much as possible.

Documentation for PyWavelets is at the bottom, under Contents, and one should look at the API Refer-

ence.

Example 6: Generation of orthogonal scaling functions and wavelets. Open up example6.py. Notice the

use of daub.py again. So daub function gives us the low pass filter coefficients (in the so-called filter bank

form).

Opening up example6.py, you’ll see that the use of function upcoef for the inverse DWT (Discrete Wavelet

Transform). It would be a good time to learn about the function upcoef from the documentation at pybytes7. In the MIT OCW 18.327 material, upcoef wasn’t discussed or documented on its usage and in this notes, I

want to improve on that by explicitly discussing the use of functions in Python (same goes with Matlab/GNU

Octave) in application in relation to what’s learned in the notes here and Handout.

Notice the use of upcoef in example6.m and the requirement to input in a synthesis lowpass filter and

synthesis highpass filter, f0, f1, respectively, in Matlab. The wavelet can be specified in terms of orthogonal

filter banks. I think the explanation can be found in Sec. 6.2. of Nguyen and Strang [2] or look at my notes

on the reading that goes along this Handout 10, Subsection 10.1.

In PyWavelet, for a custom wavelet, one must specify the entire filter bank with a lowpass decomposition, high-

pass decomposition, lowpass reconstruction and highpass reconstruction (i.e. synthesis). Given the lowpass

reconstruction and highpass reconstruction, you can get the lowpass decomposition and highpass decomposi-

tion because the coefficients are simply time-reversed.

6“Why use pip over easy install?” stackoverflow http://stackoverflow.com/questions/3220404/

why-use-pip-over-easy-install7PyWavelets API Reference http://www.pybytes.com/pywavelets/ref/idwt-inverse-discrete-wavelet-transform.html

27

Given the lowpass reconstruction and highpass reconstruction, Lo_R, Hi_R in Matlab notation 8, rec_lo,

rec_hi in PyWavelet notation 9, to obtain the lowpass decomposition filter and highpass decomposition filter,

then recall Eq. 7.1 and the discussion there.

Withc(n) = c(−n)

d(n) = d(−n), then this boils down to reversing the order of the coefficients to get c, d’s from c, d’s.

However, once the PyWavelet Object is constructed, instead of using upcoef, what you want to use is the

function wavefun. It “calculates approximations of scaling function (phi) and wavelet function (psi) at the

given level of refinement.” 10. It directly gives you the scaling function and wavelet you want.

EY : 20150703 My question is about the scaling. I would like some clarification in example6.m on the scaling

by M of the phi done after the inverse DWT. It’s unclear what choice of scaling Matlab uses. And how does

that compare with the scaling that’s done in PyWavelets?

Example 7: Generation of biorthogonal scaling functions and wavelets. biorwavf is a Biorthogonal spline

wavelet filter in Matlab.

In PyWavelets, the wavelet filters coefficients and the corresponding lowpass and highpass decomposition filters

and lowpass and highpass reconstruction filters can be accessed directly by creating a Wavelet object.

10.1. Readings. cf. Chapter 6 “Multiresolution”, Sec. 6.1. The Idea of Multiresolution [2]. Let

φ(t) = 21/2∑k

c(k)φ(2t− k) ∈ L2(R)

Let φ(t) have compact support on [2t−N, 2t] ∀ t, i.e. only c(0), c(1), . . . c(N) nonzero.

Note that c(k)’s are called approximation coefficients, which is handy to remember when using Matlab or

PyWavelets as that’s what they are referred to by, there.

Since

〈φ(t), φ(t−m)〉 =

∫ ∞−∞

dtφ(t)φ(t−m) = δ(m)

then ∑k

c(k)c(k − 2m) = δ(m)

Also, recall that ↓M : x(t) 7→ x(Mt), which is part of the “analysis” or decomposition filter bank.

Then coefficients c(k) “go into an orthonormal filter bank.” (cf. Sec. 6.1 [2]).

c(k) is involved in decomposition.

The key relations for scaling function φ(t) ∈ V 1 and wavelet φ(t) ∈ W 0 ⊂ V 1 are the following (see previous

theorems, such as Thm. 6.3.6 of Daubachies [5]):

(10.1)

φ(t) =√

2∑n

hnφ(2t− n)

ψ(t) =√

2∑n

(−1)nh−n+1φ(2t− n)

If we denote the coefficients for a wavelet d = d(k) (note that they are also called detail coefficients, from the

high-pass filter, because Matlab and PyWavelets both call these d’s detail coefficients) in

ψ(t) =√

2∑k

d(k)φ(2t− k) ∈W 1 ⊂ V 1

8upcoef, Mathworks Documentation http://www.mathworks.com/help/wavelet/ref/upcoef.html9“Using custom wavelets,” PyWavelets, http://www.pybytes.com/pywavelets/ref/wavelets.html10Approximating wavelet and scaling functions - Wavelet wavefun, PyWavelets http://www.pybytes.com/pywavelets/ref/

wavelets.html

28

Then we can conclude that the relationship between coefficients for φ and coefficients for ψ is the follow-

ing:

c(n) = d(−n+ 1)(−1)n ∀n ∈ Z

cf. Sec. 6.2. “Wavelets from Filters” [2]

The claim is this:

c(0) . . . c(N) constitute a low pass filter that determines the scaling function φ(t) =∑n 21/2c(n)φ(2t − n) ∈

V 0 ⊂ V 1

d(0) . . . d(N) constitute a high pass filter that determines the wavelets ψ(t) =∑n 21/2d(n)φ(2t−n) ∈W 0 ⊂ V 1

Now consider f j(t) =∑k a

jkφ

jk(t) ∈ V j

f(t) =∑j,k b

jkψ

jk(t) ∈W j

and now construct φ(t) using V 0 ⊂ V 1 the so-called “dilation equation.”

For V 0 ⊕W 0 with orthonormal bases, respectively,

{φ(t− k)}k∈Z, {ψ0k(t) = ψ(t− k)}k∈Z, then∑a1kφ

1k(t) =

∑a0kφ

0k(t) +

∑k

b0kψ0k(t) :=

∑k

a0kφ(t− k) +∑k

b0kψ(t− k)

φ(t− k) =∑n

21/2c(n)φ(2(t− k)− n)n=l−2k

=∑n

c(l − 2k)φ1l

ψ(t− k) =∑n

21/2d(n)φ(2(t− k)− n)n=l−2k

=∑n

d(l − 2k)φ1l

a0k = 〈f1, φ(t− k)〉 =∑

c(l − 2k)a1l

b0k = 〈f1, ψ(t− k)〉 =∑

d(l − 2k)a1l

11. Project Brief

EY : 20150701 I’m partial to wavelet collocation methods on an adaptive grid (for the spatial space) for

computational fluid dynamics (CFD), but it would be neat to include links, abstracts, and possible project

proposals here of past and future projects involving wavelets and filter banks.

However, inspired by Kyle Kastner’s post on “Wavelets” 11, there are two projects that was fun for me to

scratch the surface on and I’d be happy to see others pick up on.

11.1. Sunspots. SOFTWARE: in tools,

• sunspots_get.py

• sunspots.py

We can do a discrete wavelet transform on the number of sunspots observed over the years. The Royal

Observatory of Belgium in Brussels maintains updated data on worldwide observations of sunspot number12.

Much credit goes to them for doing this service.

sunspots_get.py gets you the latest data from SILSO on the observed sunspot number directly from SILSO

and puts the data in a Python pickle so you could save it locally on your hard drive (and so you don’t have

to keep requesting and overloading the website).

Now, I’ve used requests, BeautifulSoup, and Python 2’s urlparse to do what’s called “webscraping” of the

SILSO website - that is automating going to the website, reading in the html code, and extracting from the

website the desired links to the data. Of course, you can manually click on the website and download the

11Kyle Kastner, “Wavelets”, Garbage In, Garbage Out, http://kastnerkyle.github.io/blog/2014/04/17/wavelets/12Source: WDC-SILSO, Royal Observatory of Belgium, Brussels, http://sidc.be/silso/datafiles

29

Figure 1. Source: NASA

data, but I wanted to illustrate how web scraping can be done because you can imagine the usefulness of

automating data collection when dealing with a large amount.

sunspots.py does the actual Discrete Wavelet Transform (DWT) on the sunspot data for daily and mean

monthly observations, and plot them. While I used the single level DWT using Daubechies 2 and Daubechies

16 wavelets, there are 3 directions I see that can be fun to explore:

• Try different types of wavelets from the wavelist() wavelet list of PyWavelets, such as ’haar’, ’symlet’,

etc.

• Try different levels of resolution with wavedec

• Get the “peaks” of the wavelet decomposition (I tried scipy.signal.find_peaks_cwt from scipy.signal

but I need some work as it’s not working for me).

EY : But as of 20150704, I was able to obtain these plots of the single-level wavelet decomposition of the

observed sunspot number over 200 to 300 years, Fig. 11.2,11.2, 11.2.

11.2. Image Compression and Reconstruction. SOFTWARE: kompresni.py

11.3. Image compression and reconstruction. SOFTWARE: in tools, kompresni.py

PyWavelet, with 2-dimensional Wavelet packets (methods), offers the opportunity to do some image compres-

sion and reconstruction.13. with single level dwt2, idwt2, and 2D multilevel decomposition and reconstruction

with wavedec2 and waverec2.

Note that I was using scipy’s misc or ndimage for reading in the image and matplotlib.pyplot’s imshow to

plot it out.

EY : 20150704 right now, I’m having trouble with the code 14 in terms of using index slicing to put together

a 3-dimensional numpy array to get back the image after doing the DWT2 on each of the (3) colors, RGB

(red,green,blue). So I need help there. It’s outputting out right now multiple images after doing dwt2.

Nevertheless, here’s what I obtained so far as plots. Note that it’s tradition to use the image of Lena Soderberg,

but you should have fun and use an image you like (and use appropriately).

13PyWavelets, API Reference, http://www.pybytes.com/pywavelets/ref/2d-dwt-and-idwt.html14ernestyalumni, github, https://github.com/ernestyalumni/18-327-wavelets-filter-banks/blob/master/tools/

kompresni.py

30

31

Figure 2. Original image

32

12. Orthogonal Wavelet Bases: Connection to Orthogonal Filters; Orthogonality in the

Frequency Domain. Biorthogonal Wavelet Bases

13. Mallat Pyramid Algorithm; Handout 12

READING: Sec 1.6, 6.2 [2]

Consider

φj+1(t) ∈ V j+1 = V j ⊕W j

φj+1(t) =∑k

cj+1k φj+1

k (t) =∑k

cjkφjk(t) +

∑k

djkψjk(t)

which we can write, using the orthonormal basis {φjk(t) = 2j/2φ(2jt−k)}k∈K for V j . {ψjk}k∈K is the orthonor-

mal basis for W j .

For the scaling function φ(t) ∈ V 0, since V 0 ⊂ V 1, then we can write φ(t) as

(13.1) φ(t) =∑l

h0(l)21/2φ(2t− l)

Now, by theorem (such as Daubechies Thm. 6.3.6)

(13.2) ψ(t) =∑l

(−1)lh0(−l + 1)21/2φ(2t− l) =∑l

h1(l)21/2φ(2t− l)

So

φjk(t) = 2j/2φ(2jt− k) = 2j/2∑l

h0(l)21/2φ(2j+1t− 2k − l) =∑l

h0(l)φj+12k+l(t)

ψjk(t) = 2j/2ψ(2jt− k) = 2j/2∑l

h1(l)21/2φ(2j+1t− 2k − l) =∑l

h1(l)φj+12k+l(t)

33

Then

(13.3)

cjk = 〈φj+1(t), φjk(t)〉 =

∫ ∞−∞

dtφj+1

(t)φjk(t) =

∫ ∞−∞

dtφj+1

(t)∑l

h0(l)φj+12k+l(t) =

∑l

h0(l)cj+12k+l

u=2k+l=

∑u

h0(u− 2k)cj+1u

djk = 〈φj+1(t), ψjk(t)〉 =

∫ ∞−∞

dtφj+1

(t)∑l

h1(l)φj+12k+l(t) =

∑l

h1(l)cj+12k+l =

∑u

h1(u− 2k)cj+1u

Thus,

(13.4)

〈φj+1(t), φj+1k (t)〉 = cj+1

k =∑k′

cjk′〈φjk′(t), φ

j+1k (t)〉+

∑k′

djk′〈ψjk′(t), φ

j+1k (t)〉 =

∑k′

cjk′h0(k − 2k′) +∑k′

djk′h1(k − 2k′)

where

〈φjk′(t), φj+1k (t)〉 =

∑l

h0(l)δ(2k′ + l − k) = h0(k − 2k′)

〈ψjk′(t), φj+1k (t)〉 =

∑l

h1(l)δ(2k′ + l − k) = h1(k − 2k′)

EY : 20150707 The coefficients cjk’s djk’s live in a different space from V js, W js. These coefficients obey a

different algebra than the algebra of the scaling function and wavelets.

∀ j ∈ Z, V j is a k-algebra. V j is a ring with addition and multiplication being convolution. k is a commutative

ring with addition and multiplication (in the usual sense, like scalar addition and scalar multiplication). But

k has also addition and multiplication on its “indices.”

Proposition 1. ∀ j ∈ Z, V j is a L2(R)-module.

Proof. Sketch of a proof: ∀ f j , gj , pj ∈ V j , then under addition, it is (additive) group

so that f j + gj ∈ V j

f j + gj = gj + f j (commutativity)

(f j + gj) + pj = f j + (gj + pj) (associativity)

and it is equipped with multiplication being the convolution and this multplication is commutative

(h ∗ f j)(t) =

∫ ∞−∞

dτh(t− τ)f j(τ) = (f j ∗ h)(t)

�

Proposition 2. V j also is equipped as a kj-module over kj which itself a k-algebra of coefficients cjks, djks.

You can do addition and multiplication in kj with kj as a ring:

cjk + djk

h0(k − 2m)cjm

and you can also do addition and multiplication on the “indicies” as a commutative ring (but with no dis-

tributivity):

cjk+l = cjl+k

cj(MN)k = cj(NMk)

So V j+1 is a kj+1-module over kj+1 algebra.

V j+1 = V j ⊕W j and W j is a Dj-module over Dj algebra (consisting of djk coefficients).

I claim that kj+1 = kj ⊕Dj .34

For the filter bank representation involving coefficients cj+1k , cjk, djk, define the convolution function, that

involves the commutative ring k in the indices:

h0 ∗ cj+1k =

∑l

h0(k − l)cj+1l

Then the filter bank for this Mallot Pyramid Algorithm, or for wavelet coefficients does not involve V js at all,

but on its kj algebra:

kj+1 kj kj+1 kj+1

kj+1 kj+1

kj+1 Dj kj+1 kj+1

h0∗

h1∗

↓ 2 ↑ 2 h0∗

⊕

↓ 2 ↑ 2 h1∗ ⊕

∑l h0(k − l)cj+1

l cjk cjk/2∑l h0(k − l)cjl/2

cj+1k cj+1

k

∑l h1(k − l)cj+1

l djk djk/2∑l h1(k − l)djl/2

h0∗

h1∗

↓ 2 ↑ 2 h0∗

+

↓ 2 ↑ 2 h1∗+

with

h0 ∗ cj+1k =

∑l

h0(k − l)cj+1l

↓27→∑l

h0(2k − l)cj+1l =

∑l

h0(l − 2k)cj+1l = cjk

(confirming Eq. 13.3) and likewise,

djk =↓ 2 ◦ h1 ∗ cj+1k

and for the reconstruction part,

h0∗ ↑ 2 ◦ cjk =∑l

h0(k − l)cjl/2 =∑m

h0(k − 2m)cjm

h1∗ ↑ 2 ◦ djk =∑l

h1(k − l)djl/2 =∑m

h1(k − 2m)djm

(confirming Eq. 13.4) and the sum of the above is cj+1k .

Again, my point is that ↓ 2, ↑ 2 does not act on tR, the domain of V j as L2(R), but it acts on the commutative

ring of the indices.

14. Accuracy of Wavelet Approximations (Condition A); Vanishing Moments; Polynomial

Cancellation in Filter Banks

15. Smoothness of Wavelet Bases: Convergence of the Cascade Algorithm (Condition E);

Splines. Bases vs. Frames

16. Signal and Image Processing: Finite Length Signals; Boundary Filters and Boundary

Wavelets; Wavelet Compression Algorithms

READING: Sec 8.1-8.3, 8.5, 10.1, 11.1-11.5 [2]35

17. Guest Lecture. Physical Wavelets and their Sources: Real Physics in Complex

Spacetime

READING: ???

18. Lifting: Ladder Structure for Filter Banks; Factorization of Polyphase Matrix into

Lifting Steps; Lifting Form of Refinement Equation; Handout 16

READING: Sec 6.5 [2]

See Section 13 for the setup.

18.1. Slide 6. See Section 13 for the setup.

The lifting scheme is the following:

Proposition 3 (Lifting scheme for wavelets). In the filter bank representation on the kj algebras of which

kj-algebras V j is over,

(18.1)h]0 = h0 + u ∗ h1

h]1 = h1 − h0 ∗ u

where ] denotes the new h0 or h1, and ∗, in this algebra, is the convolution operator, but on the commutative

ring of the indices ( not the time or spatial domain of the wavelet themselves).

Proof. ] denotes new values, after the lifting transformation.

↓ 2 ◦ h]0 ∗ cj+1k =↓ 2 ◦ (h0 ∗ cj+1

k +∑l,m

u(k −m)h1(m− l)cj+1l ) = cjk +

∑l,m

u(2k −m)h1(l −m)cj+1l = c]jk

djk stays the same:

djk =↓ 2 ◦ h1 ∗ cj+1k

For the reconstruction part,

h0∗ ↑ 2 ◦ c]jk = h0∗ ↑ 2 ◦ cjk +∑l

h0(k − l)∑m,n

u(l −m)h1(n−m)cj+1n

h]1∗ ↑ 2 ◦ djk = h1∗ ↑ 2 ◦ djk − h0 ∗ u∗ ↑ 2 ◦ djk = h1∗ ↑ 2 ◦ djk −∑l,m

h0(k − l)u(l −m)djm/2 =

= h1∗ ↑ 2 ◦ djk −∑l,m

h0(k − l)u(l −m)∑n

h1(m− n)cj+1n = h1∗ ↑ 2 ◦ djk −

∑l,m,n

h0(k − l)u(l −m)h1(n−m)cj+1n

Clearly, summing the above two equations reconstructs the original cj+1k coefficient, noticing the cross term

cancels exactly. �

Note the change (and no change) in the coefficients, which I’ll recap here:

c]jk = cjk +∑l,m

u(2k −m)h1(l −m)cj+1l

djk =∑l

h1(l − 2k)cj+1l

cj+1k =

∑l

c]jl h0(k − l) +∑l

djlh]1(k − 2l)

36

Going back to Slide 6,

Recall Eqs. 13.1, 13.2

φ(t) =∑l

h0(l)21/2φ(2t− l)

ψ(t) =∑l

h1(l)21/2φ(2t− l)

With Proposition 3,

ψ](t) =∑l

h]1(l)21/2φ(2t− l) = 21/2∑l

h1(l)−∑m

h0(l −m)u(m)φ(2t− l) =

= ψ(t) + 21/2∑l,m

u(m)h0(l −m)φ(2t− l +m−m) = ψ(t) +∑m

u(m)φ(t−m/2) = ψ(t) +∑k

u(2k)φ(t− k)

These two expression are from the “reconstruction” part of the filter bank. It is clear now what the first 2

expressions on Slide 6 is for.

For the last 2 expressions of Slide 6, consider the decomposition part of the filter bank involving h0, h1.

Let’s resolve the very last expression, which is essentially

ψ(t) =∑l

h1(l)21/2φ](2t− l)

However,

φ](t) =∑l

h]0(l)21/2φ](2t− l) =∑l

(h0 + u ∗ h1)(l)21/2φ](2t− l) =

=∑l

h0(l)21/2φ](2t− l) +∑l,m

u(m)h1(l −m)φ](2t− l +m−m) =

=∑l

h0(l)21/2φ](2t− l) +∑m

u(m)ψ(t−m/2) =∑l

h0(l)21/2φ](2t− l) +∑k

u(2k)ψ(t− k)

18.2. On the Reading, Sec. 6.5, Biorthogonal Wavelets.

18.2.1. On Lifting. cf. Subsection “Filter Construction by Lifting” for Sec. 6.5 [2]

The so-called “no alias” condition for a 2-channel (i.e. ↓ 2, and the frequencies “separate into” two different

ranges, by 2π/2) filter bank is

F0(z)H0(−z) + F1(z)H1(−z) = 0

There is a choice of filters (transformations) F0, F1 that can be made. Let

F0(z) = H1(−z)

F1(z) = −H0(−z)

Thus, do this dual lifting:

H]0(z) = H0(z) + F0(−z)S(z2)

F ]1(z) = F1(z) +H1(−z)T (z2)

Given the choice above H]0, F

]1 of F0, F1, then

H]0(z) = H0(z) +H1(z)S(z2)

F ]1(z) = F1(z) + F0(z)T (z2)

37

18.3. Software: Handout16 examples.py. At this point, you should run Handout16_examples.py (I made

it myself, I don’t think the MIT OCW 18.327 material has the code for anything beyond Handout 16) and

reproduce the so-called Symmetric 5/3 Wavelets on Slide 16. They are actually the biorthogonal wavelets for

the Cohen-Daubechies-Feauveau wavelets 15. In Matlab, they are called “bior2.2”. The 2 and 2 refer to the

number of vanishing moments of the high pass filter; same in PyWavelets. Running the plots in the file, you

should reobtain the plots for Slide number 12 of Handout 16, and I also plotted the filter bank coefficients,

Figures 18.3, 18.3:

Figure 3. cf. ernestyalumni

19. Wavelets and Subdivision: Nonuniform Grids; Multiresolution for Triangular Meshes;

Representation and Compression of Surfaces; Slides and Handouts 17,18

READING: It appears that there are no reading for this session, other than Slides and Handouts 17, 18.

15“Cohen-Daubechies-Feauveau wavelet”, Wikipedia.org, https://en.wikipedia.org/wiki/Cohen-Daubechies-Feauveau_

wavelet

38

Figure 4. cf. ernestyalumni

20. Numerical Solution of PDEs: Galerkin Approximation; Wavelet Integrals (Projection

Coefficients, Moments and Connection Coefficients); Convergence. Subdivision

Wavelets for Integral Equations. Compression and Convergence Estimates

READING: Sec 11.6 [2]

21. M-band Wavelets: DFT Filter Banks and Cosine Modulated Filter Banks.

Multiwavelets

READING: Sec 7.5, 9.1-9.4 [2]39

Part 2. Extra fun

I wanted to, in this part of an open-sourced LaTeX file, welcome and elicit projects, ideas, resources beyond

the MIT OCW 18.327. You should, if you’re reading this and wanted to add to this pdf/LaTeX document,

share your projects, ideas, and resources here.

22. Fun with Quandl

Install (you can use pip install) Quandl (it’s open-source). Open up fun_w_Quandl.py that’s in the tools

directory of the github for 18-327-wavelets-filter-banks. Quandl provides open data, public data that

they’re trying to make free, open, and (easily) accessible to all.

With fun_w_Quandl.py, I’ve tried to use the Haar wavelet transform to apply discrete wavelet transforms

(dwt) of a few time-series data that seemed relevant at the time (the Shanghai Composite Index and the

Interest Rate Spread of 2 years vs. 10 years for the LCY (Local Currency Renminbi or Chinese Yuan) Bond).

The Python functions in the file, obtenir and pkl_data, gets the time-series data directly from Quandl and

“persists” the data i.e. saves it locally using Python’s pickle library. You don’t want to be making calls to

the Quandl website more than necessary.

See Figures 22, 22

Figure 5. cf. ernestyalumni

40

Figure 6. cf. ernestyalumni

Then you can read in the data onto your Python interactive interpreter with lire and then apply the dwt on all

the data sets with make_dwt and plot all of the approximation and detail coefficients with plot_dwt.

I would invite you to run the file and play around with it, alongside looking at the Python code and trying

out your own time series of interest from Quandl. I’d also would like to know if you guys have any ideas or

interpretations of what wavelets can tell us about time-series data.

Part 3. Colophon, Rationale, PitchDeck, and Help Me and Help Others

23. Why Wavelets (for me)?

One day on 18 juin 2015, I find myself needing to learn about wavelets, fast. I had just found the video

uploaded to YouTube of the talk given by Adam Lichtl and Stephen Jones (of SpaceX), given at the GPU

Tech Conference, San Jose, California, in March 17 - 20, 2015 16, and Lichtl, at the end, strongly recommended

that one look at the research of Vasilyev, Regele, Lamb, Ramaprabhu, Oefelein, and Massot. Lichtl and Jones

mentioned Wavelets and Wavelet compression in their slides and I also uncovered that one should learn and

use something called “wavelet collocation.”

I should mention that my goal is to get humanity to become a truly spacefaring civilization and to enable

all the kinds of exploration as seen in Christopher Nolan and Kip Thorne’s Interstellar (2014). I share in

SpaceX’s mission of getting humanity to become a truly spacefaring civilization, and so I want to meet the

needs of and directly help out SpaceX in anyway.17

16https://youtu.be/txk-VO1hzBY, originally posted at http://on-demand.gputechconf.com/gtc/2015/video/S5398.html17https://www.linkedin.com/in/ernestyalumni

41

I had no idea what a wavelet is, nor even the context of where wavelets come from, or used, or what not. The

first thing I now do when I want to pick up a new skill is to see if the MIT OCW has a course on it, and if any

of the websites providing MOOCs (Massive, Open, Online Courses) - edX, coursera, udacity - has anything.

There was MIT OCW 18.327, Wavelets, Filter Banks, and Applications, taught in Spring 2003 by Strang and

Amaratunga. But the problem was this.

24. The Problem (and the Solution)

24.1. The Problem with Education.

Through a sequence of things that I’ll tell you about, I got really interested in seeing if we

could make higher education available to everybody. I think it’s the most important that a

human being could do, because human resources - people - are the most important resources

on the planet and everything that you can make and do comes through education.

For you, this might be relatively trivial and obvious because you made it into Brown

[University], but if youre in Africa, in India, and in the developing world; if you have medical

problems, that might not be quite as obvious. –Sebastian Thrun, Udacity 18

If we’re to fulfill the promise of making education, at the highest levels, open, openly and freely available to

anyone, anywhere, at anytime, then we need to address these problems I had with this MIT OCW 18.327,

and in general, with MIT OCW and MOOCs.

• The code included in the tools directory of MIT OCW 18.327 used Matlab. A Matlab Individual

license is $2,150 US dollars 19. The cost of using Python is $0. I can’t afford a single individual

license, let alone, if and when I run a company, want to afford licenses for my staff. The cost of using

wavelets should be $0.

Coursera has Computational Methods for Data Analysis that has a section on wavelets, taught by

Nathan Kutz, in January-March 2015, but Matlab is exclusively used.

From my (unscientific) survey of browsing the web of disparate university course websites, courses on

wavelets uses Matlab exclusively.

I’m cognizant of GNU Octave as a direct open-source alternative to Matlab and Matlab code is

presumed to run on GNU Octave, but there are things like the Matlab Wavelet Toolkit that is exclusive

(and exclusively a proprietary black box 20) to Matlab, and loading packages in GNU Octave doesn’t

always work (nor is well-documented).

We can do better, and we can do everything and more in Python. Installing packages is especially

easy with pip and Homebrew. As great as GNU Octave is, and it is great, I can’t get the wavelet

package in GNU Octave to easily run without errors, nor all the file and website IO that I can with

Python.

Again, I want to emphasize that relying on Matlab for research and engineering deployment, and

education is not cost-effective, not amenable to anything other than a black box, nor meets the

promise of making higher education openly and freely available to everybody, anywhere, at anytime.

A Matlab Individual License is $2150. The cost of using Python is $0. Even at the proposed hourly

minimum wage in the U.S. of $15, this is 143 work-hours. The average monthly salary of a Medical

Doctor in the Czech Republic is $961 21. We cannot expect education, let alone online education, to

live up to its promise of reaching anyone, anywhere, at anytime, who desires to learn and improve

18Online Learning: The Challenge - Sebastian Thrun19http://www.mathworks.com/pricing-licensing/20http://www.mathworks.com/help/wavelet/ref/upcoef.html21http://www.worldsalaries.org/czechrepublic.shtml

42

society, if the required equipment for learning costs more than two months of a living wage - Open

Source Software (OSS), and in particular Python and its libraries that I am championing here, delivers

on that promise.

• Powerpoint(-like) slides The serious defect with using Powerpoint (or Powerpoint-like slides, in-

cluding beamer for LaTeX) is this: http://norvig.com/Gettysburg/.

We should stop using Powerpoint slides to convey technical information and ideas. At the very least,

an accompanying document should “flesh out” the slides.

The serious defect(s) when using the MIT OCW material, or any lecture slides, is the Powerpoint

template and structure itself and the fact that the presenter is not there and is not present to explain

the slides. It causes me, and anyone else, to have to reconstruct what the presenter meant in the slides

and it causes confusion and it’s inefficient in learning.

• Typography The typography of the slides and handouts is poor and doesn’t look good or pleasing

at all. Nor is the typography standardized in any way. It leads to a lot of confusion.

Yes, typography matters, because the analogy is this: before the Apple iMac, the convention was

that the PC is this beige, uninspiring plastic box. Nothing was devoted to its design, and for the

manufacturers that did attempt to, such as Sony with its VAIO line, priced design exorbitantly high.

Anyways, the convention for the Wintel PC was to be utilitarian and starkly and functionally beige.

The Apple iMac in 1998 was stunning in its design (by Jonathan Ives). You wanted to use the iMac

more. You wanted to play with it and learn more about it. You enjoyed using it more. And then you

ended up increasing your productivity because you “naturally” wanted to use it more.

The analogy, I think, applies in this case: the typography of the slides for MIT OCW 18.327 is lacking

and so are many of the MIT OCWs (e.g. MIT OCW 16.512 Rocket Propulsion). But if the typography

was better (and elegant), we can help with readability, making it easier to understand (and decrease

confusion), and make them more fun to use (and reuse).

Also, in a lot of the slides, not just for MIT OCW 18.327, but in many of the MIT OCWs, the graphs,

pictures, and figures are blurry and the plot labels are unreadable.

• Books don’t update themselves. They get outdated with progress and so do the material around

it.

• Books aren’t interactive; if you’re reading about a technical area with computer applications, the com-

puter applications are divorced, in section and in structure, from the concepts and theory presented,

when they should go hand in hand and in parallel.

• Typos in books and in the MIT OCW material do cause confusion and there’s no way for the reader

to correct them directly and share this edit with others.

• MOOCs are expensive. The cost of a MOOC is upwards to and beyond $244,000 22 and it’s hugely

time-consuming 23

• There aren’t many MOOCs that serve the graduate and post-graduate level. “Reservoir Geomechanics”

taught by Zoback 24 pulled no punches and was a really good, high, graduate-level course on Reservoir

Geomechanics, a.k.a. oil drilling and fracking. Otherwise, for graduate-level courses, I find the MIT

OCWs to be more complete, as it’s (assumedly) all the exact same material as used in the offline class.

But again, see the problems I mentioned above.

22“How and Why Institutions are Engaging with MOOCsAnswers in Report MOOCs: Expectations and Reality”, https:

//onlinelearninginsights.wordpress.com/tag/costs-of-developing-a-mooc/23Jonah Miller, “Course Review: Practical Numerical Methods with Python by George Washington University”, https:

//www.class-central.com/report/gwu-numerical-mooc-review/24https://lagunita.stanford.edu/courses/EarthSciences/GP202/Spring2014/about

43

• Prof. Strang hasn’t taught this class in over ten years and Dr. Amaratunga had left MIT and had

not responded to inquiries at his current occupation at Scaled Display Technologies. This seems like

a general problem with any class online or offline because there’s a shelf-life for the material (and the

instructors!). The hope for the online format is that there’ll be someone online that can help out.

That’s why I place a lot of hope and admiration for quora and stackexchange.

24.1.1. Solutions.

• Python. I’ve rewritten about half of the Matlab code used in MIT OCW 18.327 in Python and placed

it all on github. I’ve also written up Python scripts to accompany later Handouts (Handout 16 and

on) that are missing in the original MIT OCW 18.327 material, but it was obvious from the Handouts

that more computer-generated plots were there (but wasn’t in the tools directory).

I’ve outlined explicitly, and in conjunction with the Handouts, how and what to use with the different

Python libraries, including PyWavelets, and encourage its use and interactivity.

• In between a textbook and lecture slides: This is an excellent example of how to provide

material for a talk or lecture, and not the Powerpoint slides: Title:The singularity theorems in general

relativity. II, Speaker:Amir Aazami (Kavli IPMU), Date:Fri, May 17, 2013, 10:00 - 11:30, http:

//db.ipmu.jp/seminar/sysimg/seminar/945.pdf. Take a look at the pdf 945, “Lecture I: Basic

properties of Lorentzian manifolds.” They are something in between a textbook and Powerpoint

slides. This is what I tried to do here with this LaTeX file, that I make publicly available on github.

• Use LaTeX for typography and make it publicly available on github (and the pdf on Google Drive or

Dropbox) so people can copy, edit, and add onto the LaTeX or the pdf. Since it’s on github, students

and (older) learners can share their notes and insights directly there.

Look, I know it’s time-consuming and sometimes a pain-in-the-ass to LaTeX typeset everything.

Sometimes, I see some students make fantastic looking and very comprehensive notes of the course

lectures in LaTeX. Why not make the pdf AND the LaTeX available publicly on github (or wherever)

so that we can crowdsource this process and have more insights and questions and answers from the

bottom-up, as opposed to top-down from lecturer to students?

For plots, figures, and pictures, you can make your own plots, as I’ve meticulously outlined and

instructed here, and improve upon them, so they’d be crystal clear.

• On github, anyone can update this LaTeX file and the Python/GNU Octave code anytime.

• Python’s interactive interpreter is one of its most famous and greatest features.

• On github, anyone can make edits and correct for typos.

• Sharing stuff on github and on Google Drive or DropBox costs $0. I’m not maintaining a discussion

forum or moderating it either so that costs $0, also in time.

• This LaTeX file and related code was born out of trying to understand and get the tools for, quickly,

combustion CFD. I write for a high-level. But having this material open-sourced, if someone wants to

“fork” or make their own version with more elementary and pedagogical explanations and language,

he or she can go right ahead (just remember to share it).

• I could appreciate that university professors can’t or don’t want to (I don’t know) answer the questions

from everyone wanting to learn about wavelets. Look, I understand, we’re all busy people. Even TAs

aren’t getting paid enough, and TAs wouldn’t want to answer every emailed question from everyone,

nor moderate an online discussion forum (it’s time-consuming and can be aggravating). Why not

crowdsource and divide out this labor online?44

The kinds of discussion forums on udacity, Stanford online (was edX, now Lagunita) is great for this

and, in my opinion, very underutilized. The discussion forums are a gem and should be front and center

of the learning process that would move the office hours, TA sessions, in continental Europe what

would be called the Tutorial, Tutorium and Exercise session oder Ubungsblatter online. I recognize

that setting up and moderating a discussion forum is a time-consuming effort. Beyond a Usenet-style

email list, I wanted to try here an open-sourced, publicly available LaTeX file that anyone can copy,

edit, reuse, and add onto to further the discussion.

My rationale with this is everybody can be “on the same page,” technical, mathematical typesetting

can be done quickly (from my experience, and please tell me about anything I’m missing, but LaTeX

math typesetting online, on wordpress, a forum, stackexchange, is very time-consuming and not fun at

all - I’m well aware that there are a number of individuals who blog posts or answer on stackexchange

very clearly and voluminously math expressions with specific, custom LaTeX typesetting, but it ends

up being a one-way street of communication, because we’re not going to type that much if we can’t

copy, paste, and edit).

• One single LaTeX/pdf file for the course or subject. How many times has one had to dig up

different handouts or lectures notes on different files to reinforce a previous concept? Just throw it

all in one LaTeX/pdf file. And unlike a lot of the slides and handouts for the MIT OCWs and other

classes, with LaTeX generated pdf files; they’re completely searchable: we don’t have to waste as much

time on creating an indexing scheme.

• All the algebra steps missing during a class lecture is done explicitly and clearly here.

Plenty of questions that I have at least (and maybe some other students) involve showing the steps

explicitly. I can understand that the class time is limited and we don’t have time for that in class. We

can time-shift that to here, which is usually done in TA office hours or Ubungsblatter. But if the aim

is to make education open and freely available, to anyone, anywhere, and at anytime, then we have to

move that one-on-one learning online and in another format.

I don’t have all the answers to education, online or offline, but clearly, we can do better, and do it at a lower

cost and price point. But we cannot fail in meeting the needs and challenges of education. This is not just for

students in university. This is also for employee retraining, employee development, career retaining, whether

at a corporation, government, or a startup, or a freelance individual. Failing to bring open access to

technical education and modular skill acquisition is failing the future workers of the world. As

an American, of course I’m concerned with the huge, termed skills gap between the U.S. labor workforce,

between the portion of the U.S. workforce that finds themselves down-sized out of work, and need to acquire

new skills fast, between companies that are trying to fill and meet their human resources needs by checklisting

qualifications and with potential employees and workers who don’t have access to or know where to begin

to acquire these skills to help meet the needs of companies large and small, mature and startup. But these

issues apply all around the world, including developed countries, in particular the U.K., Ireland, France,

Germany25.

24.2. Helping Out and Helping Others. I will be restarting my crowdfunding campaign on Open/Tilt

to crowdfund for money to do more computational fluid dynamics and wavelet applications on the Mac and

Linux. I want to develop on Swift using the Accelerate framework for numerical computation, including the

wavelet stuff here. Also, I want to do CFD and wavelet stuff on a Linux box. I would be crowdfunding for

about $43,000, $16,000 in new computer equipment for the Mac and Linux, and including $20,000 to pay

myself, for my efforts here and continuing development. About 15-20% goes to taxes. It’s still a fraction of

the cost of a MOOC at $244,000 or the pay and equipment of TAs and an instructor and administrative staff

at an offline university (single high-end TA salary is $23000). I’ll launch the crowdfunding campaign as soon

25“Skills Mismatches and Labour Mobility”, europa.eu http://ec.europa.eu/europe2020/pdf/themes/27_skills_gaps_and_

labour_mobility_02.pdf

45

as I can; any and all questions and negative feedback (and funds) would be welcomed right now. However,

as I’ve mentioned several times above, I’m committed to free and open education and tools so regardless of

how much I raise (if at all), or even if I fail to get others on board to help out in this effort with Wavelets

(I’ve tried emailing Prof. Strang and Dr. Amaratunga a number of times but I’ve either gotten no substantial

help or critical feedback, or no response at all), I’ll always keep all this stuff out for free and public use and

distribution.

If you can’t help out with my own crowdfunding efforts, or I haven’t convinced you enough yet (please tell

me why so I could improve), then at the very least, please read and contribute to the Memorial fund for the

creator of Python library matplotlib, John Hunter. “On August 28 2012, John D. Hunter, the creator of

matplotlib, died from complications arising from cancer treatment, after a brief but intense battle with this

terrible illness. John is survived by his wife Miriam, his three daughters Rahel, Ava and Clara, his sisters

Layne and Mary, and his mother Sarah.”26 I used matplotlib heavily for plotting with wavelets in Python

and you should too, as it’s open-source, it costs effectively $0, and it’s very flexible and amenable to changing

needs. As of right now, I’m still seeking any kind of position at SpaceX, as I believe in their mission to the

core, and I had just gotten my Masters in Physics, and I don’t have any income right now, but I am PayPal’ing

to the John Hunter Memorial Fund next week (20150709) as soon as my bank transfer clears, with whatever I

can. If you can’t help me out, fine, but at least help out the John Hunter Memorial Fund because matplotlib

has been very useful.

References

[1] Gilbert Strang, and Kevin Amaratunga. 18.327 Wavelets, Filter Banks and Applications, Spring 2003. (Massachusetts Institute

of Technology: MIT OpenCourseWare), http://ocw.mit.edu (Accessed 19 Jun, 2015). License: Creative Commons BY-NC-

SA

[2] Gilbert Strang, Truong Nguyen. Wavelets and Filter Banks, Wellesley-Cambridge Press; 2nd edition, 1996. ISBN-13:

978-0961408879

[3] Elliott H. Lieb and Michael Loss, Analysis (Graduate Studies in Mathematics), Book 14, American Mathematical Society,

2nd edition, 2001. ISBN-13: 978-0821827833

[4] P. Wojtaszczyk, A Mathematical Introduction to Wavelets, (London Mathematical Society Student Texts), Cambridge

University Press, 1997. ISBN-13: 978-0521578943

[5] Ingrid Daubechies, Ten Lectures on Wavelets (CBMS-NSF Regional Conference Series in Applied Mathematics), SIAM:

Society for Industrial and Applied Mathematics, 1992. ISBN-13: 978-0898712742

E-mail address: ernestyalumni@gmail.com

URL: http://ernestyalumni.wordpress.com

26John Hunter Memorial Fund, Numfocus.org, http://numfocus.org/news/2012/08/28/johnhunter/

46