Functional Analysis, Part 1 - Paul Mitchener · 2017. 9. 20. · As a simple example of analysis in the world of normed spaces we present the following result. Proposition 1.16 Let

Functional Analysis, Part 1

Paul D. Mitchener

Contents

1 Normed Spaces 51.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.2 Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.3 Banach Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.4 Closures and Completions . . . . . . . . . . . . . . . . . . . . . . 131.5 Convexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2 Linear Maps and Continuity 172.1 Open Sets and Continuity . . . . . . . . . . . . . . . . . . . . . . 172.2 Bounded Linear Maps . . . . . . . . . . . . . . . . . . . . . . . . 182.3 Finite-Dimensional Spaces . . . . . . . . . . . . . . . . . . . . . . 222.4 The Open Mapping Theorem . . . . . . . . . . . . . . . . . . . . 24

3 Spaces of Continuous Functions 273.1 Dual Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.2 Zorn’s Lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.3 Proof of the Hahn-Banach Theorem . . . . . . . . . . . . . . . . 303.4 The Stone-Weierstrass Theorem . . . . . . . . . . . . . . . . . . . 32

4 Hilbert Spaces 354.1 Inner Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.2 Orthogonal Complements . . . . . . . . . . . . . . . . . . . . . . 394.3 Dual Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424.4 Adjoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

5 Orthonormal Sets 475.1 Orthonormal Sets and Bases . . . . . . . . . . . . . . . . . . . . . 475.2 Series Related to Orthonormal Sequences . . . . . . . . . . . . . 505.3 Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

3

4 CONTENTS

Chapter 1

Normed Spaces

Many problems, for example those dealing with the existence and uniquenessof solutions to systems of differential equations, naturally involve dealing withinfinite-dimensional vector spaces. Typically the elements of such spaces arefunctions.

We want to be able to do analysis on the vector speces we consider. Thereforewe assume that all vector spaces are over either the field, R, of real numbers, orthe field, C, of complex numbers. Throughout these notes we write F to denoteeither R or C.

1.1 Examples

Here are a few examples of infinite-dimensional vector spaces.

Example 1.1 Let X be a metric space. Let C(X) denote the set of continuousfunctions f :X → F. Then C(X) is a vector space under the operations ofpointwise addition and scalar multiplication of functions. By this, we mean thataddition and scalar multiplication are defined by writing

(f + g)(x) = f(x) + g(x) f, g ∈ C(X), x ∈ X,

and

(αf)(x) = αf(x) α ∈ F, f ∈ C(X), x ∈ S,

respectively.The vector space C(S) is infinite-dimensional unless the metric space X is

finite.

Example 1.2 Let p ≥ 1. Let lp denote the set of sequences (an) in the field Fsuch that the series

∞∑n=0

|αn|p

5

6 CHAPTER 1. NORMED SPACES

converges. Then lp is an infinite-dimensional vector space. The operations ofaddition and scalar multiplication are defined by writing

(an) + (bn) = (an + bn) (an), (bn) ∈ lp

and

α(an) = (αan) α ∈ F, (an) ∈ lp

respectively.

Example 1.3 Let c0 be the set of sequences, (an), in the field F such thatan → 0 as n → ∞. The operations of addition and scalar multiplication aredefined as in the above example.

1.2 Norms

In order to perform any meaningful analysis on infinite-dimensional vectorspaces it is helpful to have some additional structure. One of the simplestand most useful structures to have available is that of a norm.

Definition 1.4 Let X be a vector space over the field F. Then a norm on thespace X is a function

‖ − ‖:X → R≥0

such that:

• ‖αx‖ = |α|‖x‖ for all scalars α ∈ F and vectors x ∈ X

• ‖x+ y‖ ≤ ‖x‖+ ‖y‖ for all vectors x, y ∈ X

• Given a vector x ∈ X, if ‖x‖ = 0 then x = 0

The second of the above axioms is sometimes called the triangle inequality.A vector space equipped with a norm is referred to as a normed vector space

or merely as a normed space. If X is a normed vector space, and Y ≤ X is avector subspace, then Y is also a normed vector space; the norm on the spaceY is defined by restriction of the norm on the space X.

The following handy result is a consequence of the triangle inequality; weleave the proof as an exercise.

Proposition 1.5 Let X be a normed vector space. Let x, y ∈ X. Then:

|‖x‖ − ‖y‖| ≤ ‖x− y‖

2

We present an easy proposition as an example of playing with the axioms.

1.2. NORMS 7

Proposition 1.6 Let X be a normed vector space. Then there is a metric onthe space d defined by the formula

d(x, y) = ‖x− y‖

Proof: Certainly the supposed metric, d, is a function d:X ×X → R≥0. Weneed to check that the axioms of a metric hold.

• Let x, y ∈ X. Then

‖x− y‖ = ‖(−1)(y − x)‖ = | − 1|‖y − x‖ = ‖y − x‖

Hence d(x, y) = d(y, x)

• Let x, y, z ∈ X. Then

‖x− z‖ = ‖(x− y) + (y − z)‖ ≤ ‖x− y‖+ ‖y − z‖

Hence d(x, z) ≤ d(x, y) + d(y, z)

• Suppose that x, y ∈ X and d(x, y) = 0. Then ‖x− y‖ = 0 and so x = y.

2

The above metric is called the metric induced by the the norm. Because anynormed vector space is also a metric space, we can talk about limits in normedvector spaces, and things such as open sets, and functions being continuous.The following is left as an exercise.

Proposition 1.7 Let V be a normed vector space. Then we have a continuousmap f :V → R defined by the formula f(v) = ‖v‖. 2

Definition 1.8 Let X be a vector space. Let d be a metric on the set X.

• The metric d is said to be translation-invariant if d(x+ z, y+ z) = d(x, y)for all vectors x, y, z ∈ X

• The metric d is said to be dilation-invariant if d(αx, αy) = |α|d(x, y) forall scalars α ∈ F and vectors x, y ∈ X

Proposition 1.9 Let X be a normed vector space. Then the induced metric istranslation-invariant and dilation-invariant.

Proof: Let d denote the induced metric.

• Let x, y, z ∈ X. Then

d(x+ z, y + z) = ‖(x+ z)− (y + z)‖ = ‖x− y‖ = d(x, y)


• Let α ∈ F, x, y ∈ X. Then

d(αx, αy) = ‖αx− αy‖ = |α|‖x− y‖ = |α|d(x, y)

2

Some of the examples contained in section 1.1 are normed vector spaces.

Example 1.10 Let K be a compact metric space. Then there is a norm on thevector space C(K) defined by the formula

‖f‖ = sup{|f(x)| | x ∈ K}.

Recall that if K is a compact metric space, and g:K → Y is a continuousfunction into another metric space, then the image g[K] is compact, and there-fore bounded. In the above, since f is continuous its image is bounded, so thestated norm is defined and finite.

We will check that the axioms required for the stated function in the aboveexample to be a norm are true.

• Let f ∈ C(K) be a function, and let α ∈ F be a scalar. Then

‖αf‖ = sup{|αf(x)| | x ∈ K}= sup{|α||f(x)| | x ∈ K}= |α|‖f‖

• Let f, g ∈ C(K) be functions. Then for each point x ∈ K we have thetriangle inequality

|f(x) + g(x)| ≤ |f(x)|+ |g(x)|

Hence

‖f + g‖ = sup{|f(x) + g(x)| | x ∈ K}≤ sup{|f(x)|+ |g(x)| | x ∈ K}≤ sup{|f(x)| | x ∈ K}+ sup{|g(y)| | y ∈ K}= ‖f‖+ ‖g‖

• Suppose that f ∈ C(K) and ‖f‖ = 0. Then

sup{|f(x)| | x ∈ K} = 0

and so f(x) = 0 for all points x ∈ K. Thus f = 0.

Example 1.11 Let l1 be the set of sequences, (αn), in the field F such that∑∞n=1 |αn| converges. Then we can define a norm on the vector space l1 by the

formula

‖(αn)‖ =∞∑n=1

|αn|.

1.3. BANACH SPACES 9

Again, we check the axioms.

• Let (αn) ∈ l1, and α ∈ F. Then

‖α(αn)‖ =∞∑n=1

|ααn| = |α|∞∑n=1

|αn| = |α|‖(αn)‖.

• Let (αn), (βn) ∈ l1. Then for each natural number n we have the triangleinequality

|αn + βn| ≤ |αn|+ |βn|

Taking sums, we see

‖(αn) + (βn)‖ =∞∑n=1

|αn + βn| ≤∞∑n=1

|αn|+∞∑n=1

|βn| = ‖(αn)‖+ ‖(βn)‖.

• Suppose that (αn) ∈ l1 and ‖(α1)‖ = 0. Then

∞∑n=1

|αn| = 0

and so αn = 0 for all points n, since |αn| ≥ 0. Thus (αn) = 0.

Example 1.12 Let a < b. Then there is a norm on the space C[a, b] defined bythe formula

‖f‖ =∫ ba

|f(x)| dx

As above, the only potentially unclear part of proving that the function ‖−‖is a norm is checking the triangle inequality.

To do this, let f, g ∈ C[a, b]. Observe

‖f + g‖ =∫ ba|f(x) + g(x)| dx

≤∫ ba|f(x)|+ |g(x)| dx

=∫ ba|f(x)| dx+

∫ ba|g(x)| dx

= ‖f‖+ ‖g‖

and we are done.

1.3 Banach Spaces

The nicest type of normed vector spaces are those which are complete as metricspaces. To see what completeness involves we need to look at the notion ofconvergence in a normed vector space.


Definition 1.13 Let V be a normed vector space. Then a sequence, (xn), inthe space V converges in norm to a limit x ∈ V if for all real numbers ε > 0there exists a natural number N such that ‖xn − x‖ < ε whenever n ≥ N .

We can rephrase the above definition by saying that the sequence (xn) con-verges (in norm) to a limit x ∈ X if

limn→∞

‖xn − x‖ = 0

The proof of the following result is left as an exercise.

Proposition 1.14 Let V be a normed vector space. Let (xn) and (yn) be se-quences in V converging to points x and y respectively. Let α, β ∈ F. Then thesequence (αxn + βyn) converges to αx+ βy. 2

Definition 1.15 Let V be a normed vector space. Then a sequence, (xn), inthe space V is called a Cauchy sequence if for all real numbers ε > 0 there existsa natural number N such that ‖xm − xn‖ < ε whenever m,n ≥ N .

As a simple example of analysis in the world of normed spaces we presentthe following result.

Proposition 1.16 Let (xn) be a norm-convergent sequence in a normed vectorspace. Then the sequence (xn) is a Cauchy sequence.

Proof: Let x be the limit of the sequence (xn). Let ε > 0. Then we can finda natural number N such that if n ≥ N then ‖xn − x‖ < ε2 . In particular, ifm,n ≥ N then

‖xm − xn‖ ≤ ‖xm − x‖+ ‖xn − x‖ < ε

2

These notions can be generalised to the world of metric spaces. A sequence(xn) in a metric spaceX is said to converge to a limit x ∈ X if for all real numbersε > 0 there exists a natural number N such that d(xn, x) < ε whenever n ≥ N .A sequence (xn) in a metric space X is said to be a Cauchy sequence if for allreal numbers ε > 0 there exists a natural number N such that d(xm, xn) < εwhenever m,n ≥ N .

Proposition 1.17 Let V be a normed vector space. Let (xn) be a Cauchysequence in the space X, and suppose that the sequence (xn) has a convergentsubsequence. Then the sequence (xn) converges. 2

The proof of this result is left as an exercise.

Definition 1.18 A normed vector space V is called complete if every Cauchysequence in V converges in norm to some limit. A Banach space is a completenormed vector space.

1.3. BANACH SPACES 11

More generally, a metric space is also called complete if every Cauchy se-quence converges to some limit. For example, one of the defining properties ofthe field of real numbers, R, is that it is complete. The field of complex numbers,C, is also complete.

Example 1.19 Let K be a compact metric space. Then the space C(K), withthe norm defined in example 1.10, is complete.

Proof: Let (fn) be a Cauchy sequence in the space C(K). Let x ∈ K. Thenfor any natural numbers m,n ∈ N we have the inequality

|fm(x)− fn(x)| ≤ sup{|fm(x)− fn(x)| | x ∈ K} = ‖fm − fn‖

Hence the sequence (fn(x)) is a Cauchy sequence of numbers in the field F.By completeness of the field F the sequence (fn(x)) converges to some limit,f(x).1

Let ε > 0 be a real number. Then the sequence of functions (fn) is a Cauchysequence so we can find a natural number N such that ‖fm−fn‖ < ε2 wheneverm,n ≥ N . Let x ∈ X, and let m,n ≥ N . Then |fm(x) − fn(x)| < ε2 and so|fm(x)− f(x)| < ε if we let n→∞.

Therefore ‖fm − f‖ < ε whenever m ≥ N and the function f is the limit ofthe sequence (fn). All that remains is to show that the function f is continuous.

Again, let ε > 0. Let x ∈ X. Then we can find a neighbourhood U 3 xand a natural number N such that |fN (y) − fN (x)| < ε3 whenever y ∈ U and‖fN − f‖ < ε3 . For each point y ∈ U we therefore have the inequality

|f(y)− f(x)| ≤ |f(y)− fN (y)|+ |fN (y)− fN (x)|+ |fN (x)− f(x)| < ε

The function f is therefore continuous. 2

The above example illustrates a pattern which holds for many proofs ofcompleteness. The pattern is roughly as follows.

• Consider a Cauchy sequence in the metric space we are looking at. Pro-duce a point which looks like it should be the limit of the sequence.

• Prove that the point produced is the limit of the Cauchy sequence we areconsidering.

• If necessary, prove that the point produced belongs to the space we arelooking at.

Example 1.20 The space Fk can be equipped with a norm defined by the for-mula

‖(α(1), . . . , α(k))‖ = |α(1)|+ · · ·+ |α(k)| αi ∈ F.

If we give the space Fk this norm, it is a Banach space.1Recall that F denotes either R or C


Proof: The proof that the given formula defines a norm is similar to example1.11.

Let (vn) be a Cauchy sequence in the space Fk. Write

vn = (α(1)n , . . . , α

(k)n ).

Let j ∈ {1, . . . , k}. Observe that for all m,n ∈ N, we have |α(j)m − α(j)n | ≤‖vm − vn‖. Hence the sequence (α(j)n )∞n=1 is a Cauchy sequence in the field F.By completeness of the field F, this sequence converges to some limit, α(j). Let

v = (α(1), . . . , α(k)) ∈ Fk.

We claim that ‖vn − v‖ → 0 as n→∞. To see this,Now, observe

limn→∞

‖vn − v‖ = limn→∞

|α(1)n − α(1)|+ · · ·+ limn→∞

|α(k)n − α(k)| = 0.

So the sequence (vn) has limit v. We see that the space Fk is complete. 2

We shall see later on that any finite-dimensional normed vector space is aBanach space. Some further examples of Banach spaces include:

• The space l1 of all sequences (αn) in the field F such that the sum∑∞n=1 |αn| converges. As noted earlier, we give this space the norm defined

by the formula ‖(αn)‖ =∑∞n=1 |αn|.

• The space l∞ of all bounded sequences (αn) in the field F, with the normdefined by the formula ‖(αn)‖ = sup{|αn| | n ∈ N}.

• The subspace c0 of l∞ consisting of all sequences (αn) in the field F suchthat αn → 0 as n→∞.

The relevant proofs are left as exercises.

Example 1.21 Recall that we can define a norm on the vector space C[−1, 1]by the formula

‖f‖ =∫ 1−1|f(x)| dx

Let n ∈ N. Define fn ∈ C[−1, 1] by

fn(t) =

{(1− t)n t ≤ 01 t ≥ 0

Let

g(t) =

{0 t < 01 t ≥ 0

But

‖fn − g‖ =∫ 0−1

(1− t)n dt = 1n+ 1

→ 0

1.4. CLOSURES AND COMPLETIONS 13

as n→∞. So the sequence (fn) converges to g with respect to the above norm.In particular, the sequence (fn) is Cauchy.

But g is not continuous, and, clearly, if f ∈ C[−1, 1], then ‖f − g‖ > 0.So no function in C[−1, 1] can be the limit of the Cauchy sequence (fn) for ourgiven norm.

Thus C[−1, 1] is not a Banach space for this norm.

We have to be careful here; we have an alternative norm on C[−1, 1] definedby

‖f‖′ = sup{|f(t)| | t ∈ [−1, 1]}

for which, as we saw above, C[−1, 1] is a Banaach space.

1.4 Closures and Completions

The notion of completion is a way of passing from normed vector spaces toBanach spaces.

Definition 1.22 Let V be a normed vector space. Let A be a subset of V .Then we define the closure of A, A, to be the set of all elements v ∈ V that arenorm-limits of sequences of the space A.

We call the subset A ⊆ V closed if A = A.

Example 1.23 Let V be a normed vector space, and x ∈ V . Then the one-pointset {x} is closed.

Example 1.24 Let V be a normed vector space. Let x ∈ V and δ > 0. Thenwe define the open ball

B(x, δ) = {v ∈ V | ‖v − x‖ < δ}.

The closure is the closed ball

B(x, δ) = {v ∈ V | ‖v − x‖ ≤ δ}.

Note that when we need to be more precise, we write BV (x, δ) to denotethe open ball in the space V at the point x with radius δ. This more precisenotation is sometimes needed when we are considering more than one normedvector space in a problem.

Proposition 1.25 If U is a vector subspace of a normed vector space V , thenthe closure U is also a vector subspace.

Proof: Observe 0 ∈ U ⊆ U , so U 6= ∅.Let v, w ∈ U and α, β ∈ F. Let (vn) and (wn) be sequences in the space U

converging to v and w respectively.Then certainly, for each n ∈ N, we have αvn + βwn ∈ U , and

limn→∞

‖αvn + βwn − (αv + βw)‖ ≤ |α| limn→∞

‖vn − v‖+ |β| limn→∞

‖wn − w‖ = 0.


Hence αv+ βw ∈ U , and the closure U is a vector subspace as required. 2

We leave the proof of the following result as a straightforward exercise.

Proposition 1.26 Let V be a Banach space. Then a vector subspace of V is aBanach space if and only if it is closed. 2

Definition 1.27 Let V be a normed vector space. We call a subspace U densein V if U = V .

The notion of a subspace being dense can be rephrased as follows.

Proposition 1.28 Let V be a normed vector space, and let U ⊆ V be a sub-space. Then U is dense in V if and only if for all v ∈ V and ε > 0 there existsu ∈ U such that ‖u− v‖ < ε. 2

Example 1.29 Let U be the subspace of l1 consisting of all sequences (αn) forwhich there exists N ∈ N such that αn = 0 whenever n ≥ N .

Let v = (βn) ∈ l1. Let ε > 0. The series∑∞n=1 |βn| converges to the norm

‖v‖. Hence, looking at partial sums, we have N ∈ N such that∣∣∣∣∣N∑n=1

|βn| − ‖v‖

∣∣∣∣∣ < ε.Let

αn =

{βn n ≤ N0 n > N

.

Then u = (αn) ∈ U , and ‖u− v‖ < ε. We conclude that U is a dense subsetof l1.

Definition 1.30 Let V be a normed vector space. Then we call a Banach space,V̂ , a completion of V if V is a dense subset of V̂ .

Note that if a Banach space V has a dense vector subspace U , then V isa completion of U . We will see later on that every normed vector space has acompletion. We defer the proof to chapter 3, where we can be clever about it.

Example 1.31 Recalle that the vector space C[a, b] can be equipped with a normdefined by the formula

‖f‖ =∫ ba

|f(x)| dx

It is not complete with respect to this norm. We define the Banach spaceL1[a, b] to be the completion of C[a, b] for this particular norm.

1.5. CONVEXITY 15

1.5 Convexity

The following concept will come in handy later on, though we will not use itimmediately.

Definition 1.32 Let V be a real or complex vector space. We call a subsetC ⊆ V convex if for all x, y ∈ C and α ∈ [0, 1], we have

αx+ (1− α)y ∈ C.

In other words, for any two points x, y ∈ C, the straight line segment joiningx and y is also in C.

Example 1.33 Let V be a normed vector space, x ∈ V and δ > 0. Then theopen ball B(x, δ) is convex.

We need the notion of convexity in a few places in these notes (and inparticular in the proof of the open mapping theorem in the next chapter). Theproof of the following properties are left as exercises.

Proposition 1.34 Let V be a normed vector space, and let C ⊆ V be convex.Then the closure C is also convex. 2

Proposition 1.35 Let V and W be normed vector spaces, let T :V → W be alinear map, and let C ⊆ V be convex. Then the image T [C] ⊆W is convex. 2

Chapter 2

Linear Maps and Continuity

2.1 Open Sets and Continuity

Definition 2.1 Let V be a normed vector space. We call a subset U ⊆ V openif for all x ∈ U we have ε > 0 such that B(x, ε) ⊆ U .

Example 2.2 Let x ∈ X and δ > 0. Then the open ball B(x, δ) is itself anopen set.

The next two results are standard results from the theory of metric spaces;the details are left as an exercise.

Proposition 2.3 Let V ve a birned vector space.

• The sets ∅ and X are open.

• The intersection of finitely many open sets is open.

• The union of an arbitrary collection of open sets is open.

2

Theorem 2.4 Let X and Y be metric spaces. A map f :X → Y is continuousif and only if for any open set U ⊆ W , the inverse image f−1[U ] ⊆ V is open.2

Proposition 2.5 Let V be a metric space. Then a subset A ⊆ V is closed ifand only if the complement X\A is open.

Proof: Let A ⊆ V be closed. Let x ∈ V \A. Suppose there is no δ > 0 forwhich B(x, δ) is contained in V \A.

Then, set δ = 1n . Then we have xn ∈ A such that xn ∈ B(x,1n ), that is to

say ‖xn − x‖ < 1δ .

17

18 CHAPTER 2. LINEAR MAPS AND CONTINUITY

It follows that ‖xn − x‖ → 0 as n → ∞, so x ∈ A. But x 6∈ A, so thiscontradicts A = A. We conclude that there is some δ > 0 for which B(x, δ) ⊆V \A, and X\A is open.

Conversely, suppose V \A is open. Suppose x 6∈ A. Then x ∈ X\A, and byopenness, we have δ > 0 such that B(x, δ) ⊆ V \A.

Let (xn) be a sequence in A. Then no xn is within distance δ of x, andcertainly, (xn) does not converge to x. So x 6∈ A.

Certainly A ⊆ A. The above argument tells us that A = A, so that A isclosed. 2

The next result follows immediately from the above and proposition 2.3

Corollary 2.6 Let V be a normed vector space.

• The sets ∅ and V are closed.

• The union of finitely many closed sets is closed.

• The intersection of an arbitrary collection of closed sets is closed.

2

2.2 Bounded Linear Maps

In this chapter we investigate continuous linear maps between normed vectorspaces.

Definition 2.7 Let T :V → W be a linear map between normed vector spaces.We call T bounded if there exists M ≥ 0 such that ‖Tv‖ ≤M‖v‖ for all v ∈ V .

If required, we can always choose M > 0.Note that the above definition is not the same as the image of the map T

being a bounded subset of the space W .

Proposition 2.8 Let T :V →W be a linear map between normed vector spaces.Then the following are equivalent:

(i) T is continuous.

(ii) T is continuous at the point 0.

(iii) T is bounded.

Proof:

(i)⇒ (ii): This is trivial.

2.2. BOUNDED LINEAR MAPS 19

(ii)⇒ (iii): Let us take ε = 1 in the definition of continuity at the point 0. Then wecan find δ > 0 such that d(v, 0) < δ implies d(Tv, 0) < 1, ie: ‖v‖ < δimplies ‖Tv‖ < 1.

Take M = 2δ . Let v ∈ V \{0}. Then∥∥∥∥ δv2‖v‖∥∥∥∥ = δ2 < δ

Hence ‖T (δv/2‖v‖)‖ < 1. By linearity

δ

2‖v‖‖Tv‖ < 1 and ‖Tv‖ < 2

δ‖v‖ = M‖v‖

Certainly, if v = 0, then ‖Tv‖ = 0 ≤M‖v‖. So the map T is bounded.

(iii)⇒ (i): Let T :V → W be bounded. Then we can find M > 0 such that ‖Tv‖ ≤M‖v‖ for all v ∈ V .

Choose ε > 0. Write δ = ε/M . Then, for v, w ∈ V with d(v, w) < δ, wehave

‖v − w‖ < εM

and

‖Tv − Tw‖ = ‖T (v − w)‖ ≤M‖v − w‖ < MεM

= ε

We see that the map T is continuous.

2

Definition 2.9 Let V and W be normed vector spaces, with V 6= {0}, and letT :V → W be a bounded linear map. Then we define the operator norm of theoperator T by the formula

‖T‖ = supv∈V,v 6=0

‖Tv‖‖v‖

Certainly ‖T‖ ≥ 0. By the above definition, we have the inequality

‖Tv‖ ≤ ‖T‖‖v‖

for all v ∈ V , and the value ‖T‖ is the smallest number where we have such aninequality.

Actaully, our definition breaks down in the case V = {0}. In this case, wesimply define ‖T‖ = 0. The proof of the following is left as an exercise.


Proposition 2.10 Let T :V →W be a bounded linear map. Then

‖T‖ = supv∈V,‖v‖≤1

‖Tv‖.

If V 6= {0}, we have‖T‖ = sup

v∈V,‖v‖=1‖Tv‖.

2

When it comes to computing norms, we have the following.

Proposition 2.11 • Let T :V →W be a linear map, where we have a con-stant M ≥ 0 such that ‖Tv‖ ≤M‖v‖ for all v ∈ V . Then T is a boundedlinear map, and ‖T‖ ≤M .

• Let T :V → W be a bounded linear map. Suppose we have some w ∈ V ,w 6= 0, where ‖Tw‖ = A‖w‖. Then ‖T‖ ≥ A.

Proof:

• By definition, the existence of the constant M maps T a bounded linearmap. Now, by definition of the norm, assuming V 6= {0}:

‖T‖ = supv∈V,v 6=0

‖Tv‖‖v‖

≤ supv∈V,v 6=0

M‖v‖‖v‖

= M.

• By definition of the norm

‖T‖ = supv∈V,v 6=0

‖Tv‖‖v‖

≥ ‖Tw‖‖w‖

=A‖w‖‖w‖

= A.

2

Definition 2.12 Let V and W be normed vector spaces. Then we writeHom(V,W ) to denote the set of all bounded linear maps from V to W .

The following is straightforward.

Proposition 2.13 The set Hom(V,W ) is a normed vector space. The opera-tions of addition and scalar multiplication defined by the formulae

(S + T )v = Sv + Tv S, T ∈ Hom(V,W ), v ∈ V

and(αT )v = αT (v) T ∈ Hom(V,W ), α ∈ F, v ∈ V

The norm is defined by the above operator norm. 2

2.2. BOUNDED LINEAR MAPS 21

Example 2.14 Let w ∈ W . Let Tw:F → W be the linear map defined by theformula Tw(α) = αw. Then, for any scalar α ∈ F, we have

‖Tw(α)‖ = ‖αw‖ = |α|‖w‖

Hence the linear map Tw is bounded, and ‖Tw‖ = ‖w‖.

Note that any linear map T :F → W takes the form Tw for some w ∈ W .Hence we can identify the space Hom(F,W ) with the normed vector space W .

Proposition 2.15 Let V be a normed vector space, and let W be a Banachspace. Then the space Hom(V,W ) is a Banach space.

Proof: Let (Tn) be a Cauchy sequence in the space Hom(V,W ). Let v ∈ V .Observe, for all m,n ∈ N, we have

‖Tmv − Tnv‖ ≤ ‖Tm − Tn‖ · ‖v‖.

Hence the sequence (Tnv) is a Cauchy sequence in the space W . But thespace W is complete, so we have a norm limit

Tv = limn→∞

Tnv.

Let α, β ∈ F, and u, v ∈ V . Then

T (αu+βv) = limn→∞

Tn(αu+βv) = α limn→∞

Tn(u)+β limn→∞

Tn(v) = αT (u)+βT (v)

and the map T is linear.Since the norm is a continuous map, the sequence (‖Tn‖) is a Cauchy se-

quence in R. Since the real numbers are complete, we have a limit

M = limn→∞

‖Tn‖.

For v ∈ V , we have

‖Tv‖ = limn→∞

‖Tn(v)‖ ≤ limn→∞

‖Tn‖ · ‖v‖ = M‖v‖.

So the map T is bounded. We claim that T is the norm limit of theCauchy sequence (Tn) in the space Hom(V,W ), which tells us that the spaceHom(V,W ) is complete, and finishes the proof.

Let ε > 0. Then there is a natural number N ∈ N such that ‖Tm−Tn‖ < ε/2whenever m,n ≥ N . Let v ∈ V and ‖v‖ ≤ 1. Then for m,n ≥ N , we have

‖Tm(v)− Tn(v)‖ <ε

2.

Taking the limit as m→∞, we see for n ≥ N that

‖T (v)− Tn(v)‖ ≤ε

2.


Hence‖T − Tn‖ = sup

v∈V,‖v‖≤1‖Tv − Tnv‖ ≤

ε

2< ε.

It follows that T is the norm limit of the sequence (Tn) and we are done. 2

The following result is often convenient. We leave the proof as an exercise.

Proposition 2.16 Let T :V → W be a bounded linear map betweeen normedvector spaces. Then the kernel

kerT = {v ∈ V | Tv = 0}

is a closed subspace of V . 2

2.3 Finite-Dimensional Spaces

As a matter of convenience, when we mention the space Fn in this section, weequip it with the norm

(α1, . . . , αn) = |α1|+ · · ·+ |αn|.

Lemma 2.17 Let V be a normed vector space, and let T :Fn → V be a linearmap. Then the map T is bounded.

Proof: Let {e1, . . . , en} be the standard basis for Fn. Let v ∈ Fn, and write

v = α1e1 + · · ·+ αnen αi ∈ F

Let M = max(‖Te1‖, . . . , ‖Ten‖), where ‖ − ‖ is the norm on V . Then

‖Tv‖ = ‖α1Te1 + · · ·+ αnTen‖≤ |α1|‖Te1‖+ · · ·+ |αn|‖Ten‖≤ |α1|M + · · ·+ |αn|M= M‖v‖

.

So the map T is bounded. 2

Recall that if K is a metric space, we call K compact if every sequence (xn)has a convergent subsequence with a limit in the space K. The following resultis known as the Heine-Borel theorem.

Theorem 2.18 A subset K ⊆ Fn is compact if and only if it is closed andbounded. 2

Here, by a metric spae X being bounded, we mean there is a constant C ≥ 0such that d(x, y) ≤ C for all x, y ∈ X. This notion is different to the notion ofa linear map being bounded.


2.3. FINITE-DIMENSIONAL SPACES 23

Proposition 2.19 Let X and Y be metric spaces, let K ⊆ X be compact, andlet f :X → Y be a continuous map. Then the image f [K] ⊆ Y is compact. 2

Lemma 2.20 Let K be a compact subset of a normed vector space such that0 6∈ K. Then we can find m > 0 such that ‖a‖ ≥ m for all a ∈ K.

Proof: We have already seen that we have a continuous map p:K → R definedby the formula p(a) = ‖a‖. Hence the image p[K] is compact, and thereforeclosed and bounded.

Let m = inf p[K] = inf{‖a‖ | a ∈ K}. Then by definition, ‖a‖ ≥ m for alla ∈ K. By definition of infemum, there is a sequence of elements, (yn), in p[K]converging to m. Set yn = ‖an‖. Since K is compact, the sequence (an) has aconvergent subsequence, (ank), with limit a ∈ K.

By definition of the norm

‖a‖ = limk→∞

‖ank‖ = limk→∞

mnk = m.

Since a ∈ K, a 6= 0 and therefore m > 0. 2

Let V and W be normed vector spaces. We call a bounded linear mapT :V →W an isomorphism if T is invertible, and T−1:W → V is also a boundedlinear map.

Theorem 2.21 Let V be an n-dimensional normed vector space. Then V isisomorphic to the space Fn.

Proof: Since the spaces Fn and V have the same dimension, there is aninvertible linear map T :Fn → V . By lemma 2.17, the map T is continuous. Wewant to show that the inverse T−1 is also continuous.

Since the norm is a continuous map, and the map T is continuous, thecomposition

F T→ V ‖−‖→ R

is continuous.Let S = {x ∈ Fn | ‖x‖ = 1} be the unit sphere in the space Fn. Then

the subset S is closed and bounded, and therefore compact by the Heine-Boreltheorem. It follows by lemma 2.20 that we can find m > 0 such that ‖Tx‖ ≥ mfor all x ∈ S.

Write M = 1m . Observe that ‖T−1(0)‖ = 0 ≤ M‖0‖. Choose v ∈ V \{0}.

Then T−1v 6= 0, and T−1v

‖T−1v‖ ∈ S. Hence

T

(T−1v

‖T−1v‖

)≥ m

and ‖v‖ ≥ m‖T−1v‖.We see that ‖T−1v‖ ≤M‖v‖ for all v ∈ V . Thus the map T−1 is bounded,

and we are done. 2


The above result tells us that a finite-dimensional vector space has the sameopen sets for any choice of norm. In particular, when looking at properties suchas continuity, limits, or which subsets are compact, the choice of norm on afinite-dimensional vector space is irrelevant.

Corollary 2.22 Let V and W be normed vector spaces, where V is finite-dimensional. Let T :V →W be a linear map. Then T is continuous.

Proof: By the above theorem, we have a linear homeomorphism φ:Fn → V .By lemma 2.17, the map T ◦ φ:Fn →W is continuous.

The inverse of the homeomorphism φ is also continuous. Hence the mapT = T ◦ φ ◦ φ−1:V →W is also continuous, and we are done. 2

2.4 The Open Mapping Theorem

Definition 2.23 A mapping f :V → W between two normed vector spaces issaid to be open if for every open set U ⊆ V , the image f [U ] ⊆W is open.

Note that this is different to the definition of continuity in terms of opensets, which asserts that a function is continuous if the inverse image of an openset is open. So, if we have an open bijective map, f , then the inverse f−1 iscontinuous.

The following result is called the open mapping theorem. The results in theprevious section establish it in the finite-dimensional case; we go into more detailin the exercises. We will not prove the general infinite-dimensional version inthese notes. The proof is rather technicall, and involves a result called the Bairecategory theorem in the general theory of metric spaces.

Theorem 2.24 (The Open Mapping Theorem) Let V and W be Banachspaces, and let T :V →W be a surjective bounded linear map. Then T is open.

The following consequence of the open mapping theorem is called the Banachisomorphism theorem.

Corollary 2.25 Let V and W be Banach spaces. Let T :V →W be a bijectivebounded linear map. Then T is an isomorphism. 2

We can sometimes use the open mapping theorem to show a normed vectorspace is not complete, as in the following example.

Example 2.26 Let C∞[0, 1] be the vector space of all infinitely-differentiablefunctions f : [0, 1]→ F. Equip C∞[0, 1] with the norm

‖f‖ = sup{|f(t)| | t ∈ [0, 1]}.

Then C∞[0, 1] is not a Banach space.

2.4. THE OPEN MAPPING THEOREM 25

Suppose that C∞[0, 1] is a Banach space.To see this, let C∞0 [0, 1] be the subspace of all functions f ∈ C∞[0, 1] such

that f(0) = 0. Define E:C∞[0, 1] → F by the formula E(f) = f(a). Observethat E is linear, and

|E(f)| = |f(a)| ≤ sup{|f(t)| | t ∈ [a.b]} = ‖f‖.

so E is a bounded linear map. We have C∞0 [0, 1] = kerE, which is closed, soC∞0 [0, 1] is also a Banach space.

Define I:Cinfty[0, 1]→ C∞0 [0, 1] by the formula

I(f)(x) =

∫ x0

f(t) dt x ∈ [0, 1].

Then

‖I(f)‖ = sup{∣∣∣∣∫ x

0

f(t) dt

∣∣∣∣ | x ∈ [0, 1]} ≤ ∫ 10

|f(t)| dt ≤ ‖f‖,

so I is a bounded linear map. By the fundamental theorem of calculus, I hasan inverse D:C∞0 [0, 1]→ C∞[0, 1] defined by the formula

D(f)(x) = f ′(x).

By the open mapping theorem, the map D must be a bounded linear map.Now,let fn(t) = t

n, n ∈ N. Then

‖fn‖ = sup{|tn| | t ∈ [0, 1]} = 1

and‖D(fn)‖ = sup{|ntn−1| | t ∈ [0, 1]} = n.

Hence, if D is bounded, ‖D‖geqn. And this is true for all n. This is acontradiction; if D is bounded, ‖D‖ is finite.

Therefore our assumption that C∞[0, 1] is a Banach space is not correct; thespace C∞[0, 1] is not complete.

Definition 2.27 Let A and B be sets, and let f :A→ B be a map. We definethe graph of f to be the set

Gr(f) = {(x, f(x)) | x ∈ A} ⊆ A×B.

Proposition 2.28 Let X and Y be metric spaces. Let f :X → Y be a contin-uous map. Then the graph Gr(f) is a closed subset of X × Y .

Proof: Let (xn, f(xn)) be a sequence in Gr(f) that converges to some point(x, y) ∈ X × Y . Then xn → x as n→∞.

So by continuity, f(xn) → f(x) as n → ∞. But by the above, f(xn) → yas n → ∞. So, by the uniqueness of limits in metric spaces, y = f(x), whichmeans (x, y) ∈ Gr(f).


It follows that Gr(f) is a closed subset of X × Y . 2

The above result has a converse in the case of Banach spaces. This converseis called the closed graph theorem.

Theorem 2.29 Let V and W be Banach spaces, and let T :V →W be a linearmap with a closed graph. Then T is continuous.

Proof: Observe that Gr(T ) is a closed linear subspace of V ×W , and thereforea Banach space. Define a linear map P :Gr(T )→ V by the formula

P ((x, Tx)) = x x ∈ V.

The map P is clearly bijective. Observe that

P (x, Tx)‖ = ‖x‖ ≤ ‖(x, Tx)‖.

so the map P is continuous. By the Banach isomorphism theorem, the inverseP−1:V → Gr(T ) is also continuous.

Similarly, the map Q:Gr(T )→W defined by the formula Q(x, Tx) = Tx iscontinuous. Therefore the composite

T = Q ◦ P−1:V →W

is also continuous, and we are done. 2

Chapter 3

Spaces of ContinuousFunctions

3.1 Dual Spaces

Definition 3.1 Let V be a normed vector space over the field F. Then a linearfunctional is a bounded linear map f :V → F.

Example 3.2 Let K be a compact metric space. Consider the normed vectorspace C(K) from example 1.10. Fix x ∈ K. Then we have a bounded linearmap Ex:C(K)→ F defined by the formula

Ex(f) = f(x) f ∈ C(K).

Example 3.3 The integration function

f 7→∫ 1

0

f(x) dx

defines a linear functional on the space L1[0, 1]

Definition 3.4 Let V be a normed vector space. Then we define the dual space,V ∗, to be the vector space of bounded linear functionals f :V → F.

We have V ∗ = Hom(V,F), which is a normed vector space when it is giventhe operator norm. By theorem 2.15, since the field F is complete, the space V ∗is a Banach space.

Definition 3.5 Let V be a normed vector space, and let W ≤ V be a subspace.Suppose we have a bounded linear functional f :W → F. Then a bounded linearfunctional F :V → F is called an extension of f if F (v) = f(v) whenever v ∈W .

Our fundamental result on dual spaces is called the Hahn-Banach theorem.

27

28 CHAPTER 3. SPACES OF CONTINUOUS FUNCTIONS

Theorem 3.6 Let V be a real normed vector space, and let W ≤ V be a sub-space. Let f ∈W ∗ be a linear functional. Then there exists an extension F ∈ V ∗of f such that ‖F‖ = ‖f‖.

We prove this result in section 3.3. The Hahn-Banach theorem has a signif-icant application when we consider the second dual of a normed vector space.The second dual is the space (V ∗)∗ consisting of all bounded linear functionalsg:V ∗ → F. We have a map τ :V → (V ∗)∗ given by the formula

τ(v)(f) = f(v) v ∈ V, f ∈ V ∗.

This makes sense, as if τ(v) ∈ (V ∗)∗, then τ(v) is a bounded linear mapfrom V ∗ to F. It is easy to see that τ(v) is linear; as for bounded, observe

‖τ(v)(f)‖ = ‖f(v)‖ ≤ ‖v‖‖f‖.

Indeed, the above calculation tells us that τ is a bounded linear map, and‖τ(v)‖ ≤ ‖v‖ for all v ∈ V .

Proposition 3.7 The map τ is an isometry, that is to say that ‖τ(v)‖ = ‖v‖for all v ∈ V .

Proof: Let v ∈ V . We have seen that ‖τ(v)(f)‖ ≤ ‖v‖‖f‖ for all f ∈ V ∗. Wetherefore need to find some F ∈ V ∗ such that ‖τ(v)(F )‖ = ‖v‖ · ‖F‖, and weare done.

The result is obvious if v = 0. So let v 6= 0. Let W = Span{v} = {αv | α ∈F}.

Define f :W → F by the formula

f(αv) = α‖v‖.

Then f ∈W ∗, and

‖f‖ = supα∈F,α6=0

|α|‖v‖‖αv‖

= 1.

By the Hahn-Banach theorem, we have a functional F ∈ V ∗ such that ‖F‖ =1, and F (αv) = α‖v‖ for all α ∈ F.

Observe‖τ(v)(F )‖ = ‖F (v)‖ = ‖v‖ = ‖v‖ · ‖F‖.

Hence ‖τ(v)‖ = ‖v‖, and we are done. 2

So we have an isometric linear map τ :V → (V ∗)∗. Since the map τ preservesboth the linear structure and the norm, we can consider V to be a vectorsubspace of (V ∗)∗.

This leads us back to an earlier promised result.

Theorem 3.8 Let V be a normed vector space. Then V has a completion.

3.2. ZORN’S LEMMA 29

Proof: Recall that any dual space is complete. Hence (V ∗)∗ is complete. Bythe above, we regard V as a subspace of (V ∗)∗ through the isometry τ .

Let V̂ be the closure of V in (V ∗)∗. Since (V ∗)∗ is complete, the closure V̂is complete. And V is a dense subspace of its closure, so V̂ is the completion ofV . 2

3.2 Zorn’s Lemma

Zorn’s Lemma is an alternative formulation of the axiom of choice. We need itto prove the Hahn-Banach theorem.

Definition 3.9 A set P is said to be partially ordered if there is a relation,writtten ≤, on the elements of P such that the following axioms are satisfied:

• x ≤ x for every element x ∈ P

• If x ≤ y and y ≤ z then x ≤ z

• If x ≤ y and y ≤ x then x = y

A partially ordered set P is said to be totally ordered if for any two elementsx, y ∈ P either x ≤ y or y ≤ x.

Example 3.10 Let S be any set, and let P be the collection of subsets of S.Then the set P is partially ordered by the relation ⊆ of set inclusion.

If the set S has more than one element then the set P is not totally ordered.

Example 3.11 The set of real numbers, R, is a totally ordered set, where therelation ≤ has the usual meaning.

Definition 3.12 Let P be a partially ordered set. Then an element x ∈ P issaid to be maximal if for all elements y ∈ P the relation x ≤ y implies thatx = y.

Let Q be a subset of P . Then an element x ∈ P is said to be an upper boundfor Q if q ≤ x for all elements q ∈ Q.

Example 3.13 Let P be the collection of subsets of some set S. Then the setS itself is both an upper bound for P and a maximal element of P .

Example 3.14 Let B be a bounded subset of the real numbers. Then the supre-mum of B, α, is an upper bound for B. If α ∈ B then α is a maximal elementof B.

We are now ready to state Zorn’s lemma which can be considered a funda-mental axiom of set theory.


Zorn’s Lemma

Let P be a partially ordered set. Suppose that every totally ordered subset hasan upper bound. Then the set P has a maximal element.

The following result provides an example of how Zorn’s lemma is used.

Proposition 3.15 Let X be a vector space, and let Y be a subspace of X. Thenwe can find a subspace Z such that X = Y ⊕ Z.

Proof: Let P be the set of subspaces, A, such that Y ∩A = {0}. We can definea partial ordering on the set P by writing A ≤ B if the space A is contained inthe space B.

Suppose that T is a totally ordered subset of P . Form the set

M =⋃A∈T

A

Certainly M ∩ Y = {0}. Let α, β ∈ F and v, w ∈ V . Because the set T istotally ordered, there is some A ∈ T where v, w ∈ A, and since A is a vectorspace, αv + βw ∈ A ⊆M . Certainly 0 ∈M , so the set M is a subspace of V .

Hence we have an element M ∈ P that is an upper bound for the subset T .By Zorn’s lemma the set P must have a maximal element, Z.

We know that Y ∩ Z = {0}. Suppose that Y ⊕ Z 6= X. Then we canfind a vector x ∈ X\(Y ⊕ Z). The vector space Z ⊕ 〈x〉 is an element of theset P , contains the space Z, and is not equal to Z. This fact contradicts themaximality of the space Z.

Therefore Y ⊕ Z = X and we are done. 2

3.3 Proof of the Hahn-Banach Theorem

.We first establish the Hahn-Banach theorem for real normed vector spaces.

Theorem 3.16 Let V be a real normed vector space, and let W ≤ V be asubspace. Let f ∈W ∗ be a linear functional. Then there is an extension F ∈ V ∗such that ‖F‖ = ‖f‖.

Proof: The proof proceeds by showing that we can extend the linear functionalf one dimension at a time and then applying Zorn’s lemma. Suppose withoutloss of generality that ‖f‖ = 1.

Let Z be a vector space containing W and let g:Z → R be an extension ofthe linear functional f such that ‖g‖ = 1.

Suppose that Z 6= X. Choose a point x1 ∈ X\Z and form the space

Z1 = Z ⊕ 〈x1〉

3.3. PROOF OF THE HAHN-BANACH THEOREM 31

Let x, y ∈ Z. Then:

g(x) + g(y) = g(x+ y) ≤ ‖x+ y‖ ≤ ‖x− x1‖+ ‖x1 + y‖

so we have the inequality

g(x)− ‖x− x1‖ ≤ ‖y + x1‖ − g(y)

We can therefore find a real number β such that:

supx∈Z

(g(x)− ‖x− x1‖) ≤ β ≤ infx∈Z

(‖x+ x1‖ − g(x))

Define a linear functional g1:Z1 → R by the formula

g1(x+ αx1) = g(x) + αβ

for all vectors x ∈ X and scalars α ∈ R. We claim that g1(x+αx1) ≤ ‖x+αx1‖.There are three cases to consider.

• α = 0:g1(x+ αx1) = g(x) ≤ ‖x‖ = ‖x+ αx1‖

• α > 0:

g1(x+ αx1) = α(g(α−1x) + β) ≤ α‖α−1x+ x1‖ = ‖x+ αx1‖

• α < 0:

g1(x+ αx1) = −α(g(−α−1x) + β) ≤ −α‖ − α−1x− x1‖ = ‖x+ αx1‖

Replacing the vector x + αx1 by −(x + αx1) we see that |g1(x + αx1)| ≤‖x + αx1‖. Thus ‖g1‖ ≤ 1, and g is an extension of g. Therefore we also have‖g‖ ≥ ‖g1‖ = 1, so ‖g‖ = 1. This completes the first step of the proof.

Now let P be the set of pairs (Z, g) such that Z is a subspace of V thatcontains W , and g:Z → R is an extension of the linear functional f such that‖g‖ = 1. Define a partial ordering on the set P by writing (Z, g) ≤ (Z ′, g′)if the set Z ′ contains the set Z and the linear functional g′ extends the linearfunctional g.

Let T = {(Zλ, gλ) | λ ∈ Λ} be a totally ordered subset of the set P . Thenwe can form a vector space

Z =⋃λ∈Λ

Zλ

and a linear functional g:Z → R defined by the formula g(x) = gλ(x) wheneverx ∈ Zλ. The pair (Z, g) is an upper bound for the set T , so by Zorn’s lemmathe set P has a maximal element, (M,F ).

Suppose that M 6= V . Then by the first step of this proof we have a pair(M1, F1) such that (M,F ) ≤ (M1, F1) and (M,F ) 6= (M1, F1). This contradictsmaximality of the element (M,F ).


Therefore M = V and we have an extension F :X → R such that ‖F‖ = 1.2

The complex version of the Hahn-Banach theorem follows as a simple corol-lary.

Corollary 3.17 Let V be a complex normed vector space, and let W ≤ V be asubspace. Let f ∈W ∗ be a linear functional. Then there is an extension F ∈ V ∗such that ‖F‖ = ‖f‖.

Proof: The spaces V and W can be considered real vector spaces. We havea real-valued linear functional g:W → R defined by writing g(x) =

3.4. THE STONE-WEIERSTRASS THEOREM 33

Thus a subalgebra of C(X) is a linear subspace that is closed under theproduct.

Lemma 3.19 Let A be a closed unital subalgebra of the real algebra CR(X). Letf, g ∈ A. Then the functions max(f, g) and min(f, g), defined by the formulae

max(f, g)(x) = max(f(x), g(x)) min(f, g)(x) = min(f(x), g(x))

respectively, belong to the algebra A.

Proof: Since 2 min(f, g) = |f − g| + (f + g) and −min(f, g) = max(−f,−g)it suffices to show that |f | ∈ A whenever f ∈ A. By multiplying by a constant,let us assume that ‖f‖ < 1.

Let g(t) = (1 − t) 12 . By looking at the Taylor series of the function g(t),given ε > 0 there is a polynomial p(t) such that

|g(t)− p(t)| < ε

for all t ∈ [0, 1].Now let t = 1 − f(x)2. Then t ∈ [0, 1], and g(t) = (1 − t) 12 = |f(x)|. We

therefore have the inequality

| |f(x)| − p(1− f(x)2)| < ε

for all x ∈ X. Hence ‖ |f | − p(1− f2)‖ ≤ ε.Since the set A is a unital algebra, and p is a polynomial, the function

p(1− f2) belongs to A. Since the set A is closed, it follows that |f | ∈ A and weare done. 2

The following handy result is known as the Stone-Weierstrass theorem

Theorem 3.20 Let A ⊆ CR(X) be a unital subalgebra that separates points.Then A is a dense subset of CR(X).

Proof: Note that if A ⊆ CR(X) is a unital subalgebra that seperates points,then the closure A is a closed unital subalgebra that seperates points. Further,if A is dense, then A = CR(X). Thus it suffices to prove that if A is a closedunital subalgebra that seperates points, then A = CR(X).

Consider a function f ∈ CR(X), and a real number ε > 0. We will find afunction g ∈ A such that ‖f − g‖ ≤ ε.

Since the algebra A is unital and separates points, for any two points x, y ∈ Xwith x 6= y we can find a function gx,y ∈ A such that

gx,y(y) = f(y) +ε

2gx,y(x) > f(x)

For each point x ∈ X there is an open neighbourhood Ux 3 x such thatgx,y(z) > f(z) for all z ∈ Ux. Since the space X is compact, we have a finitecover of the form {Ux1 , . . . , Uxm}.


Define gy = max(gx1 , . . . , gxm). Then

gy(y) = f(y) +ε

2gy > f

and gy ∈ A by lemma 3.19. For each point y ∈ X there is an open neighbourhoodVy 3 y such that gy(z) < f(z) + ε for all z ∈ Vy. Since the space X is compact,we have a finite cover of the form {Vy1 , . . . , Vyn}.

We can define a function g = min(gy1 , . . . , gyn). By construction we knowthat f ≤ g ≤ g + ε, and so ‖f − g‖ ≤ ε. But lemma 3.19 tells us that g ∈ A sowe are done. 2

To formulate the complex version of this result, note that we have an op-eration f 7→ f∗ on the space CC(X) defined by writing f∗(x) = f(x) for allx ∈ X.

Definition 3.21 We call a subalgebra A ⊆ CC(X) a ∗-subalgebra if f∗ ∈ Awhenever f ∈ A.

The complex version of the Stone-Weierstrass theorem is then as follows.

Theorem 3.22 Let A ⊆ CC(X) be a unital ∗-subalgebra that separates points.Then A is a dense subset of CC(X). 2

The classical Weierstrass approximation theorem follows as a corollary.

Corollary 3.23 Let f be a continuous function on the interval [0, 1]. Let ε > 0.Then there is a polynomial p such that |f(t)− p(t)| for all t ∈ [0, 1].

Proof: Let A be the set of polynomials on the interval [0, 1]. Clearly, A is a∗-subalgebra of the space C[0, 1], and 1 ∈ A so A is unital.

Let x, y ∈ [0, 1] with x 6= y. We have a polynomial defined by writing p(t) = tfor all t ∈ [0, 1]; certainly p(x) 6= p(y). So the subalgebra A seperates points.

Hence, given ε > 0 and a continuous function f ∈ C[0, 1], we have a polyno-mial p such that ‖f − p‖ < ε. By definition of the norm on C[0, 1], this meansthat |f(t)− p(t)| < ε for all t ∈ [0, 1]. 2

The following result will be handy to us later on when we consider Fourierseries, and is shown similarly. We leave the details as an exercise.

Proposition 3.24 LetT = {z ∈ C | |z| = 1}.

Let A be the span of the set of functions ek:T → T given by the formulaek(z) = z

k, where k ∈ Z. Then A is dense in C(T). 2

Chapter 4

Hilbert Spaces

4.1 Inner Products

If norms provide a measure of distance within a vector space, inner productsprovide a measure of angle. In the following definition, given a scalar α ∈ F,when we write α, we mean the complex conjugate. So, if x, y ∈ R, then x+ iy =x− iy. If α ∈ R, then obviously α = α.

Definition 4.1 Let V be a vector space over the field F. An inner product onV is a map

〈−,−〉:V × V → F

such that:

• Let u, v, w ∈ V , and α, β ∈ F. Then 〈u, αv + βw〉 = α〈u, v〉+ β〈u,w〉.

• Let u, v ∈ V . Then 〈u, v〉 = 〈v, u〉.

• Let v ∈ V . Then 〈v, v〉 ∈ R≥0, and 〈v, v〉 = 0 if and only if v = 0.

We call a vector space V equipped with an inner product an inner productspace.

Example 4.2 Let x, y ∈ Cn. Write x = (u1, . . . , un) and y = (v1, . . . , vn).Then we can define an inner product on the space Cn by the formula

〈x, y〉 = u1v1 + · · ·+ unvn.

• Let x, y, z ∈ Cn, and α, β ∈ C. Write x = (u1, . . . , un), y = (v1, . . . , vn),and z = (w1, . . . , wn). Then

〈x, αy + βz〉 = u1(αv1 + βw1) + · · ·+ un(αvn + βwn)= α(u1v1 + · · ·+ unvn) + β(u1w1 + · · ·+ unwn)= α〈x, y〉+ β〈x, z〉.

35

36 CHAPTER 4. HILBERT SPACES

• Let x = (u1, . . . , un) and y = (v1, . . . , vn) ∈ Fn. Then

〈x, y〉 = u1v1 + · · ·+ unvn= v1u1 + · · ·+ vnun= 〈y, x〉.

• Let x = (u1, . . . , un) ∈ Fn. Then

〈x, x〉 = |u1|2 + · · ·+ |un|2.

Observe 〈x, x〉 ∈ R≥0. Each of the terms |ui|2 is non-negative. So if〈x, x〉 = 0, we must have |ui|2 = 0 for all i, meaning x = 0.

We call the above the standard inner product on Cn.Now, suppose we consider elements of the space Cn to be column vectors.

Thus, given x, y ∈ Cn, we write x = (u1, . . . , un)T and y = (v1, . . . , vn)T . Thenby definition of matrix multiplication

〈x, y〉 = xT y.

This idea can be a useful way of viewing the standard inner product forcalculations.

Definition 4.3 Let V be an inner product space, and let v ∈ V . Then we write

‖v‖ =√〈v, v〉.

We shall see soon that this formula defines a norm, as our notation suggests.

Theorem 4.4 (The Cauchy-Schwarz inequality) Let V be an inner prod-uct space. Let u, v ∈ V . Then

|〈u, v〉| ≤ ‖u‖ · ‖v‖.

Proof: If v = 0, then clearly 〈u, v〉 = 0, and ‖v‖ = 0, so the result holds. Sosuppose v 6= 0. Then 〈v, v〉 6= 0.

For any scalar α ∈ F, we have

〈u− αv, u− αv〉 ≥ 0.

But〈u− αv, u− αv〉 = 〈u, u〉 − α〈u, v〉α〈u, v〉+ |α|2〈v, v〉.

Set

α =〈u, v〉〈v, v〉

.

Then we see

〈u, u〉 − |〈u, v〉|2

〈v, v〉≥ 0.

4.1. INNER PRODUCTS 37

Rearranging, we see

|〈u, v〉|2 ≤ 〈u, u〉 · 〈v, v〉.

Taking the square root, the desired result follows. 2

Corollary 4.5 Let V be an inner product space. Then ‖ − ‖ is a norm on V .

Proof: Certainly ‖v‖ ≥ 0 for all v ∈ V . We simply check the axioms for anorm.

• Let α ∈ F and v ∈ V . Then

‖αv‖2 = 〈αv, αv〉 = |α|2〈v, v〉.

so ‖αv‖ = |α|‖v‖.

• Let u, v ∈ V . Then, by the Cauchy-Schwarz inequality

‖u+ v‖2 = ‖u‖2 + 〈u, v〉+ 〈u, v〉+ ‖v‖2= ‖u‖2 + 2


Proposition 4.8 Let V be an inner product space. Let (un) and (vn) be se-quences in V , with norm limits u and v respectively. Then

limn→∞

〈un, vn〉 = 〈u, v〉.

Proof: Observe, by the Cauchy-Schwarz inequality, that

|〈un, vn〉 − 〈u, v〉| = |〈un, vn〉 − 〈u, vn〉+ 〈u, vn〉 − 〈u, v〉|≤ |〈un − u, vn〉|+ |〈u, vn − v〉|≤ ‖vn‖ · ‖un − u‖+ ‖u‖ · ‖vn − v‖.

By convergence of the sequences (un) and (vn), we have ‖un − u‖ → 0 and‖vn − v‖ → 0 as n → ∞. By continuity of the norm, ‖vn‖ → ‖v‖ as n → ∞.Hence

limn→∞

|〈un, vn〉 − 〈u, v〉| = 0

and we are done. 2

It follows that the inner product defines a continuous map 〈−,−〉:V ×V → F.

Definition 4.9 An inner product space that is complete (as a normed vectorspace) is called a Hilbert space.

Example 4.10 Recall that we define l2 to be the set of sequences(αn) in thefield F such that

∞∑n=1

|αn|2

4.2. ORTHOGONAL COMPLEMENTS 39

Therefore the formula

〈(αn), (βn)〉 =∞∑n=1

αnβn

is well-defined. We can check it defines an inner product as in example 4.2.We have associated norm given by the above formula

‖α‖ =

( ∞∑n=1

|αn|2) 1

2

(αn) ∈ l2.

We saw in the chapter 1 exercises that the space l2, with this norm, iscomplete. Hence the space l2 is a Hilbert space.

Definition 4.11 Let V be an inner product space. Then we call a Hilbert space,V̂ , a completion of V if V is a dense subset of V̂ , with the same inner product.

Just as in the normed vector space case, it is a theorem that any innerproduct space has a completion.

Example 4.12 It is shown in the exercises that the space C[0, 1] can be equippedwith an inner product defined by the formula

〈f, g〉 =∫ 1

0

f(t)g(t) dt.

The inner product defines a norm

‖f‖ :=√〈f, f〉 =

(∫ 10

|f(t)|2 dt) 1

2

.

Thus the space C[0, 1] is not complete (exercise). However, we can form thecompletion, which is a Hilbert space. We call this Hilbert space L2[0, 1].

4.2 Orthogonal Complements

Definition 4.13 Let H be a Hilbert space. Let S be a subset of H. Then wedefine the orthogonal complement of S:

S⊥ = {v ∈ H | 〈x, v〉 = 0 for all x ∈ S}

It is left as an exercise to show that

‖x+ v‖2 = ‖x‖2 + ‖v‖2

whenever x ∈ S and v ∈ S⊥.


Proposition 4.14 The orthogonal complement S⊥ is a closed linear subspaceof H.

Proof: It is left as an exercise to show that S⊥ is a linear subspace of H. Butwe will check it is closed here.

To see this, let v ∈ S⊥. Then there is a sequence, (vn), in S⊥, converging innorm to v.

Let x ∈ S. Then 〈x, vn〉 = 0 for all n. By continuity of the inner product, ifwe let n→∞, we see 〈x, v〉 = 0.

It follows that v ∈ S⊥. So S⊥ = S⊥ and we are done. 2

The proof of the following result is again left as an exercise.

Proposition 4.15 Let H be a Hilbert space, and let V ⊆ H be a subspace.Then (V )⊥ = V ⊥. 2

We call a vector space V the direct sum of two subspaces V1 and V2 ifV1 ∩ V2 = {0}, and for every element v ∈ v, we can write v = v1 + v2 wherev1 ∈ V1 and v2 ∈ V2. We write V = V1⊕V2 when V is the direct sum of V1 andV2.

Our main aim in the rest of this section is to prove that given a closedsubspace, V , of a Hilbert space H, we have H = V ⊕ V ⊥. We call this resultthe projection theorem.

The following lemma, which is needed in the proof, is sometimes called thenearest point theorem.

Lemma 4.16 Let H be a Hilbert space, let x ∈ H, and let C ⊆ H be a closedconvex subset. Set

d(x,C) = inf{‖x− v‖ | v ∈ C}.

Then there is a unique point y ∈ C such that ‖x− y‖ = d(x,C).

Proof: Let d = d(x,C). Then we have a sequence, (yn), in C such that‖x − yn‖ → d as n → ∞. Set zn = x − yn. Then ‖zn‖ ≥ d for all n, and‖x− yn‖ → d as n→∞.

By the parallelogram law

‖yn − ym‖2 = ‖zn − zm‖2 = 2(‖zm‖2 + ‖zn‖2)− ‖zn + zm‖2.

But

‖zn + zm‖2 = ‖2x− ym − yn‖2 = 4‖x−1

2(ym + yn)‖2

and 12 (ym + yn) ∈ C because the set C is convex and ym, yn ∈ C. Hence

‖ym − yn‖2 ≤ 2(‖zm‖2 + ‖zn‖2)− 4d2 → 0

as m,n→∞.

4.2. ORTHOGONAL COMPLEMENTS 41

It follows that the sequence (yn) is Cauchy. Hence, as H is a Hilbert space,and so complete, there is a point y ∈ H such that yn → y as n→∞.

But by hypothesis, the set C is closed. So y ∈ C. Now, by definition of thesequence (yn), we had ‖x − yn‖ → d as n → ∞. Hence, by continuity of thenorm, we have ‖x− y‖ = d, which is exactly what we need.

Uniqueness of the point y is left as an exercise. 2

Note in particular that any linear subspace of H is convex. Then givenx ∈ H, and a closed linear subspace, V , of H, we have y ∈ V such that‖x− y‖ = d(x, V ).

Theorem 4.17 Let H be a Hilbert space. Let V ⊆ H be a closed linear sub-space. Then H = V ⊕ V ⊥.

Proof: Firstly, observe that V and V ⊥ are both subspaces of H. Let v ∈V ∩ V ⊥. Then 〈v, v〉 = 0, which implies v = 0. Thus V ∩ V ⊥ = {0}.

Now, let x ∈ H. Let d = d(x, V ). By the nearest point formula, we have aunique point y ∈ V such that ‖x − y‖ = d. We claim that v − y ∈ V ⊥. If thisclaim holds, then

x = y + (x− y) y ∈ V, x− y ∈ V ⊥

and we are done.Suppose x − y 6∈ V ⊥. Then we have a point v ∈ V such that 〈x − y, v〉 =

α 6= 0.Let z = x− y and let λ ∈ F. Observe

‖z − λv‖2 = ‖z‖2 − λ〈z, v〉 − λ〈z, v〉+ |λ|2‖v‖2= d2 − λα− λα+ |λ|2‖v‖2 .

Let

λ =α

‖v‖2.

Then ‖z − λv‖2 = d2 − 2|α|2, so

‖z − λv‖ = ‖x− (y + λv)‖ < d.

But y + λv ∈ V , and d = d(x.V ) = inf{‖x−w‖ | w ∈ V }. So the above is acontradiction. It follows that x− y ∈ V ⊥.

This observation completes the proof. 2

Thus, if H is a Hilbert space, and V ⊆ H is a closed linear subspace, anelement x ∈ H can be written uniquely as a sum x = v + w where v ∈ V andw ∈ V ⊥. We can therefore define a map P :H → H by writing P (x) = v.

We call the map P the orthogonal projection onto V . The above theoremguarantees the existence of the map P , which is why we call it the projectiontheorem.

The following result is left as an exercise.


Proposition 4.18 The map P is a bounded linear map with the property P 2 =P . 2

Corollary 4.19 Let H be a Hilbert space, and let V be a linear subspace of H.Then (V ⊥)⊥ = V .

Proof: Let v ∈ V . Then for any vector w ∈ V ⊥, we have 〈w, v〉 = 0. Hencev ∈ (V ⊥)⊥. As the space (V ⊥)⊥ is closed by proposition 4.14, it follows thatV ⊆ (V ⊥)⊥.

Conversely, let x ∈ (V ⊥)⊥. By proposition 4.15, we have (V )⊥ = V ⊥.Hence, by the projection theorem, we can write

H = V ⊕ V ⊥.

Hence x = v + w, where v ∈ V ⊆ (V ⊥)⊥, and w ∈ V ⊥. It follows thatw = x− v ∈ (V ⊥)⊥.

But w ∈ V ⊥, and certainly V ⊥ ∩ (V ⊥)⊥ = {0}. Therefore w = 0, andx = v ∈ V .

We have therefore shown that (V ⊥)⊥ ⊆ V , which completes the proof. 2

In particular, if V is a closed subspace of H, then (V ⊥)⊥ = H.

Corollary 4.20 Let H be a Hilbert space, and let V be a linear subspace of H.Then V is dense in H if and only if V ⊥ = {0}.

Proof: Let V be dense in H. Then V = H. By proposition 4.15, we have(V )⊥ = V ⊥. Hence, by the projection theorem, we can write

H = V ⊕ V ⊥.

It follows that V ⊥ = {0}.Conversely, suppose V ⊥ = {0}. Then (V ⊥)⊥ = H. But by the above

corollary, we have (V ⊥)⊥ = V . Thus V = H, and we are done. 2

4.3 Dual Spaces

Let H be a Hilbert space. Let v ∈ H. Then we can define a map Jv:H → F bythe formula

Jv(x) = 〈v, x〉.

Proposition 4.21 The map Jv is a bounded linear map, with norm ‖Jv‖ =‖v‖.

Proof: It is easy to see that the map Jv is linear. By the Cauchy-Schwarzinequality, for any point x ∈ H we have

|Jv(x)| ≤ ‖v‖ · ‖x‖

4.3. DUAL SPACES 43

which tells us that the operator Jv is bounded, with norm ‖Jv‖ ≤ ‖v‖.Observe

|Jv(v)| = 〈v, v〉 = ‖v‖2

so ‖Jv‖ ≥ ‖v‖. We conclude that ‖Jv‖ = ‖v‖. 2

Thus the map Jv is an element of the dual space, H∗

We call a map T :V → W between two vector spaces over the field Fconjugate-linear if

T (αu+ βv) = αT (u) + βT (v)

for all α, β ∈ F and u, v ∈ V . Observe that if F = R, conjugate-linearity is thesame as linearity.

The following is straightforward to check.

Proposition 4.22 Let H be a Hilbert space. Then we have a conjugate-linearisometry J :H → H∗ defined by writing J(v) = Jv, that is

J(v)(x) = 〈v, x〉 v ∈ H, x ∈ H.

2

Our next result is called the Riesz representation theorem; it tells us thatthe above map is bijective.

Theorem 4.23 Let f :H → F be a bounded linear map. Then we have a uniquevector Rf ∈ H such that

f(x) = 〈Rf , x〉

for all x ∈ H.

Proof: Since f is a continuous map, ker f is closed. If f = 0, the result istrivial. Otherwise, we can find u ∈ H such that f(u) = 1.

By the projection theorem, we can write

u = v + w v ∈ ker f, w ∈ (ker f)⊥.

Observe that f(u) = f(w), so f(w) = 1. More generally, let x ∈ H. Then

f(x− f(x)w) = f(x)− f(x)f(w) = 0

so x− f(x)w ∈ ker f . It follows that 〈w, x− f(x)w〉 = 0, so

〈w, x〉 = f(x)〈w,w〉.

Set Rf = w/‖w‖2. Then by the above

〈Rf , x〉 = f(x)

and we have established existence of the vector Rf .


Let R′f ∈ H be another vector where f(x) = 〈R′f , x〉 for all x ∈ H. Letx = Rf −R′f . Then

〈x, x〉 = 〈Rf , x〉 − 〈R′f , x〉 = 0.

Therefore x = 0, which tells us that Rf = R′f and establishes uniqueness.

2

We leave the following two results as exercises.

Proposition 4.24 We have a conjugate-linear isometry R:H∗ → H defined bythe formula f 7→ Rf . 2

Proposition 4.25 The above map R is the inverse of the map J :H → H∗.

4.4 Adjoints

Theorem 4.26 Let H be a Hilbert space, and let T :H → H be a bounded linearmap. Then there is a unique map T ∗H → H such that

〈x, Ty〉 = 〈T ∗x, y〉

for all x, y ∈ H.

Proof: Let x ∈ H. Then we have a bounded linear map y 7→ 〈x, Ty〉. Hence,by the Riesz representation theorem, we have a unique element T ∗x ∈ H suchthat

〈T ∗x, y〉 = 〈x, Ty〉

for all y ∈ H. 2

We call the operator T ∗ the adjoint of T .

Example 4.27 Let I:H → H be the identity map. Then for all x, y ∈ H, wehave

〈Ix, y〉 = 〈x, y〉 = 〈x, Iy〉.

Hence I∗ = I.

Some general properties follow immediately from the definition. The precisedetails of the following two results are left as exercises.

Proposition 4.28 The map T ∗:H → H is a bounded linear map, with norm‖T ∗‖ = ‖T‖. 2

Proposition 4.29 Let S, T :H → H be bounded linear maps, and let α, β ∈ F.Then:

• (αS + βT )∗ = αS∗ + βT ∗.

4.4. ADJOINTS 45

• (S∗)∗ = S.

• (ST )∗ = T ∗S∗.

2

Example 4.30 Let A = (aij) be an n × n complex matrix. Then A defines abounded linear map Cn → Cn by matrix multiplication on a column vector.

Thus, if we consider column vectors u, v ∈ Cn, then

〈u,Av〉 = uTAv = (ATu)T v = 〈ATu, v〉.

Therefore A∗ = AT

.

Example 4.31 Define bounded linear operators R,L: l2 → l2 by writing

R(a1, a2, a3, . . .) = (0, a1, a2, a3, . . .)

and

L(a1, a2, a3, . . .) = (a2, a3, . . .)

respectively.

Observe

〈L(a1, a2, a3, . . .), (b1, b2, b3, . . .)〉 = a2b1 + a3b2 + · · ·

and

〈(a1, a2, a3, . . .), R(b1, b2, b3, . . .)〉 = a2b1 + a3b2 + · · ·

We conclude that R∗ = L. Further, L∗ = (R∗)∗ = R.

Definition 4.32 Let H and H ′ be Hilbert spaces. We call a bounded linearoperator U :H → H ′ unitary if U∗U = IH and UU∗ = IH′ .

Example 4.33 A unitary n×n matrix, U , defines a unitary operator U :Cn →Cn.


Proposition 4.34 Let T :H → H ′ be an invertible bounded linear map. Thenthe map T is a unitary if and only if

〈Tu, Tv〉 = 〈u, v〉

for all u, v ∈ H. 2


Example 4.35 Let CP (R) be the vector space of all continuous functionsf :R → C such that f(t + 2π) = f(t) for all t ∈ R. We call elements CP (R)the periodic functions. The space CP (R) has an inner product defined by theformula

〈f, g〉 =∫ 2π

0

f(t)g(t) dt

It turns out that CP (R) is dense in the space L2[0, 2π]; for details, see theexercises.

RecallT = {z ∈ C | |z| = 1}

and we can define an inner product on C(T) by the formula

〈f, g〉 =∫ 2π

0

f(eit)g(eit) dt.

Let L2(T) be the completion. Observe that we can define a linear mapU :C(T)→ CP (R) by the formula

U(f)(t) = f(eit).

The map U is invertible, with inverse V given by the formula V (g)(eit) =g(t); the inverse is well-defined since g is periodic.

By definition of the inner products involved, U satisfies the formula

〈Uf,Ug〉 = 〈f, g〉

for all f, g ∈ C(T).Since the inner product is continuous, and C(T), CP (R) are dense subsets

of L2(T) and L2[0, 2π] respectively, the map U extends to a unitary map

U :L2(T)→ L2[0, 2π].

Chapter 5

Orthonormal Sets

5.1 Orthonormal Sets and Bases

Let H be a Hilbert space. Then we call two elements x, y ∈ H orthogonal if〈x, y〉 = 0. It is easy to see that if x and y are orthogonal, then

‖x+ y‖2 = ‖x‖2 + ‖y‖2.

Certainly, if x ∈ S and y ∈ S⊥, then x and y are orthogonal.

Definition 5.1 Let H be a Hilbert space. We call a subset M ⊂ H an orthog-onal set if 〈x, y〉 = 0 whenever x, y ∈M and x 6= y.

An orthonormal set is an orthogonal set where each element has norm one.

Thus a subset M ⊆ H is orthonormal if for elements x, y ∈ H, we have

〈x, y〉 ={

1 x = y0 x 6= y .

The following result lets us construct orthonormal sets from orthogonal sets.It is easy to check.

Proposition 5.2 Let M be an orthogonal set. Let

M ′ = {v/‖v‖ | v ∈M}.

Then the set M ′ is orthonormal. 2

Example 5.3 Let H = L2[0, 2π]. Set

fn(t) = sin(nt)

where n ∈ N.Now, let m,n ∈ N, with m 6= n. Without loss of generality, suppose that

n > m.

47

48 CHAPTER 5. ORTHONORMAL SETS

Observe

〈fm(t), fn(t)〉 =∫ 2π

0

sin(mt) sin(nt) dt =1

2

∫ 2π0

cos((n−m)t) dt−12

∫ 2π0

cos((n+m)t) dt = 0.

Therefore the set {fn | n ∈ N} is orthogonal.Now, observe

‖f‖2 =∫ 2π

0

sin2(nt) dt =1

2

∫ 2π0

1− cos(2nt) dt = π.

Set

gn(t) =1√π

sin(nt).

Then the set {gn | n ∈ N} is orthonormal.

Now, we call a sequence of vectors, (en), in a Hilbert space H an orthonormalsequence if ‖en‖ = 1 for all n, and 〈em, en〉 = 0 if m 6= n. Thus, in the aboveexample, we have an orthonormal sequence, (gn), in the space L

2[0, 2π].The proof of the following is left as an exercise.

Proposition 5.4 Any orthogonal set of vectors is linearly independent. 2

The following result (or rather, its proof) tells us how to construct an or-thonormal sequence out of a linearly independent sequence. It is called theGram-Schmidt orthonormalisation process.

Theorem 5.5 Let H be a Hilbert space. Let {vn | n ∈ N} be a linearly inde-pendent set of vectors in H. Then we have an orthonormal sequence, (en), suchthat

Span{e1, . . . , en} = Span{v1, . . . , vn}.

for all n.

Proof: We set

e1 =1

‖v1‖v1.

Then ‖e1‖ = 1, and certainly Span{e1} = Span{v1}..Let

w2 = v2 − 〈e1, v2〉e1and

e2 =1

‖w2‖w2.

Observe that 〈e1, w2〉 = 0, so 〈e1, e2〉 = 0. Clearly ‖e2‖ = 1.Certainly e1, e2 ∈ Span{v1, v2}. Since the vectors e1 and e2 are orthogonal,

they are linearly independent, so their span has dimension equal to the span ofv1 and v2. But this means that Span{e1, e2} = Span{v1, v2}.

5.1. ORTHONORMAL SETS AND BASES 49

Now, suppose we have an orthonormal set {v1, . . . , vn−1} such that

Span{e1, . . . , en−1} = Span{v1, . . . , vn−1}.

Letwn = vn − 〈e1, vn〉e1 − · · · − 〈en−1, vn〉.

and

en =1

‖wn‖wn.

Then as above, the set {e1, . . . , en} is orthonormal, and Span{e1, . . . , en} =Span{v1, . . . , vn}. The result therefore follows by induction. 2

In general, in a vector space V and a subset S, the span of S, Span(S) isthe smallest linear subspace of V that contains S. More concretely,

Span(S) = {α1v1 + · · ·+ αnvn | αi ∈ F, vi ∈ V, n ∈ N}.

Thus the span of a subset of the set of all finite linear combinations ofelements in that set. Note that we only include finite linear combinations, evenwhen the set is infinitely large.

Definition 5.6 A subset, M , of a normed vector space V is called total if thespan of M is a dense subset of V , that is to say

Span(M) = V.

A total orthonormal set of a Hilbert space, H, is called an orthonormal basisof H.

The notion of orthonormal basis is central to computations, as we will seemainly in the semester two notes. It is an interesting and not altogether easyproblem as to how to construct orthonormal bases, or to check that a givenorthonormal set is total. We will look into the example of Fourier series laterin this chapter.

Proposition 5.7 Let H be a Hilbert space, and let M ⊆ H. Then M is totalif and only if M⊥ = {0}.

Proof: Let M be total. Then Span(M) = H. By proposition 4.15, we

know that Span(M)⊥

= Span(M)⊥. It is shown in the exercises that M⊥ =Span(M)⊥. Hence

M⊥ = H⊥ = {0}.

Conversely, suppose that M⊥ = {0}. Then Span(M)⊥ = {0}, and bycorollary 4.19

Span(M) = (Span(M)⊥)⊥ = {0}⊥ = H.

This completes the proof. 2


5.2 Series Related to Orthonormal Sequences

The following result is called Bessel’s inequality.

Proposition 5.8 Let (en) be an orthonormal sequence in a Hilbert space, H.Let x ∈ H. Then the series

∑∞n=1 |〈en, x〉|2 converges, and

∞∑n=1

|〈en, x〉|2 ≤ ‖x‖2.

Proof: Since |〈en, x〉|2 ≥ 0 for all n, it suffices to show that for any N ∈ N,we have

N∑n=1

|〈en, x〉|2 ≤ ‖x‖2.

Set

y =

N∑n=1

〈en, x〉en z = x− y.

Then x = y + z. Observe

〈z, y〉 = 〈x− y, y〉 = 〈x, y〉 − 〈y, y〉.

But

〈x, y〉 =N∑n=1

〈en, x〉〈x, en〉 =N∑n=1

|〈en, x〉|2 = 〈y, y〉.

We conclude that z and y are orthogonal. Hence ‖x‖2 = ‖y‖2 + ‖z‖2. Inparticular, it follows that

‖y‖2 =N∑n=1

|〈en, x〉|2 ≤ ‖x‖2

and we are done. 2

Lemma 5.9 Let H be a Hilbert space. Let an ∈ F. Then the series∑∞n=1 anen

converges in norm in H if and only if the series∑∞n=1 |an|2 converges in R.

Proof: Let

sN =

N∑n=1

anen SN =

N∑n=1

|an|2.

Then for M,N ∈ N with N > M we have

‖sN − sM‖2 = ‖N∑

n=M+1

anen‖2 =N∑

n=M+1

|an|2 = SN − SM .

5.2. SERIES RELATED TO ORTHONORMAL SEQUENCES 51

Suppose the series∑∞n=1 anen converges. Then the sequence of partial sums,

(sn), is Cauchy. By the above, it follows that the real sequence (Sn) is Cauchy,and therefore converges. Hence the series

∑∞n=1 |an|2 converges.

Conversely, suppose the series∑∞n=1 |an|2 converges. Then the sequence of

partial sums, (Sn), is Cauchy. By the above, the sequence of partial sums (sn)is also Cauchy. Since the space H is a Hilbert space, and therefore complete,this means the sequence (sn) converges, that is to say the series

∑∞n=1 anen

converges. 2

Theorem 5.10 Let H be a Hilbert space, and let (en)∞n=1 be an orthonormal

basis. Let x ∈ H. Set an = 〈en, x〉. Then the series∑∞n=1 anen converges in

norm, and

x =

∞∑n=1

anen.

Proof: By Bessel’s inequality, the series∑∞n=1 |an|2 converges in norm. The

above lemma tells us that the series∑∞n=1 anen therefore converges.

Let

y =

∞∑n=1

anen z = x− y.

Observe

〈z, y〉 = 〈x− y, y〉 = 〈x, y〉 − 〈y, y〉.

But by continuity of the inner product

〈x, y〉 =∞∑n=1

〈en, x〉〈x, en〉 =∞∑n=1

|〈en, x〉|2 = 〈y, y〉.

Therefore 〈z, y〉 = 0. Let M = Span{en | n ∈ N}. Then y ∈M , so z ∈M⊥

.

But proposition 5.7 tells us that M⊥

= {0}. Therefore z = x− y = 0, so

x = y =

∞∑n=1

anen.

2

Our last result in this section is called Parseval’s law.

Corollary 5.11 Let H be a Hilbert space, and let (en)∞n=1 be an orthonormal

basis. Then for any element x ∈ H, we have

‖x‖2 =∞∑n=1

|〈en, x〉|2.


Proof: By the above theorem, we have

x =

∞∑n=1

〈en, x〉en.

Let

sN =

N∑n=1

〈en, x〉en.

Then the sequence (sN ) converges to x. By continuity of the norm, it followsthat ‖sN‖2 → ‖x‖2 as N →∞.

But

‖sN‖2 =N∑n=1

|〈en, x〉|2.

So if we let N →∞, we see

‖x‖2 = limN→∞

‖sN‖2 =∞∑n=1

|〈en, x〉|2

as required. 2

5.3 Fourier Series

Recall that T = {z ∈ C | |z| = 1} and we define L2(T) to be the completion ofthe space of continuous functions C(T) with respect to the inner product givenby the formula

〈f, g〉 =∫ 2π

0

f(eit)g(eit) dt.

So we have a norm on the space L2(T) defined by writing

‖f‖2 =

√∫ 2π0

|f(eit)|2 dt.

Note that we have different norm on the space C(T) defined by writing

‖f‖∞ = sup{|f(z)| | z ∈ T}.

Comparison of these two norms enables us to apply the Stone-Weierstrasstheorem to prove when certain subspaces of the space L2(T) are total.

Lemma 5.12 Let M be a dense subset of the Banach space (C(T), ‖ − ‖∞).Then M is also a dense subset of the Hilbert space L2(T).

5.3. FOURIER SERIES 53

Proof: Let f ∈ L2(T), and ε > 0. Then we have g ∈ C(T) such that‖g − f‖2 < ε2 .

By hypothesis, we also have h ∈M such that ‖h− g‖∞ < ε2√2π . Observe

‖h− g‖22 =∫ 2π

0

|h(eit)− g(eit)|2 dt ≤ 2π‖h− g‖2∞

so‖h− g‖2 <

ε

2.

It follows that

‖h− f‖2 ≤ ‖h− g‖2 + ‖g − f‖2 < ε

and we are done. 2

We are now ready to apply our theory to Fourier series.

Proposition 5.13 The set of functions {ek | k ∈ Z} where

ek(z) =1√2πzk

is an orthonormal basis for the space L2(T).

Proof: Observe

〈ek, el〉 =1

2π

∫ 2π0

ei(l−k)t dt.

If k = l, then

〈ek, el〉 =1

2π

∫ 2π0

1 dt = 1.

If k 6= l, then〈ek, el〉 =

1

2π(e2πi(l−k) − e0) = 0.

Hence the set {ek | k ∈ Z} is orthonormal. By proposition 3.24, we see thatthe set

A = Span{ek | k ∈ Z}

is a dense subset of C(T), with the supremum norm. By the above lemma, theset A is therefore also a dense subset of L2(T). It follows that {ek | k ∈ Z} is abasis as required. 2

We can put this basis into a form more typical of Fourier series. First notethe following general result.

Lemma 5.14 Let U :H → H ′ be a unitary map, and let S be an orthonormalbasis for the Hilbert space H. Set S′ = {Ux | x ∈ S}. Then S′ is an orthonormalbasis for the Hilbert space H ′


Proof: Let x, x′ ∈ S. Then

〈Ux,Ux′〉 = 〈x, x′〉

so the set S′ is orthonormal, since the set S is orthonormal.Note that Span(S) is dense in H, and U is a continuous surjection, so

Span(S′) = U [Span(S)] = U [Span(S)] = U [H] = H ′.

It follows that the set S′ is total, and we are done. 2

Now we have a unitary U :L2(T)→ L2[0, 2π] given by the formula

(Uf)(t) = f(eix) x ∈ [0, 2π].

Here

Uek(x) =1√2πeikx = cos(kx) + i sin(kx).

By the above lemma, the following is immediate.

Proposition 5.15 Let

fk(x) =1√2πeikx.

Then the set {fk | k ∈ Z} is an orthnormal basis for the space L2[0, 2π]. 2

Let A be the set of all functions of the form

x 7→ a0 +N∑n=1

(an cos(nx) + bn sin(nx))

where n ∈ N and an, bn ∈ F. Then A = U Span{ek | k ∈ Z} so as in the abovelemma, the set A is a dense subset of the space L2[0, frm−epi].

Proposition 5.16 The sequence 1√2π, 1√

πcosx, 1√

πsinx, 1√

πcos(2x), 1√

πsin(2x),

1√π

cos(3x), 1√π

sin(3x), . . . forms an orthonormal basis of the Hilbert space

L2[0, 2π].

Proof: Let

M = { 1√2π,

1√π

cosx,1√π

sinx,1√π

cos(2x),1√π

sin(2x),1√π

cos(3x),1√π

sin(3x), . . .}.

It is a straightforward calculation to check that M is orthonormal. If A isthe above set, then Span(M) = A, which means Span(M) is dense in L2[0, 2π].Hence M is an orthonormal basis of the Hilbert space L2[0, 2π]. 2

Combining the above with theorem 5.10 immediately yields the followingresult.

5.3. FOURIER SERIES 55

Corollary 5.17 Let f ∈ C[0, 2π]. Set

an =1

π

∫ 2π0

f(x) cos(nx) dx bn =1

π

∫ 2π0

f(x) sin(nx) dx.

Then the series

a02

+

∞∑n=1

(an cos(nx) + bn sin(nx))

converges to the function f in the space L2[0, 2π]. 2

The numbers an and bn are called the Fourier coefficients of f . We can alsoformulate this result in terms of the functions eikx.

Corollary 5.18 Let f ∈ C[0, 2π]. Set

ck =1√2π

∫ 2π0

f(x)e−ikx dx.

Then the series∞∑

k=−∞

ckeikx

converges to the function f in the space L2[0, 2π]. 2

Parseval’s Law gives us the following result

Corollary 5.19 Let f ∈ L2[0, 2π]. Let ck be as in the above result. Then∫ 2π0

|f(x)|2 dx =∞∑

k=−∞

|ck|2.

2

This last result can lead to some interesting summation formulae. The fol-lowing is an example.

Proposition 5.20

∞∑n=1

1

n2=π2

6.

Proof: Let f(x) = x and k 6= 0. Observe, if we integrate by parts:

ck =1√2π

∫ 2π0

xe−ikx dx =i

k√

2π

(xe−ikx

)2π0− ik√

2π

∫ ∞0

e−ikx dx.

Hence

ck =i√

2π

k|ck|2 =

2π

k2.


For k = 0:

c0 =1√2π

∫ 2π0

x dx =4π2

2√

2π|c0|2 =

16π4

8π= 2π3.

Now ∫ 2π0

|f(x)|2 dx =∫ 2π

0

x2 dx =8π3

3.

Putting everything together, we see

8π3

3= |c0|2 + 2

∞∑k=1

|ck|2 = 2π3 + 4π∞∑k=1

1

k2.

Rearranging:∞∑k=1

1

k2=

2π2

3− π

2

2=π2

6.

2

Documents

Functional Analysis, Part 1 - Paul Mitchener · 2017. 9. 20. · As a simple example of analysis in the world of normed spaces we present the following result. Proposition 1.16 Let