CONVERGENCE OF SOME GREEDY ALGORITHMS IN BANACH SPACES

CONVERGENCE OF SOME GREEDY ALGORITHMSIN BANACH SPACES

S. J. DILWORTH, DENKA KUTZAROVA AND V. N. TEMLYAKOV

Abstract. We consider some theoretical greedy algorithms forapproximation in Banach spaces with respect to a general dictio-nary. We prove convergence of the algorithms for Banach spaceswhich satisfy certain smoothness assumptions. We compare thealgorithms and their rates of convergence when the Banach spaceis Lp(Td) (1 < p < ∞) and the dictionary is the trigonometricsystem.

Contents

1. Introduction 2

2. Convergence results for the WCGA 5

3. Convergence of the WCGA in H1(T) 11

4. Convergence results for the WDGA 14

5. An application of the WCGA 16

5.1. Convergence 16

5.2. Rate of approximation 17

5.3. Constructive approximation of function classes 18

References 20

Date: January 24, 2002.1991 Mathematics Subject Classification. Primary: 41A65. Secondary: 42A10,

46B20.Key words and phrases. greedy algorithms; Banach spaces; trigonometric

approximation.The research of the third author was supported by NSF Grant DMS 9970326

and by ONR Grant N00014-91-J1343.1

2 S. J. DILWORTH, DENKA KUTZAROVA AND V. N. TEMLYAKOV

1. Introduction

We continue the investigation of nonlinear m-term approximation.

In this paper we concentrate on studying convergence and rate of ap-

proximation of some theoretical greedy algorithms in Banach spaces.

Let X be a Banach space with norm ‖ · ‖. We say that a set of

elements (functions) D from X is a dictionary if each g ∈ D has norm

one (‖g‖ = 1),

g ∈ D implies −g ∈ D,

andX is the closed linear span of D. We study in this paper a particular

case of the following Weak Chebyshev Greedy Algorithm with respect

to D [17]. For an element f ∈ X we denote by Ff a norming (or ‘peak’)

functional for f :

‖Ff‖ = 1, Ff(f) = ‖f‖.

The existence of such a functional is guaranteed by the Hahn-Banach

theorem. Let τ := {tk}∞k=1 be a given sequence of positive numbers with

tk ≤ 1 (k ∈ N). We define the Weak Chebyshev Greedy Algorithm

(WCGA), which is a generalization for Banach spaces of the Weak

Orthogonal Greedy Algorithm for Hilbert spaces defined and studied

in [16] (see also [4] for the Orthogonal Greedy Algorithm), as follows.

Weak Chebyshev Greedy Algorithm (WCGA). We define

f c0 := f c,τ

0 := f . Then, for each m ≥ 1, we inductively define

1). ϕcm := ϕc,τ

m ∈ D is any element of D satisfying

Ffcm−1

(ϕcm) ≥ tm sup

g∈DFfc

m−1(g).

2). Define

Φm := Φτm := Span {ϕc

j}mj=1,

and define Gcm := Gc,τ

m to be the best approximant to f from Φm.

3). Denote

f cm := f c,τ

m := f −Gcm.

We will consider the case τ = {t} with 0 < t ≤ 1 here. To stress this

we will write t instead of τ in the notation.

In Section 2 we discuss the question of convergence of the WCGA in

Banach spaces which satisfy certain smoothness hypotheses. In order

CONVERGENCE OF SOME GREEDY ALGORITHMS IN BANACH SPACES 3

not to disrupt the flow of this Introduction unduly, we defer until Sec-

tion 2 the precise definitions and comparisons of the various notions of

smoothness which will be used.

It is known (see [17]) that the WCGA with τ = {t} converges for all

elements f in a uniformly smooth Banach space. In uniformly smooth

Banach spaces we can even allow tn → 0 as n→ ∞ without losing the

convergence property [17]. It is also known [1, 5] that the assumption

of Gateaux smoothness is necessary to ensure that

infc∈R,g∈D

‖f − cg‖ < ‖f‖

for any f ∈ X and any dictionary D. This means that Gateaux smooth-

ness is necessary for convergence of the WCGA for an arbitrary dic-

tionary. However, we do not know if it is a sufficient condition for

convergence in the case τ = {t}.In Section 2 we show that the WCGA converges in a large class

of reflexive Banach spaces satisfying some mild additional smoothness

conditions. In particular, we prove the following general convergence

result.

Theorem 2.6. Let X be a separable reflexive Banach space. Then X

admits an equivalent norm for which the WCGA converges for every

dictionary D and for every f ∈ X.

In Section 3 we discuss convergence of the WCGA in the Hardy

space H1(T). This provides an interesting test case because H1(T) is

nonreflexive and does not satisfy the smoothness assumptions made in

Section 2. Nevertheless, we are able to prove convergence of the WCGA

for a large class of dictionaries, including the trigonometric system.

In Section 4 we present some results on convergence of the Weak

Dual Greedy Algorithm (WDGA), which is the analogue for Banach

spaces of the Weak Greedy Algorithm in Hilbert space. Here is the

definition of the algorithm for the special case τ = {t} discussed in

Section 4.

Weak Dual Greedy Algorithm (WDGA). We define fD0 := fD,t

0 :=

f . Then for each m ≥ 1 we inductively define

1). ϕDm := ϕD,t

m ∈ D is any element of D satisfying

FfDm−1

(ϕDm) ≥ t sup

g∈DFfD

m−1(g).


2). Define am as follows:

‖fDm−1 − amφ

Dm‖ = min

a∈R‖fD

m−1 − aφDm‖.

3). Denote

fDm := fD,t

m := fDm−1 − amφ

Dm.

Convergence of the WDGA is not very well understood, and we can

prove only a few scattered results in this direction.

In Section 5 we discuss greedy type approximation with regard to a

basis. Let X be a Banach space with basis Ψ = (ψk)∞k=1, ‖ψk‖ = 1.

We compare the efficiency of the WCGA with the following theoreti-

cal greedy algorithm which we call the Thresholding Greedy Algorithm

(TGA) (see [11]).

Thresholding Greedy Algorithm (TGA). For a given f ∈ X, we

consider the expansion

f =∞∑

k=1

ck(f)ψk.

Suppose that f ∈ X and that ρ, ρ(j) = kj (j = 1, 2, . . . ), is a permu-

tation of the positive integers. We say that ρ is decreasing and write

ρ ∈ D(f) if

|ck1(f)| ≥ |ck2(f)| ≥ . . . .

In the case of strict inequalities here, D(f) consists of only one per-

mutation. We define the m-th greedy approximant of f with regard to

the basis Ψ corresponding to a permutation ρ ∈ D(f) by the formula:

Gm(f,Ψ) := GXm(f,Ψ) := GX

m(f,Ψ, ρ) :=

m∑

j=1

ckj(f)ψkj

.

The general conclusion is that the WCGA is more efficient than the

TGA for approximation in Lp for 2 < p < ∞. We also point out in

Section 5 that the WCGA provides a constructive (algorithmic) way

to get best possible m-term trigonometric approximation for different

smoothness classes of functions [3].

We use standard Banach space notation and terminology (see e.g.

[12]) and basic results from functional analysis (see e.g. [6]). For clarity,

however, we recall here the notation and results which will be used most

heavily. Let X be a Banach space. The unit sphere of X, denoted


S(X), is the set {x ∈ X : ‖x‖ = 1}. The dual space of X, denoted X∗,

is the Banach space of all continuous linear functionals F equipped

with the norm:

‖F‖ = sup{F (x) : x ∈ S(X)}.The closed linear span of a set A ⊆ X (resp., a sequence (xn)) is denoted

[A] (resp. [xn]). A normalized Schauder basis for X is a sequence of

unit vectors Ψ = (ψk)∞k=1 such that every f ∈ X has a unique expansion

as a norm-convergent series

f =

∞∑

k=1

ck(f)ψk.

We say that a sequence (xn) in X converges strongly (resp. weakly)

to x ∈ X if ‖x − xn‖ → 0 (resp. for all F ∈ X∗, F (x − xn) → 0)

as n → ∞. We say that a sequence (Fn) in X∗ converges weak-star

to F if Fn(x) → F (x) as n → ∞ for all x ∈ X. A Banach space is

reflexive if its closed unit ball is weakly compact (equivalently, if every

bounded sequence has a weakly convergent subsequence). It follows

from the Banach-Alaoglu theorem (see e.g. [6, p. 424]) that if X

is separable, then the closed unit ball of X∗ is weak-star sequentially

compact (i.e. every bounded sequence in X∗ has a weak-star convergent

subsequence).

Finally, more specialized notions from Banach space theory, includ-

ing the various notions of smoothness used throughout, will be intro-

duced as needed.

2. Convergence results for the WCGA

In this section we discuss convergence of the WCGA. It is known

that the WCGA converges (for an arbitrary dictionary) in all uni-

formly smooth Banach spaces [17]. The Banach spaces which admit

a uniformly smooth renorming (equivalent to the original norm) have

been studied extensively: they are the so-called super-reflexive spaces;

they coincide, in particular, with the spaces which admit an equivalent

uniformly convex renorming [7].

The class of super-reflexive spaces is properly contained, as the term

suggests, in the class of reflexive spaces: for example, the projective

tensor product `p⊗`q (1/p + 1/q < 1) is a separable reflexive space

that is not super-reflexive. We prove below that the WCGA converges


in all reflexive spaces which satisfy some mild additional smoothness

hypotheses. In particular, we show that every separable reflexive space

admits an equivalent norm for which the WCGA converges.

We start by recalling several notions of smoothness for Banach spaces

(see e.g. [2, p. 2]). For x, y ∈ X, we define

ρx,y(u) =‖x+ uy‖ + ‖x− uy‖ − 2‖x‖

2(u ∈ R).

We say that x is a point of Gateaux smoothness if

(1) limu↓0

ρx,y(u)

u= 0 (y ∈ X).

It is known that (1) is equivalent to the uniqueness of the norming

functional Fx. If every x 6= 0 is a point of Gateaux smoothness then X

is said to be smooth. The local modulus of smoothness at x is defined

as follows:

(2) ρx(u) = sup{ρx,y(u) : y ∈ S(X)} (u ∈ R).

We say that x is a point of Frechet differentiability if

(3) limu↓0

ρx(u)

u= 0.

It is known that (3) is equivalent to

‖x+ y‖ = ‖x‖ + Fx(y) + o(‖y‖) (y ∈ X).

If every x 6= 0 is a point of Frechet differentiability then X is said to

be Frechet differentiable. Finally, we define the modulus of smoothness

of X:

(4) ρX(u) = sup{ρx(u) : x ∈ S(X)} (u ∈ R).

X is said to be uniformly smooth if

(5) limu↓0

ρX(u)

u= 0.

It is known that (5) is equivalent to the following:

(6) ‖x+ y‖ = ‖x‖ + Fx(y) + ‖y‖ε(x, y),

where ε(x, y) → 0 as y → 0 uniformly for x ∈ S(X).


Suppose that 0 6= f ∈ X and that (Gc,tn ), (φc,t

n ), and Ffc,tn

, are as

defined in the Introduction. To simplify notation, we shall drop the

superscript t, so that Gcn := Gc,t

n , etc. It is clear that

‖f cn‖ ↓ α ≥ 0

as n→ ∞. The WCGA converges if and only if α = 0. Note also that

(7) α = dist(f, [φcn]∞n=1).

Lemma 2.1. Suppose that X is smooth and that α > 0. Then there

exists β > 0 such that Ffcn(φc

n+1) > β for all sufficiently large n.

Proof. Since Gcn is the best approximant to f from [φc

i ]ni=1, it follows

that there exists Fn ∈ S(X∗) such that Fn(φci) = 0 for 1 ≤ i ≤ n and

Fn(f cn) = ‖f c

n‖. Clearly Fn is a norming functional for f cn, and since X

is smooth it follows that Fn = Ffcn. Thus,

Ffcn(φc

i) = Fn(φci) = 0 (1 ≤ i ≤ n).

Let D1 = D \ {±φcn : n ≥ 1}. Since f ∈ [D] there exist N ≥ 1, m ≥ 1,

f ′ ∈ [φci ]

Ni=1, and

(8) f1 =m∑

i=1

aigi

(for some gi ∈ D1 and scalars ai 6= 0 (1 ≤ i ≤ m)) such that

‖f − f ′ − f1‖ < α/2.

Suppose that n ≥ N . Then Ffcn(φc

i) = 0 for 1 ≤ i ≤ N , and so

Ffcn(f ′) = 0. Thus,

Ffcn(f1) = Ffc

n(f ′ + f1) ≥ Ffc

n(f) − ‖f − f ′ − f1‖

≥ α− α/2 = α/2.

From (8) and the above, we obtainm∑

i=1

|ai||Ffcn(gi)| ≥ α/2,

and hence

max1≤i≤m

|Ffcn(gi)| ≥

α

2mmax1≤i≤m |ai|.

Finally,

Ffcn(φc

n+1) ≥ t supg∈D

Ffcn(g) ≥ t max

1≤i≤mFfc

n(gi).


So we can take β = tα/(2mmax1≤i≤m |ai|).�

Proposition 2.2. Suppose that X is Frechet differentiable. Then (Gcn)

converges strongly to f if and only if (Gcn) has a strongly convergent

subsequence.

Proof. Assume, to derive a contradiction, that some subsequence (Gcnk

)

converges strongly to y 6= f . Then y−f is a point of Frechet differentia-

bility, and so by Smulyan’s Lemma [2, Thm. 1.4] the duality mapping

x 7→ Fx from X to X∗ is norm-to-norm continuous at y − f . Thus,

FGcnk

−f → Fy−f strongly as k → ∞. For each n ≥ 1, we have

Fy−f (φcn) = lim

k→∞FGc

nk−f(φ

cn) = 0.

Thus,

FGcnk

−f (φcnk+1) = (FGc

nk−f − Fy−f )(φ

cnk+1) ≤ ‖FGc

nk−f − Fy−f‖ → 0

as k → ∞. But by Lemma 2.1, FGcnk

−f (φcnk+1) > β > 0, which is a

contradiction. Thus, y = f , which implies that (Gcn) converges strongly

to f . �

Remark 2.3. Following the method of [17], an alternative proof of

Proposition 2.2 can be given which uses the local modulus of smooth-

ness ρx(u) defined in (2).

Let us recall that X has the Kadec-Klee property (known also as

the Radon-Riesz property or property H ) if the following holds: when-

ever a sequence (xn) in X converges weakly to x ∈ X, and ‖x‖ =

limn→∞ ‖xn‖, then (xn) converges strongly to x. It is known that every

separable Banach space admits an equivalent norm with the Kadec-

Klee property (see e.g. [2, Th. 2.6]).

Proposition 2.4. Suppose that X has the Kadec-Klee property and a

Frechet differentiable norm. Then (Gcn) converges strongly to f if and

only if (Gcn) has a weakly convergent subsequence.

Proof. Suppose that some subsequence (Gcnk

) converges weakly to y.

Note that y ∈ [φcn]

∞n=1, since [φc

n]∞n=1 is weakly closed. Hence (7) gives

‖y − f‖ ≥ dist(f, [φcn]∞n=1) = α.


On other hand, since (Gcnk

) converges weakly to y, we have

‖y − f‖ = Fy−f(y − f)

= limk→∞

Fy−f (Gcnk

− f)

≤ limk→∞

‖Gcnk

− f‖ = α.

Since X has the Kadec-Klee property, it follows that (Gcnk

− f) con-

verges strongly to y − f , i.e., (Gcnk

) converges strongly to y. Finally,

by Proposition 2.2, it follows that (Gcn) converges strongly to f . �

Proposition 2.5. Suppose that X is a reflexive Banach space which

has both the Kadec-Klee property and a Frechet differentiable norm.

Then the WCGA converges for every dictionary D and for every f ∈ X.

Proof. Since X is reflexive, (Gcn) has a weakly convergent subsequence.

Proposition 2.4 now yields the result. �

Now we deduce the main result of this section.

Theorem 2.6. Let X be a separable reflexive Banach space and let

ε > 0. There exists an equivalent norm | · | on X, with

‖x‖ ≤ |x| ≤ (1 + ε)‖x‖ (x ∈ X),

for which the WCGA converges in (X, | · |) for every dictionary D and

every f ∈ X.

Proof. Every separable reflexive Banach space admits such a norm | · |which is both Frechet differentiable and has the Kadec-Klee property

(see e.g. [2, Thm. 2.6]). Proposition 2.5 now yields the result. �

Finally, we prove a result for non-reflexive Banach spaces. Suppose

that X is a smooth Banach space and that y ∈ S(X). Recall that X

is said to be uniformly Gateaux differentiable in the direction y if

(9) limu↓0

ρx,y(u)

u= 0

uniformly for x ∈ S(X); equivalently,

‖x+ uy‖ = ‖x‖ + u(Fx(y) + o(u)) (u ∈ R)

uniformly for x ∈ S(X).

We shall say that a dictionary D is minimal if D has no proper

subdictionary. Recall that every separable Banach space contains a


system (wn) ⊂ S(X) satisfying the following: {wn : n ≥ 1} has dense

span and no proper subset has dense span. We shall say that (wn) is

quasi-minimal. (In fact, (wn) can be chosen to be both minimal and

total [12, p. 43], although we don’t need this). Clearly, (wn)∞n=1 ⊂S(X) is quasi-minimal if and only if D = {±wn : n ≥ 1} is a minimal

dictionary.

Theorem 2.7. Let X be a separable Banach space and let ε > 0. There

exists an equivalent norm | · | on X, with

‖x‖ ≤ |x| ≤ (1 + ε)‖x‖ (x ∈ X),

such that (X, | · |) has a minimal dictionary D for which the WCGA

converges for every f ∈ X.

Proof. Every separable Banach space admits such an equivalent norm

| · | which is uniformly Gateaux differentiable in every direction [2,

Cor. II.6.9]. Choose y ∈ S(X, | · |) and choose a quasi-minimal system

(wn) ⊂ S(X, | · |), with w1 = y. Let yn = (1/n)wn+1 + y (n ≥ 1), so

that (yn) converges strongly to y. Then (yn) is also quasi-minimal, and

hence D = {±yn/|yn| : n ≥ 1} is a minimal dictionary.

We show that the WCGA converges for D. So assume that f ∈ X,

and, to derive a contradiction, assume that |f cn| ↓ α > 0. There exists

a choice of signs εn = ±1 such that (εnφcn) converges strongly to y.

By Lemma 2.1, there exists β > 0 such that Ffcn(φc

n+1) > β for all

sufficiently large n. Hence

(10) Ffcn(εny) ≥ β for all sufficiently large n.

Since X is uniformly Gateaux differentiable in the direction y, it follows

from (10) and (9) that there exists s0 > 0 such that

|f cn| − |f c

n − εns0y| ≥βs0

2for sufficiently large n. Since (εnφ

cn) converges strongly to y, we obtain

|f cn| − |f c

n − εs0φcn+1| ≥

βs0

3for all sufficiently large n, which implies that

|f cn+1| ≤ |f c

n| −βs0

3.

But this contradicts our assumption that |f cn| ↓ α.

�


3. Convergence of the WCGA in H1(T)

In this section we shall discuss convergence of the WCGA in the

Hardy space H1(T) defined below. This space is nonreflexive and is

nowhere Frechet differentiable [2, Prop. 4.5], and so Proposition 2.5

does not apply. Nevertheless, we are able to prove that the WCGA

converges for a large class of dictionaries, including the trigonometric

system.

The WCGA was defined in the Introduction for real Banach spaces.

But since H1(T) is a complex Banach space, we must first explain how

the algorithm is to be applied in complex spaces. So suppose that X is

a complex Banach space with complex dual space X∗. Let XR be the

real Banach space obtained from X by restricting scalar multiplication

to real scalars. To apply the WCGA in X, we simply apply the WCGA

in the real space XR. It follows from the complex version of the Hahn-

Banach theorem (due to Bohnenblust and Sobczyk [6, p. 63]) that the

mapping x∗ 7→ < x∗ defines a real linear isometry from (X∗)R onto

(XR)∗. In particular, the norming functionals in (XR)∗ are precisely

the real parts of norming functionals in X∗.

Now let us recall the definition of H1(T). Let T denote the unit

circle in the complex plane and let m denote the normalized Lebesgue

measure on T. Let C(T) denote the Banach space of all complex-

valued continuous functions equipped with the supremum norm. The

disc algebra A(T) is the complex closed linear span of {zn : n ≥ 0} in

C(T). Let A0 denote the subspace of A(T) consisting of the mean zero

functions. The Hardy space H1(T) is the (complex) closed linear span

of {zn : n ≥ 0} in the complex Lebesgue space L1(T).

First we present an example which shows that the WCGA does not

converge in general in L1(T) for the trigonometric dictionary D =

{±zn,±izn : n ≥ 0}. (A similar example works for the usual trigono-

metric system {cosnθ}n≥0 ∪ (sinnθ)n≥1.)

Example 3.1. Let f(eiθ) = 4χ[3π/4,5π/4](θ) (0 ≤ θ < 2π). Then, for

every g ∈ D, we have

‖f + sg‖1 ≥ 1 +|s|2

(s ∈ R).

Thus, s = 0 gives the unique best approximant to f from D. Hence

f c,t1 = f , which of course implies that f c,t

n = f for all n ≥ 1.


Before we can proceed, it is necessary to recall some facts about

H1(T) as a Banach space.

Fact 3.2. (see e.g. [9, p. 137]) H1(T) is the dual space of C(T)/A0(T),

with the duality given by

〈f, g〉 =

∫

Tfg dm (f ∈ C(T), g ∈ H1(T)).

Weak-star convergence in Fact 3.4 and in the proof of Theorem 3.5

below refers to the weak-star topology on H1(T) induced by its predual

C(T)/A0.

Fact 3.3. H1(T) is a smooth Banach space. For 0 6= f ∈ H1(T), the

unique (real) norming functional Ff is given by

Ff(g) = <∫

Tsgn fg dm.

Here sgn 0 := 0 and sgn z := z/|z| for z 6= 0.

Fact 3.4. [14] H1(T) has the weak-star Kadec-Klee property, i.e., if (fn)

converges weak-star to f and ‖f‖ = limn→∞ ‖fn‖, then (fn) converges

strongly to f .

Recall that a subset K ⊂ L1(m) is uniformly integrable if for every

ε > 0 there exists δ > 0 such that if E ⊆ T satisfies m(E) < δ then∫

E

|f | dm ≤ ε (f ∈ K).

Theorem 3.5. Suppose that the dictionary D ⊂ H1(T) satisfies the

following two conditions:

(a) D is uniformly integrable;

(b) [E] is weak-star closed whenever E ⊆ D.

Then the WCGA converges for every f ∈ H1(T).

Proof. With the notation introduced above, suppose, to derive a con-

tradiction, that ‖f cn‖ ↓ α > 0. By Lemma 2.1 and Fact 3.3 there exists

β > 0 such that

(11) Ffcn(φc

n+1) ≥ β (n ≥ 1).


There is a subsequence (Gcnk

)∞k=1 which converges weak-star to some y ∈H1(T). Let E = {φc

n : n ≥ 1}. Then, by assumption (b), [E] is weak-

star closed in H1(T), and so y ∈ [E]. Thus ‖y − f‖ ≥ dist(f, [E]) = α.

On the other hand, by weak-star lower semi-continuity of ‖ · ‖, we have

‖y − f‖ ≤ limk→∞ ‖Gcnk

− f‖ = α. So

‖y − f‖ = limk→∞

‖Gcnk

− f‖ = α.

Since H1(T) has the weak-star Kadec-Klee property (Fact 3.4), it fol-

lows that (Gcnk

) converges strongly to y. SinceH1(T) is smooth (Fact 3.3),

the mapping x 7→ Fx is norm-to-weak-star continuous. So

(12) Ff−y(φcn) = lim

k→∞Ffc

nk(φc

n) = 0 (n ≥ 1).

Recall that m({z : f(z) = 0}) = 0 for all f ∈ H1(T) (see e.g. [9, p.

52]). So, by passing to a subsequence if necessary, we may assume that

sgn(Gcnk

− f) → sgn(y − f) a.e.

Hence the uniform integrability of D (assumption (b)) gives

(13)

∫

T(sgn(Gc

nk− f) − sgn(y − f))g(z) dm→ 0

as k → ∞ uniformly for all g ∈ D. Now (11), (12), (13) and Fact 3.3

yield

0 = limk→∞

Ff−y(φcnk+1)

= limk→∞

<∫

Tsgn(y − f)φc

nk+1(z) dm

= limk→∞

<∫

Tsgn(Gc

nk− f)φc

nk+1(z) dm

= limk→∞

Ffcnk

(φcnk+1) ≥ β > 0,

which is the desired contradiction. �

We now show that the trigonometric system D = {±zn,±izn : n ≥0} satisfies the assumptions of Theorem 3.5. Uniform integrability is

clear. The second condition is verified in the next lemma.

Lemma 3.6. Suppose that A,B ⊆ N. Let X denote the real closed

linear span of {zn : n ∈ A} ∪ {izn : n ∈ B} in H1(T). Then X is a

weak-star closed subspace of H1(T).


Proof. For n ∈ Z, let an(f) denote the n-th Fourier coefficient of f .

It is clear that the weak-star closure of X consists precisely of those

f ∈ H1(T) such that < an(f) = 0 for all n ∈ N \A and = an(f) = 0 for

all n ∈ N \B. So the Fejer polynomials

pn =

n∑

k=0

(1 − k

n

)ak(f)zk (n ≥ 0)

all belong to X whenever f lies in the weak-star closure of X. But (pn)

converges strongly to f . Thus, f ∈ X. �

Theorem 3.7. In H1(T) the WCGA converges for the trigonometric

system D = {±zn,±izn : n ≥ 0} and for every f ∈ H1(T).

Remark 3.8. Hoffmann [10] proved that the multivariate Hardy space

H1(Td) has the weak-star Kadec-Klee property with respect to its nat-

ural predual. Using this fact it is a straightforward matter to extend

Theorem 3.5 to the multivariate case. In particular, Theorem 3.7 is

valid for H1(Td) and the multivariate trigonometric system.

4. Convergence results for the WDGA

We have only a few isolated results for convergence of the WDGA.

Proposition 4.1. Let X be a uniformly smooth Banach space. Then,

for every increasing sequence (nk) ⊆ N, either (GDnk

) converges weakly

in X to f or (FfDnk

) has a weakly null subsequence in X∗.

Proof. Suppose that (GDnk

) does not converge weakly to f . Since X

is uniformly smooth, and hence reflexive, by passing to a subsequence

and relabelling we may assume that (GDnk

) converges weakly to y 6= f

and that (FfDnk

) converges weakly to F , say. Suppose, to derive a

contradiction, that F 6= 0. Then there exists g ∈ D such that F (g) =

β > 0. By weak convergence FfDnk

(g) > β/2, and hence

FfDnk

(φDnk+1) >

tβ

2

for sufficiently large k. Since X is uniformly smooth, it follows from

(6) that there exists s0 > 0 such that

‖fDnk

− s0φDnk+1‖ ≤ ‖fD

nk‖ −

s0FfDnk

(φDnk+1)

2


for all k. Thus, for sufficiently large k, we have

‖fDnk+1‖ ≤ ‖fD

nk‖ − tβs0

2,

which contradicts the fact that ‖fDn ‖ ↓ α > 0. Thus F = 0. �

Theorem 4.2. Suppose that X is uniformly smooth and satisfies the

following: (xn) ⊂ S(X) is weakly null whenever (Fxn) is weakly null.

Then (GDn ) converges weakly to f .

Proof. From Proposition 4.1, the hypothesis implies that every subse-

quence of (GDn ) has a further subsequence which converges weakly to

f . But this implies that (GDn ) converges weakly to f . �

Since `p satisfies the hypotheses of Theorem 4.2, we get the following

corollary.

Corollary 4.3. The WDGA converges weakly in `p (1 < p < ∞) for

every dictionary D and for every f ∈ `p.

We have only one result for strong convergence of the WDGA. Say

that a Schauder basis (ψn) is strictly supression 1-unconditional if, for

every finite A ⊂ N, for every n ∈ N \ A, and for all scalars (ai)i∈A, we

have

‖∑

i∈A

aiψi‖ < ‖∑

i∈A

aiψi + en‖.

The standard basis of an Orlicz or Lorentz sequence space satisfies this

condition.

Theorem 4.4. Suppose that X is Frechet differentiable and that Ψ =

(ψn) is a strictly suppression 1-unconditional basis for X. Then the

WDGA converges for D = {±ψn : n ≥ 1} and for every f ∈ X.

Proof. The fact that Ψ is strictly suppression 1-unconditional implies

that GDn = Gc

n = Pn(f), where Pn is the natural projection of X onto

[ψci ]

ni=1. In particular, (GD

n ) coincides with one possible ‘branch’ (Gcn)

for the WCGA. Moreover, since Ψ is unconditional, (Pn(f)) converges

strongly to P (f), where P is the natural projection onto [ψci ]∞i=1. By

Proposition 2.4, (Gcn) (and hence (GD

n )) converges strongly to f �

Remark 4.5. Suppose that X has an unconditional basis and is Frechet

differentiable. It is not known whether there exists an equivalent


Frechet differentiable norm for which the basis is strictly suppression

1-unconditional. We thank the referee for this remark.

5. An application of the WCGA

In this section we consider a particular case: the Banach space

X = Lp(Td) and the dictionary D = RT – the real trigonomet-

ric system {1/2, sinx, cos x, . . . } and its d-dimensional version RT d =

RT × . . .RT . It is more convenient for us to consider the real Lp(Td)

and the real trigonometric system because the Weak Chebyshev Greedy

Algorithm is defined and studied in a real Banach space. Note that

the system RT is not normalized in Lp but only semi-normalized:

C1 ≤ ‖f‖p ≤ C2 for any f ∈ RT with absolute constants C1, C2,

1 ≤ p ≤ ∞. This is sufficient for application of the general meth-

ods developed in [17]. We will compare performance of the WCGA

with performance of the Thresholding Greedy Algorithm (TGA). It is

proved in [15] that in the case of the complex trigonometric system

T d = {ei(k,x)}, for any f ∈ Lp(Td), we have that

(14) ‖f −Gm(f, T d)‖p ≤ Cm|1/2−1/p|σm(f, T d)p, 1 ≤ p ≤ ∞,

with an absolute constant C. The same proof works for RT d and gives

(15) ‖f −Gm(f,RT d)‖p ≤ Cm|1/2−1/p|σm(f,RT d)p, 1 ≤ p ≤ ∞.

5.1. Convergence. It is shown in [15, Remark 4] that Gm(·, T ) may

fail to converge in Lp when p 6= 2. The same is true for Gm(·,RT ).

The convergence of WCGA for 1 < p <∞ follows from general results

(see [17, Th. 2.1] cited in Introduction). The divergence of Gm(·, T )

and Gm(·,RT ) can be fixed by adding the “Chebyshev step” in these

algorithms. We describe this in the general case of Gm(·,Ψ). At the

step m instead of taking the partial sum

Gm(f,Ψ) =m∑

j=1

ckjψkj

,

we take the best approximation Bm(f,Ψ, X) to f from Span {ψk1, . . . , ψkm}.It is easy to see that in this case we have

‖f − Bm(f,Ψ, X)‖X → 0


as m → ∞. Thus, in the sense of convergence, the Weak Chebyshev

Greedy Algorithm and the Chebyshev Thresholding Greedy Algorithm

(CTGA) defined above are both effective in Lp, 1 < p <∞.

5.2. Rate of approximation. We will compare the rate of approxi-

mation of TGA, CTGA, and WCGA for the class

A1 := A1(RT ) := {f :

∞∑

k=0

|ak(f)| + |bk(f)| ≤ 1},

where ak, bk are the corresponding Fourier coefficients. From the gen-

eral results on convergence rate of the Weak Chebyshev Greedy Algo-

rithm (see [17, Th.2.2]) we get the following lemma.

Lemma 5.1. For f ∈ A1 we have

(16) ‖f c,tm ‖p ≤ C(p, t)m−1/2, 2 ≤ p <∞.

This estimate and (15) imply, for f ∈ A1 and 2 < p <∞, that

(17) ‖f −Bm(f,RT , Lp)‖p ≤ ‖f −Gm(f,RT )‖p ≤ C(p, t)m−1/p,

which is weaker than (16).

Let us give an example showing that (17) can not be improved in

the sense of order. Consider for a given m

f := (2m)−1

2m∑

k=1

(1 − k

4m) cos kx ∈ A1.

Then Bm(f,RT , Lp) is the best approximation to f in Lp from

Span {cos x, . . . , cosmx}. It is not difficult to see that

(18) ‖f − Bm(f,RT , Lp)‖p ≥ C(p)m−1/p.

This proves that (17) cannot be improved in the sense of the order of

m.

We will show now that the constant C(p, t) in (17) can be replaced

by 1. We denote p′ := p/(p− 1) and use the Hausdorff-Young theorem

(see [18, Ch. 12, Sec. 2]): for g ∈ Lp′ , 2 ≤ p <∞, we have

(19) (∞∑

k=0

|ak(g)|p + |bk(g)|p)1/p ≤ ‖g‖p′.


Then we have for any f ∈ A1

‖f −Gm(f,RT )‖p = supg,‖g‖p′≤1

|〈f −Gm(f,RT ), g〉|

= supg,‖g‖p′≤1

|∑

k/∈Λc

ak(f)ak(g) +∑

l /∈Λs

bl(f)bl(g)|,(20)

where Λc and Λs are such that (here # denotes cardinality)

Gm(f,RT ) =∑

k∈Λc

ak(f) cos kx +∑

l∈Λs

bl(f) sin kx, #Λc + #Λs = m.

Using the Holder inequality we continue (20)

≤ sup‖g‖p′≤1

(∑

k/∈Λc

|ak(f)|p′ +∑

l /∈Λs

|bl(f)|p′)1/p′(∑

k/∈Λc

|ak(g)|p +∑

l /∈Λs

|bl(g)|p)1/p.

By (19) we have

≤ (∑

k/∈Λc

|ak(f)|p′ +∑

l /∈Λs

|bl(f)|p′)1/p′

Using the definition of A1 and Λc,Λs, we finish the estimate

≤ m1/p′−1 = m−1/p.

Thus we have proved that for f ∈ A1 the inequality

(21) ‖f −Gm(f,RT )‖p ≤ m−1/p, 2 ≤ p <∞,

holds.

The relations (16) and (18) show that the WCGA gives a better rate

of approximation in Lp (2 ≤ p <∞) than the CTGA.

5.3. Constructive approximation of function classes. In this sub-

section we discuss the question of efficiency of the algorithms TGA and

WCGA for some function classes. As above we confine ourselves to the

case of m-term approximation with regard to the trigonometric system.

For a function class F , we denote

σm(F, T d)p := supf∈F

σm(f, T d)p.

The inequality (14) implies that for any function class F we have

supf∈F

‖f −Gm(f, T d)‖p ≤ Cm|1/2−1/p|σm(F, T d)p, 1 ≤ p ≤ ∞.

We would like to understand for what function classes F the construc-

tive algorithms TGA and WCGA provide an optimal order of approx-

imation, the order of σm(F, T d)p. The major point of the discussion


that follows is that in the case of standard smothness classes and the

Lp spaces with 2 ≤ p ≤ ∞, the WCGA provides a constructive way of

m-term approximation as efficient as the best m-term approximation.

We will need some known results on σm(F, T d)p in the discussion. For

the reader’s convenience these results are formulated as theorems (see

Theorem 5.2 and 5.3).

In the paper [3] two types of function classes were studied from the

point of view of best m-term trigonometric approximation. We begin

with the first class. For 0 < α <∞ and 0 < q ≤ ∞, let Fαq denote the

class of those functions in L1(Td) such that

|f |Fαq

:= (∑

k∈Zd

(max(1, |k1|, . . . , |kd|)αq|f(k)|q)1/q ≤ 1.

The following theorem was proved in [3].

Theorem 5.2. If α > 0 and λ := α/d + 1/q − 1/2, then for all 1 ≤p ≤ ∞ and all 0 < q ≤ ∞

C1m−λ ≤ σm(Fα

q , T d)p ≤ C2m−λ, α > d(1 − 1/q)+,

with C1, C2 > 0 constants depending only on d, α, q.

The second class is defined as follows. Let α > 0, 0 < τ, s ≤ ∞ and

Bαs (Lτ ) denote the class of functions such that there exist trigonometric

polynomials Tn of coordinate degree 2n with the properties

f =

∞∑

n=0

Tn, ‖{2nα‖Tn‖τ}∞n=0‖`s(Z) ≤ 1.

The following theorem concerning these classes was proved in [3].

Theorem 5.3. Let 1 ≤ p ≤ ∞, 0 < τ, s ≤ ∞; and define

α(p, τ) :=

{d(1/τ − 1/p)+, 0 < τ, s ≤ ∞ and 1 ≤ p ≤ τ ≤ ∞max(d/τ, d/2), otherwise.

Then for α ≥ α(p, τ), we have

C1m−µ ≤ σ(Bα

s (Lτ ), T d)p ≤ C2m−µ,

where

µ := α/d− (1/τ − max(1/p, 1/2))+

and C1, C2 depend only on α, p, τ , and d.

Remark 5.4. Theorems 5.2 and 5.3 hold with T d replaced by RT d.


It was proved in [17] that in the case 1 ≤ p ≤ 2 the rate of best m-

term approximation in Theorem 5.2 can be realized by Gm(·, T d) for

the TGA, that is by a constructive method (the same is true for RT d).

In the same case 1 ≤ p ≤ 2 the rate of σm(Bαs (Lτ ), T d)p can be realized

by approximating by trigonometric polynomials of degree m1/d in each

variable. Thus, in the case 1 ≤ p ≤ 2, there exist constructive methods

which provide the optimal rate in Theorems 5.2 and 5.3. However, for

the case 2 < p ≤ ∞ which is the most interesting case in Theorems

5.2 and 5.3 from the point of view of upper estimates (they do not

depend on p in this case) there was not a constructive proof of the

upper estimates. The existing methods in this case are based either

on a probabilistic approach (Yu. Makovoz, [13], 2 < p < ∞) which

does not cover the most interesting case p = ∞ or on the geometry of

finite-dimensional Banach spaces ([3], 2 < p ≤ ∞) which covers the

case p = ∞. Both approaches contain a nonconstructive step. In [3]

this step is hidden in the following inequality (see [3, Corollary 5.1])

(22) σm(A1(T dn ), T d)∞ ≤ Cm−1/2(1 + ln+ nd

m)1/2,

where T dn denotes the subsystem of the trigonometric system T d which

forms a basis for the space of trigonometric polynomials of coordinate

degree n. The inequality (22) was proved in [3] with the help of a

theorem of Gluskin [8].

The major purpose of this section is to note that in the case 2 < p <

∞ the Weak Chebyshev Greedy Algorithm provides a constructive way

to get an analog of (22). This follows immediately from Lemma 5.1:

for f ∈ A1(RT dn) we have

(23) ‖f c,tm ‖p ≤ C(p, t)m−1/2, 2 ≤ p <∞.

Thus the only nonconstructive step in the proof of upper estimates in

Theorems 5.2 and 5.3 can be made constructive for p <∞. We do not

know a constructive proof of (22).

References

[1] C. Darken, M. Donahue, L. Gurvitz and E. Sontag, Rate of convex approx-imation in non-Hilbert spaces, Constr. Approx. 13 (1997), 187–220.

[2] R. Deville, G. Godefroy and V. Zizler, Smoothness and renormings in Ba-nach spaces, Longman Scientific & Technical, Harlow, 1993.


[3] R. A. DeVore and V. N. Temlyakov, Nonlinear approximation by trigono-metric sums, J. Fourier Anal. Appl. 2 (1995), 29–48.

[4] R. A. DeVore and V. N. Temlyakov, Some remarks on Greedy Algorithms,Adv. Comput. Math. 5 (1996), 173–187.

[5] V. V. Dubinin, Greedy Algorithms and Applications, Ph.D. thesis, Univer-sity of South Carolina, 1997.

[6] N. Dunford and J. Schwartz, Linear Operators, Vol. I, Interscience, NewYork, 1963.

[7] Per Enflo, Banach spaces which can be given an equivalent uniformly convexnorm, Israel J. Math. 13 (1972), 281–288.

[8] E. D. Gluskin, Extremal properties of orthogonal parallelpipeds and theirapplication to the geometry of Banach Spaces, Math. USSR-Sb. 64 (1989),85–96.

[9] Kenneth Hoffman, Banach Spaces of Analytic Functions, Prentice-Hall, Ea-glewood Cliffs, N.J., 1962.

[10] L. D. Hoffmann, Pseudo-uniform convexity of H1 in several variables, Proc.Amer. Math. Soc. 26 (1970), 609–614.

[11] S. V. Konyagin and V. N. Temlyakov, A remark on greedy approximationin Banach spaces, East J. Approx. 5 (1999), 1–15.

[12] J. Lindenstrauss and L. Tzafriri, Classical Banach spaces I, Sequence Spaces,Springer-Verlag, Berlin-Heidelberg, 1977.

[13] Y. Makovoz, On trigonometric n-widths and their generalization, J. Approx.Theory 41 (1984), 361–366.

[14] D. J. Newman, Pseudo-uniform convexity in H1, Proc. Amer. Math. Soc.14 (1963), 676-679.

[15] V. N. Temlyakov, Greedy Algorithm and m-term Trigonometric Approxi-mation, Constr. Approx. 14 (1998), 569–587.

[16] V. N. Temlyakov, Weak greedy algorithms, Adv. Comput. Math. 12 (2000),213–227.

[17] V. N. Temlyakov, Greedy algorithms in Banach spaces, Adv. Comput. Math,14 (2001), 277–292.

[18] A. Zygmund, Trigonometric Series, Cambridge University Press, Cam-bridge, 1959.

Department of Mathematics, University of South Carolina, Columbia,SC 29208, U.S.A.

E-mail address : [email protected]

Institute of Mathematics, Bulgarian Academy of Sciences, Bul-garia

Current address : Department of Mathematics, University of Illinois at Urbana-Champaign, Urbana, IL 61801, U.S.A.


Department of Mathematics, University of South Carolina, Columbia,SC 29208, U.S.A.


Documents

CONVERGENCE OF SOME GREEDY ALGORITHMS IN BANACH SPACES