Compressive Samplingand Frontiers in Signal Processing

Emmanuel Candes

New directions short course, IMA, University of Minnesota, June 2007

Lecture 4: The uniform uncertainty principle and its implications

The uniform uncertainty principle (UUP)The UUP and general signal recovery from undersampled dataExamples of measurements obeying the UUP

Gaussian measurementsBinary measurementsRandom orthonormal projectionsBounded orthogonal systems

So far...

Last time: exact recovery of sparse signals

1 We need to deal with compressible signals (not exactly sparse)2 We need to deal with noise

We deal with the first issue todayWe will deal with the second issue next time

So far...

Last time: exact recovery of sparse signals

1 We need to deal with compressible signals (not exactly sparse)2 We need to deal with noise

We deal with the first issue todayWe will deal with the second issue next time

So far...

Last time: exact recovery of sparse signals

1 We need to deal with compressible signals (not exactly sparse)2 We need to deal with noise

We deal with the first issue todayWe will deal with the second issue next time

Last time: uncertainty relation

Key by-product of signal recovery result: Rmn (sensing matrix)T arbitrary set of size SNear isometry: for all x supported on T

12

mnx2`2 x

2`2

32

mnx2`2

We interpreted this as an uncertainty relation; e.g.If x is supported on TThen the energy of x on a set of size m is just about proportional to m

Last time: uncertainty relation

Key by-product of signal recovery result: Rmn (sensing matrix)T arbitrary set of size SNear isometry: for all x supported on T

12

mnx2`2 x

2`2

32

mnx2`2

We interpreted this as an uncertainty relation; e.g.If x is supported on TThen the energy of x on a set of size m is just about proportional to m

The uniform uncertainty principle

Definition (Restricted isometry constant S)

For each S = 1, 2, . . . , S is the smallest quantity such that

(1 S)x2`2 x2`2 (1 + S) x

2`2 , S-sparse x

Or equivalently for all T with |T| S

1 S min(TT) max(TT) 1 + S

T Rm|T| columns with indices in T, |T| S

T

Sparse subsets of column vectors are approximately orthonormal

The uniform uncertainty principle

Definition (Restricted isometry constant S)

For each S = 1, 2, . . . , S is the smallest quantity such that

(1 S)x2`2 x2`2 (1 + S) x

2`2 , S-sparse x

Or equivalently for all T with |T| S

1 S min(TT) max(TT) 1 + S

T Rm|T| columns with indices in T, |T| S

T

Sparse subsets of column vectors are approximately orthonormal

The uniform uncertainty principle

Definition (Restricted isometry constant S)

For each S = 1, 2, . . . , S is the smallest quantity such that

(1 S)x2`2 x2`2 (1 + S) x

2`2 , S-sparse x

Or equivalently for all T with |T| S

1 S min(TT) max(TT) 1 + S

T Rm|T| columns with indices in T, |T| S

T

Sparse subsets of column vectors are approximately orthonormal

Why is this is an uncertainty principle?

Suppose = n

m RF

F is the n by n Fourier isometry is a set of frequencies

Suppose S = 1/2Arbitrary support T with |T| SArbitrary signal supported on T

12

mnx2`2 1 x

2`2

32

mnx2`2

Uniform because holds for all T

Why is this is an uncertainty principle?

Suppose = n

m RF

F is the n by n Fourier isometry is a set of frequencies

Suppose S = 1/2Arbitrary support T with |T| SArbitrary signal supported on T

12

mnx2`2 1 x

2`2

32

mnx2`2

Uniform because holds for all T

Why is this is an uncertainty principle?

Suppose = n

m RF

F is the n by n Fourier isometry is a set of frequencies

Suppose S = 1/2Arbitrary support T with |T| SArbitrary signal supported on T

12

mnx2`2 1 x

2`2

32

mnx2`2

Uniform because holds for all T

Foundational result of CS?

min s`1 s = y = x

xS : best S-term approximation of x (S largest entries)

Theorem (C., Tao (2004)a)aalthough the statement below is due to C. (2007)

Assume 2S