Robust Compressed Sensing

  • View
    24

  • Download
    2

Embed Size (px)

DESCRIPTION

CS optimization techniques

Text of Robust Compressed Sensing

Robust Signal Recovery: Designing Stable MeasurementMatrices and Random SensingRagib MorshedAmi RadunskayaApril 3, 2009Pomona CollegeDepartment of MathematicsSubmitted as part of the senior exercise for the degree ofBachelor of Arts in MathematicsAcknowledgementsI would like to thank my family for pushing me so far in life, my advisors, Ami Radunskayaand Tzu-Yi Chen for all their help and motivation, my friends and everyone else who hasmade my college life really amazing. This thesis is not just a product of my own eort, butmore.AbstractIn recent years a series of papers have developed a collection of results and theories showingthat it is possible to reconstruct a signal f Rnfrom a limited number of linear mea-surements of f. This broad collection of results form the basis of the intriguing theory ofcompressive sensing, and has far reaching implications in areas of medical imaging, compres-sion, coding theory, and wireless sensor network technology. Suppose f is well approximatedby a linear combination of m vectors taken from a known basis . Given we know nothingin advance about the signal, f can be reconstructed with high probability from some limitednonadaptive measurements. The reconstruction technique is concrete and involves solving asimple convex optimization problem.In this paper, we look at the problem of designing sensing or measurements matrices.We explore a number of properties that such matrices need to hold. We prove that oneof these properties in an NP-Complete problem, and give an approximation algorithm forestimating that property. We also discuss the relation of randomness and random matrixtheory to measurement matrices, develop a template based on eigenvalue distribution thatcan help determine random matrix ensembles that are suitable as measurement matrices,and look at deterministic techniques for designing measurement matrices. We prove thesuitability of a new random matrix ensemble using the template. We develop approaches toquantifying randomness in matrices using entropy, and a computational technique to identifygood measurement matrices. We also briey discuss some of the more recent applications ofcompressive sensing.Contents1 Introduction 41.1 Nyquist-Shannon Theorem and Signal Processing . . . . . . . . . . . . . . . 41.2 Compressive Sensing: A Novel sampling/sensing Mechanism . . . . . . . . . 61.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 Background 72.1 The Sensing Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.2 Sparsity and Compressible Signal . . . . . . . . . . . . . . . . . . . . . . . . 82.2.1 Transform Coding and its ineciencies . . . . . . . . . . . . . . . . . 82.3 Incoherence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.4 What Compressive Sensing is trying to solve? . . . . . . . . . . . . . . . . . 102.5 Relation to Classical Linear Algebra . . . . . . . . . . . . . . . . . . . . . . . 102.6 l2 norm minimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.7 l0 norm minimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.8 Basis Pursuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 Signal recovery from incomplete measurements 144 Robust Compressive Sensing 155 Designing Measurement Matrices 165.1 Randomness in Sensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165.2 Restricted Isometry Property (RIP) . . . . . . . . . . . . . . . . . . . . . . . 175.2.1 Statistical Determination of Suitability of Matrices . . . . . . . . . . 205.3 Uniform Uncertainty Principle (UUP) and Exact Reconstruction Principle(ERP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215.4 Random Matrix Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235.4.1 Gaussian ensemble . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235.4.2 Laguerre ensemble . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285.4.3 Jacobi ensemble . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295.5 Deterministic Measurement Matrices . . . . . . . . . . . . . . . . . . . . . . 295.6 Measurement of Randomness . . . . . . . . . . . . . . . . . . . . . . . . . . 3115.6.1 Entropy Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315.6.2 Computational Approach . . . . . . . . . . . . . . . . . . . . . . . . . 335.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346 Applications of Compressive Sensing 357 Conclusion 362List of Figures1.1 How Shannons theorem is used? . . . . . . . . . . . . . . . . . . . . . . . . 52.1 Example of sparsity in digital images. The expansion of the image on a waveletbasis is shown to the right of the image. . . . . . . . . . . . . . . . . . . . . 92.2 l2 norm minimization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.3 l0 norm minimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.4 l1 norm minimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135.1 Typical measurement matrix dimensions . . . . . . . . . . . . . . . . . . . . 185.2 Reduction using Independent Set. Vertex a and c form an independent set ofthis graph. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195.3 Statistical determination of Restricted Isometry Property [23] . . . . . . . . 215.4 Eigenvalue distribution of a Wishart matrix. Parameters: n=50, m=100 . . . 265.5 Eigenvalue distribution of a Manova matrix. Parameters: n=50, m1=100,m2=100 (nm1 and nm2 are the dimensions of the matrix G forming thetwo Wishart matrices respectively) . . . . . . . . . . . . . . . . . . . . . . . 273Chapter 1IntroductionSignals are mathematical functions of independent variables that carry information. One canalso view signals as an electrical representation of time-varying or spatial-varying physicalquantities, the so called digital signal. Signal processing is the eld that deals with theanalysis, interpretation, and manipulation of such signals. The signals of interest can beof the category of sound, video, images, biological signals such as in MRIs, wireless sensornetworks and others. Due to the myriad applications of signal processing systems in our dayto day life, it turns out to be an important eld of research.The concept of compression has also enabled us to store and transmit such signals inmany modern-day applications. For example, image compression algorithms help reducedata sets by orders of magnitude, enabling the advent of systems that can acquire extremelyhigh-resolution images [3]. Signals and compression are apparently interlinked, and thathas made it feasible to develop all sorts of modern-day innovations. In order to acquire theinformation in the signal, the signal needs to be sampled. Conventionally, sampling signalsis determined by Shannons celebrated theorem.1.1 Nyquist-Shannon Theorem and Signal ProcessingTheorem 1. Suppose f is a continuous-time signal whose highest frequency is less thanW/2, thenf(t) =

nZ f( nW)sinc(Wt n).where sinc(x) = sin(x)x .f has a continuous Fourier transform and f( nW) are the samples of f(t) taken at intervalsof nW. We exactly reconstruct the signal f from its sam