Applied and Numerical Harmonic Analysis
Series Editor John J. Benedetto University of Maryland
Editorial Advisory Board
Akram Aldroubi Vanderbilt University
Ingrid Daubechies Princeton University
Christopher Heil Georgia Institute of Technology
James McClellan Georgia Institute of Technology
Michael Unser Ecole Polytechnique Federale de Lausanne
M. Victor Wickerhauser Washington University, St. Louis
Douglas Cochran Arizona State University
Hans G. Feichtinger University of Vienna
Murat Kunt Ecole Polytechnique Federale de Lausanne
Wim Sweldens Lucent Technologies Bell Laboratories
Martin Vetterli Ecole Polytechnique Federale de Lausanne
Modern Sampling Theory Mathematics and Applications
John 1 Benedetto Paulo lS.G. Ferreira
Editors
With 37 Figures
Springer Science+Business Media, LLC
John 1. Benedetto Department of Mathematics University of Maryland CoIIege Park, MD 20742 USA
Paulo 1.S.G. Ferreira Departamento de Electronica
e Telecommunicacoes Universidade de Aveiro Aveiro 3810-193 Portugal
Library of Congress Cataloging-in-Publication Data Benedetto, John.
Modern sampling theory : mathematics and applications I John J. Benedetto, Paulo J.S.O. Ferreira.
p. cm.-(Applied and numerica! harmonic ana!ysis) Includes bibliographica! references and index. ISBN 978-1-4612-6632-7 ISBN 978-1-4612-0143-4 (eBook) DOI 10.1007/978-1-4612-0143-4 1. Sampling (Statistics) I. Ferreira, Pauio 1.S.0. II. Title. III. Series.
QA276.6 .B46 2000 5 1 9.5'2--dc2 1 00-060867
Printed on acid-free paper.
© 2001 Springer Science+Business Media New York Originally published by Birkhăuser Boston in 2001
Softcover reprint of the hardcover lst edition 2001
CIP
All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher Springer Science+Business Media, LLC,
except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieva!, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former are not especiall y identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.
ISBN 978-1-4612-6632-7 SPIN 10645585
Produc ti an managed by Louise Farkas; manufacturing supervised by Erica Bresler. Typeset by the authars in TeX.
9 8 7 6 5 432 1
Contents
Preface xi
Contributors xiii
1 Introduction 1 JOHN J. BENEDETTO AND PAULO J. S. G. FERREIRA 1.1 The Classical Sampling Theorem . . 1 1.2 Non-Uniform Sampling and Frames. . . . . . . . 10 1.3 Outline of the Book ................ 21
1.3.1 Sampling, Wavelets, and the Uncertainty Principle .................. 22
1.3.2 Sampling Topics from Mathematical Analysis 23 1.3.3 Sampling Tools and Applications . . . . . . . 24
2 On the Transmission Capacity of the "Ether" and Wire in Electrocommunications 27 V. A. KOTEL'NIKOV (TRANSLATED BY V. E. KATSNELSON)
I Sampling, Wavelets, and the Uncertainty Principle 47
3 Wavelets and Sampling 49 GILBERT G. WALTER 3.1 Introduction ...... 49
3.1.1 Background .. 49 3.1.2 The RKHS Setting 50 3.1.3 The Wavelet Setting 51
3.2 Sampling in Other Spaces 52 3.2.1 Impulse Train Convergence 53 3.2.2 RKHS and Sampling . . . . 57
3.3 Sampling in Wavelet Subspaces .. 58 3.3.1 Elements of Wavelet Theory. 58 3.3.2 Sampling Functions ..... 60 3.3.3 Sampled Values as Coefficients 63
3.4 Interpolating Multiwavelets ...... 66 3.4.1 Properties of Hermite Sampling Functions 67
Vl Contents
4 Embeddings and Uncertainty Principles for Generalized Modulation Spaces 73 J. A. HOGAN AND J. D. LAKEY
4.1 Introduction......................... 73 4.1.1 Weighted Fourier Inequalities Imply Uncertainty
Principle Inequalities. . . . . . . . . . . . 75 4.1.2 Sharp Constants and Endpoint Estimates . . . . 76 4.1.3 Embeddings for Modulation Spaces. . . . . . . . 77
4.2 The Short-Time Fourier Transform and Modulation Spaces 77 4.2.1 Weighted Fourier Inequalities Imply Modulation
Embeddings. . . . . . . . . . . . . . . . . . . . . .. 80 4.2.2 Representation Theory and the Link with Littlewood-
Paley Theory . . . . . . . . . . . . . . . . . . 81 4.2.3 More Remarks on the Uncertainty Principle. 82
4.3 Modulation Embeddings and Uncertainty Principles 82 4.3.1 Grochenig's Two-Sided Embedding Theorem 82 4.3.2 Lieb's Sharp Single-Sided Embedding Theorem 88
4.4 Symmetric Localization . . . . . . . . . . . . . . . . . 89 4.5 Generalized Modulation Spaces . . . . . . . . . . . . . 93
4.5.1 Generalized Square-Integrability, Frames, and the Metaplectic Group . . . . . . . . 94
4.5.2 The Metaplectic Group .... . . . . . . . . . 95 4.5.3 Generalized Square Integrability . . . . . . . . 96 4.5.4 Connection with Time-Frequency Localization 97 4.5.5 Metaplectic Frames. . . . . . . . . . . . . 97
4.6 Generalized Modulation Spaces and Embedding. 100
5 Sampling Theory for Certain Hilbert Spaces of Bandlimited Functions 107 JEAN-PIERRE GABARDO
5.1 Introduction................. 107 5.2 Notation................... 108 5.3 Positive-Definite Distributions on (-R, R) 109 5.4 The Case Where MR(/1) Has Finite Codimension in L~ 115 5.5 The General Case. . . . 129 5.6 Riesz Bases and Frames . . . . . . . . . . . . . . . . . . 134
6 Shannon-Type Wavelets and the Convergence of Their Associated Wavelet Series 135 AHMED I. ZAYED
6.1 Introduction.... 135 6.2 Preliminaries . . . 136 6.3 Shannon's Wavelet 138 6.4 Generalized Shannon Wavelets 140 6.5 Pointwise Convergence of Shannon-type Wavelet Series. 147
Contents Vll
II Sampling Topics from Mathematical Analysis 153
7 Non-Uniform Sampling in Higher Dimensions: From Trigonometric Polynomials to Bandlimited Functions 155 KARL HEINZ GROCHENIG
7.1 Introduction......................... 155 7.2 Non-Uniform Sampling With Trigonometric Polynomials. 157 7.3 Toward Bandlimited Functions 161 7.4 Proof of Theorem 7.1 . . . . . . . . . . . . . . . . . . . .. 167
8 The Analysis of Oscillatory Behavior in Signals Through Their Samples 173 RODOLFO H. TORRES
8.1 Introduction and Motivation. . . . . . . . . . . 173 8.2 Besov Spaces of Functions . . . . . . . . . . . . 178 8.3 Sampling Theorem for Besov Spaces and Their
Discrete Counterpart . . . . . . . . . . . . . . . 182 8.4 Other Characterizations of the Discrete Besov Spaces 185 8.5 Nonlinear Approximation of Band Limited Signals
Using Samples ... 188 8.6 Concluding Remarks . . . . . . . . . . . . . . . . . 192
9 Residue and Sampling Techniques in Deconvolution 193 STEPHEN CASEY AND DAVID WALNUT
9.1 Introduction........................ 193 9.1.1 Statement of the Problem . . . . . . . . . . . . 193 9.1.2 The Coprime Condition and Local Deconvolution. 194 9.1.3 Other Types of Deconvolvers 195 9.1.4 Example: Cubes in]Rd . 196 9.1.5 Example: Balls in ]Rd. . . . . 197 9.1.6 Practical Deconvolution . . . 198
9.2 Residue Methods for Deconvolution. 199 9.3 Sampling Methods for Deconvolution. 209
9.3.1 Deconvolution and Sets of Uniqueness in cm~2[-R,R] . . . . . . . . . . . . . . . . 210
9.3.2 Nonperiodic Frames and Bases for L2 [-R, R] 215 9.3.3 Deconvolution and the Completeness of:F . . 217
10 Sampling Theorems from the Iteration of Low Order Differential Operators 219 J. R. HIGGINS
10.1 Introduction. . . . . . . . . . . 219 10.2 Preliminary Facts and Methods 220 10.3 Examples of the Method . . . 223 10.4 Remarks and Open Problems . 227
viii Contents
11 Approximation of Continuous Functions by Rogosinski-Type Sampling Series 229 AND! KIVINUKK
11.1 Notation and Introduction to the Sampling Series. 229 11.2 Rogosinski-Type Sampling Series . . . . . . . 232 11.3 Applications to Generalized Sampling Series. 239 11.4 Conclusion ................... 244
III Sampling Tools and Applications
12 Fast Fourier Transforms for Nonequispaced Data: A Tutorial DANIEL POTTS, GABRIELE STEIDL, AND MANFRED TASCHE
12.1 Introduction .................... . 12.2 NDFT for Nonequispaced Data Either in Time or
Frequency Domain . . . . . . . . . . . . . . . 12.3 NDFT for Nonequispaced Data in Time and
Frequency Domain . . 12.4 Roundoff Errors 12.5 Fast Bessel Transform
245
247
247
250
258 261 265
13 Efficient Minimum Rate Sampling of Signals with Frequen-cy Support over Non-Commensurable Sets 271 CORMAC HERLEY AND PING WAH WONG
13.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . 271 13.2 Periodic Non-Uniform Sampling. . . . . . . . . . . . . . . . 273 13.3 Reconstruction of a Discrete-Time Signal from N Samples
Out of M . . . . . . . . . . . . . . . . . . . . . 275 13.4 Filter Design Using POCS ........... . 13.5 Minimum Rate Sampling of Multiband Signals 13.6 Slicing the Spectrum ......... .
13.6.1 Dividing the Bands into Slices 13.6.2 Freedom in Pairing. 13.6.3 Pairing the Edges ...... .
14 Finite- and Infinite-Dimensional Models for Oversampled
283 284 288 290 290 291
Filter Banks 293 THOMAS STROHMER
14.1 Introduction. . . . . . . . . . . . . . . . . . . 293 14.1.1 Oversampled Filter Banks and Frames 295
14.2 Finite-Dimensional Models for Filter Banks . 296 14.2.1 The Finite Section Method for Oversampled
Filter Banks .................. 297
Contents ix
14.2.2 Finite Sections, Laurent Operators, and Polyphase Matrices . . . . . . . . . . . . . . . . . . . . . . . . . 298
14.2.3 Polyphase Matrices and "Perfect Symbol Calculus" in Finite Dimensions . . . . . . . . . . . . . . 300
14.3 Convergence and Rates of Approximation for Finite Dimensional Approximations . . . . . . . . . . . . . 301 14.3.1 Rates of Approximation for Oversampled Filter
Banks. . . . . . . . . . . . . . . . . . . . . . . . 302 14.4 Convergence Using Polyphase Representation . . . . . . 305 14.5 Finite-Dimensional Approximation of Paraunitary Filter
Banks Via S-! . . . . . . . . . . . . . . . . . . . . . . . 309 14.6 Oversampled DFT Filter Banks - Beyond
Polyphase Representation . . . . . . . . . . . . . . . 311 14.6.1 Matrix Factorization of the Frame Operator. 312
15 Statistical Aspects of Sampling for Noisy and Grouped Da-ta 317 M. PAWLAK AND U. STADTMULLER
15.1 Introduction. . . . . . . . . . . . . . . . . . . . . 317 15.2 Sampling from Noisy Data and Signal Recovery. 319 15.3 Signal Recovery from Grouped Data 324 15.4 Statistical Accuracy 327 15.5 Concluding Remarks 334 15.6 Proofs . . . . . . . . 336
16 Reconstruction of MRI Images from Non-Uniform Sampling and Its Application to Intrascan Motion Correction in Functional MRI 343 MARC BOURGEOIS, FRANK T. A. W. WAJER,
DIRK VAN ORMONDT, AND DANIELLE GRAVERON-DEMILLY
16.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . 343 16.2 MRI Sampling Trajectories . . . . . . . . . . . . . . . 344 16.3 Motion Influence and Correction of Intrascan Motion
Artifacts . . . . . . . . . . . . . . . . . . . . . . . 346 16.3.1 Physiological and Subject Motions . . . . 346 16.3.2 Reduction of Motion Artifacts in Images. 347 16.3.3 Correction of Intrascan Motions in fMRI . 347
16.4 Image Reconstruction from Non-Uniform Sampling 350 16.4.1 Gridding from a Non-Uniform Grid to a Cartesian
Grid and Vice Versa . . . . . . . . . . . . . 350 16.4.2 Density Correction and Voronoi Algorithm 16.4.3 Limits of Gridding .
16.5 Bayesian Image Estimation ... . 16.5.1 Preamble ......... . 16.5.2 The Most Probable Image .
351 352 353 353 354
x Contents
16.5.3 The Likelihood . 355 16.5.4 The Prior . . . . 355 16.5.5 Calculation of Ti 358
16.6 Applications of Intrascan Motion Correction to fMRI . 358 16.6.1 Shepp-Logan Simulation. . . 358 16.6.2 Simulated fMRI Experiment 359
16.7 Conclusions. . . . . . . . . . . . . . 362
17 Efficient Sampling of the Rotation Invariant Radon Transform 365 LAURENT DESBAT AND CATHERINE MENNESSIER
17.1 Introduction. . . . . . . . . . . . . 365 17.1.1 Principle ............ 365 17.1.2 Inverse Problem Formalism . . 366
17.2 Efficient Sampling in 2D Tomography 369 17.2.1 Results in Standard Tomography 369 17.2.2 Generalization to the Rotation Invariant
Radon Transform. . . . . . . 371 17.3 Application to Doppler Imaging. . . . . . . . . . 374
17.3.1 Null Inclination Case: 0: = O. . . . . . . . 374 17.3.2 Perpendicular Inclination Case: 0: = 7r /2 . 374
17.4 Discussion. . . . . . . . . . . . . . . . . . . . . . 376
References 379
Index 414
Preface
This book was planned at the close of SampTA'97, an international workshop on sampling theory and applications held in 1997 in Aveiro, Portugal. These workshops are held biennially. The first-named co-editor attributes the remarkable success of the conference in A veiro exclusively to the extraordinary effort and ability of his fellow co-editor.
A major feature of this book is Victor Katsnelson's English translation from the Russian of Kotel'nikov's classical sampling paper, On the Transmission Capacity of the "Ether" and Wire in Electrocommunications (Chapter 2). It is probably safe to say that, until now, Kotel'nikov's paper has been referenced more than it has been read by the Western scientific community. Our Introduction is Chapter 1, and it contains a mathematical history and perspective on sampling theory, as well as an outline of the volume and an overview of each chapter.
The remainder of the book is divided into three parts reflecting, respectively, the role of wavelet theory in sampling, a broad range of other modern mathematical methods used in sampling theory, and some current and important tools and applications. We deeply appreciate the excellent contributions by the authors, and it has been a pleasure working with each of them.
It is natural to envisage a sequel to this volume devoted more extensively to tools and applications. In fact, sampling theory is a natural component in the broad emerging area of mathematical engineering. For example, it is realistic to assert that mathematical engineering will be to today's mathematics departments what mathematical physics was to those a century ago; and it is no extravagance to note the fundamental impact of mathematics in engineering subjects such as speech and image processing, information theory, and biomedical engineering, to name but a few. Of course, Birkhiiuser's book series, Applied and Numerical Harmonic Analysis, in which this volume appears, is devoted to such an interleaving of ideas and disciplines.
Xll Preface
We have been fortunate beneficiaries of patience and professionalism, par excellence, from Birkhiiuser's editorial offices in Boston. In particular, it is a great pleasure to acknowledge the help and guidance of Wayne Wheeler, Wayne Yuhasz, and, most extensively, Lauren Lavery. She tolerated our idiosyncracies, corrected our errant tendencies, and delivered the book. We hope you enjoy it.
John J. Benedetto College Park, Maryland
Paulo J. S. G. Ferreira A veiro, Portugal
Contributors John J. Benedetto
Marc Bourgeois
Stephen Casey
Laurent Desbat
Paulo J. S. G. Ferreira
Jean-Pierre Gabardo
Department of Mathematics, University of Maryland, College Park, MD 20742, USA.
E-mail: j jb@math. umd. edu
Laboratoire de RMN, CNRS UPRESA 5012, Universite Claude Bernard Lyon I - CPE, 43 Boulevard du 11 Novembre 1918, 69622 Villeurbanne Cedex, France.
E-mail: [email protected]
Department of Mathematics and Statistics, American University, 4400 Massachusetts Ave., N.W., Washington, DC 20016-8050, USA. E-mail: [email protected]
TIMC-lMAG, UMR CNRS 5525, lAB, Faculte de Medecine, UJF, 38706 La Tronche, France.
E-mail: [email protected]
Departamento de Electronica e Telecomunica<;oes / IEETA, Universidade de Aveiro, 3810-193 Aveiro, Portugal.
E-mail: [email protected]
Department of Mathematics and Statistics, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S 4K1, Canada.
E-mail: [email protected]
Danielle Graveron-Demilly Laboratoire de RMN, CNRS UPRESA 5012, Universite Claude Bernard Lyon I - CPE, 43 Boulevard du 11 Novembre 1918, 69622 Villeurbanne Cedex, France. E-mail: [email protected]
X1V Contributors
Karlheinz Grochenig
Cormac Herley
J. R. Higgins
J. A. Hogan
V. E. Katsnelson
Andi Kivinukk
J. D. Lakey
Catherine Mennessier
Dirk van Ormondt
Department of Mathematics, U-9, The University of Connecticut, Storrs, CT 06269-3009, USA.
E-mail: groch@math. uconn. edu
Microsoft Research, One Microsoft Way, Redmond, WA 98052, USA.
E-mail: [email protected]
Department of Sciences, Anglia Polytechnic University, East Road, Cambridge CBl IPT, UK.
E-mail: [email protected] . uk
School of MPCE, Macquarie University, NSW 2109, Australia.
E-mail: [email protected]
Department of Mathematics, The Weizmann Institute, Israel.
E-mail: [email protected]
Department of Mathematics and Computer Science, Tallinn Pedagogical University, Narva Str. 25, 10 120 Tallinn, Estonia.
E-mail: [email protected]
College of Arts and Sciences, Department of Mathematical Sciences, Box 30001, Dept. 3MB, Las Cruces, New Mexico 88003-8001, USA.
E-mail: j lakey@nmsu. edu
TIMC-IMAG, UMR CNRS 5525, lAB, FacuM de Medecine, UJF, 38706 La Tronche, France.
E-mail: [email protected]
Spin Imaging Group, Department of Applied Physics, Delft University of Technology, PO Box 5046, 2600 GA Delft, The Netherlands.
E-mail: [email protected]
M. Pawlak
Daniel Potts
U. Stadt muller
Gabriele Steidl
Thomas Strohmer
Manfred Tasche
Rodolfo H. Torres
Frank T. A. W. Wajer
David Walnut
Gilbert G. Walter
Contributors xv
Department of Electrical and Computer Engineering, University of Manitoba, Winnipeg, Man. R3T 5V6, Canada.
E-mail: pawlak©ee.umanitoba.ca
Medical University Lubeck, WallstraJ3e 40, 23560 Lubeck, Germany.
E-mail: potts©math.mu-luebeck.de
Abteilung fUr Mathematik III, University of Ulm, D 89069 Ulm, Germany.
University of Mannheim, D 7, 29, 68131 Mannheim, Germany.
E-mail: steidl©math.uni-mannheim.de
Department of Mathematics, 1 Shields Ave., University of California, Davis, CA 95616-8633, USA.
E-mail: strohmer©math. ucdavis. edu
Medical University Lubeck, WallstraJ3e 40, 23560 Lubeck, Germany.
E-mail: [email protected]
Department of Mathematics, University of Kansas, Lawrence, KS 66045, USA.
E-mail: torres©math. ukans. edu
Spin Imaging Group, Department of Applied Physics, Delft University of Technology, PO Box 5046, 2600 GA Delft, The Netherlands.
E-mail: waje©si.tn.tudelft.nl
Department of Mathematical Sciences, George Mason University, Fairfax, VA, 22030, USA.
E-mail: dwalnut©gmu. edu
Department of Mathematical Sciences, University of Wiscousin, PO Box 413 Milwaukee WI 53201, USA.
E-mail: ggw©csd. uwm. edu
XVi Contributors
Ping Wah Wong
Ahmed 1. Zayed
Hewlett Packard, 11000 Wolfe Road, Cupertino, CA 95014, USA.
Department of Mathematics, University of Central Florida, Orlando, FL 32816, USA.
E-mail: zayed@pegasus. ee. uef . edu