Compressed Sensing - beyond the Shannon Paradigm .Compressed Sensing - beyond the Shannon ... The

  • View
    212

  • Download
    0

Embed Size (px)

Text of Compressed Sensing - beyond the Shannon Paradigm .Compressed Sensing - beyond the Shannon ... The

  • Compressed Sensing - beyond the ShannonParadigm

    Wolfgang Dahmen

    Institut fur Geometrie und Praktische MathematikRWTH Aachen

    joint work with Albert Cohen and Ron DeVore

    ISWCS-11, Aachen, Nov.8, 2011

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 1 / 30

  • Contents

    1 Motivation, Background

    2 A New Paradigm

    3 Back to the 70s - An Information Theoretic Bound

    4 Randomness Helps...

    5 An Optimal Decoder

    6 Electron-Tomography

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 2 / 30

  • Contents

    1 Motivation, Background

    2 A New Paradigm

    3 Back to the 70s - An Information Theoretic Bound

    4 Randomness Helps...

    5 An Optimal Decoder

    6 Electron-Tomography

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 2 / 30

  • Contents

    1 Motivation, Background

    2 A New Paradigm

    3 Back to the 70s - An Information Theoretic Bound

    4 Randomness Helps...

    5 An Optimal Decoder

    6 Electron-Tomography

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 2 / 30

  • Contents

    1 Motivation, Background

    2 A New Paradigm

    3 Back to the 70s - An Information Theoretic Bound

    4 Randomness Helps...

    5 An Optimal Decoder

    6 Electron-Tomography

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 2 / 30

  • Contents

    1 Motivation, Background

    2 A New Paradigm

    3 Back to the 70s - An Information Theoretic Bound

    4 Randomness Helps...

    5 An Optimal Decoder

    6 Electron-Tomography

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 2 / 30

  • Contents

    1 Motivation, Background

    2 A New Paradigm

    3 Back to the 70s - An Information Theoretic Bound

    4 Randomness Helps...

    5 An Optimal Decoder

    6 Electron-Tomography

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 2 / 30

  • The Nyquist-Shannon Theorem

    A bandlimited signal f (t), t R, with bandwidth 2B can bereconstructed exactly from the samples f (k), < 1/2B, ...

    X (f )() :=R

    f (t)ei2tdt = 0, || > B,

    f (t) =

    k=f (k)

    sin(1t k)(1t k)

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 3 / 30

  • The Nyquist-Shannon Theorem

    A bandlimited signal f (t), t R, with bandwidth 2B can bereconstructed exactly from the samples f (k), < 1/2B, ...

    X (f )() :=R

    f (t)ei2tdt = 0, || > B,

    f (t) =

    k=f (k)

    sin(1t k)(1t k)

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 3 / 30

  • The Nyquist-Shannon Theorem

    A bandlimited signal f (t), t R, with bandwidth 2B can bereconstructed exactly from the samples f (k), < 1/2B, ...

    X (f )() :=R

    f (t)ei2tdt = 0, || > B,

    f (t) =

    k=f (k)

    sin(1t k)(1t k)

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 3 / 30

  • A Dilemma...

    Common features of many signals:

    ...they are far from bandlimited

    ...however, they are sparse, i.e. their information content is smallcompared with their size

    ...measurements are difficult/expensive/harmful

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 4 / 30

  • A Dilemma...

    Common features of many signals:

    ...they are far from bandlimited

    ...however, they are sparse, i.e. their information content is smallcompared with their size

    ...measurements are difficult/expensive/harmful

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 4 / 30

  • A Dilemma...

    Common features of many signals:

    ...they are far from bandlimited

    ...however, they are sparse, i.e. their information content is smallcompared with their size

    ...measurements are difficult/expensive/harmful

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 4 / 30

  • A Dilemma...

    Common features of many signals:

    ...they are far from bandlimited

    ...however, they are sparse, i.e. their information content is smallcompared with their size

    ...measurements are difficult/expensive/harmful

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 4 / 30

  • (Information-)Sparse Signals

    ...are those sparse?

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 5 / 30

  • Attempt of a Shortcut

    employ more general measurements than classical sampling

    Question: can one sample at a rate comparable to informationcontent rather than signal size?...what exactly would one like to know?

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 6 / 30

  • Attempt of a Shortcut

    employ more general measurements than classical samplingQuestion: can one sample at a rate comparable to informationcontent rather than signal size?

    ...what exactly would one like to know?

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 6 / 30

  • Attempt of a Shortcut

    employ more general measurements than classical samplingQuestion: can one sample at a rate comparable to informationcontent rather than signal size?...what exactly would one like to know?

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 6 / 30

  • Attempt of a Shortcut

    employ more general measurements than classical samplingQuestion: can one sample at a rate comparable to informationcontent rather than signal size?...what exactly would one like to know?

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 6 / 30

  • An Instructive Example: Polynomials

    t1 t2

    P1

    t1 t2 t3

    P2

    t1 t4t2

    P3

    t3

    PN1(t) = x1t0 + x2t1 + x3t2 + + xN1tN2 + xN tN1

    yi = PN1(ti), i = 1, . . . ,N,

    x0 . . . , xN

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 7 / 30

  • An Instructive Example: Polynomials

    t1 t2

    P1

    t1 t2 t3

    P2

    t1 t4t2

    P3

    t3

    PN1(t) = x1t0 + x2t1 + x3t2 + + xN1tN2 + xN tN1

    yi = PN1(ti), i = 1, . . . ,N,

    x0 . . . , xN

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 7 / 30

  • An Instructive Example: Polynomials

    t1 t2

    P1

    t1 t2 t3

    P2

    t1 t4t2

    P3

    t3

    PN1(t) = x1t0 + x2t1 + x3t2 + + xN1tN2 + xN tN1

    yi = PN1(ti), i = 1, . . . ,N, x0 . . . , xN

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 7 / 30

  • An Instructive Example: Polynomials

    t1 t2

    P1

    t1 t2 t3

    P2

    t1 t4t2

    P3

    t3

    PN1(t) = x1t0 + x2t1 + x3t2 + + xN1tN2 + xN tN1

    yi = PN1(ti), i = 1, . . . ,N, x0 . . . , xN

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 7 / 30

  • An Instructive Example: Polynomials

    t1 t2

    P1

    t1 t2 t3

    P2

    t1 t4t2

    P3

    t3

    PN1(t) = x1t0 + x2t1 + x3t2 + + xN1tN2 + xN tN1

    yi = PN1(ti), i = 1, . . . ,N, x0 . . . , xN

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 7 / 30

  • Sparse Polynomials

    PN1(t) = x1t0 + x2t1 + x3t2 + + xN1tN2 + xN tN1

    yi = PN(ti) =N

    j=1

    (t j1i ) =:i,j

    xj

    =: i x

    , i = 1, . . . ,N,

    x0 . . . , xN

    1

    xi2

    ikik1i3i2i1

    xik

    xik1

    xi3

    xi1

    N + 12 N

    yi = PN(ti), i = 0, . . . ,

    ? =

    n

    < N

    ?= xi1 . . . , xik

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 8 / 30

  • Sparse Polynomials

    PN1(t) = x1t0 + x2t1 + x3t2 + + xN1tN2 + xN tN1

    yi = PN(ti) =N

    j=1

    (t j1i ) =:i,j

    xj

    =: i x

    , i = 1, . . . ,N,

    x0 . . . , xN

    1

    xi2

    ikik1i3i2i1

    xik

    xik1

    xi3

    xi1

    N + 12 N

    yi = PN(ti), i = 0, . . . ,

    ? =

    n

    < N

    ?= xi1 . . . , xik

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 8 / 30

  • Sparse Polynomials

    PN1(t) = x1t0 + x2t1 + x3t2 + + xN1tN2 + xN tN1

    yi = PN(ti) =N

    j=1

    (t j1i ) =:i,j

    xj =: i x , i = 1, . . . ,N,

    x0 . . . , xN

    1

    xi2

    ikik1i3i2i1

    xik

    xik1

    xi3

    xi1

    N + 12 N

    yi = PN(ti), i = 0, . . . ,

    ? =

    n

    < N

    ?= xi1 . . . , xik

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 8 / 30

  • Sparse Polynomials

    PN1(t) = x1t0 + x2t1 + x3t2 + + xN1tN2 + xN tN1

    yi = PN(ti) =N

    j=1

    (t j1i ) =:i,j

    xj =: i x , i = 1, . . . ,N, x0 . . . , xN

    1

    xi2

    ikik1i3i2i1

    xik

    xik1

    xi3

    xi1

    N + 12 N

    yi = PN(ti), i = 0, . . . ,

    ? =

    n

    < N

    ?= xi1 . . . , xik

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 8 / 30

  • Sparse Polynomials

    PN1(t) = x1t0 + x2t1 + x3t2 + + xN1tN2 + xN tN1

    yi = PN(ti) =N

    j=1

    (t j1i ) =:i,j

    xj =: i x , i = 1, . . . ,N, x0 . . . , xN

    1

    xi2

    ikik1i3i2i1

    xik

    xik1

    xi3

    xi1

    N + 12 N

    yi = PN(ti), i = 0, . . . ,

    ? =

    n

    < N

    ?= xi1 . . . , xik

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 8 / 30

  • Sparse Polynomials

    PN1(t) = x1t0 + x2t1 + x3t2 + + xN1tN2 + xN tN1

    yi = PN(ti) =N

    j=1

    (t j1i ) =:i,j

    xj =: i x , i = 1, . . . ,N, x0 . . . , xN

    1

    xi2

    ikik1i3i2i1

    xik

    xik1

    xi3

    xi1

    N + 12 N

    yi = PN(ti), i = 0, . . . ,

    ? =

    n

    < N

    ?= xi1 . . . , xik

    W. Dahmen (RWTH Aachen) Compressed Sensing 28.10.2011 8 / 30

  • Sparse Polynomials

    PN1(t) = x1t0 + x2t1 + x3t2 + + xN1tN2 + xN