Direct Exact Inverse Pseudo-Polar FFT and Radon Transform Using Orthogonalizing Weights

Direct Exact Inverse Pseudo-Polar FFT and Radon Transform Using Orthogonalizing Weights

Ofer LeviOfer Levi

Department of Industrial Engineering and ManagementBen-Gurion University of the NegevBeer Sheva, Israel

On sabbatical leave at the Institute for Computational and Mathematical Engineering, Stanford University

Department of Industrial Engineering and ManagementBen-Gurion University of the NegevBeer Sheva, Israel

On sabbatical leave at the Institute for Computational and Mathematical Engineering, Stanford University

April 21, 2023

Summer School at Inzell

Mathematical and statistical modeling of physical and biological processes and systems

Inverse problems solving using Sparse Representation and Compressed Sensing

High dimensional image analysis and processing Large scale Optimization, Parallel and Distributed Computing Numerical Analysis and Matrix Computation

My research interests

3

Lecture StructureLecture Structure

Background• Rectilinear DFT: General Definition and Properties

• Polar DFT: Importance and difficulties

• The Pseudo-Polar FFT (PPFFT): Definition and Properties

• The Fast Slant-Stack Algorithm

Direct and Exact Inverse PPFFT (IPPFFT) Conclusions

4

1D DFT: General Definition and Properties1D DFT: General Definition and Properties

]1,0[,1

1

0

/2ˆ

ˆ)(,:

nkn

n

j

nkjijk

nn

exx

xxFCCF

nkjijk eAAxx

n

/21,ˆ

kjjk AAIAA ,*

Background – 1D DFT

Matrix-vector notation

]1,0[,1 1

0

/2ˆ

njn

n

k

nkjijk exx

Reconstruction (IDFT)

)sin()cos( xixeix

5

1D DFT: Computability

Direct evaluation of the 1D DFT costs o(n2)

FFT – an n·log(n) DFT Algorithm

12

,0ˆˆˆ

ˆˆˆ/2

2

/2

nk

k

nki

knk

k

nki

kk

zwx

zwx

ee

]1,0[,1

1

0

/2ˆ

nkn

n

j

nkjijk exx

12

,012

2 nj

jj

jj

xz

xw

Background – 1D DFT

6

Example – Spectral Decomposition

+ +

=

7

Example – Spectral Decomposition

Time Domain Frequency Domain

8

Example - Denoising

Time Domain Frequency Domain

9

2D DFT – Cartesian Grid2D DFT – Cartesian Grid

]1,0[2,]1,0[1,

1

0

1

0

)//(2

2 1

2211

2121

1ˆ̂

ˆ̂)(,:

nkmk

m

j

n

j

nkjmkjijjkk

mnmn

exmn

x

xxFCCF

)/(21

0

1

0

)/(2 22

2 1

11

2121

11ˆ̂ mkjim

j

n

j

mkjijjkk eex

nmx

Background – 2D DFT

Direct evaluation → o(m2n2)

10

2D FFT

A total of o(Nlog(N)), N=mn

1. Apply 1D FFT for each column

n times mlog(m)

2. Apply 1D FFT for each row

m times nlog(n)

Same for 2D IFFT and for higher dimensions

Background – 2D DFT

Compression, Denoising

Spectral Analysis

11

Fast Convolution/Correlation - n·log(n) instead of n2

Trigonometric Interpolation – Shift Property

Applications of rectilinear 2D FFT

YXZYXZ ˆˆˆ* Convolution Theorem -

Background – 2D DFT

12

Polar DFT

],0[],12/,2/[,)sin(,)cos(

,

1

0

1

0

/)(2

21

2 1

2211

2121

1ˆ̂

nnrrkrk

n

j

n

j

nkjkjijjkk ex

nx

1

0

1

0

/))sin()cos((2

2 1

21

21

1ˆ̂n

j

n

j

njjrijjr ex

nx

Difficulties:

1 – Impossible to separate to series of 1D FFTs

2 – Non Orthogonal (Ill Conditioned)

Background – Polar DFT

13

Polar DFT

Direct Polar DFT is impractical – o(N2) and no direct inverse

Trade off between time and accuracy

Common Solution – Interpolation to and from Cartesian Grid with Oversampling

Background – Polar DFT

14

Importance of Polar DFT

MRI – Reconstruction from Polar Grid CT – Reconstruction from Projections

)(ˆ̂

)(

),(),(

)sin(),cos(

)sin()cos(

11

rr

yxr

fFrR

dxdyyxfrRy x

Projection-Slice Theorem

Background – Polar DFT

Accurate Rotation (Shift in Polar Coordinates) Rigid body Registration

15

1

2,

2;,]1,[;

2 12

nnm

n

mlknnl

lkBV

The Pseudo-Polar FFT (PPFFT)(Averbuch et. al.)The Pseudo-Polar FFT (PPFFT)(Averbuch et. al.)

1

2,

2;,]1,[;

2 21

nnm

n

mlknnl

lkBH

Pseudo-Polar FFT

Basically

Horizon

tal

Basically Vertical

- Concentric squares

- Equally sloped lines

A total of 4n2 grid points

16

The Pseudo Polar FFT

1

0

1

0

/)2(BH

2 1

221

21

1ˆ̂n

j

n

j

nmjnjlijjlm ex

nx

n

leex

nnx nmji

n

j

n

j

nljijjlm

;11ˆ̂ // 2

2 1

1

21

21

0

1

0

)2(2BH

12

,2

,1

1

0

/2ˆ nnkn

n

j

nkjijk exx

Fractional FFT – o(nlog(n))

Pseudo-Polar FFT

17

Fractional FFT Algorithm (D. Bailey and P. Swarztrauber 1990)

]1,0[

]1,0[

]12,[

]1,0[

]12,[

]1,0[

;ˆ.4

;.3

;

;.2

;0

;.1

/

12

0

/)2(

/

/

2

2

2

2

nk

nk

nnj

nj

nnj

nj

knki

k

jk

n

jjk

nnji

nji

j

njij

j

wex

zyw

e

ez

exy

]1,0[;ˆ1

0

/2

nkn

j

njkijk exx

18

Some basic facts about Toeplitz Matrices

T has a Toeplitz structure if Tjk=f(j-k), i.e. T has constant diag’s

Any n by n Toeplitz matrix T can be expanded into a 2n by 2n circulant Matrix C as follows:

TS

STC Where S is also Toeplitz

A circulant Matrix C can be diagonalyzed using the DFT Matrix F as follows: C=FDF-1, D=Diag(v) where v is the Fourier transform of the first column of C

Sx

Txx

TS

ST

0

Tx can be computed in nlog(n) flops by doing the following

This procedure is very similar to the FFFT Algorithm!

19

FFFT and Structured Matrices

nkjijk eVVxx

n

/21,ˆ FFFT in Matrix-vector notation

What is the structure of V ? V is symmetric Vandermonde

Theorem : A symmetric Vandermonde Matrix V can be decomposed as V=DTD where T is Toeplitz

Proof: If V is symmetric Vandermonde then there exist a unique scalar β such that Vjk= β-2jk

Define Dj= βjk and T=DVD 2)( kj

jkT

Reminder: if V=V(a) then Vjk=ajk

V=Vt => Vjk=ajk

20

The PPFFT Algorithm

A total of o(Nlog(N)), N=n2

1. 1D FFT for each 0-padded column

n times 2nlog(2n)

2. Apply Fractional FFT for each row

With α=l/n 2n times nlog(n)

Pseudo-Polar FFT

Repeat the same procedure for the transposed image matrix to compute the BH coefficients

21

The PPFFT – Matrix notation

A can be implicitly applied in O(Nlog(N)) operations

221BV

21

221BH

21

22

/)2(,

/)2(,

4

11,

,

nnjmjlijjlm

nmjnjlijjlm

BV

BHnn

en

Aen

A

A

AACA

Denote the Adjoint PPFFT by A*A* can be also implicitly applied in o(Nlog(N))

Pseudo-Polar FFT

22

Use CGLS or LSQR for the Normal Equations

Inverse PPFFTInverse PPFFT

yAAxASolve **

)(minarg2

yAuxu

yWAxAWA 22* *~

Pseudo-Polar FFT

A problem: A is ill conditioned, k(A) is proportional to n

Solution: Solve

W is diagonal when each diagonal element is the grid point radius of the corresponding PPFFT coefficient

If a zero residual solution exists then xx ~

23

Weighted PPFFTWeighted PPFFT

Pseudo-Polar FFT

- Each coefficient is multiplied by its grid point radius

- The weights compensates for the non-uniform grid sampling

- Experimental result:

2.1)( 2* AWAK

The weighted IPPFFT converges within 4-5 iterations

24

Applications of the PPFFT

Fast and Accurate interpolation to PFT Fast and Accurate interpolation to Spiral FT Fast Slant-stack – An nlog(n) Radon Transform Fast and Accurate Rotations 3D Pseudo Spherical FFT and Radon

Pseudo-Polar FFT

25

Theorem:

There exists W, a Real Positive Diagonal Matrix

such that:

Direct Inverse PPFFTDirect Inverse PPFFT

IAWA 2*

Direct IPPFFT

yWAx 2*If y belongs to the image of A then:

The elements of W can be rapidly pre-calculated for any given n

26

Problem formulationProblem formulation

2222 4442* ,, nnnn RARWIAWA

- 4n4 constraints

- 4n2 variables (W is diagonal)

The problem is over-determined

There is no solution for an arbitrary A

and a diagonal W

Direct IPPFFT

27

Finding the Ideal weights

1. Experiments showed that the over-determined system is solvable for n≤8

)(),(, 244

2*

24

IvecbWdiagxCM

bMxIAWAnn

222 442 ~,

~

~~

nnn RbCM

bxMbMx

System Reduction

2. The under-determined reduced system is solvable. The weights could be computed numerically for n≤323. The reduced system could be solved by a fast iterative FFT based solver within o(n2log(n)) operations

Direct IPPFFT

28

The ideal WeightsThe ideal Weights

Direct IPPFFT

29

Fast solver for the ideal weights

nosnosnnosos CCA 21

21:,

os1n slopes and os2n radiuses

Define: PPFFTos1,os2 = PPFFT with over-sampling ration = os1·os2

The Conventional PPFFT can be denoted as PPFFT2,2

otherwise

nnj

bRb

Wdiagx

jn

osososos

;02

;1~,

~

)(

2

2,,

2

2121

Define:

bxA osososos

~2121 ,

*2/,2/ If x exists it satisfies -

Direct IPPFFT

30

Existence of ideal weights

Direct IPPFFT

bxA osososos

~2121 ,

*2/,2/

*2/,2/ 21 ososA

The system has a solution if

is invertible

There is no solution for PPFFT2,2 in its standard form

since is singular*1,1A

For a slightly modify PP grid solution exists. It can be verified using the Vandermonde structure of A and the fact that the modified grid has distict set of points

31

Modified PP Grid

Direct IPPFFT

32

Error analysis

))((minarg

)(minarg

22

21

yAuWx

yAux

u

u

yAxryy ~,~

rWAxrWAAxWArWAyWAyWAx

rAAAxrAAAAxAAA

rAAAyAAAyAAAx

2*2*2*2*2*2*2

*1**1**1*

*1**1**1*1

~)()()(

)(~)()(

12*1*2* )( xxxxAAAWA

Conclusions

Matrix approach can be valuable for better understanding and analysis of discrete signal and image transformations

33

The Pseudo-Polar Fourier Transform (PPFFT) posses several attractive computational properties

The PPFFT combines the best properties of both the Cartesian FT and the Polar FT

Can be generalized to higher dimensions

Provides fast and accurate discrete Radon Transform

Direct inverse

34

Thank You!

35

The Slow Slant-Stack Transform

1. Zero-pad the image from the left and the right (BV lines)

2. Shear the padded image in a θ angle using trigonometric interpolation via FFT

3. Sum the sheared image array column-wise to get the θ-projection

36

The Fast Slant-Stack Algorithm

Backprojections of SS coefficients

)/sin(

)sin(/

12/

2/2

12/

2/

)(

)(),(),(~

),(~

),(

ltl

tltil

n

nun

n

nu

etD

vyDvuIyuI

tsuuIItsxySS

Dl(t) is an interpolating kernel

1

2,

2,,),(

~),(

nnvuvuIvuI

37

The Fast Slant-Stack transform

Projection-Slice Theorem

For a given n by n discrete image

1. Compute the PPFFT coefficients

2. Apply 1D IFFT to each vector of “same slope” coefficients in the PPFFT coefficients array

1,,1,,2

1,,2

1,1

1,,1,,2

1,,2

1,1

;

;

nnntnn

nnntnn

stsyxBV

stsxyBH

Slant-Stack LinesUniform horizontal /vertical spacing in each projection !

38

ConclusionsConclusions

- A direct solution for an important inverse problem

- A direct result – Direct inverse for the Fast Slant-Stack

- Can be generalized to higher dimensions

- A new methodology for LS problem – pre-computation of weights

- Various Applications