Hybrid Enriched Bidiagonalization forDiscrete Ill-Posed Problems

Per Christian HansenTechnical University of Denmark

Joint work with

Kuniyoshi Abe - Gifu Shotoku Gakuen University

Yiqiu Dong - Technical University of Denmark

With thanks toHenrik Garde and Nao Kuroiwa

Japan, Spring 20202/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Setting the Stage – Overview of Talk

Forward problem:b = A x

Inverse problem:solve Ax = b

Japan, Spring 20203/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Inverse Problems Ill-Conditioning

Japan, Spring 20204/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Regularization Algorithms

Japan, Spring 20205/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Krylov Subspaces and Semi-Convergence

Japan, Spring 20206/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Illustration of Semi-Convergence

Japan, Spring 20207/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Hybrid Methods

Projectedproblem

Regularizedprojectedproblem

Japan, Spring 20208/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Augmented Krylov Subspace

Japan, Spring 20209/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Example: Augmented GMRES

GMRES Augmented GMRES

Japan, Spring 202010/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Overview of Methods

Japan, Spring 202011/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Enriched CGNR

Japan, Spring 202012/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Towards our Algorithm

Japan, Spring 202013/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Setting the Stage for Our Algorithm

Japan, Spring 202014/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

More Details

Projectedproblem

Japan, Spring 202015/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Basic Enriched Bidiagonalization

Japan, Spring 202016/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Efficient and Stable Implementation

Algorithm HYBR (Chung, Nagy, O’Leary 2008) also uses full reorthogonalization.

Japan, Spring 202017/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

And Now: a Hybrid Algorithm

Japan, Spring 202018/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

HEB: Hybrid Enriched Bidiagonalization

Regularized projected problem

Japan, Spring 202019/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Terrible Computational Details of Step 6

Japan, Spring 202020/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Numerical Examples

Japan, Spring 202021/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Large Component in Augment. Subspace

Results next page

Japan, Spring 202022/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Large Component in Augment. Subspace

Japan, Spring 202023/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

1D Deconvolution and “Inpainting”

Results next page

Japan, Spring 202024/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

1D Deconvolution and “Inpainting”

Japan, Spring 202025/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

2D Image Deblurring and Inpainting

Results next page

Japan, Spring 202026/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

2D Image Deblurring and Inpainting

LSQR

HEB

Japan, Spring 202027/27 P. C. Hansen – Hybrid Enriched Bidiagonalization for Discrete Ill-Posed Problems

Conclusions We augment the bidiagonalization algorithm underlying LSQR.

Our algorithm uses an enriched subspace:the Krylov subspace plus a low-dimensional linear subspace.

We add standard-form Tikhonov regularization, thus arriving at a hybrid enriched bidiagonalization algorithm.

We choose the regularization parameter adaptively in each iteration, e.g., by means of GCV.

Possible extension: use general-form Tikhonov regularization.