89
×××××××× ×××××××× ××××××× ×××××× ××××× ×××× ××× ×× × On Deflations in Extended QR Algorithms Thomas Mach Raf Vandebril Dept. Computer Science, KU Leuven, Belgium IWASEP 10 Dubrovnik, 3 June 2014 Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 1/20

On Deflations in Extended QR Algorithms

Embed Size (px)

Citation preview

Page 1: On Deflations in Extended QR Algorithms

×××××××××××××××××××××××××××××××××××××××××××

×

On Deflations in Extended QRAlgorithms

Thomas Mach Raf Vandebril

Dept. Computer Science, KU Leuven, Belgium

IWASEP 10Dubrovnik, 3 June 2014

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 1/20

Page 2: On Deflations in Extended QR Algorithms

Rotation

11

1

11

c

c

s

−s= ��

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 2/20

Page 3: On Deflations in Extended QR Algorithms

Hessenberg Matrix

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

0×××××

��

QR factorization of the Hessenberg matrix

Rotation of the form:

[c s−s c

], |c |2 + |s|2 = 1

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 3/20

Page 4: On Deflations in Extended QR Algorithms

Hessenberg Matrix

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

0××××

����

QR factorization of the Hessenberg matrixRotation of the form:

[c s−s c

], |c |2 + |s|2 = 1

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 3/20

Page 5: On Deflations in Extended QR Algorithms

Hessenberg Matrix

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

0×××

����

��

QR factorization of the Hessenberg matrixRotation of the form:

[c s−s c

], |c |2 + |s|2 = 1

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 3/20

Page 6: On Deflations in Extended QR Algorithms

Hessenberg Matrix

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

0××

����

����

QR factorization of the Hessenberg matrixRotation of the form:

[c s−s c

], |c |2 + |s|2 = 1

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 3/20

Page 7: On Deflations in Extended QR Algorithms

Hessenberg Matrix

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

��

QR factorization of the Hessenberg matrixRotation of the form:

[c s−s c

], |c |2 + |s|2 = 1

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 3/20

Page 8: On Deflations in Extended QR Algorithms

Hessenberg Matrix

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××0

����

����

����

QR factorization of the Hessenberg matrixRotation of the form:

[c s−s c

], |c |2 + |s|2 = 1

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 3/20

Page 9: On Deflations in Extended QR Algorithms

Hessenberg Matrix

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

QR factorization of the Hessenberg matrix

Rotation of the form:

[c s−s c

], |c |2 + |s|2 = 1

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 3/20

Page 10: On Deflations in Extended QR Algorithms

Extended Hessenberg Matrix

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××××× ×

××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

There is a relation to extended Krylov subspaces.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 4/20

Page 11: On Deflations in Extended QR Algorithms

Rotators

Fusion:

�� �� = c1−s1

s1c1

c2−s2

s2c2

= c3−s3

s3c3

= ��

Turnover:

����

��=

Ö

Ö

Ö

Ö

Ö

Ö

Ö

Ö

Ö

= ����

��

Transfer Through Upper Triangular:

����

��Ö

Ö Ö Ö Ö

Ö Ö Ö

Ö Ö=

��

��Ö

Ö Ö Ö Ö

Ö Ö Ö

Ö Ö Ö=

��

��

Ö

Ö

Ö

Ö

Ö

Ö

Ö

Ö

Ö

Ö

� �

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 5/20

Page 12: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

1) Compute bulge.

A) Eigenvalues of trailing matrix provide shift.B) x = p(A)e1 = (A− µmI ) · · · (A− µ1I )e1.C) Q0x = γe1.D) Apply Q0.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 13: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

1) Compute bulge.

A) Eigenvalues of trailing matrix provide shift.B) x = p(A)e1 = (A− µ1I )e1.C) Q0x = γe1.D) Apply Q0.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 14: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

��

� �

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

� �

��

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 15: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

×=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

2) Bulge chasing.

A) Compute zeroing similarity transformation.B) Apply transformation.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 16: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

×=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

2) Bulge chasing.

A) Compute zeroing similarity transformation.B) Apply transformation.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 17: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

��

� �

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��� �

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 18: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

× =

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

� �

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 19: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

× =

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 20: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

× =

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 21: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

��

� �

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

� �

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 22: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

×=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

� �

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 23: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

×=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 24: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

×=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 25: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

��

� �

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

� �

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 26: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

×

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

� �

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 27: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

×

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 28: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

×

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 29: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

��

� �

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

� �

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 30: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××××××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

� �

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 31: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××××××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 32: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××××××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

������

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 33: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××××××

��

� �

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

������

� �

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 34: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

� �

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 35: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

���� ��

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 36: On Deflations in Extended QR Algorithms

Francis’s Implicit Hessenberg QR Algorithm[Francis 1961]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

Why Francis’s Algorithm Works [Watkins 1982, ’07, ’08, ’11]

Power method.

Krylov subspaces.

Shifts accelerate convergence.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 6/20

Page 37: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 38: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

��

� �

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

� �

��

Fusion

Transfer through up. triang.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 39: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

×=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

Turnover

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 40: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

×=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

Similarity transformation

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 41: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

��

� �

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��� �

Similarity transformation

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 42: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

×××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

� �

Transfer through up. triang.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 43: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

×××

×=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

Move pattern

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 44: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

×××

×=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

Turnover

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 45: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

×××

×=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

Transfer through up. triang.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 46: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

×××

×=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

� �

Similarity transformation

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 47: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

×××

��

� �

×=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

� �

��

Similarity transformation

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 48: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

×××

×=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

Turnover

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 49: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

×××

×=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

Transfer through up. triang.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 50: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

×××

×=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

� �

Similarity transformation

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 51: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

×××

��

� �

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

� �

��

Similarity transformation

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 52: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

×××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

Move pattern

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 53: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

×××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

����

����

��

Turnover

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 54: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

××× ×××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

��

��

������

Similarity transformation

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 55: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×× × ×

×

××× ×××

��

� �

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

��

��

������

� �

Similarity transformation

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 56: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×

××

××

×

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

��

��

����

� �

Transfer through up. triang.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 57: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×

××

××

×

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

��

��

���� ��

Turnover

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 58: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×

××

××

×

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

��

��

����

��

Similarity transformation

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 59: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×

××

××

��

� �

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

��

��

����

��

� �

Similarity transformation

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 60: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×

××

××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

��

��

����

� �

Transfer through up. triang.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 61: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×

××

××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

����

��

��

����

��

Fusion

Move pattern

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 62: On Deflations in Extended QR Algorithms

Extended QR Algorithm[Vandebril ’11, Watkins, Vandebril ’12]

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××× ×

××

××

=

× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

��

��

����

����

Why the Extended QR Algorithm Works

Power method.

Rational Krylov subspaces.

Shifts accelerate convergence.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 7/20

Page 63: On Deflations in Extended QR Algorithms

Deflation Criterion

× × × × ×× × × ×× × ×× ××

××××

=

����

����

× × × × ×× × × ×× × ×× ××

A) |H(k + 1, k)| ≤ ε ‖H‖F

B) |H(k + 1, k)| ≤ ε (|H(k , k)|+ |H(k + 1, k + 1)|)

A = Gτ(1)Gτ(2) · · ·Gτ(n−1)R

∥∥I − Gτ(i)

∥∥2≤ ε

A = Gτ(1)Gτ(2) · · ·Gτ(i−1)Gτ(i+1) · · ·Gτ(n−1)R

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 8/20

Page 64: On Deflations in Extended QR Algorithms

Bauer-Fike Theorem

Bauer-Fike Theorem [Bauer, Fike 1960]

minµ∈Λ(A)

|λ− µ| ≤ κ2(X )‖A− A‖2

‖A− A‖ ≤ ‖Gτ(1) · · ·Gτ(i−1)‖ ‖Gτ(i) − I‖ ‖Gτ(i+1) · · ·Gτ(n−1)‖ ‖R‖≤ ‖Gτ(i) − I‖ ‖R‖≤ ‖Gτ(i) − I‖ ‖A‖

Lemma

Let A be diagonalizable, A = XΛX−1, and λ an eigenvalue of A.Then there exists an eigenvalue µ of A, so that

|λ− µ| ≤ κ2(X )‖Gτ(i) − I‖2‖A‖2.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 9/20

Page 65: On Deflations in Extended QR Algorithms

But we can do better:

Theorem [Eisenstat, Ipsen 1995]

A = A1A2, E = A− A

mini

|λ−µi ||µi | ≤ κ2(X )

∥∥A−11 EA−1

2

∥∥2.

A1 = Gτ(1) · · ·Gτ(i) and A2 = Gτ(i+1) · · ·Gτ(n−1)R

⇒ A−11 EA−1

2 = I − G−1τ(i)

‖I − G−1τ(i)‖2 = ‖G−1

τ(i)(Gτ(i) − I )‖2 ≤ ‖Gτ(i) − I‖2

Lemma

Let A be diagonalizable, A = XΛX−1, and λ an eigenvalue of A.Then there exists an eigenvalue µ of A, so that

|λ−µ||µ| ≤ κ2(X )‖Gτ(i) − I‖2.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 10/20

Page 66: On Deflations in Extended QR Algorithms

Conditioning of One Eigenpair

Theorem [Wilkinson 1965]

Let

µ ∈ Λ(A), Ax = µx , yHA = µyH , be a simple eigenvalue,

E = εF , with ‖F‖2 = 1.

For µ = µ(ε) ∈ Λ(A + E ) with µ(0) = λ, we have

|µ− λ| ≤ ‖E‖2

yH x+O(‖E‖2

2).

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 11/20

Page 67: On Deflations in Extended QR Algorithms

Relative Perturbation Estimate (1)

Following the proofs of [Eisenstat, Ipsen 1995] we can proof thefollowing theorem.

Theorem [M., Vandebril ’13]

Let

A be invertible,

µ ∈ Λ(A), Ax = µx , yHA = µyH , be a simple eigenvalue,

E = εF , with ‖F‖2 = 1.

For µ = µ(ε) ∈ Λ(A + E ) with µ(0) = λ, we have

|µ−λ||λ| ≤

‖A−1E‖2

yH x+O(

∥∥A−1E∥∥2

2).

If A = A1A2 and A1 and A2 invertible, then

|µ−λ||λ| ≤

‖A−11 EA−1

2 ‖2

yH x+O

(‖A−1

1 EA−12 ‖2

2

).

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 12/20

Page 68: On Deflations in Extended QR Algorithms

Relative Perturbation Estimate (2)

Corollary [M., Vandebril ’13]

Let A = Gτ(1)Gτ(2) · · ·Gτ(n−1)R and

A = Gτ(1)Gτ(2) · · ·Gτ(i−1)Gτ(i+1) · · ·Gτ(n−1)R. Let further A be

nonsingular, and let λ be a single eigenvalue of A. Then thereexists an eigenvalue µ of A, so that

|µ−λ||λ| ≤

‖I−Gτ(i)‖2

xH y+O

(‖I − Gτ(i)‖2

2

),

with x the right and y the left eigenvector to λ.

The deflation criterion∥∥I − Gτ(i)

∥∥2≤ ε is good.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 13/20

Page 69: On Deflations in Extended QR Algorithms

Aggressive Early Deflation (AED)[Braman, Byers, Mathias ’02]

× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

×××××××××

A

× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

××××××⊗⊗⊗

A′ = QHAQ

VH

× × × ×× × ×× ××

V

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 14/20

Page 70: On Deflations in Extended QR Algorithms

AED for Extended QR Algorithms

����

����

����

����

��

× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 15/20

Page 71: On Deflations in Extended QR Algorithms

AED for Extended QR Algorithms

× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 15/20

Page 72: On Deflations in Extended QR Algorithms

AED for Extended QR Algorithms

× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ×××××

×××

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 15/20

Page 73: On Deflations in Extended QR Algorithms

AED for Extended QR Algorithms

× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

VH V

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 15/20

Page 74: On Deflations in Extended QR Algorithms

AED for Extended QR Algorithms

× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

���� ��

�� �� ��

���� ��

�� �� ��

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 15/20

Page 75: On Deflations in Extended QR Algorithms

AED for Extended QR Algorithms

× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

���� ��

�� �� ��

������

������

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 15/20

Page 76: On Deflations in Extended QR Algorithms

AED for Extended QR Algorithms

× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

���� ��×

�� �� ��× ×

������ ×

���� ××��

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 15/20

Page 77: On Deflations in Extended QR Algorithms

AED for Extended QR Algorithms

× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 15/20

Page 78: On Deflations in Extended QR Algorithms

Wedge-to-Broadhead Transformation

����

�� =�� �

���

����

����

����

��=

����

�� ��

��

����

=

�� ��

��

��

Test for deflations.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 16/20

Page 79: On Deflations in Extended QR Algorithms

The effect of aggressive early deflation for n = 500

1020

3040

50

020

4060

80100

0

200

400

*

*

# shifts#

deflation window

run

tim

ein

s

aggressive early deflationstandard deflation only

best without early deflation best with early deflation savingsn sh t #it·sh sh m t #it·sh

100 4 6.00 s 215 10 18 5.62 s 223 6% time100 2 6.42 s 198 2 28 8.60 s 163 18% iterations500 6 148.49 s 803 20 20 100.45 s 727 32% time500 2 161.38 s 732 2 70 314.85 s 391 46% iterations

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 17/20

Page 80: On Deflations in Extended QR Algorithms

The effect of aggressive early deflation for n = 500

1020

3040

50

020

4060

80100

0

200

400

*

*

# shifts#

deflation window

run

tim

ein

s

aggressive early deflationstandard deflation only

best without early deflation best with early deflation savingsn sh t #it·sh sh m t #it·sh

100 4 6.00 s 215 10 18 5.62 s 223 6% time100 2 6.42 s 198 2 28 8.60 s 163 18% iterations500 6 148.49 s 803 20 20 100.45 s 727 32% time500 2 161.38 s 732 2 70 314.85 s 391 46% iterations

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 17/20

Page 81: On Deflations in Extended QR Algorithms

Middle Deflation

× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 18/20

Page 82: On Deflations in Extended QR Algorithms

Middle Deflation

× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 18/20

Page 83: On Deflations in Extended QR Algorithms

Middle Deflation

× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 18/20

Page 84: On Deflations in Extended QR Algorithms

Middle Deflation

× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

V VH V VH

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 18/20

Page 85: On Deflations in Extended QR Algorithms

Middle Deflation

���� ��

�� �� ��×

× ×

���� ��

������×××

����������

��

× ××

�� �� ���� ��

��

× ××

× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 18/20

Page 86: On Deflations in Extended QR Algorithms

Middle Deflation

× × × × × × × × × ×× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×× ××

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 18/20

Page 87: On Deflations in Extended QR Algorithms

Predicting Middle Deflations

One needs to define a deflation window that most likely leads to amiddle deflation.

g∏i=f

|si | < ε2

For random matrices a successful middle deflation window of size0.75 · n × 0.75 · n is found after a few iterations.

A ∈ C1500×1500, random, three multishift steps (150 shifts),deflation window 363:1026, leads to middle deflation

The corresponding deflation windows for other test-matrices(tols2000, pde2961, . . . ) have not been successful.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 19/20

Page 88: On Deflations in Extended QR Algorithms

Predicting Middle Deflations

One needs to define a deflation window that most likely leads to amiddle deflation.

g∏i=f

|si | < ε2

For random matrices a successful middle deflation window of size0.75 · n × 0.75 · n is found after a few iterations.

A ∈ C1500×1500, random, three multishift steps (150 shifts),deflation window 363:1026, leads to middle deflation

The corresponding deflation windows for other test-matrices(tols2000, pde2961, . . . ) have not been successful.

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 19/20

Page 89: On Deflations in Extended QR Algorithms

Conclusions & Acknowledgments

We have investigated deflations for extended QR algorithms.

Deflating almost diagonal rotations preserves absolute andrelative accuracy also with respect to the conditioning ofsingle eigenvalues.

We have seen aggressive early deflation in extended QRalgorithms.

These ideas can be used for middle deflations in special cases.

The research was partially supported by the Research Council KULeuven (CREA-13-012, F+/13/020, OT/11/055, CoEEF/05/006), by DFG research stipend MA 5852/1-1, by Fund forScientific Research–Flanders (Belgium) project G034212N, and byBelgian Network DYSCO – Dynamical Systems, Control, andOptimization

Thomas Mach, Raf Vandebril, On Deflations in Extended QR Algorithms 20/20