Embed Size (px)
DESCRIPTION
Ill-posed Computational Algebra with Approximate Data. Zhonggang Zeng Northeastern Illinois University, USA. Feb. 21, 2006, Radon Institute for Computational and Applied Mathematics. = 4 nullity = 2. Rank. null space. basis. - PowerPoint PPT Presentation
Citation preview
Ill-posed Computational Algebra with Approximate Data
Zhonggang Zeng
Northeastern Illinois University, USA
Feb. 21, 2006, Radon Institute for Computational and Applied Mathematics
Example: Matrix rank and nulspace
Rank= 4nullity = 2
+ E = 6nullity = 0
null space
basis2
A well-posed problem: (Hadamard, 1923) the solution satisfies
• existence• uniqueness• continuity w.r.t data
Ill-posed problems are common in applications
- image restoration - deconvolution - IVP for stiction damped oscillator - inverse heat conduction- some optimal control problems - electromagnetic inverse scatering- air-sea heat fluxes estimation - the Cauchy prob. for Laplace eq. … …
3
An ill-posed problem is infinitely sensitive to perturbation
tiny perturbation huge error
Ill-posed problems are common in algebraic computing
- Multiple roots of polynomials
- Polynomial GCD
- Factorization of multivariate polynomials
- The Jordan Canonical Form
- Multiplicity structure/zeros of polynomial systems
- Matrix rank
4
Problem: Polynomial factorization (E. Kaltofen, Challenges of symbolic computation, 2000)
01296288648726481681 2222444 yxyxzyx
Exact factorization no longer exists under perturbation
Can someone recover the factorization when the polynomial is perturbed?
0)3621849)(3621849( 222222 zyxzyx
5
“attainable” roots1.072753787571903102973345215911852872073…0.422344648788787166815198898160900915499…0.422344648788787166815198898160900915499…2.603418941910394555618569229522806448999…2.603418941910394555618569229522806448999 …2.603418941910394555618569229522806448999 …1.710524183747873288503605282346269140403…1.710524183747873288503605282346269140403…1.710524183747873288503605282346269140403…1.710524183747873288503605282346269140403…
Inexact coefficients 2372413541474339676910695241133745439996376-21727618192764014977087878553429208549790220 83017972998760481224804578100165918125988254-175233447692680232287736669617034667590560780 228740383018936986749432151287201460989730170-194824889329268365617381244488160676107856140 110500081573983216042103084234600451650439720-41455438401474709440879035174998852213892159 9890516368573661313659709437834514939863439-1359954781944210276988875203332838814941903 82074319378143992298461706302713313023249
9355
Exact coefficients 2372413541474339676910695241133745439996376-21727618192764014977087878553429208549790220 83017972998760481224804578100165918125988254-175233447692680232287736669617034667590560789 228740383018936986749432151287201460989730173-194824889329268365617381244488160676107856145 110500081573983216042103084234600451650439725-41455438401474709440879035174998852213892159 9890516368573661313659709437834514939863439-1359954781944210276988875203332838814941903 82074319378143992298461706302713313023249
Exact roots1.072753787571903102973345215911852872073…0.422344648788787166815198898160900915499…0.422344648788787166815198898160900915499…2.603418941910394555618569229522806448999…2.603418941910394555618569229522806448999 …2.603418941910394555618569229522806448999 …1.710524183747873288503605282346269140403…1.710524183747873288503605282346269140403…1.710524183747873288503605282346269140403…1.710524183747873288503605282346269140403…
-6-
Problem: polynomial root-finding
Example: JCF computation
t
s
s
r
r
r
1
1
1
,2r
Special case:
,3s 5t
Maple takes 2 hours
On a similar 8x8 matrix, Maple and Mathematica run out of memory
Question: can we compute the JCF using approximate data?
7
21
21
2)(222
)()21(213)(2)2(21
10
10
10
10
222222
222222222222222
Lancaster matrix derived from stability problem )1(
, , , ,0 spectrum ,0For iii
i
i
i
i
i
i
JCF
1
1
0
10
,0For
Question:
Can we find the JCFapproximately when
?0
8
0 10 3 0 -1 -1 -4 0 0 -5 -5 0 1 0 0 -1 0 -5 -1 0 -3 -1 0 5 9 -1 3 -2 -1 1 1 -2 -2 1 -1 1 1 2 -1 -1 1 0 -1 1 3 1 7 2 -2 -11 1 0 6 -4 -3 6 0 5 -1 0 -3 -2 -1 0 0 0-1 1 5 2 3 1 -1 0 0 0 0 0 -1 0 1 2 0 0 1 0 -1 1-4 -2 -9 -2 6 19 -2 0 -8 8 6 -8 1 -7 1 -2 4 4 2 0 0 -1 0 -1 1 -1 1 2 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 9 -2 4 -3 3 3 1 -2 -2 1 0 1 2 1 -1 -1 1 0 -1 0 1 0 1 0 0 -2 0 3 4 0 0 3 0 2 0 0 -1 0 0 0 0 0 1 -4 -2 0 1 4 1 0 3 5 4 0 -2 0 0 1 0 3 1 0 1 1-1 1 -2 1 -1 3 -1 -1 -3 3 0 -3 0 -2 -1 0 1 0 0 0 0 0 5 2 6 2 -3 -16 1 0 12 -5 -1 12 0 9 -1 0 -5 -3 -2 0 0 0-1 4 0 1 -2 -4 -1 0 0 -5 -4 3 4 0 -1 -2 0 -3 -1 0 -1 -2 1 0 1 0 0 -2 0 0 2 0 0 2 3 2 0 0 -1 0 0 0 0 0 0 -1 4 -3 3 -1 1 1 0 0 0 0 -2 3 3 1 0 0 0 0 0 1 0 2 12 -1 2 -7 0 0 2 -4 -3 2 -3 2 4 6 -1 -2 0 0 -1 3-4 -1 -5 -2 2 12 -1 0 -7 4 3 -7 0 -6 1 3 4 2 1 0 0 0 0 11 8 1 -2 -12 -3 0 6 -9 -8 6 1 5 0 -1 0 -7 -2 0 -3 -1-2 0 7 -2 5 1 -1 1 -2 0 0 -2 0 -1 1 1 0 4 3 -1 -1 0 3 2 6 2 -2 -7 1 0 2 -5 -4 2 -2 2 0 3 -1 -3 1 2 0 2 5 -12 -10 2 -3 1 5 -1 0 6 6 0 0 0 -2 -1 0 6 0 3 5 0 4 -9 0 1 0 1 4 -1 0 4 4 0 -4 0 0 4 0 4 0 1 6 4 2 0 2 0 0 -3 0 0 3 0 0 3 0 3 0 0 -2 0 0 0 0 3
Problem: Jordan Canonical Form (JCF) 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3
JCF
)10( 15 EA
3)( A
9
Problem: Multiplicity structure of a polynomial system
0)( xP
A zero x* can only be approximate, even if P is exact (Abel’s Impossibility Theorem)
An approximate zero x0 is an exact solution to
0)()()(~
0 xPxPxPx0 degrades to a simple zero, even if x* is multiple
Can we recover the lost multiplicity structure?10
P : Data SolutionP
P
Data
Solution
P
Challenge in solving ill-posed problems:
Can we recover the lost solution when data is inexact?
11
If the answer is highly sensitive to perturbations, you have probably asked the wrong question.
Maxims about numerical mathematics, computers, science and life, L. N. Trefethen. SIAM News
A numerical algorithm seeks the exact solution of a nearby problem
Wilkinson’s Turing Award (1970):
Ill-posed problems are infinitely sensitive to data perturbation
However:
Conclusion: Numerical computation is incompatible
with ill-posed problems.
Solution: Ask the right question.
12
Trivia quiz: Is the following matrix nonsingular?
110
1
10
110
1
(matrix derived from polynomial division by x+10 )
Answer: It is nonsingular in pure math. Not so in Numerical Analysis.
(Distance to singular matrices)110
1 aluesingular vsmallest
n
13
1 nullity
2 nullity
3 nullity A
• nearness• max-nullity• mini-distance
)(min)( BrankArankAB
i.e. the worst rank of nearby matrices
)()( BNAN with
2)()(2min ACAB
ArankCrank
I.e. the nulspace of the nearest matrix with that rank
B
14
Rank problem: Ask the right question
0 nullity
Rank
= 4nullity = 2
+ E = 6nullity = 0
null space
basis
+ E = 4nullity = 2
98.40
1
26.61))()(( EANANdist
Rank
= 4nullity = 2
After reformulating the rank:
Ill-posedness is removed successfully.
15
Computing approximate rank/nullspace:
General purpose: Singular value decomposition (SVD)
Software package RankRev available
Recursive Inverse iteration/deflation (T.Y. Li & Z. Zeng, SIMAX 2005)
Low-nullity and low-rank cases:
16
which is an important application problem in its own right.
Applications: Robotics, computer vision, image restoration, control theory, system identification, canonical transformation, mechanical geometry theorem proving,hybrid rational function approximation … …
gwu
fvu),( gfGCDu
For given polynomials f and g, find u, v, w such that
Problem: Polynomial GCD
17
The question: How to determine the GCD structure
If ,vuf wug
then 0)()( vwuwvu
v
wgf ,
0)(),(
v
wgCfC
At heart of the GCD-finding is calculating rank and nulspace with an approximate matrix
The Sylvester Resultant matrix
is approximately rank-deficient
nulspace
18
1 )u~deg(
2 )u~deg(
3 )u~deg(
• nearness• max-degree• mini-distance
))~,~
(deg(max)~deg(),()~,
~(
gfGCDugfgf
i.e. the max degree of nearby polyn’s
)~,~
(),( gfEGCDgfAGCD with
),()ˆ,ˆ(min),()~,~
()~deg())ˆ,ˆ(deg(2
gfgfgfgfugfGCD
I.e. the GCD of the nearest polyn pair with that degree
19
ts PP -- space of polynomial pairs)~,
~( gf
),( gf0 )u~deg(
)~,~
( gf
perturbed
polynomial pair
original polynomial pair
),( gf
computed polynomial pair )ˆ,ˆ( gf
ts PP -- space of polynomial pairs
manifold
krs PgfGCDPPgf ),(:),(
Minimize2
)ˆ,ˆ()~,~
( gfgf --- a least squares problem
A two staged approach:
Determine the GCD structure (or manifold)
Reformulate – solve a least squares problemwell-posed, and even well conditioned
20
Given ts PPgf ),(
1)(u
gwu
fvu
Set up a system for ktksk PPPwvu ),,(
Or, in vector form bwvuF
),,( where
uwC
uvC
ur
wvuF
k
k
H
)(
)(),,(
g
fb
1
and
minimize2
2),,( bwvuF
and the GCD degree k
The strategy: Reformulate a well-posed problem
21
,
)(
)(),,(
uwC
uvC
ur
wvuF
k
k
H
Theorem: The Jacobian is of full rank
if v and w are co-prime
)()(
)()(),,(
wCwC
uCvC
r
wvuJ
knk
kmk
H
Its Jacobian:
In other words: if the AGCD degree k is correct, then
2
2),,(
),,(min bwvuFknkmk PPPwvu
is a regular, well-posed problem!22
For univariate polynomials :
Stage I: determine the GCD degree
S1(f,g) = QR S2(f,g) QR
until finding the first rank-deficient Sylvester submatrix
Stage II: determine the GCD factors ( u, v, w )
by formulating bwvuF;
),,(
and the Gauss-Newton iteration
23
Start: k = n
Is GCD of degree kpossible?no
k := k-1
Successful?
no
k := k-1
Refine with G-N Iteration
probably
yes
Output GCD
Univariate AGCD algorithm
Max-degree
Min-distancenearness
24
Software packages
Zeng:
uvGCD – univariate GCD computation mvGCD – multivariate GCD computation
Corless-Watt-Zhi
Gao-Kaltofen-May-Yang-Zhi
Kaltofen-Yang-Zhi
etc
25
Identifying the multiplicity structure
p(x)= (x-1)5(x-2) 3(x-3) = [(x-1)4(x-2) 2] [(x-1)(x-2)(x-3)]
p’(x)= [(x-1)4(x-2) 2] [ (x-1)(x-2)+5(x -2)(x -3)+3(x -1)(x -3) ]GCD(p,p’) = [(x-1)4(x-2) 2]
u0(x) = [(x-1)4(x-2) 2] [(x-1)(x-2)(x-3)]
u1(x) = [(x-1)3(x-2) ] [(x-1)(x-2)]
u2(x) = [(x-1)2] [(x-1)(x-2)]
u3(x) = [(x-1)] [(x-1)]
u4(x) = [1] [(x-1)]
distinct roots:
* * *
* *
* *
*
*
-----------------------------------------
multiplicities 5 3 1u0 = pum =GCD(um-1, um-1’)
26
A squarefree factorization of f:
u0 = f
for j = 0, 1, … while deg(uj) > 0 do
uj+1 = GCD(uj, uj’)
vj+1 = uj/uj+1
end do
with vj’s being squarefreeOutput : f = v1 v2 … vk
- The number of distinct roots: m = deg(v1)
kj ,,2,1
- The multiplicity structure
,1)deg(|max jmvtl tj
],,,[ 21 kllll
- Roots of vj’s are initial approximation to the roots of f
the key
27
For the ill-posed multiple root problem with inexact data
The key:
Remove the ill-posedness by reformulating the problem
Original problem: calculating the roots
Reformulated problem: finding the nearestpolynomial on a proper pejorative manifold
-- A constraint minimization
28
Let ( x - z1 l1 x - z2 ) l2 ... ( x - zm ) lm =
xn + g1 ( z1, ..., zm ) xn-1+...+gn-1 ( z1, ..., zm ) x + gn ( z1, ..., zm )
Then, p(x) = ( x - z1 l1 x - z2 ) l2 ... ( x - zm ) lm <==>
g1 ( z1, ..., zm ) =a1
g2( z1, ..., zm ) =a2
... ... ...
gn ( z1, ..., zm ) =an
I.e. An over determined polynomial system
G(z) = a
(m<n)(degree) n
m (number of distinct roots)
To calculate roots of p(x)= xn + a1 xn-1+...+an-1 x + an
29
aazz l ˆˆ abbal ˆ
abl 2
given polynomial
b ~ q(x)
b
azGl ˆ)ˆ( computed polynomiala
original polynomial Gl(z) = a ~ pa
It is now a well-posed problem! 30
5101520 )4()3()2()1( xxxxFor polynomial
with (inexact ) coefficients in machine precision
Stage I results:
The backward error: 6.05 x 10-10
Computed roots multiplicities
1.000000000000353 202.000000000030904 153.000000000176196 104.000000000109542 5
Stage II results:
The backward error: 6.16 x 10-16
Computed roots multiplicities
1.000000000000000 201.999999999999997 153.000000000000011 103.999999999999985 5
Software: MultRoot, Zeng, ACM TOMS 2004 31
JCF computing: For a given matrix A
find X, J:
AX = XJ Segre characteristics:
[3,2,2,1]
Staircase form with Weyr characteristics:
[4,3,1]
1
1 1
1
J =
find U, S,
AU = U(I+S)
3 2 2 1
4
31
Ferrer’s diagram
I+S=
+ + ++ + ++ + ++ + +
++++
+++
32
find U, S,
AU = U(I+S)I+S=
+ + ++ + ++ + ++ + +
++++
+++
If the Weyr characteristics is known:
Theorem: Solving system (1) for a least squares solution is well-posed!
AU- U(I+S) = 0 UHU – I = 0
leads to
-- Eigenvalues are no longer flying!-- Gauss-Newton iteration converges locally
BHU = 0
for , U, S(1)
with a staircase structureconstraint
33
A two-stage strategy for computing JCF:
Stage I: determine the Jordan structure and aninitial approximation
Stage II: Solve the reformulated least squares problemat each eigenvalue, subject to the structuralconstraint, using the Gauss-Newtion iteration.
34
eigenvalue Segre characteristics 5 3 2 2 3 3 1 2
Minimal polynomials:
214
12
213
32
312
23
32
511
)()(
)()()(
)()()(
)()()()(
p
p
p
p
Let v be a coefficient vector of p1. Then for a random x
0,,, 10 vxAAxx
A rank/nulspace problem, again 35
By a Hessenberg reduction, with A1 = A, and x being the first column of Q1,
Rank/nulspace leads top1()
(the 1st minimal polynomial)
leads to
p2(), p3(), …
recursively
36
Minimal polynomials
214
12
213
32
312
23
32
511
)()(
)()()(
)()()(
)()()()(
p
p
p
p
Ill-posed root-finding with approximate data
Computing multiple roots of inexact polynomials, Z. Zeng, ISSAC 03
Rank-revealing GCD Root-finding JCF
37
When matrix A is possibly approximate: EAA ~
A~
perturbed
matr
ix
“nearest” matrix on A
nnC -- space of matrices
manifold
structure JCF fixeda withnnCB
original matrix
A
Nearness: JCF of AMax-defectiveness: on
Mini-distance: from to A~
We can obtain an approximate JCF of A~
Software AJCF is in testing stage
38
A two-stage strategy for removing ill-posedness:
Stage I: determine the structure of the desired solution this structure determines a pejorative manifold of data
P-1 = D | P(D) = S fits the structure }
Stage II: formulate and solve a least squares problem
For a problem(data ----> solution)
SDP : with data D0
2
01 )( DSP
minimizeS
by the Gauss-Newton iteration
39
The multiplicity structure of polynomial systemsExample:
0
0422
21
31
xxx
x What’s the multiplicity of (0,0)?
422
21
3121 ,]],[[dim xxxxxxCC
(algebraic geometry)
Intersection multiplicity =
• Dual space D(0,0) (I) spanned by
1
2306
1
2205
2
020413
2
0312
3
021120
2
0110
1
00 ,, ,, ,, , ,, ,
ji
ji
ij xxji 21!!
1
• Multiplicity = 12
• Hilbert function = {1,2,3,2,2,1,1,0,…}
0 1 2 3 4 5 6Differential orders
depth = 6
breadth=2 Multiplicity is not just a number!
For a univariate polynomial p(x) at zero x0:
Multiplicity = m 0)()(')( 0)1(
00 xpxpxp m
Differential functionals
1210 ,,,, mVanish on the ideal <p>, and span the dual space of <p> at x0
Duality approach (Lasker, Macaulay, Groebner):Multiplicity is the dimension of the dual space, spannedby the differential functionals that vanish on the ideal
tNj
jjtx ffffcccffDs
,,,0)(,, 110The dual space
defines the multiplicity structure of the ideal at x0 41
A differential functional c of order
zj
jjcc
with ss Njjj ,,1
For a polynomial system, or ideal tff ,,1
The functional c is in the dual space if and only if
,0)( ipfc tixxCp s ,,1],,,[ 1
at zero z
Or, equivalently siNkzfzxc s
ji
kjj ,,1,for0)()(
0cS
A rank/nulspace problem, again!42
Multiplicity matrices S and nullspaces SN
0S
(0,0)at , 2221
2121 xxxxxxI
,, span )( 02112010011000)0,0( ID 43
If the polynomial system is inexact, or
if the system is exact, but the zero is approximate
determining the multiplicity structure goes back to solving
Ill-posed rank/nulspace problem with approximate data.
Algorithm:• for = 1, 2, … do
- construct matrix S - compute- if W = {0}, break
• end do
)()( 1 SNSNW
A reliable algorithm for calculating the multiplicity structurewith approximate data Dayton & Zeng, ISSAC 2005 Bates, Peterson & Sommese, 2005 44
Conclusion
- Ill-posed problems could be regularized as well posed least squares problems
- Regularized ill-posed problems permit approximated data
- Rank/nulspace determination is at the heart of solving ill-posed problems
Software package PolynMat : RankRev uvGCD mvGCD MultRoot AJCF dMult
