tropical presentation short · Tropical mathematics (and why you should care) Manos Theodosis Harvard University School of Engineering and Applied Sciences

Tropical mathematics(and why you should care)

Manos Theodosis

Harvard UniversitySchool of Engineering and Applied Sciences

OutlineIntroduction Motivation (algebra and geometry) General definitions Eigenvalues and polynomials Dilations and erosions Curves and surfaces Applications Neural networks Curve fitting Spectral analysis Parameter reduction


Introduction


Multidimensional manifolds

Geometric algorithms

Neural networks

Motivation (geometry)


Figure 1Complex manifolds.

(a) (b) (c)

Figure 2DBSCAN algorithm.

(a) Unconnected components. (b) Connected components.

(a) Decision region of a morphological neural network.

(b) Newton polytope of a tropical polynomial.

Figure 3Tropical neural networks and their regions.

Shortest path problems Scheduling problems

Bonus: Matrix representation (vector spaces) Piecewise linear solution spaces


start

destination

Motivation (algebra)

Figure 4General formulation of a shortest path problem.

Figure 5Scheduling problem of connecting flights.

General definitionsSimilar to linear algebra, but replace the pair with . Matrix-vector multiplication becomes In general two variants; min-plus algebra and max-plus algebra. Example


( + , × ) (min , + )

(Ax)j= ∑

i

(Aji ⋅ xi) ⇔ (A ⊞′ x)j= min

i(Aji + xi)

A ⊞′ b = [3 41 −26 2 ] ⊞′ [1

3] =min(3 + 1, 4 + 3)

min(1 + 1, − 2 + 3)min(6 + 1, 2 + 3)

= [415]

Ab = [3 41 −26 2 ] [1

3] =(3 ⋅ 1) + (4 ⋅ 3)

(1 ⋅ 1) + (−2 ⋅ 3)(6 ⋅ 1) + (2 ⋅ 3)

=15−512

vs

Eigenvalues and polynomialsEigenvalues Two very important matrices Solutions to the generalized (sub)eigenvalue problems Polynomials Exponents change meaning General form of a tropical polynomial We can allow for non integer (or negative) exponents

Γ(A) = min(A, A2, . . . ) =∞

minn=1

An, Δ(A) = min(I, A, A2, . . . ) =∞

minn=0

An

A ⊞′ x = λ + x, A ⊞′ x ≥ λ + x


xa = x ⋅ x ⋅ … ⋅ x ⇔ xa = x + x + … + x = a ⋅ x

linear tropical

y = mini

aix + ci ⇒ y = mini

aTi x

y = min(0.3x + 4, − 5x − 5,15

x)

Distributivity Erosions distribute over infima, dilations over suprema Formulation A unique adjunction pair Fundamental property For any dilation, erosion, lattice element Tropical erosions and dilations In our context where and is the max-plus multiplication.

⋀i

ε(Xi) = ε (⋀i

Xi), ⋁i

δ(Xi) = δ (⋁i

Xi)

Dilations and erosions

δ(X) ≤ Y ⇔ X ≤ ε(Y )


δ(ε(X)) ≤ X ≤ ε(δ(X))

δA(x) = A ⊞ x, εA(x) = A* ⊞′ x

A* = − AT ⊞

Euclidean curves Tropical curves


Curves: Polynomials

Figure 6Euclidean polynomials.

(a) y = ax + b (b) y = ax2 + bx + c (c) y = ax3 + bx2 + cx + d

Figure 7Tropical polynomials.

(b� a) (c� b)

(2b� a)

c

a

(b) y = min(a + 2x, b + x, c)(b� a)

b

a

(a) y = min(a + x, b)(b� a) (c� b) (d� c)

d

a

2c� b

3b� 2a

(c) y = min(a + 3x, b + 2x, c + x, d )

1D Halfspaces 2D Halfspaces


Curves: Halfspaces and polytopes

Figure 8Halfspaces and polytopes in 1D. For example, the red line is .y = min(1 + x,2)

�2 0 2 40

1

2

3

4

Rr�

Rr

x

y

(a) Tropical line and its halfspaces.

0 50

1

2

3

4

RP

Rb�

Rg�

Rr

x

y

(b) Tropical polytope; intersection of tropical halfspaces.

Figure 9Tropical polytopes in 2D. Blue is and red is .z2 = min(5 + x,7 + y,9)

z1 = max(0 + x,2 + y,7)

05

100

10

0

10

xy

z

(b) Second view.

0 5 100

10

0

10

x

y

z

(a) First view.

Curves: Tropical Loci


2x < min(y, c) c < min(2x, y)

y < min(2x, c)

c/2

c

Figure 10Tropical locus of the equation

and the resulting space clustering.y = min(2x, c)

2y + c

2x+ a x+ y + b

y + d

x+ f

e

Figure 11Tropical locus of the general quadratic tropical 2-D polynomial

.z = min(2x + a, x + y + b,2y + c, y + d, x + f, e)

Applications


Neural networksNeural networks Traditional morphological perceptron ReLUs and maxout units are tropical polynomials Idea: find a bound the number of linear regions/vertices of the solution space.


τ(x) = maxi

wi + xi = wT ⊞ x

Figure 12Regions of a morphological perceptron for binary classification.

ReLU(x) = max(0,wTx + b), maxout(x) = maxj

(WTj x + bj)

Neural networks: Linear regionsTropical polynomials Remember the definition Newton polytope Defined as the convex hull of the coefficient vectors . How do operations on polynomials affect the Newton polytope?

ai

⋅ Newt(y1 + y2) = P1 ⊕ P2⋅ Newt(max(y1, y2)) = conv(P1, P2)


y = mini

aTi x

P1 � P2

(0, 3)

(0, 1)

(1, 3)

(3, 2)

(3, 1)P1

P2

conv(P1 [ P2)

(0, 0)

(�1, 1)

(�1, 0)

P1

(3, 1)

(1, 2)

P2

(0, 0)

(0, 1)(�1, 1)

(�1, 0)

P1

(3, 1)(1, 1)

(1, 2)

P2

Figure 13Newton polytopes of the tropical polynomials and , their Minkwoski sum, and their convex hull.

z1 = max(−x, − x + y, y,0)z2 = max(x + y,3x + y, x + 2y)

Minkwoski sum

Modeling Suppose we fit a model For more data Fundamental property Remember that In the tropical case Optimal solutions Optimal solution for Optimal unconstrained solution

w

Curve fitting


δ(ε(X)) ≤ X ≤ ε(δ(X))

X ⊞ w ≤ b

δX(εX(b)) ≤ b ⇔ X ⊞ (−XT ⊞′ b) ≤ b

y = max(aT1 x + w1, …, aT

k x + wk) hyperplanes

w = − XT ⊞′ b

w∞ = − XT ⊞′ b + μ μ =12

∥X ⊞ w − b∥∞

Tropical models Arbitrary models


Curve fitting: 1D examples

!10 !5 0 5 10!5

0

5

10

!3x !2x !1x

0x

1x 2x 3x

�2 �1 0 1 2�2

0

2

4

6

8

Input Data

TargetData

Max-plus Tropical Line Fitting

Tropic GLETropic MMAE

0 2 4 6 8 10 120

2

4

6

8

AMTmi .�i�

h�`;

2i.

�i�

J�t@THmb h`QTB+�H GBM2 6BiiBM;

h`QTB+ :G1h`QTB+ JJ�11m+HB/ Ga1

0 2 4 6 8 10 120

2

4

6

8

AMTmi .�i�

h�`;

2i.

�i�

J�t@THmb h`QTB+�H GBM2 6BiiBM;

h`QTB+ :G1h`QTB+ JJ�11m+HB/ Ga1

Figure 14Euclidean vs tropical fitting for the max-plus line under different noises.

y = max(x − 2,3)

Figure 15Toy circle fitting; highlights the various tropical lines.

Figure 16Optimal fitting of the max-afine curve given

by .y = max(−6x − 6,12

x,15

x5 +12

x)

2D surfaces Questions Number of terms? Complexity? How to compute the slopes?

�1 �0.5 0 0.5 1�1

0

1

0

1

2

x

y

z

�1 �0.5 0 0.5 1�1

0

1

0

1

2

x

y

zHarvard UniversitySchool of Engineering and Applied Sciences

Curve fitting: 2D examples

Figure 17Optimal fitting (left) of parabolic data and the corresponding unconstrained solution (right).

Spectral Analysis of WFSTsWeight pushing Can be written in tropical algebra

with . Epsilon removal Can be written in tropical algebra

Do they remind you something?

λ′ = Λ ⊞′ v∞, ρ′ = P ⊞′ (−v∞), A′ = V− ⊞′ A ⊞′ V+

v∞ = Δ(A) ⊞′ ρ

A′ = Δ(E) ⊞′ Aε, ρ′ = Δ(E) ⊞′ ρ


Figure 18Example showcasing the weight pushing algorithm for WFSTs.

Figure 19Example showcasing the epsilon removal algorithm for WFSTs.

Spectral Analysis of WFSTsEigenproblems

provides the solution to Solution set is Interpretation For weight pushing ; eigenvalue of zero. For epsilon removal ; eigenvalues of zero. Idea: what about other eigenvalues?

Δ(A)

v∞ = Δ(A) ⊞′ ρ

A′ = Δ(E) ⊞′ Aε, ρ′ = Δ(E) ⊞′ ρ


A ⊞′ x ≥ λ + x

V*(A, λ) = {Δ(−λ + A) ⊞′ u, u ∈ Rn}

Parameter ReductionEuclidean What’s the minimum number of hyperplanes required for a closed region?

: three lines : four planes : (?) hyperplanes

Proof: 1. hyperplanes can bound -dimensional space (axes + a -dimensional hyperplane passing through points) 2. hyperplanes can’t bound the -dimensional space (induction)

How many parameters? parameters per hyperplane; total parameters.

Tropical hyperplanes of parameters each; total parameters.

Common ground For a polytope of vertices Euclidean needs parameters. Tropical needs parameters.

d = 2d = 3d = n n + 1

d + 1 d dd

d d

d d(d + 1)

2 d 2d

2d

⋅ 2d2

⋅ 2d



Thanks!

Documents

tropical presentation short · Tropical mathematics (and why you should care) Manos Theodosis Harvard University School of Engineering and Applied Sciences