I. Mezic: Spectral Theory of Nonlinear Fluid Flows Based on the Koopman Operator

Igor Mezić(presenter: Marko Budišić)

Spectral Theory of Nonlinear Fluid Flows

Based on the Koopman operator

IUTAM Symposium on 50 Years of ChaosKyoto, 30 Nov 2011

Wednesday, Nov 30, 2011 Igor Mezić; Spectral Theory of Nonlinear Flows using the Koopman Operator (presenter: Marko Budišić)

3-D. Finite-time. Actively controlled.

2

Mixing and transport in engineering

Integrated Building Systems

Energy Efficiency in Transportation

Microfluidics and drug discovery

Drifting sensors in geophysical flows

Unifying challenges:


Key concepts

3

Introduction

• Ergodic-theory-based Visualization Methods

• Mesohyperbolicity

• Quantifying ergodicity and mixing

• Control-theory linked with ergodic-theory based methods

Featured technical tools:

• choice of measurement functions, observables

• averaging

• spectral quantities


B.O. Koopman and J. von Neumann, “Dynamical Systems of Continuous Spectra”, PNAS (1932)

Operator theory: history

Motivation: Mixing

Operator and Measure Theory Necessary

Low-DOF Systems

Measure-theoretic (modern) mixing


B.O. Koopman and J. von Neumann, “Dynamical Systems of Continuous Spectra”, PNAS (1932)

Operator theory: history

Motivation: Mixing

Operator and Measure Theory Necessary

Low-DOF Systems

Perron-Frobenius (transfer) Operator: dynamics of measuresLasota and Mackey, “Chaos, fractals, and noise: stochastic aspects of dynamics”,David Ruelle, Lai-Sang Young, Vivian Baladi,Michael Dellnitz, Oliver Junge, Erik Bollt, Gary Froyland…

Koopman Operator: dynamics of observablesStay tuned...

Two operators:

Measure-theoretic (modern) mixing

f : M → C


Koopman operator:

Observables:

B.O. Koopman “Hamiltonian Systems and Transformations in Hilbert Space”, PNAS (1931)

Operator theory: setup

Uf(x) = [f ◦ T ](x), T : M → M Iterative map (discrete time)

f : M → C


Koopman operator:

Observables:




U tf(x) = [f ◦ Φt](x), Φt : M → M Flow (continuous time)

∂f

∂t+ v ·∇f = 0, x = v(x) ODE

Koopman o. is a linear operator capturing the full nonlinear flow.

f : M → C


Koopman operator:

Observables:




U tf(x) = [f ◦ Φt](x), Φt : M → M Flow (continuous time)

∂f

∂t+ v ·∇f = 0, x = v(x) ODE

On the attractor, U is unitary and admits a spectral decomposition.

U = Up + Uc

Up =�

j

ei2πωjPωj

T1

i

orthogonal projections to algebraic eigenspaces

Koopman o. is a linear operator capturing the full nonlinear flow.

g : M → CJ

g(xn) = g(Tnx) = Ung(x)

Wednesday, Nov 30, 2011 Igor Mezić; Spectral Theory of Nonlinear Flows using the Koopman Operator (presenter: Marko Budišić) 6

Evolution of physically motivated observables

Visualization: Koopman Modes

Identification of natural modes of dynamics

g : M → CJ






Uϕk = λkϕk

g(x) =

...gj(x)

...

=∞�

k=1

...vj...

k

ϕk(x)

Expansion in Koopman eigenfunctions

g : M → CJ




g(xn) =

...gj(xn)

...

=∞�

k=1

...vj...

k

λnkϕk(x)

Koopman modesare coefficients of expansion and are time-invariant.



Uϕk = λkϕk

g(x) =

...gj(x)

...

=∞�

k=1

...vj...

k

ϕk(x)


g : M → CJ




g(xn) =

...gj(xn)

...

=∞�

k=1

...vj...

k

λnkϕk(x)

Koopman modesare coefficients of expansion and are time-invariant.



Computation ofKoopman modes

• Record observable as system evolves.• Use observables’ snapshots as a basis for

the Krylov subspace in Arnoldi algorithm.C.W. Rowley, I. Mezic, S. Bagheri, P. Schlatter, and D.S. Henningson JFM 2009 )

Uϕk = λkϕk

g(x) =

...gj(x)

...

=∞�

k=1

...vj...

k

ϕk(x)



Spectral analysis of nonlinear flows 9

(a) (b)

Figure 3. Positive (red) and negative (blue) contour levels of the streamwise velocity compo-nents of two Koopman modes. The wall is shown in gray. (a) Mode 2, with �v2� = 400 andSt2 = 0.141. (b) Mode 6, with �v6� = 218 and St6 = 0.0175.

are compared to the frequencies obtained directly from the Ritz eigenvalues (red verticallines). The shedding frequencies and a number of higher harmonics are in very goodagreement with the frequencies of the Koopman modes. In particular, the dominantKoopman eigenvalues match the frequencies for the wall mode (St = 0.017) and theshear-layer mode (St = 0.14). Note that the probe signals are local measures of thefrequencies at one spatial point, whereas the Koopman eigenvalues correspond to globalmodes in the flow with time-periodic motion.

The streamwise velocity component u of Koopman modes 2 and 6 are shown in Fig-ure 3. Each mode represents a flow structure that beats with one single frequency, andthe superposition of several of these modes results in the quasiperiodic global system.The high-frequency mode 2 (Figure 3(a)) can be associated with the shear layer vortices;along the jet trajectory there is first a formation of ring-like vortices that eventuallydissolve into smaller scales due to viscous dissipation. Also visible are upright vortices:on the leeward side of the jet, there is a significant structure extending towards the wall.This indicates that the shear-layer vortices and the upright vortices are coupled and os-cillate with the same frequency. The spatial structures of modes 4 and 8 are very similarto those of mode 2, as one expects, since the frequencies are very close.

On the other hand, the low-frequency mode 6 shown in figure 3(b) features large-scale positive and negative streamwise velocity near the wall, which can be associatedwith shedding of the wall vortices. However, this mode also has structures along the jettrajectory further away from the wall. This indicates that the shedding of wall vorticesis coupled to the jet body, i.e. the low frequency can be detected nearly anywhere in thevicinity of the jet since the whole jet is oscillating with that frequency.

4.2. Comparison with linear global modes and POD modes

The linear global eigenmodes of the Navier-Stokes equations linearized about an unstablesteady state solution were computed by Bagheri et al. (2009) for the same flow parametersas the current study. They computed 22 complex conjugate unstable modes using theArnoldi method combined with a time-stepper approach. The frequency of the mostunstable (anti-symmetric) mode associated with the shear-layer instability was St =0.169, not far from the value St = 0.14 observed for the DNS. However, the mode withthe lowest frequency associated with the wall vortices was St = 0.043, quite far fromthe observed frequency of St = 0.017. These linear frequencies can capture the dynamicsonly in an neighborhood of the unstable fixed point, while the Koopman modes correctlycapture the behavior on the attractor.

We also compared the Koopman modes with modes determined by Proper OrthogonalDecomposition (POD) of the same dataset, and although we do not show the resultsexplicitly, we comment on the main similarities and differences. The POD modes capturesimilar spatial structures, but the most striking distinction is in the time coefficients:

Spectral analysis of nonlinear flows 9

(a) (b)

Figure 3. Positive (red) and negative (blue) contour levels of the streamwise velocity compo-nents of two Koopman modes. The wall is shown in gray. (a) Mode 2, with �v2� = 400 andSt2 = 0.141. (b) Mode 6, with �v6� = 218 and St6 = 0.0175.

are compared to the frequencies obtained directly from the Ritz eigenvalues (red verticallines). The shedding frequencies and a number of higher harmonics are in very goodagreement with the frequencies of the Koopman modes. In particular, the dominantKoopman eigenvalues match the frequencies for the wall mode (St = 0.017) and theshear-layer mode (St = 0.14). Note that the probe signals are local measures of thefrequencies at one spatial point, whereas the Koopman eigenvalues correspond to globalmodes in the flow with time-periodic motion.

The streamwise velocity component u of Koopman modes 2 and 6 are shown in Fig-ure 3. Each mode represents a flow structure that beats with one single frequency, andthe superposition of several of these modes results in the quasiperiodic global system.The high-frequency mode 2 (Figure 3(a)) can be associated with the shear layer vortices;along the jet trajectory there is first a formation of ring-like vortices that eventuallydissolve into smaller scales due to viscous dissipation. Also visible are upright vortices:on the leeward side of the jet, there is a significant structure extending towards the wall.This indicates that the shear-layer vortices and the upright vortices are coupled and os-cillate with the same frequency. The spatial structures of modes 4 and 8 are very similarto those of mode 2, as one expects, since the frequencies are very close.

On the other hand, the low-frequency mode 6 shown in figure 3(b) features large-scale positive and negative streamwise velocity near the wall, which can be associatedwith shedding of the wall vortices. However, this mode also has structures along the jettrajectory further away from the wall. This indicates that the shedding of wall vorticesis coupled to the jet body, i.e. the low frequency can be detected nearly anywhere in thevicinity of the jet since the whole jet is oscillating with that frequency.

4.2. Comparison with linear global modes and POD modes

The linear global eigenmodes of the Navier-Stokes equations linearized about an unstablesteady state solution were computed by Bagheri et al. (2009) for the same flow parametersas the current study. They computed 22 complex conjugate unstable modes using theArnoldi method combined with a time-stepper approach. The frequency of the mostunstable (anti-symmetric) mode associated with the shear-layer instability was St =0.169, not far from the value St = 0.14 observed for the DNS. However, the mode withthe lowest frequency associated with the wall vortices was St = 0.043, quite far fromthe observed frequency of St = 0.017. These linear frequencies can capture the dynamicsonly in an neighborhood of the unstable fixed point, while the Koopman modes correctlycapture the behavior on the attractor.

We also compared the Koopman modes with modes determined by Proper OrthogonalDecomposition (POD) of the same dataset, and although we do not show the resultsexplicitly, we comment on the main similarities and differences. The POD modes capturesimilar spatial structures, but the most striking distinction is in the time coefficients:

8 C.W. Rowley, I. Mezic, S. Bagheri, P. Schlatter, and D.S. Henningson

!1 !0.5 0 0.5 1!1

!0.5

0

0.5

1

0 0.03 0.06 0.09 0.12 0.15 0.18 0.21 0.240

100

200

300

400

(a) (b)

St

�vj�

Re{λj}

Im{λj}

Figure 2. (a) The empirical Ritz values λj . The value corresponding to the first Koopman modeis shown with the blue symbol. (b) The magnitudes of the Koopman modes (expect the firstone) at each frequency. In both figures, the colors vary smoothly from red to white, dependingon the magnitude of the corresponding mode.

spectrum shows the frequency content u1(ω) of u1(t). The peak frequency corresponds

to a vortex shedding of wake vortices with the Strouhal number St ≡ fD/Vjet = 0.0174.

In figure 1(d,f), a second probe located a few jet diameters along the jet trajectory

x2P = (12, 6, 2), shows a second oscillation that can be identified with the shedding of

the shear-layer vortices. The peak frequency beats with St = 0.141 which is nearly one

order of magniture larger than the low-frequency mode. Note that the peak frequencies

of the power spectra vary slightly depending on the location of the probe.

4.1. Koopman modes and frequencies

In this section we compute the Koopman modes and show that they directly allow an

identification of the various shedding frequencies. The empirical Ritz values λj and the

empirical vectors vj of a sequence of flow-fields {u0,u1, . . . ,um−1} = {u(t = 200),u(t =

202), . . . ,u(t = 700)} with m = 251 are computed using the algorithm described earlier.

Thus, the transient time (t < 200) is not sampled and only the asymptotic motion in

phase space is considered.

Figure 2(a) shows that nearly all the Ritz values are on the unit circle |λj | = 1

indicating that the sample points ui lie on or near an attracting set. The Koopman

eigenvalue corresponding to the first Koopman mode is the time-averaged flow and is

depicted with blue symbol in figure 2(a). This mode, shown in figure 1(b), captures the

steady flow structures as discussed previously. In figure 2(a), the other (unsteady) Ritz

values vary smoothly in color from red to white, depending on the magnitude of the

corresponding Koopman mode. The magnitudes defined by the global energy norm �vj�,and are shown in figure 2(b) with the same coloring as the spectrum. In figure 2(b) each

mode is displayed with a vertical line scaled with its magnitude at its corresponding

frequency ωj = Im{log(λj)}/∆t (with ∆t = 2 in our case). Only the ωj ≥ 0 are shown,

since the eigenvalues come in complex conjugate pairs. Ordering the modes with respect

to their magnitude, the first (2-3) and second (4-5) pair of modes oscillate with St2 =

0.141 and St4 = 0.136 respectively, whereas the third pair of modes (6-7) oscillate with

St6 = 0.017. All linear combinations of the frequencies excite higher modes, for instance,

the nonlinear interaction of the first and third pair results in the fourth pair, i.e. St8 =

0.157 and so on.

In figures 1(e) and (f) the power spectra of the two DNS time signals (black lines)

Mixing by injection of fluid through a hole into a steady cross-flow.

Ritz values: approx. Koopman spectrum8 C.W. Rowley, I. Mezic, S. Bagheri, P. Schlatter, and D.S. Henningson

!1 !0.5 0 0.5 1!1

!0.5

0

0.5

1

0 0.03 0.06 0.09 0.12 0.15 0.18 0.21 0.240

100

200

300

400

(a) (b)

St

�vj�

Re{λj}

Im{λj}

Figure 2. (a) The empirical Ritz values λj . The value corresponding to the first Koopman modeis shown with the blue symbol. (b) The magnitudes of the Koopman modes (expect the firstone) at each frequency. In both figures, the colors vary smoothly from red to white, dependingon the magnitude of the corresponding mode.

spectrum shows the frequency content u1(ω) of u1(t). The peak frequency corresponds

to a vortex shedding of wake vortices with the Strouhal number St ≡ fD/Vjet = 0.0174.

In figure 1(d,f), a second probe located a few jet diameters along the jet trajectory

x2P = (12, 6, 2), shows a second oscillation that can be identified with the shedding of

the shear-layer vortices. The peak frequency beats with St = 0.141 which is nearly one

order of magniture larger than the low-frequency mode. Note that the peak frequencies

of the power spectra vary slightly depending on the location of the probe.

4.1. Koopman modes and frequencies

In this section we compute the Koopman modes and show that they directly allow an

identification of the various shedding frequencies. The empirical Ritz values λj and the

empirical vectors vj of a sequence of flow-fields {u0,u1, . . . ,um−1} = {u(t = 200),u(t =

202), . . . ,u(t = 700)} with m = 251 are computed using the algorithm described earlier.

Thus, the transient time (t < 200) is not sampled and only the asymptotic motion in

phase space is considered.

Figure 2(a) shows that nearly all the Ritz values are on the unit circle |λj | = 1

indicating that the sample points ui lie on or near an attracting set. The Koopman

eigenvalue corresponding to the first Koopman mode is the time-averaged flow and is

depicted with blue symbol in figure 2(a). This mode, shown in figure 1(b), captures the

steady flow structures as discussed previously. In figure 2(a), the other (unsteady) Ritz

values vary smoothly in color from red to white, depending on the magnitude of the

corresponding Koopman mode. The magnitudes defined by the global energy norm �vj�,and are shown in figure 2(b) with the same coloring as the spectrum. In figure 2(b) each

mode is displayed with a vertical line scaled with its magnitude at its corresponding

frequency ωj = Im{log(λj)}/∆t (with ∆t = 2 in our case). Only the ωj ≥ 0 are shown,

since the eigenvalues come in complex conjugate pairs. Ordering the modes with respect

to their magnitude, the first (2-3) and second (4-5) pair of modes oscillate with St2 =

0.141 and St4 = 0.136 respectively, whereas the third pair of modes (6-7) oscillate with

St6 = 0.017. All linear combinations of the frequencies excite higher modes, for instance,

the nonlinear interaction of the first and third pair results in the fourth pair, i.e. St8 =

0.157 and so on.

In figures 1(e) and (f) the power spectra of the two DNS time signals (black lines)

• DNS• Observable: pointwise (Eulerian)

velocities• 256 X 201 X 144 gridpoints• 250 time snapshots

Strouhal No. ～ Mode freq.

Koopman Modemagnitudes


Coherent features in physical space


Coherent features in the state space

89

Analytic (visualization):• 3D: plotting trajectories is messy

• 3D+time: even Poincaré map is 3D

• nD: how do you take a 2D slice?

• in nD – how do you take a 2D slice?

Design:It matters WHAT we identify as coherent structure.Ergodic invariant sets identify regions between which mixing is slow/nonexistent.

Visualization: Ergodic Partition

f(x) := limN→∞

1N

N−1�

n=0

f(Tn(x))

Uf = fP 0T f = f

(x, y) ∈ T2, ε ∈ [0, 1)

x+ = x + ε sin 2πy

y+ = x + y + ε sin 2πy


Level sets of a TAF form a stationary partition of the state space.

Trajectory-averaged functions (TAF)

[Mezić, Wiggins, 1999]


Choice of observables is not physically motivated.

f(x) := limN→∞

1N

N−1�

n=0

f(Tn(x))

Uf = fP 0T f = f

(x, y) ∈ T2, ε ∈ [0, 1)

x+ = x + ε sin 2πy

y+ = x + y + ε sin 2πy


Level sets of a TAF form a stationary partition of the state space.

Averaging different observables reveals different invariant sets in the state space.

Trajectory-averaged functions (TAF)

[Mezić, Wiggins, 1999]


Choice of observables is not physically motivated.

f!

1

f! 2

-0.5 0 0.5-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6


f1(x, y) = cos(2πx + 7πy)

f2(x, y) = cos(9πx + πy)[Levnajić, Mezić, 2010]f∗

1

f∗2


f!

1

f! 2

-0.5 0 0.5-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6




1

f∗2


f!

1

f! 2

-0.5 0 0.5-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6




1

f∗2


f!

1

f! 2

-0.5 0 0.5-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6




1

f∗2


f!

1

f! 2

-0.5 0 0.5-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6




1

f∗2



f!

1

f! 2

-0.5 0 0.5-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

Ergodic partition: partition into smallest invariant sets – ergodic sets.

Ergodic quotient: representation of the ergodic partition as a subset of a sequence space.

Any coherent structure will be

made up of ergodic sets

Sequences are “easy”, just

remember Fourier coefficients.


limit with infinity of

observables

limit with infinity of

observables

[Budišić, Mezić, submitted to Physica D, 2012]


Swirled Unsteady Hill’s VortexHill’s Vortex

Swirl

Perturbation

(R, z, θ) ∈ R+ × R× T

Rzθ

=

2Rz

1− 4R− z2c2R

+ ε

√2R sin θz√2R

sin θ

2 cos θ

sin 2πt

ε = c > 0


[Budišić, Mezić, see poster presentation]


Small perturbation: KAM behavior


Swirl

Perturbation

(R, z, θ) ∈ R+ × R× T

Rzθ

=

2Rz

1− 4R− z2c2R

+ ε

√2R sin θz√2R

sin θ

2 cos θ

sin 2πt

ε = c > 0




Small perturbation: KAM behavior


Swirl

Perturbation

(R, z, θ) ∈ R+ × R× T

Rzθ

=

2Rz

1− 4R− z2c2R

+ ε

√2R sin θz√2R

sin θ

2 cos θ

sin 2πt

ε = c > 0

ε = c = 0.2800Large perturbation: new bifurcation uncovered

R

zθ




• Choice of observables is physically motivated: Lagrangian velocity.

• Averaging time is finite.

Mesohyperbolicity

Poje, Haller, Mezić, Phys. Flu. , 1999

Mesohyperbolicity

x(t0)Φ

t0+Tt0−−−−→ x(t0 + T )

x(t0 + T ) = x(t0) + T v(x0, t0, T )

J(x0, t0, T ) = DΦt0+Tt0 (x)

∇v(x0, t0, T )


Mesohyperbolicity

x(t0 + T ) = x(t0) +

� t0+T

t0

v[t0 + τ,Φτt0(x0)]dτ Finite time transport

Averaged velocity

“Time-sampling”: from ODE to a map using averaged velocity

Defines hyperbolicity/ellipticity.

v(x0, t0, T ) =1

T

� t0+T

t0

v[t0 + τ,Φτt0(x0)]dτ

Goal: deduce properties of the finite-time flow, by studying only averaged velocity.

Practically accessible.


Mesohyperbolicity

J(x0, t0, T ) = DΦt0+Tt0 (x)

“meso” – mean, average“mesohyperbolic” – hyperbolic on average“mesoelliptic” – elliptic on average

Essentially, a finite-time analogues of known behaviors.

1

i

Mesohyperbolicity

Mesoellipticity

Mesoellipticity

Incompressibility constrains the spectrum.

Spectral properties of


Mesohyperbolicity

J(x0, t0, T ) = DΦt0+Tt0 (x)



1

i

Mesohyperbolicity

Mesoellipticity

Mesoellipticity



(T 2 det∇v(x0, t0, T )− 4) det∇v(x0, t0, T ) > 0

(T 2 det∇v(x0, t0, T )− 4) det∇v(x0, t0, T ) < 0det∇v

4

T 20

∇v(x0, t0, T )

Mesoellipticity

Mesohyperbolicity

Mesoellipticity

MesohyperbolicityMesohyperbolicity

Translated to


Compared to LCS, provides distinction between elliptic/hyperbolic.

Mesohyperbolicity

J(x0, t0, T ) = DΦt0+Tt0 (x)



1

i

Mesohyperbolicity

Mesoellipticity

Mesoellipticity



(T 2 det∇v(x0, t0, T )− 4) det∇v(x0, t0, T ) > 0

(T 2 det∇v(x0, t0, T )− 4) det∇v(x0, t0, T ) < 0det∇v

4

T 20

∇v(x0, t0, T )

Mesoellipticity

Mesohyperbolicity

Mesoellipticity

MesohyperbolicityMesohyperbolicity

Translated to


Mesohyperbolicity

“Sanity check”: cellular gyres

Steady

Unsteady

Mesoellipticity

Mesohyperbolicity

Mesohyperbolicity

Mesoellipticity

Mesohyperbolicity

Mesohyperbolicity

det∇v


With P. Hogan – ONR Stennis, Science (Express), 2010

Success story: Gulf Oil Spill transport route prediction

Mesohyperbolicity


Ergodicity and mixing: From yes/no to continuous indicators

18

Controlling Mixing and Ergodicity

Example: ErgodicityClassically: two trajectories are in the same ergodic set if for any given observable there is no difference in averages along trajectories.

Continuous criterion: measure how much different the time-averages are.

Time

Question: how do we quantify “how much different”?Cf. work with Scott, Redd, Kuznetsov, Jones, Physica D 238, 1668-1679


Criterion: Sobolev norms

19


WNf(x) =1

N

N−1�

n=0

f(Tn(x))Finite-time averages of observables

WNf(x)N→∞−−−−→ �f, µx�

FTA are weak representatives of empirical measures

WNf(x)− �f, ν�We can compare them to “target” measure we want to sample

c2(t) =�

k∈Zd

Λ(k)s|WNfk(x)− �fk, ν� |2

fk = exp(ik · x) Λ(k) = (1 + |k|2)Observables: Weight:

Choice of the target measure and negative order s determine the effect of the criterion.

c(t)t−→ 0Sobolev norm can be used as the continuous criterion:


Application: Searching an area by UAVs

20

Second-order dynamics

Optimal feedback criterion in a closed form

Future: apply it to navigation of ocean/sea drifters



Acknowledgments: Zoran Levnajić, M. Budišić (visualization), George Mathew (Ergodicity and Mixing)L. Petzold, U.Vaidya, S. Grivopoulos, F. Bottausci (microfluidics experiment)Funding: ONR, AFOSR, DARPA

21

Summary and Conclusions

New visualization techniques based on spectral properties of the Koopman operator.

Meso-____: criteria for finite-time analogues for hyperbolicity and ellipticity.

Moving away from YES/NO concepts of ergodicity and mixing to continuous analogues: opportunities for control.

Koopman has a place in dynamical transport!

(just Google it)

Documents

I. Mezic: Spectral Theory of Nonlinear Fluid Flows Based on the Koopman Operator