Embed Size (px)
Citation preview
John DoyleControl and Dynamical
Systems Caltech
Theory of
Complex
Networks
FinanceTransportation
Energy
Information
Consumer
Utilities
Manufacturing
Commerce
Health
Our lives are run by/with networks
Emergency
ConvergentNetworks
Transportation
Energy
Information
Consumer
Manufacturing
CommerceHealth
Environment
Emergency
Finance
Utilities
Convergent networking: the promise
Ubiquitous computing, communications, and
control• that is embedded and
intertwined• via sensors and actuators • in complex networks of
networks, with layers of protocols and feedback.
Resulting in:
• Seamless integration and automation of everything
• Efficient and economic operation
• Robust and reliable services
ConvergentNetworks
Transportation
Energy
Information
ConsumerManufacturing
CommerceHealth
Environment
Emergency
Finance
Utilities
Convergent networking: the reality
• Right now, back in Los Angeles, we can experience (in addition to smog, earthquakes, fires, floods, riots, lawyers,…)– Widespread and prolonged power outages from lightning strikes in
Washington (or just “nonequilibrium market fluctuations”).
– Widespread and prolonged flight delays from weather or ATC software glitches in Chicago or Atlanta.
– Internet meltdowns caused by hackers in Moscow.
– Financial meltdowns caused by brokers in Singapore.
• What can we expect?– Widespread and prolonged meltdowns of integrated power,
transportation, communication, and financial networks caused by lightning strikes in Singapore or a new release of MS Windows 2020?
Elements of systems
• Sense the environment and internal state
• Extract what’s novel
• Communicate or store what’s novel
• Extract what’s useful
• Compute decisions based on what’s useful
• Take action
• Evaluate consequences
• Repeat
DataIs not novel informationIs not useful Information
Is not knowledge Is not understanding
Is not wisdomIs not action Is not results
HARDER
We want results
Two great abstractions of the 20th Century
1. Separate systems engineering into control, communications, and computing
– Theory
– Applications
2. Separate systems from physical substrate• Facilitated massive, wildly successful, and explosive
growth in both mathematical theory and technology…
• …but creating a new Tower of Babel where even the experts do not read papers or understand systems outside their subspecialty.
Tower of Babel
• Issues for theory– Rigor– Relevance– Accessibility
• Spectacular success on the first two• Little success on the last one, which is critical for a
multidisciplinary approach to systems biology• Perhaps all three is impossible?• (In contrast, there are whole research programs in
“complex systems” devoted exclusively to accessibility. They have been relatively “popular,” but can be safely ignored in biology.)
Biology and advanced technology
• Biology– Integrates control, communications, computing– Into distributed control systems– Built at the molecular level
• Advanced technologies will do the same• We need new theory and math, plus
unprecedented connection between systems and devices
• Two challenges for greater integration:– Unified theory of systems– Multiscale: from devices to systems
Compute
Communicate Communicate
StoreCommunicate
Communications and computing
Compute
Sense
EnvironmentEnvironment
Act
Communicate Communicate
StoreCommunicate
Computation
Devices
Dynamical SystemsDynamical Systems
DevicesCommunication Communication
Control
From• Software to/from human• Human in the loop
To• Software to Software• Full automation• Integrated control,
comms, computing• Closer to physical
substrateCompute
Communicate Communicate
Store
Communicate
Computation
Devices
Dynamical SystemsDynamical Systems
Devices
Communication Communication
Control
• New capabilities & robustness• New fragilities & vulnerabilities
Theoretical foundations
• Computational complexity: decidability, P-NP-coNP
• Information theory: source and channel coding
• Control theory: feedback, optimization, games
• Dynamical systems: dynamics, bifurcation, chaos
• Statistical physics: phase transitions, critical phenomena
• Unifying theme: uncertainty management
• Different abstractions and relaxations
• Integrating these theories involves new math, much not traditionally be viewed as “applied,” e.g..– Perturbation theory of operator Banach algebras
– Semi-algebraic geometry
Uncertainty management
• Each domain faces similar abstract issues and tradeoffs, but with differing details:
• Sources of uncertainty
• Limited resources
• Robust strategies
• Fundamental tradeoffs
• Ignored issues
Control theory
• Sources of uncertainty: plant uncertainty and sensor noise
• Limited resources: sensing, actuation, and computation
• Robust strategies: feedback control and related methods
• Fundamental tradeoffs: Bode’s integral formula, RHP zeros, saturations, …
• Ignored issues: communications in distributed control, software reliability
Information theory
• Sources of uncertainty: source and channel
• Limited resources: storage, bandwidth, and computation
• Robust strategies: coding
• Fundamental tradeoffs: capacity, rate-distortion
• Ignored issues: feedback and dynamics
Computation complexity
• Sources of uncertainty: intractability, problem instance
• Limited resources: computer time and space
• Robust strategies: algorithms
• Fundamental tradeoffs: P/NP/Pspace/undecidable
• Ignored issues: real-time, uncertainty in physical systems
Software correctness
• Sources of uncertainty: bugs, user inputs
• Limited resources: computer time and space
• Robust strategies: formal verification
• Fundamental tradeoffs: computational complexity
• Ignored issues: real-time, uncertainty in physical systems
Multiscale physics
• Sources of uncertainty: initial conditions, unmodeled dynamics, quantum mechanics
• Limited resources: computer time and space, measurements
• Robust strategies: coarse graining, renormalization??
• Fundamental tradeoffs: energy/matter, entropy, quantum, etc…
• Ignored issues: robustness, rigor, computation, etc• (This looks mostly fixable.)
Unified theory of uncertainty management
• Sources of uncertainty: plant, multiscale physics, sensors, channels, bugs, user inputs
• Limited resources: computer time and space, energy, materials, bandwidth, actuation
• Robust strategies: ??
• Fundamental tradeoffs: ??
• Ignored issues: human factors
Progress
• Unified view of web and internet protocols– Good place to start
– Add feedback and dynamics to communications
– Observations: fat tails (Willinger)
– Theory: Source coding and web layout (Doyle)
– Theory: Channel coding and congestion control (Low)
• Unified view of robustness and computation– Anecdotes from engineering and biology
– New theory (especially Parrilo)
– Not enough time today…
Bonus!
• “Unified systems” theory helps resolve fundamental unresolved problems at the foundations of physics
• Ubiquity of power laws (statistical mechanics)• Shear flow turbulence (fluid dynamics)• Macro dissipation and thermodynamics from micro
reversible dynamics (statistical mechanics)• Quantum-classical transition• Quantum measurement• Thus the new mathematics for a unified theory of
systems is directly relevant to multiscale physics• The two challenges (unify and multiscale) are
connected.
Network protocols.
HTTP
TCP
IP
Routers
Files
packetspacketspacketspacketspacketspackets
Web servers
web traffic
Is streamed out on the net.
Creating internet traffic
Webclient
Web/internet traffic
Web servers
web traffic
Is streamed out on the net.
Creating internet traffic
Webclient
Let’s look at some web traffic
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
Frequency
Decimated dataLog (base 10)
Forest fires1000 km2
(Malamud)
WWW filesMbytes
(Crovella)
Data compression
(Huffman)
Los Alamos fire
Cumulative
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
FrequencyFires
Web filesCodewords
Cumulative
Log (base 10)
-1/2
-1
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
Frequency
Decimated dataLog (base 10)
Forest fires1000 km2
(Malamud)
WWW filesMbytes
(Crovella)
Data compression
(Huffman)
Los Alamos fire
Cumulative
>1e5 files
>4e3 fires
10-2
10-1
100
100
101
102
20th Century’s 100 largest disasters worldwide
US Power outages (10M of customers)
Natural ($100B)
Technological ($10B)
10-2
10-1
100
100
101
102
Log(Cumulative frequency)
Log(size)
= Log(rank)
0 2 4 6 8 10 12 140
20
40
60
80
100
size
rank
Natural ($100B)
Technological ($10B)
100
101
102
1
2
3
10
100
10-2
10-1
100
Log(size)
Log(rank)
100
101
102
20th Century’s 100 largest disasters worldwide
US Power outages (10M of customers)
Natural ($100B)
Technological ($10B)
Slope = -1(=1)
10-2
10-1
100
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
Frequency
Decimated dataLog (base 10)
Forest fires1000 km2
WWW filesMbytes
Data compression
Cumulative
-1/2
-1
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
Frequency Forest fires1000 km2
WWW filesMbytes
Data compression
Cumulative
-1/2
-1
exponential
100
101
102
103
10-4
10-3
10-2
10-1
100
loglog
.5
1
semilogy
20 40 60 80 100
0.2
0.6
1
linearPlotting power laws
and exponentials
exp
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
Frequency Forest fires1000 km2
WWW filesMbytes
Data compression
Cumulative
exponential
All events are close in size.
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
Frequency Forest fires1000 km2
WWW filesMbytes
Data compression
Cumulative
-1/2
-1
Most events are small
But the large events are huge
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
Frequency Forest fires1000 km2
WWW filesMbytes
Data compression
Cumulative
-1/2
-1
Most events are small
But the large events are huge
Robust
Yet Fragile
Robustness of HOT systems
Robust
Fragile
Robust(to known anddesigned-foruncertainties)
Fragile(to unknown
or rareperturbations)
Uncertainties
Large scale phenomena is extremely non-Gaussian
• The microscopic world is largely exponential
• The laboratory world is largely Gaussian because of the central limit theorem
• The large scale phenomena has heavy tails (fat tails) and power laws
Size of events x vs. frequency
log(size)
)1()( xxpdx
dPlog(probability)
log(Prob > size)
xPlog(rank)
-1 0 1 2 3 4 5
0
-1
-2
-3
-4
log10(x)
log10(P)
x integer
1e3 samples from a known distribution:
x10
10
=1
xP
10( )
10P X x
x
)( xXP
Slope = -
=1
=0
=1
=0Cumulative Distributions
dx
dPxp )(
Slope = -(+1)Noncumulative
Densities
=0
=1
Cumulative Distributions
NoncumulativeDensities
Correct
Wrong
=0
The physics view
• Power laws are “suggestive of criticality”• Self-organized criticality (SOC)• Examples where this holds:
– Phase transitions in lab experiments– Percolation models– Rice pile experiments
• No convincing examples in technology, biology, ecology, geophysical, or socio-economic systems
• Special case of “new science of complexity”• Complexity “emerges” at a phase transition or
bifurcation “between order and disorder.”• Doesn’t work outside the lab.
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
WWWDC
Data + Model/Theory
Forest fire
SOC = .15
SOC = .15
= .15Cumulative distributions
=.15Noncumulative densities, logarithmic binning
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
FrequencyFires
Web filesCodewords
Cumulative
Log (base 10)
-1/2
-1
The HOT view of power laws(w/ Jean Carlson, UCSB)
• The central limit theorem gives power laws as well as Gaussians
• Many other mechanisms (eg multiplication noise) yield power laws
• A model producing a power law is per se uninteresting• A model should say much more, and lead to new
experiments and improved designs, policies, therapies, treatments, etc.
The HOT view of power laws
• Engineers design (and evolution selects) for systems with certain typical properties:
• Optimized for average (mean) behavior
• Optimizing the mean often (but not always) yields high variance and heavy tails
• Power laws arise from heavy tails when there is enough aggregate data
• One symptom of “robust, yet fragile”
HOT and fat tails?
• Surprisingly good explanation of statistics (given the severity of the abstraction)
• But statistics are of secondary importance
• Not mere curve fitting, insights lead to new designs
• Understanding design
Examples of HOT fat tails?
• Power outages• Web/Internet file traffic• Forest fires• Commercial aviation delays/cancellations• Disk files, CPU utilization, …• Deaths or dollars lost due to man-made or natural
disasters?• Financial market volatility?• Ecosystem and specie extinction events?• Other mechanisms, examples?
Detailed simulations
Examples with additional mechanisms?
• Word rank (Zipf’s law)
• Income and wealth of individuals and companies
• Citations, papers
• Social and professional networks
• City sizes
• Many others….
• (Simon, Mandelbrot, …)
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
WWWDC
Data
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
WWWDC
Data + Model/Theory
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
Frequency
Decimated dataLog (base 10)
WWW filesMbytes
(Crovella)
Cumulative Most files are small
(mice)
Most packets are in large files (elephants)
NetworkNetwork
Sources
Mice
Elephants
Router queues
NetworkNetwork
Sources
Mice
Elephants
Router queues
Delay sensitive
Bandwidth sensitive
Log(bandwidth)
Log(delay)
cheap
Expensive
• We’ll focus to begin with on similar tradeoffs in internetworking between bandwidth and delay. • We’ll assume TCP (via retransmission) eliminates loss, and will return to this issue later.
Delay
BW
BW = Bandwidth sensitive trafficDelay = Delay sensitive traffic
Log(bandwidth)
Log(delay)
Delay
BWBulk transfers (most packets)
Web navigation, voice (most files)
• Mice: many small files of few packets which the user presumably wants ASAP• Elephants: few large files of many packets for which average bandwidth will be more important than individual packet delay• Most files are mice but most packets are in elephants…•…which is the manifestation of fat tails in the web and internet.
Log(bandwidth)
Log(delay)
Delay
BWBulk transfers (most packets)
Web navigation, voice (most files)
Claim I: Current traffic dominated by these two types of flows
Claim II: Intrinsic feature of many future network applications
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
WWWDC
Data
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
WWWDC
Data + Model/Theory
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
Frequency
Decimated dataLog (base 10)
WWW filesMbytes
(Crovella)
Cumulative Most files are small
(mice)
Most packets are in large files (elephants)
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
Frequency
WWW filesMbytes
Data compression
Cumulative
exponential
All events are close in size.
Source coding for data compression
Based on frequencies of source word occurrences,
Select code words.
To minimize message length.
Source coding for data compression
Objectives:• Optimally compress file• Tractable compression• Tractable decompression
Shannon:• Optimally compress ensemble• Tractable compression• Tractable decompression
Kolmogorov:• Optimally compress file• Undecidable compression• Intractable decompression
• Surprise: natural and practical• Stochastic relaxation
• Philosophically important• Turing, Godel, Chaitin, …
Shannon coding
• Ignore value of information, consider only “surprise”• Compress average codeword length (over stochastic
ensembles of source words rather than actual files)• Constraint on codewords of unique decodability• Equivalent to building barriers in a zero dimensional tree• Optimal distribution (exponential) and optimal cost are:
DataCompression
length log( )
exp( )i i
i i
l p
p cl
Avg. length =
log( )
i i
i i
p l
p p
Shannon source coding
1i i iJ p l r Minimize expected
length
source words with probabilities pi
length of codewords li
unique decodability
2 11
log( )
il
i
i i
rl r
Kraft’s inequality
2 11
log( )
il
i
i i
rl r
Kraft’s inequality =Prefix-less code
0
110
11
100
101
110
111
1110
1111
11100
1110111110
11111
010010111011100111011111011111
Codewords
2 1il
2 11
log( )
il
i
i i
rl r
0
110
11
100
101
110
111
1110
1111
11100
1110111110
11111
010010111011100111011111011111
Codewords
2 1il
0 dimensional (discrete) tree
Kraft’s inequality =Prefix-less code
cut in a 0-dim tree
2 1il
Kraft’s inequality =Prefix-less code
Channel noise
Coding = building barriers
Source coding Channel coding
Control = building barriers
( ) log( )l r r
log( )i il p
1i i iJ p l r
Leads to optimal solutions for codeword lengths.
With optimal cost log( )i iJ p p
Minimize
Equivalent to optimal barriers on a discrete tree (zero dimensional).
( ) log( )l r r
log( )i il p
1i i iJ p l r
log( )i iJ p p • Compressed files look like white noise.• Compression improves robustness to limitations in
resources of bandwidth and memory.• Compression makes everything else much more fragile:
– Loss or errors in compressed file– Statistics of source file
• Information theory also addresses these issues at the expense of (much) greater complexity
0 1 2-1
0
1
2
3
4
5
6
DC
Data
Avg. length =
log( )
i i
i i
p l
p p
How well does the model predict the data?
length log(
exp( )
)i i
i i
l p
p cl
0 1 2-1
0
1
2
3
4
5
6
DC
Data + Modellength log(
exp( )
)i i
i i
l p
p cl
Avg. length =
log( )
i i
i i
p l
p p
How well does the model predict the data?
Not surprising, because the file was compressed using
Shannon theory.
Small discrepancy due to integer lengths.
Why is this a good model?
• Lots of models will reproduce an exponential distribution
• Shannon source coding lets us systematically produce optimal and easily decodable compressed files
• Fitting the data is necessary but far from sufficient for a good model
Web layout as generalized “source coding”
• Keep parts of Shannon abstraction:– Minimize downloaded file size– Averaged over an ensemble of user access
• Equivalent to building 0-dimensional barriers in a 1- dimensional tree of content
document
split into N files to minimize download time
A toy website model(= 1-d grid HOT design)
# links = # files
Optimize 0-dimensional cuts in a 1-dimensional document
More complete website models
(Zhu, Yu, Effros)
• Necessary for web layout design• Statistics consistent with simpler models• Improved protocol design (TCP)• Commercial implications still unclear
Generalized “coding” problems
Web
Data compression
• Optimizing d-1 dimensional cuts in d dimensional spaces…
• To minimize average size of files • Models of greatly varying detail all give a consistent
story.• Power laws have 1/d.• Completely unlike criticality.
PLR optimization
RrlpJ iiiMinimize
expected loss
P: uncertain events with probabilities pi
L: with loss li
R: limited resources ri
P L R
DC source codewords decodability
WWW user access files web layout
document
split into N files to minimize download time
1l r r = density of links or filesl = size of files
pi = Probability
of event
drl
li = volume enclosed
ri = barrier density
d-dimensional
i
d
id
i lrl
1
,
Resource/loss relationship:
1)(
r
crl
RrlpJ iii
PLR optimization
d
= 0 data compression = 1 web layout
= “dimension”
RrlpJ iii
PLR optimization
= 0 data compression
=0 is Shannon
source coding
0
0
1
)log(
)(
r
c
r
rl
0
0)log()log(
1
1
1
ii
iiii
pRc
ppRpp
J
p
01
0)log()(
r
cr
rl
1
)1/(1
)1/(1
j
j
ii
p
Rpcl
RrlpJ iii
11
ipcR
Minimize average cost using standard Lagrange multipliers
With optimal cost
Leads to optimal solutions for resource allocations and the relationship between the event probabilities and sizes.
01
0)log()(
r
cr
rl
1
)1/(1
)1/(1
j
j
ii
p
Rpcl
RrlpJ iii
11
ipcR
Minimize average cost using standard Lagrange multipliers
With optimal cost
Leads to optimal solutions for resource allocations and the relationship between the event probabilities and sizes.
011
0)log(
1
1
1
i
ii
p
pp
J
1
)1/(1
)1/(1
j
j
ii
p
Rpcl
To compare with data.
ip
Forward engineering
il ir
Reverse engineering
(1 1/ )
1 2
ˆ i
i
p
c l c
1
)1/(1
)1/(1
j
j
ii
p
Rpcl
ip il ir
Reverse engineering
To compare with data.
(1 1/ )
1 2
ˆ i
i
p
c l c
1
)1/(1
)1/(1
j
j
ii
p
Rpcl
1
ˆ
ˆi
i i ik i
P
p l l
Cumulative
ii Pl ˆ,plot
sizesfromdata
computeusingmodel
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
WWWDC
Data
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
WWWDC
Data + Model/Theory
Typical web traffic
log(file size)
> 1.0log(freq > size)
p s-
Web servers
Heavy tailed web traffic
Is streamed out on the net.
Creating fractal Gaussian internet traffic (Willinger,…)
2
3 H
Fat tail web traffic
Is streamed onto the Internet
creating long-range correlations with 2
3 H
time
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
WWWDC
Data + Model/Theory
Are individual websites distributed like this?
Roughly, yes.
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
WWWDC
Data + Model/Theory
How has the data changed since 1995?
Steeper. Consistent with more use of cross hyperlinks.
More complete website models
(Zhu, Yu, Effros)
• More complex hyperlinks leads to steeper distributions with 1< < 2
• Optimize file sizes within a fixed topology:• Tree: 1• Random graph: 2
• No analytic solutions
The broader Shannon abstraction
• Information = surprise… and therefore ignoring– Value or timeliness of information
– Topology of information
• Separate source and channel coding– Data compression
– Error-correcting codes (expansion)
• Eliminate time and space– Stochastic relaxation (ensembles)
– Asymptopia
• Brilliantly elegant and applicable, but brittle• Better departure point than Kolmogorov, et al
What can we keep?
• Separation:– Source and channel
– Congestion control and error correction
– Estimation and control
• Tractable relaxations– Stochastic embeddings
– Convex relaxations
• Add to information:– Value
– Time and dynamics
– Topology
– Feedback
• More subtle treatment of computational complexity
• Naïve formulations intractable
What must we change?
Log(bandwidth)
Distortion
achievable
notRate distortion theory
studies tradeoffs between bandwidth and distortion
from lossy coding.
Log(bandwidth)
Log(delay)
cheap
Expensive
• We’ll focus to begin with on similar tradeoffs in internetworking between bandwidth and delay. • We’ll assume TCP (via retransmission) eliminates loss, and will return to this issue later.
Delay
BW
BW = Bandwidth sensitive trafficDelay = Delay sensitive traffic
Log(bandwidth)
Log(delay)
Delay
BWBulk transfers (most packets)
Web navigation, voice (most files)
• Mice: many small files of few packets which the user presumably wants ASAP• Elephants: few large files of many packets for which average bandwidth will be more important than individual packet delay• Most files are mice but most packets are in elephants…•…which is the manifestation of fat tails in the web and internet.
Log(bandwidth)
Log(delay)
Delay
BWBulk transfers (most packets)
Web navigation, voice (most files)
Claim I: Current traffic dominated by these two types of flows
Claim II: Intrinsic feature of many future network applications
NetworkNetwork
Sources
Mice
Elephants
Router queues
NetworkNetwork
Sources
Mice
Elephants
Router queues
Delay sensitive
Bandwidth sensitive
Log(bandwidth)
Log(delay)
Delay
BWBulk transfers (most packets)
Web navigation, voice (most files)
Claim (channel): We can tweak TCP using ECN and REM to make these flows co-exist.
Currently: Delays are aggravated by queuing delay and packet drops from congestion caused by BW traffic?
Specifically:• Keep queues empty (ECN/REM). • BW slightly improved (packet loss)• Delay greatly improved (queuing)• Provision network for BW• “Free” QOS for Delay• Network level stays simple
Log(bandwidth)
Log(delay)
Delay
BW
Claim (source): Many (future) applications are natural and intrinsically coded into exactly this kind of fat-tailed traffic.
Expensive
The rare traffic that can’t or won’t will be expensive, and essentially pay for the rest.
Fat tailed traffic is “intrinsic”
• Two types of application traffic are important: communications and control
• Communication to and/or from humans (from web to virtual reality)
• Sensing and/or control of dynamical systems• Claim: both can be naturally “coded” into fat-tailed
BW + delay traffic • This claim needs more research
Log(bandwidth)
Log(delay)
Delay
BW
Expensive
Abstraction I
• Separate source and channel coding• Source is coded into
– Delay sensitive mice
– Bandwidth sensitive elephants
• “Channel coding” = congestion control
Log(bandwidth)
Log(delay)
Delay
BW
Expensive
Putting loss back into the picture
• Packet loss can be handled by coding (application) or retransmission (transport)
• Need coherent theory to perform tradeoffs• Currently, congestion control and reliable transport are
intertwined• What benefits would derive from some decoupling,
enabled by ECN or other explicit congestion control strategies?
Log(BW)
Log(d)
Loss?
Optimization/control framework
• Application specific cost functions J(app,delay,loss,BW) (assume to be minimized)
• Network resources:lines, routers, queues (energy, spectrum, deployment, repair, stealth, security, etc)
• Comm/control network is embedded in other networks (transportation, energy, military action, …)
• Robustness to uncertainties in users and resources• Need to flesh out details for future scenarios
Optimization/control framework• Global optimal allocation sets lower bound on
achievable performance• Control problem is to find decentralized strategies
(eg TCP/IP) with (provably) near optimal performance and robustness in dynamical setting
• Duality theory key to using network • Coding and control interact in unfamiliar ways• Naïve formulations intractable:
– Computation intractable– Requires too much information not available to
decentralized agents
• Key is to find tractable relaxations
Optimization/control framework• Pioneered by Kelly et al and extended by
Low et al.• Ambitious goal: foundation for (much?)
more unified theory of computation, control, and communications
• Hoped for outcome:– Rich theoretical framework– Motivated by practical problems– Yielding principled design of new protocols– And methods for deploying and managing
complex networks
Scalable Congestion Control
LINKS
SOURCES
( )fR s
( )TbR s
ROUTING + DELAY
p : link prices
y : aggregate link flows
x : source rates
q : aggregate prices per source
(Paganini, Doyle & Low ’01)
Robustness, evolvability/scalability, verifiability
Ideal performance
Robustness
Evolvability
Verifiability
Typical design IP
Robustness of HOT systems
Robust
Fragile
Robust(to known anddesigned-foruncertainties)
Fragile(to unknown
or rareperturbations)
Uncertainties
Feedback and robustness
• Negative feedback is both the most powerful and most dangerous mechanism for robustness.
• It is everywhere in engineering, but appears hidden as long as it works.
• Biology seems to use it even more aggressively, but also uses other familiar engineering strategies:– Positive feedback to create switches (digital systems)
– Protocol stacks
– Feedforward control
– Randomized strategies
– Coding
The Internet hourglass
IP
Web FTP Mail News Video Audio ping napster
Applications
TCP SCTP UDP ICMP
Transport protocols
Ethernet 802.11 SatelliteOpticalPower lines BluetoothATM
Link technologies
From Hari Balakrishnan
The Internet hourglass
IP
Web FTP Mail News Video Audio ping napster
Applications
TCP SCTP UDP ICMP
Transport protocols
Ethernet 802.11 SatelliteOpticalPower lines BluetoothATM
Link technologies
From Hari Balakrishnan
Everythingon IP
IP oneverything
Commodities,Hardware
Consumers,Applications
RobustMesoscale
Robust, yet fragile
Hardware
Applications
TCP/IP
UncertaintyUncertainty
UncertaintyUncertaintyCommodities,
Hardware
Consumers,Applications
RobustMesoscale
Robust
Commodities,Hardware
Consumers,Applications
RobustMesoscale
Yet fragile
Difficult to change
Yet fragile
Protocols allow for the creation of large complex networks, with rare but catastrophic cascading failures.
Software
Hardware
Early computing
Analogsubstrate
Variousfunctionality
Digital
Software
Hardware
Hardware
Applications
OperatingSystem
ModernComputing
Robust, yet fragile
Analogelectronics
Variousfunctionality
Digital
Uncertainsubstrate
Variedfunctionality
Robustmesoscale
Commodities
Consumers
Money
Commodities
Consumers
Barter
Commodities
Consumers
Money
Investments
Investors
Markets,Insitutions
The hourglass
Dress Shirt Slacks Lingerie Coat Scarf Tie
Garments
Cloth
Sewing
Wool Cotton NylonRayon Polyester
Material technologies
Producers
Consumers
Energy
Energy
• 110 V, 60 Hz AC• Gasoline• ATP, glucose, etc• Proton motive force
Hardware
Applications
TCP/IP
• Decentralized• Asynchronous
Robust to:• Network topology• Application traffic• Delays, link speeds
High performanceNecessity:
Essentially only onedesign is possible
Hardware
Applications
TCP/IP
• Decentralized• Asynchronous
Robust to:• Network topology• Application traffic• Delays, link speeds
High performanceNecessity:
Essentially only onedesign is possible
The existing designis incredible, but…
It’s a product of evolution,and is not optimal.
1 dimension
All
None
desi
gnControl Theory
Statistical Physics
Dynamical Systems
Information TheoryComputational
Complexity
Theory ofComplex systems?
1 dimension
All
None
desi
gnControl Theory
Statistical Physics
Dynamical Systems
Information TheoryComputational
Complexity
Biology• Non-equilibrium• Highly tuned or optimized• Finite but large dimension
1 dimension
All
None
desi
gnControl Theory
Statistical Physics
Dynamical Systems
Information TheoryComputational
Complexity
Theory needs• Integrated horizontally and vertically• Horizontal: control, communications, computing• Vertical: multiscale physics
• Status: nascent but promise results• Bonus: unexpected synergy
• Ubiquity of power laws• High shear turbulence• Dissipation• Quantum/classical transition• Quantum measurement
1 dimension
All
None
desi
gnControl Theory
Statistical Physics
Dynamical Systems
Information TheoryComputational
Complexity
