Lower Bounds on the Communication of Distributed Graph Algorithms:
Progress and Obstacles
Rotem OshmanADGA 2013
Overview: Network ModelsCONGESTED CLIQUE
ASYNC MESSAGE-PASSING
LOCAL
CONGEST / general network
X
Talk Overview
I. Lower bound techniquesa. CONGEST general networks: reductions from 2-
party communication complexityb. Asynchronous message passing: reductions from
multi-party communication complexityII. Obstacles on proving lower bounds for the
congested clique
Communication Complexity
đ đ
= ?
Example: DISJOINTNESS
đâ {1 ,âŚ,đ} đâ {1 ,âŚ,đ}
đâŠđ=â ?
bitsneeded
[Kalyanasundaram and Schnitger, Razborov â92]
DISJ :
Applying 2-Party Communication Complexity Lower Bounds
Textbook reduction:Given algorithm for solving task âŚ
Solution for answer for DISJOINTNESS
bits
đđ
Based on
Based on
Simulate
Example: Spanning Trees
⢠Setting: directed, strongly-connected network⢠Communication by local broadcast with
bandwidth ⢠UIDs ⢠Diameter 2⢠Question: how many rounds to find a rooted
spanning tree?
New Problem: PARTITION
⢠Inputs: , with the promise that
⢠Goal: Alice outputs ,Bob outputs such that partition .
đ đ
The PARTITION Problem
⢠Trivial algorithm:â Alice sends her input to Bobâ Alice outputs all tasks in her inputâ Bob outputs all remaining tasks
⢠Communication complexity: bits⢠Lower bound?
Reduction from DISJ to PARTITION
⢠Given input for DISJ :â Notice: iff â To test whether :⢠Try to solve PARTITION on ⢠Ensure ⢠Check if is a partition of : Alice sends Bob hash(), Bob
compares it to hash()
4 65
From PARTITION to Spanning Tree
a b
1 2 3
đ={1,2,3 } đ={2,4,5,6 }
Given a spanning tree algorithm âŚ
4 65
From PARTITION to Spanning Tree
a b
1 2 3
đ={1,2,3 } đ={2,4,5,6 }
Simulating one round of :
Node aâs message
Node bâs message
4 65
From PARTITION to Spanning Tree
a b
1 2 3
đ={1,2,3 } đ={2,4,5,6 }
When outputs a spanning tree:
From PARTITION to Spanning Tree
⢠If runs for rounds, we use bits
⢠One detail: randomnessâ Solution: Alice and Bob use public randomness
When Two Players Just Arenât Enough
⢠No bottlenecks in the network
When Two Players Just Arenât Enough
⢠Too much information revealed
Multi-Player Communication Complexity
⢠Communication by shared blackboard⢠Number-on-forehead⢠Number-in-hand
??
The Message-Passing Model
⢠players⢠Private channels⢠Private -bit inputs ⢠Private randomness
⢠Goal: compute ⢠Cost: total communication
The Coordinator Model
⢠players, one coordinator⢠The coordinator has no input
Message-Passing vs. Coordinator
â
Prior Work on Message-Passing
⢠For players with -bit inputsâŚâ˘ Phillips, Verbin, Zhang â12:â for bitwise problems (AND/OR, MAJ, âŚ)
⢠Woodruff, Zhang â12, â13:â for threshold and graph problems
⢠Braverman, Ellen, O., Pitassi, Vaikuntanathan â13: for
Set Disjointness
Disjđ ,đ =Âż đ=1ÂżđÂż đ=1Âżđ đ đđÂż
?
đ 1đ 2
đ 3
đ 4đ 5
Notation
⢠: randomized protocolâ Also, the protocolâs transcriptâ : player âs view of the transcript
⢠worst-case communication of
in the worst case
Entropy and Mutual Information
⢠Entropy:
⢠A lossless encoding of requires bits⢠Conditional entropy:
Entropy and Mutual Information
⢠Mutual information:
⢠Conditional mutual information:
Information Cost for Two Players[Chakrabarti, Shi, Wirth, Yao â01], [Bar-Yossef, Jayram, Kumar, Sivakumar â04], [Braverman, Rao â10], âŚ
Fix a distribution , ⢠External information cost:
⢠Internal information cost:
Extension to the coordinator model:
Why is Info Complexity Nice?
⢠Formalizes a natural notionâ Analogous to causality/knowledge
⢠Admits direct sum theorem:
âThe cost of solving independent copies of problem is times the cost of
solving â
Example
Example (Work in Progress)
⢠Triangle detection in general congested graphs⢠âIs there a triangleâ =
âis a triangleâ
Application of DISJ Lower Bound
⢠Open problem from Woodruff & Zhang â13:â Hardness of computing the diameter of a graph
⢠We can show: bits to distinguish diameter 3 from diameter
⢠Reduction from DISJ : given ,â Notice: disjoint iff
Application of DISJ Lower Bound
⢠Diameter ⢠Diameter
đ1 đ3
đ2
đ4
1
32 4
5
6
đ 3
Part II: The Power of the Congested Clique
CONGESTED CLIQUE
Conversion from Boolean Circuit
⢠Suppose we have a Boolean circuit â Any type of gate, inputsâ Fan-in â Depth = , #gates and wires =
⢠Step 1: reduce the fan-out to â Convert large fan-out gates to âcopying treeââ Blowup: depth, size
⢠Step 2: convert to a layered circuit
Conversion from Boolean Circuit
⢠Now we have a layered circuit of depth and size = â With fan-in and fan-out
⢠Design a CONGEST protocol:â Fix partition of inputs of size eachâ Assign each gate to a random CONGEST nodeâ Simulate the circuit layer-by-layer
Simulating a Layer
⢠If node âownsâ gate on layer , it sends âs output to the nodes that need it on layer
⢠Size of layer size of layer ⢠What is the load on edge ?â For each wire from layer to layer ,
â At most wires in totalâ By Chernoff, w.h.p. the load is
Conversion from Boolean Circuit
⢠A union-bound finishes the proof⢠Corollary: explicit lower bounds in the
congested clique imply explicit lower bounds on Boolean circuits with polylogarithmic depth and nearly-linear size.
⢠Even worse:â Reasons to believe even bound hard
ConclusionCONGESTED CLIQUE
ASYNC MESSAGE-PASSING
LOCAL
CONGEST / general network
X