Download pdf - Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/TIFR-CRG.pdf · of 17 Randomness Processing Industry Dispersers, Extractors, Merges, Condensers,

of 17May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 1

Communication Complexity of Randomness Manipulation

Madhu SudanHarvard University

Joint work with Mitali Bafna (Harvard), Badih Ghazi (Google), and Noah Golowich (Harvard)

of 17

Randomness Processing Industry

Dispersers, Extractors, Merges, Condensers, PRGs …

Interest not in the exact function. (E=RSA most boring.)

Rather in its manipulation of distributions … Distribution of 𝑋𝑋 vs. that of 𝐸𝐸(𝑋𝑋)

Long history … omitted. Key ingredients single processor “𝐸𝐸” unknown source 𝑋𝑋 ∈ 𝒳𝒳

May 29, 2019 TIFR: Comm. Comp. Randomness Manipulation 2

𝐸𝐸𝑋𝑋 𝐸𝐸(𝑋𝑋)

of 17

2-player setting:

Is 𝛿𝛿 = 0 possible? If not minimize 𝛿𝛿! Etc. Alice + Bob can use private randomness Zero communication version: “Non-Interactive

Simulation” (NIS).

Distributed Randomness Processing


Alice 𝑈𝑈

Bob𝑌𝑌 𝑉𝑉

≤ 𝐶𝐶 bits, ≤ 𝑟𝑟 Rounds

𝑋𝑋𝑋𝑋1,𝑋𝑋2, …

𝑌𝑌1,𝑌𝑌2, …

𝑋𝑋,𝑌𝑌 ∼ 𝑃𝑃 Goal: 𝑈𝑈,𝑉𝑉 ≈𝛿𝛿 𝑄𝑄

Infinite sequence

of 17

A classical example

Zero communication 𝑋𝑋,𝑌𝑌 ∼ Unif 0,0 , 0,1 , 1,0 𝑈𝑈,𝑉𝑉 ∼ 𝜌𝜌-correlated bits.

𝑈𝑈,𝑉𝑉 ∼ Unif 0,1 , Pr 𝑈𝑈 = 𝑉𝑉 = 1+𝜌𝜌2

[Witsenhausen ‘70s]: Can’t achieve 𝜌𝜌 = 1. Not even 𝜌𝜌 = .51

(Naïve strategy achieves 𝜌𝜌 = 14)

Can achieve 𝜌𝜌 = 13

by a general method.

Best 𝜌𝜌? Open!!


of 17

Witsenhausen …

Intermediate Problem: Output (𝐴𝐴,𝐵𝐵) w. 𝐴𝐴,𝐵𝐵 ∼ 𝑁𝑁 0,1with maximum correlation.

Definition: Max-Corr 𝜌𝜌 𝑋𝑋,𝑌𝑌 ≜ max𝑓𝑓,𝑔𝑔

𝔼𝔼𝑋𝑋,𝑌𝑌 𝑓𝑓 𝑋𝑋 𝑔𝑔(𝑌𝑌)

Where 𝑓𝑓,𝑔𝑔:Ω → ℝ,𝔼𝔼𝑋𝑋 𝑓𝑓 𝑋𝑋 = 𝔼𝔼𝑌𝑌 𝑔𝑔 𝑌𝑌 = 0𝔼𝔼𝑋𝑋 𝑓𝑓 𝑋𝑋 2 = 𝔼𝔼𝑌𝑌 𝑔𝑔 𝑌𝑌 2 = 1

Thm 1: For any 𝑈𝑈,𝑉𝑉 output, 𝜌𝜌 𝑈𝑈,𝑉𝑉 ≤ 𝜌𝜌(𝑋𝑋,𝑌𝑌) Thm 2: For Gaussian output (𝐴𝐴,𝐵𝐵), can achieve

𝜌𝜌 𝐴𝐴,𝐵𝐵 = 𝜌𝜌(𝑋𝑋,𝑌𝑌) Thm 3: For binary 𝑈𝑈,𝑉𝑉 , can achieve

1 −2 ⋅ cos−1 𝜌𝜌 𝑋𝑋,𝑌𝑌

𝜋𝜋≤ 𝜌𝜌 𝑈𝑈,𝑉𝑉 ≤ 𝜌𝜌 𝑋𝑋,𝑌𝑌


For Bernoulli/GaussiansMax-Correlation = Correlation

of 17

“Tensorization”

“Single-letter” Problems: Given (1) structure 𝑆𝑆 (graph, distribution, game) (2) product operation 𝑆𝑆⊗𝑡𝑡, (3) measure 𝑀𝑀, compute lim

𝑡𝑡→∞𝑀𝑀 𝑆𝑆⊗𝑡𝑡

Shannon capacity, Compression length, Channel capacity, parallel repetition value of 2-prover game, Direct sum complexity, NIS

“Tensorizing bound”: Typically: 𝑀𝑀 𝑆𝑆⊗𝑡𝑡 ≥ 𝑀𝑀(𝑆𝑆). Find: ℳ(⋅) s.t. 𝑀𝑀 𝑆𝑆 ≤ ℳ(𝑆𝑆) and ℳ 𝑆𝑆⊗𝑡𝑡 = ℳ(𝑆𝑆)

Max-Correlation “Tensorizes” If 𝑋𝑋,𝑌𝑌 ∼ 𝜇𝜇 & 𝑋𝑋𝑡𝑡,𝑌𝑌𝑡𝑡 ∼ 𝜇𝜇⊗𝑡𝑡 then 𝜌𝜌 𝑋𝑋𝑡𝑡 ,𝑌𝑌𝑡𝑡 = 𝜌𝜌(𝑋𝑋,𝑌𝑌) Proof idea: 𝜇𝜇 described by matrix 𝑃𝑃; 𝜌𝜌 related to its

singular values, 𝜇𝜇⊗𝑡𝑡 described by 𝑃𝑃⊗𝑡𝑡


of 17

Single-letter EmbarassmentProblem Base Single-letter

versionCompressibility P P

Channel capacity NP P


Glossary: 0 ≤ P ≤ 𝑁𝑁𝑃𝑃 ≤ EXP ≤ Computable≤ CA Computably Approximable ≤ ∞

Problem Base Single-letterversion

Compressibility P P


Independent set/Shannon capacity

NP [0,∞]

Value of 2-prover game NP [NP,∞]


Compressibility P P



NP [0,∞]


Non-interactive-simulation (“correlation”)

NP [0,𝐶𝐶𝐴𝐴]

Communication Complexity [NP,Exp?] 0,𝐶𝐶𝐴𝐴


Compressibility P P



NP [0,∞]


Non-interactive-simulation (“correlation”)

NP [0,𝐶𝐶𝐴𝐴]

Communication Complexity [NP,Exp?] 0,𝐶𝐶𝐴𝐴

? P [NP,∞]

of 17

Complexity of NIS

“Computably approximable” [Ghazi,Kamath,S.], [De,Mossel,Neeman]2,

[Ghazi,Kamath,Raghavendra] There exists a finite time algorithm determining if we can

get 𝜖𝜖-close to (𝑈𝑈,𝑉𝑉) from i.i.d. samples of (𝑋𝑋,𝑌𝑌)

Idea: “Invariance Priniciple” [Mossel] Either there is a method using few samples of (𝑋𝑋,𝑌𝑌) or

many samples 𝑋𝑋1,𝑌𝑌1 , … , 𝑋𝑋𝑛𝑛,𝑌𝑌𝑛𝑛 each with low influence on 𝑈𝑈,𝑉𝑉 .

If latter, can replace 𝑋𝑋,𝑌𝑌 by finite # Gaussians. Gives computable upper bound on #samples needed to

get close to optimal strategy.


of 17

Communication?

Restrict to “Common Randomness Generation” (𝑘𝑘-CRG): Output (𝑈𝑈 = 𝑉𝑉), with 𝑈𝑈 ∼ Unif( 0,1 𝑘𝑘)

First considered ~70 [Ahlswede-Csizar]: “Characterization”: Can communicate 𝑅𝑅 ⋅ 𝑘𝑘 +𝑜𝑜 𝑘𝑘 bits, where 𝑅𝑅 = min

ΠICint(Π)ICext Π

Computable? Computably approximable? For one-way communication: 𝑅𝑅 = 1 − 𝜌𝜌 𝑋𝑋,𝑌𝑌 2

[Zhao-Chia] (see also [Guruswami-Radhakrishnan, Ghazi-Jayram, S.-Tyagi-Watanabe])


Alice 𝑈𝑈

Bob 𝑉𝑉

≤ 𝐶𝐶 bits, ≤ 𝑟𝑟 Rounds

𝑋𝑋1,𝑋𝑋2, …

𝑌𝑌1,𝑌𝑌2, …

𝑋𝑋,𝑌𝑌 ∼ 𝑃𝑃 Goal: 𝑈𝑈,𝑉𝑉 ≈𝛿𝛿 𝑄𝑄

of 17

This talk: Rounds in CRG

Does increasing #rounds ease CRG? (For some (𝑋𝑋,𝑌𝑌)?) … Is the following true?

Question: ∀𝜖𝜖 > 0 , 𝑟𝑟, ∃(𝑋𝑋,𝑌𝑌) s.t. ∀𝑘𝑘 ∃ 𝑟𝑟, 𝜖𝜖𝑘𝑘 -protocol for 𝑘𝑘-CRG 𝑋𝑋,𝑌𝑌 No 𝑟𝑟 − 1, 1 − 𝜖𝜖 .𝑘𝑘 -protocol for 𝑘𝑘-CRG(𝑋𝑋,𝑌𝑌)

Still don’t know. Partial progress. Thm. [BGGS’19] : ∀𝑛𝑛,𝑘𝑘, 𝑟𝑟 there exists 𝑋𝑋,𝑌𝑌 s.t.

∃ 𝑟𝑟,𝑂𝑂(𝑟𝑟 log 𝑛𝑛 -protocol for 𝑘𝑘-CRG(𝑋𝑋,𝑌𝑌).

No 𝑟𝑟2− 3, min 𝑘𝑘, 𝑛𝑛

polylog 𝑛𝑛-protocol for 𝑘𝑘-CRG(𝑋𝑋,𝑌𝑌)


of 17

Our Source:


𝑖𝑖0

𝜋𝜋1

𝜋𝜋2

𝜋𝜋3

𝜋𝜋𝑟𝑟

⋮

𝐴𝐴1 𝐴𝐴2 𝐴𝐴3 𝐴𝐴𝑛𝑛 𝐵𝐵1 𝐵𝐵2 𝐵𝐵3 𝐵𝐵𝑛𝑛… …

Alice’s Inputs𝑋𝑋 = (𝜋𝜋1,𝜋𝜋3, … ;𝐴𝐴1, … ,𝐴𝐴𝑛𝑛)

Bob’s Inputs𝑌𝑌 = (𝑖𝑖0;𝜋𝜋2,𝜋𝜋4, … ;𝐵𝐵1, … ,𝐵𝐵𝑛𝑛)

𝑖𝑖0 ∼ [𝑛𝑛]

𝜋𝜋1, … ,𝜋𝜋𝑛𝑛 ∼ 𝑆𝑆𝑛𝑛𝐴𝐴1, … ,𝐴𝐴𝑛𝑛 ∼ 0,1 𝑘𝑘

𝐵𝐵1, … ,𝐵𝐵𝑛𝑛 ∼ 0,1 𝑘𝑘

𝐴𝐴𝑖𝑖𝑟𝑟 = 𝐵𝐵𝑖𝑖𝑟𝑟where 𝑖𝑖𝑗𝑗 = 𝜋𝜋𝑗𝑗 𝑖𝑖𝑗𝑗−1

“Pointer Chasing Source” (PCS)

of 17

CRG from Pointer Chasing Source

Easy direction easy: (𝑟𝑟, 𝑟𝑟 log𝑛𝑛)-protocol for 𝑘𝑘-CRG Hardness?

Pointer chasing is hard: [Duris-Galil-Schnitger, Nisan-Wigderson,…].

CRG from PCS requires pointer chasing? No! Can solve problem without pointers! Small non-deterministic complexity! Hardness needs to use hardness of

disjointness? And of pointer-chasing? How to combine modularly?


of 17

Our solution: Pointer Verification Problem

PVP: Distinguish 𝑋𝑋,𝑌𝑌 ∼ 𝑃𝑃𝑉𝑉YES from 𝑋𝑋,𝑌𝑌 ∼ 𝑃𝑃𝑉𝑉No 𝑃𝑃𝑉𝑉YES:𝑋𝑋 = 𝜋𝜋1,𝜋𝜋3, …𝜋𝜋𝑟𝑟−1 ; 𝑌𝑌 = 𝑖𝑖0, 𝑖𝑖𝑟𝑟; 𝜋𝜋2,𝜋𝜋4, …𝜋𝜋𝑟𝑟 𝑃𝑃𝑉𝑉NO:𝑋𝑋 = 𝜋𝜋1,𝜋𝜋3, …𝜋𝜋𝑟𝑟−1 ; 𝑌𝑌 = 𝑖𝑖0, 𝑖𝑖𝑟𝑟′ ; 𝜋𝜋2,𝜋𝜋4, …𝜋𝜋𝑟𝑟

YES instance satisfy: 𝑖𝑖𝑗𝑗 = 𝜋𝜋𝑗𝑗 𝑖𝑖𝑗𝑗−1 NO instance: 𝑖𝑖𝑟𝑟′ random

Claims: Hardness of (Unique) Set Disjointness ⇒

Protocol for 𝑘𝑘-CRG(PCS) solves PVP.

No 𝑟𝑟2− 𝑂𝑂 1 , 𝑛𝑛

polylog 𝑛𝑛-protocol for PVP.


of 17

PVP ≤ CRG

Let 𝜇𝜇𝑋𝑋𝑌𝑌 denote dist. of PCS source. 𝜇𝜇𝑋𝑋, 𝜇𝜇𝑌𝑌 marginals

CRG Hard ⇐ 𝜇𝜇𝑋𝑋𝑌𝑌 indist. from 𝜇𝜇𝑋𝑋 × 𝜇𝜇𝑌𝑌 (to 𝑟𝑟2

,𝐶𝐶 -protocols)

Intermediate distribution: 𝜇𝜇mid 𝑋𝑋 = (𝜋𝜋odd,𝐴𝐴1, … ,𝐴𝐴𝑛𝑛); 𝑌𝑌 = 𝑖𝑖0,𝜋𝜋even,𝐵𝐵1, … ,𝐵𝐵𝑛𝑛 ,

with 𝐴𝐴𝑗𝑗 = 𝐵𝐵𝑗𝑗 for random 𝑗𝑗. (Correlation exists but pointers don’t point to it.)

Claims: 𝜇𝜇𝑋𝑋𝑌𝑌 indist. from 𝜇𝜇mid ⇐ PVP is hard 𝜇𝜇mid indist. from 𝜇𝜇𝑋𝑋 × 𝜇𝜇𝑌𝑌 ⇐ (Unique) disjointness

hard.


𝑖𝑖

⋮

of 17

Bulk of paper: PVP is hard

Main idea: “Round elimination” a la NW ’93 No black-box use No simple variant

Induction on #rounds Many invariants … roughly

𝐻𝐻 𝜋𝜋odd,𝜋𝜋even transcript) ≈ 𝑀𝑀𝑀𝑀𝑀𝑀 − 𝑂𝑂 𝐶𝐶 𝐻𝐻 𝑖𝑖𝑡𝑡 transcript) ≈ log𝑛𝑛 − 𝑜𝑜 1 𝑋𝑋 ⫫ 𝑌𝑌 | path, transcript


of 17

Conclusions

Many interesting problems in distributed randomness manipulation.

Complexity of the “Single-letter characterizations”.

Tight characterization of round complexity of CRG.


of 17

Thank You!
