Fourier analysis of Boolean functions: Some beautiful examples

Fourier analysis of Boolean functions:Some beautiful examples

Ronald de Wolf

Centrum voor Wiskunde en Informatica

Amsterdam

Fourier analysis of Boolean functions: Some beautiful examples – p.1/13

Fourier analysis

Many applications in math, physics, engineering,

Fourier analysis

. . . and in computer science

Fourier analysis

. . . and in computer science:

Signal processing

Fourier analysis

Signal processing

Data compression

Fourier analysis

Signal processing

Data compression

Multiplying two polynomials

Fourier analysis

Signal processing

Data compression

These examples use Fourier analysis over cyclic groups

Fourier analysis

Signal processing

Data compression

We will focus on Fourier analysis over the Boolean cube

Fourier analysis

Signal processing

Data compression

We will focus on Fourier analysis over the Boolean cube= 0, 1n, set of all n-bit strings

This has been very useful in CS

Fourier coefficients measure correlations with parities

Analysis of error-correcting codes

Learning a function from examples

The influence of variables on a function

Sensitivity of a function to noise on the inputs

PCPs, NP-hardness of approximation

Cryptography

Lower bounds on communication complexity

Cryptography

Threshold phenomena in random graphs

Cryptography

Quantum computing

Cryptography

Quantum computing. . .

Fourier analysis over the Boolean cube

Consider the space of functions f : 0, 1n → R

Consider the space of functions f : 0, 1n → R, with

normalized inner product 〈f, g〉 =1

x∈0,1n

f(x)g(x)

x∈0,1n

f(x)g(x)

The parity-functions χs(x) = (−1)x·s =∏

i:si=1(−1)xi

form an orthonormal basis of this space

x∈0,1n

f(x)g(x)

i:si=1(−1)xi

Hence we can write f =∑

s∈0,1n

f(s)χs

x∈0,1n

f(x)g(x)

i:si=1(−1)xi

s∈0,1n

f(s)χs

with f(s) = 〈f, χs〉 =1

x∈0,1n

f(x)χs(x)

x∈0,1n

f(x)g(x)

i:si=1(−1)xi

s∈0,1n

f(s)χs

x∈0,1n

f(x)χs(x)

Map f 7→ f is proportional to unitary (length-preserving)

x∈0,1n

f(x)g(x)

i:si=1(−1)xi

s∈0,1n

f(s)χs

x∈0,1n

f(x)χs(x)

Map f 7→ f is proportional to unitary (length-preserving)

f(x)2 =∑

f(s)2 (Parseval’s identity)

Examples

OR on 2 bits:

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

f(00) = 14

∑x∈0,1n f(x)χ00(x)

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

f(00) = 14

∑x∈0,1n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1)

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

f(00) = 14

∑x∈0,1n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 34

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

f(00) = 14

∑x∈0,1n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 34

f(01) = −14 ,

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

f(00) = 14

∑x∈0,1n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 34

f(01) = −14 , f(10) = −1

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

f(00) = 14

∑x∈0,1n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 34

f(01) = −14 , f(10) = −1

4 , f(11) = −14

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

f(00) = 14

∑x∈0,1n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 34

f(01) = −14 , f(10) = −1

4 , f(11) = −14

Note: f(00) = Expx[f(x)]

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

f(00) = 14

∑x∈0,1n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 34

f(01) = −14 , f(10) = −1

4 , f(11) = −14

Note: f(00) = Expx[f(x)], and f(00) =∑

s f(s)

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

f(00) = 14

∑x∈0,1n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 34

f(01) = −14 , f(10) = −1

4 , f(11) = −14

s f(s)

Parseval:1

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

f(00) = 14

∑x∈0,1n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 34

f(01) = −14 , f(10) = −1

4 , f(11) = −14

s f(s)

Parseval:1

f(x)2 =3

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

f(00) = 14

∑x∈0,1n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 34

f(01) = −14 , f(10) = −1

4 , f(11) = −14

s f(s)

Parseval:1

f(x)2 =3

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

f(00) = 14

∑x∈0,1n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 34

f(01) = −14 , f(10) = −1

4 , f(11) = −14

s f(s)

Parseval:1

f(x)2 =3

PARITY on n bits

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

f(00) = 14

∑x∈0,1n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 34

f(01) = −14 , f(10) = −1

4 , f(11) = −14

s f(s)

Parseval:1

f(x)2 =3

PARITY on n bits, with TRUE= −1, FALSE= +1:

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

f(00) = 14

∑x∈0,1n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 34

f(01) = −14 , f(10) = −1

4 , f(11) = −14

s f(s)

Parseval:1

f(x)2 =3

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

f(00) = 14

∑x∈0,1n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 34

f(01) = −14 , f(10) = −1

4 , f(11) = −14

s f(s)

Parseval:1

f(x)2 =3

f(x) = χ1n(x) = (−1)|x|

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

f(00) = 14

∑x∈0,1n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 34

f(01) = −14 , f(10) = −1

4 , f(11) = −14

s f(s)

Parseval:1

f(x)2 =3

f(x) = χ1n(x) = (−1)|x|, so f(1n) = 1

Examples

OR on 2 bits:

f(x1, x2) = OR(x1, x2) ∈ 0, 1

f(00) = 14

∑x∈0,1n f(x)χ00(x) = 1

4(0 + 1 + 1 + 1) = 34

f(01) = −14 , f(10) = −1

4 , f(11) = −14

s f(s)

Parseval:1

f(x)2 =3

f(x) = χ1n(x) = (−1)|x|, so f(1n) = 1

all other f(s) are 0

(1) Approximating functions with parities

Suppose f : 0, 1n → ±1 has small Fourier degree d

Suppose f : 0, 1n → ±1 has small Fourier degree d:

f =∑

s:|s|≤d

f(s)χs

f =∑

s:|s|≤d

f(s)χs

Then there exists a parity-function on at most d bits thathas non-trivial correlation with f

f =∑

s:|s|≤d

f(s)χs

f =∑

s:|s|≤d

f(s)χs

Why?∑

s:|s|≤d

f =∑

s:|s|≤d

f(s)χs

Why?∑

s:|s|≤d

f(s)2 =1

x∈0,1n

f(x)2 = 1 (Parseval).

f =∑

s:|s|≤d

f(s)χs

Why?∑

s:|s|≤d

f(s)2 =1

x∈0,1n

This is a sum over ≤ nd terms.

f =∑

s:|s|≤d

f(s)χs

Why?∑

s:|s|≤d

f(s)2 =1

x∈0,1n

This is a sum over ≤ nd terms. Hence ∃s with

f =∑

s:|s|≤d

f(s)χs

Why?∑

s:|s|≤d

f(s)2 =1

x∈0,1n

This is a sum over ≤ nd terms. Hence ∃s with1

nd≤ f(s)2

f =∑

s:|s|≤d

f(s)χs

Why?∑

s:|s|≤d

f(s)2 =1

x∈0,1n

nd≤ f(s)2 = |

x∈0,1n

f(x)χs(x)|2

f =∑

s:|s|≤d

f(s)χs

Why?∑

s:|s|≤d

f(s)2 =1

x∈0,1n

nd≤ f(s)2 = |

x∈0,1n

f(x)χs(x)|2

So χs (or its negation) has non-trivial correlation with f

(2) Learning from uniform examples

A Fourier coefficient is just a uniform expectation

A Fourier coefficient is just a uniform expectation:

f(s) =1

x∈0,1n

f(x)χs(x)

f(s) =1

x∈0,1n

f(x)χs(x) = Expx[f(x)χs(x)]

f(s) =1

x∈0,1n

We can approximate this given uniformly randomexamples (x1, f(x1)), . . . , (xm, f(xm)):

f(s) =1

x∈0,1n

We can approximate this given uniformly randomexamples (x1, f(x1)), . . . , (xm, f(xm)):1

f(xi)χs(xi)

f(s) =1

x∈0,1n

f(xi)χs(xi) → f(s)

f(s) =1

x∈0,1n

Converges fast if |f(s)| is not too small (Chernoff)

f(s) =1

x∈0,1n

Hence we can quickly learn (approximate) an unknownfunction f that is dominated by a few large coefficients

f(s) =1

x∈0,1n

Hence we can quickly learn (approximate) an unknownfunction f that is dominated by a few large coefficients(example from LMN 89: AC0-circuits)

(3) List-decoding of Hadamard code

Error-correcting code: E : 0, 1n → 0, 1m

If all codewords have distance d(E(x), E(y)) ≥ 2e + 1,then we can uniquely recover x from corruptedcodeword w ∈ 0, 1m with e errors

If all codewords have distance d(E(x), E(y)) ≥ 2e + 1,then we can uniquely recover x from corruptedcodeword w ∈ 0, 1m with e errors (d(w,E(x)) = e)

Hadamard code (m = 2n):

Hadamard code (m = 2n): E(x)y = x · y mod 2

All codewords are at distance m/2

All codewords are at distance m/2 ⇒ given w withe < m/4 errors, there is a unique x with d(w,E(x)) ≤ e

Problem: if e ≥ m/4 errors, then there may be manydifferent x with d(w,E(x)) ≤ e

Example: w = 03m/41m/4 could’ve come from codewordsE(0n) = 0m or E(10n−1) = 0m/21m/2

List-decoding: output the whole list (hopefully small)

List-decoding of Hadamard code (cntd)

List-decoding: given corrupted codeword w ∈ 0, 1m

and error bound e, output list x : d(w,E(x)) ≤ e

For Hadamard code: if e ≤ (1/2 − ε)m,then this list has only O(1/ε2) elements!

Why? Fourier analysis!

Why? Fourier analysis!1. View w as function w : 0, 1n → ±1, and E(s) = χs

2. If d(w,E(s)) ≤ (1/2 − ε)m, then w(s) ≥ 2ε

s w(s)2 = 1 (by Parseval)

s w(s)2 = 1 (by Parseval),hence at most 1

4ε2 different s satisfy w(s) ≥ 2ε

Goldreich and Levin show how to find this list efficiently

There are codes with much better rate that are stillefficiently list-decodable

There are codes with much better rate that are stillefficiently list-decodable (e.g. Reed-Solomon)

(4) Influence of variables

Consider a Boolean function f : 0, 1n → 0, 1

The influence of variable i is the probability that xi

determines the function value

determines the function value:

Inff (i) = Prx∈0,1n

[f(x) 6= f(x ⊕ ei)]

For things like voting and distributed coin-flipping:would like to find a balanced f where each Inff (i) ≈ 1/n

[f(x) 6= f(x ⊕ ei)]

KKL 88: if f is balanced, thenthere always is an i with Inff (i) ≥ log(n)/n

[f(x) 6= f(x ⊕ ei)]

KKL 88: if f is balanced, thenthere always is an i with Inff (i) ≥ log(n)/n

This implies there is a set of O(n/ log(n)) variables thatcontrols f with high probability

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei)

The KKL proof

Define fi(x) = f(x) − f(x ⊕ ei) ∈ −1, 0,+1

The KKL proof

Then fi(s) = 2f(s) if si = 1

The KKL proof

Then fi(s) = 2f(s) if si = 1 , and fi(s) = 0 if si = 0

The KKL proof

Inff (i)

The KKL proof

Inff (i) = Pr[fi 6= 0]

The KKL proof

Inff (i) = Pr[fi 6= 0] = Exp[f2i ]

The KKL proof

Inff (i) = Pr[fi 6= 0] = Exp[f2i ] =

fi(s)2

The KKL proof

Inff (i) = Pr[fi 6= 0] = Exp[f2i ] =

fi(s)2 = 4

s:si=1

The KKL proof

Inff (i) = Pr[fi 6= 0] = Exp[f2i ] =

fi(s)2 = 4

s:si=1

If L =∑

s:|s|>log n f(s)2 ≥ 1/3

The KKL proof

Inff (i) = Pr[fi 6= 0] = Exp[f2i ] =

fi(s)2 = 4

s:si=1

If L =∑

s:|s|>log n f(s)2 ≥ 1/3, then∑n

i=1 Inff (i)

The KKL proof

Inff (i) = Pr[fi 6= 0] = Exp[f2i ] =

fi(s)2 = 4

s:si=1

If L =∑

i=1 Inff (i)

= 4∑

s |s|f(s)2

The KKL proof

Inff (i) = Pr[fi 6= 0] = Exp[f2i ] =

fi(s)2 = 4

s:si=1

If L =∑

i=1 Inff (i)

= 4∑

s |s|f(s)2 ≥ Ω(log n)

The KKL proof

Inff (i) = Pr[fi 6= 0] = Exp[f2i ] =

fi(s)2 = 4

s:si=1

If L =∑

i=1 Inff (i)

= 4∑

s |s|f(s)2 ≥ Ω(log n) ⇒ maxi Inff (i) ≥ Ω(log(n)/n)

The KKL proof

Inff (i) = Pr[fi 6= 0] = Exp[f2i ] =

fi(s)2 = 4

s:si=1

If L =∑

i=1 Inff (i)

= 4∑

If L < 1/3, then use KKL inequality (special case ofBonami-Beckner):

The KKL proof

Inff (i) = Pr[fi 6= 0] = Exp[f2i ] =

fi(s)2 = 4

s:si=1

If L =∑

i=1 Inff (i)

= 4∑

If L < 1/3, then use KKL inequality (special case ofBonami-Beckner): ∀g : 0, 1n → −1, 0,+1, δ ∈ [0, 1]

s∈0,1n

δ|s|g(s)2 ≤ Pr[g 6= 0]2/(1+δ)

The KKL proof

Inff (i) = Pr[fi 6= 0] = Exp[f2i ] =

fi(s)2 = 4

s:si=1

If L =∑

i=1 Inff (i)

= 4∑

If L < 1/3, then use KKL inequality (special case ofBonami-Beckner): ∀g : 0, 1n → −1, 0,+1, δ ∈ [0, 1]

s∈0,1n

δ|s|g(s)2 ≤ Pr[g 6= 0]2/(1+δ)

A calculation shows maxi Inff (i) ≥ Ω(log(n)/n)

Summary

Fourier analysis of Boolean functions is an increasinglyprominent tool in theoretical computer science

Summary

We showed a few simple but beautiful examples:

Summary

1. Approximating low-degree functions by parities

Summary

2. List-decoding of Hadamard codes

Summary

3. Learning under the uniform distribution

Summary

3. Learning under the uniform distribution

4. The influence of variables on Boolean functions

Warning: these are powerful techniques!

Fourier analysis of Boolean functions: Some beautiful examples

Documents

Boolean and non-Boolean conjunction 1 - univie.ac.at › ... › texte › Boolean_and_non-Boolean_conju… · Boolean and non-Boolean conjunction 1 Viola Schmitt University of Vienna

Ramakrishna Mission Vivekananda College · 2019. 10. 18. · FOURIER SERIES, FOURIER INTEGRALS, FOURIER TRANSFORMS AND Z-TRANSFORMS / FOURIER SERIES FOURIER INTEGRALS AND FOURIER

Boolean Algebra – II. Outline Basic Theorems of Boolean Algebra Boolean Functions Complement of Functions Standard Forms

Fourier analysis and sampling theoryFourier transforms Convolution, while a bit cumbersome looking, actually has a beautiful structure when viewed in terms of Fourier analysis. We

Boolean expressions created from: Boolean logic

Limits of Boolean algebras and Boolean valued … › matteo_viale › guichardaz.pdfLimits of Boolean algebras and Boolean valued models Relatore: Candidata: Prof. Matteo Viale Fiorella

Boolean Algebra – I. Outline Introduction Digital circuits Boolean Algebra Two-Valued Boolean Algebra Boolean Algebra Postulates Precedence

Boolean Functions and Boolean Maps - uni-mainz.de€¦ · Boolean Functions and Boolean Maps Klaus Pommerening Fachbereich Physik, Mathematik, Informatik der Johannes-Gutenberg-Universit

Boolean Algebra, Gates and Circuits - Radboud Universiteit · Equivalence of Boolean Expressions Two Boolean expressions that represent the same Boolean function are called equivalent

New 1. Boolean Logic1 · 2002. 10. 14. · Boolean Functions and Boolean Algebra Boolean algebra deals with Boolean (or binary) values that are typically labeled true/false, 1/0,

Laws of Boolean Algebra Commutative Laws of Boolean

Boolean Algebra1 BOOLEAN ALGEBRA Boolean Algebra2 BOOLEAN ALGEBRA -REVIEW Boolean Algebra was proposed by George Boole in 1853. Basically AND,OR NOT

Boolean Retrieval - Simon Fraser University slides/L16 - Boolean retrieval.pdf · J. Pei: Information Retrieval and Web Search -- Boolean Retrieval 10 The Boolean Retrieval Model

Fast Fourier Sparsity Testing over the Boolean Hypercube Grigory Yaroslavtsev Joint with Andrew Arnold (Waterloo), Arturs Backurs (MIT),

CHAPTER 3 Boolean Algebra and Digital Logic€¦ · Boolean Algebra and Digital Logic . 3.1 Introduction 121 . 3.2 Boolean Algebra 122 . 3.2.1 Boolean Expressions 123 . 3.2.2 Boolean

An Application of Fourier analysis on Boolean functions in ... · additive combinatorics to show the quasi-polynomial Freiman-Ruzsa theorem originally proven by Sanders. Using this,

Correlation of Boolean Functions - MIT CSAILpeople.csail.mit.edu/.../Coded_Aperture/Correlation_Boolean_Functio… · of Boolean Functions Desired Cryptographic Properties of Boolean

Logika Matematika - blogs.unpas.ac.id fileAljabar Boolean Dua Nilai Sifat-Sifat Aljabar Boolean Fungsi Boolean Penyederhanaan Fungsi Boolean Kalkulus Proposisi

1. Boolean Algebra Fachgebiet Rechnersysteme 1 1 Boolean

Fourier Analysis of Boolean Maps { A Tutorial