Higher-Order Fourier Analysis: Applications to Algebraic...

Higher-Order Fourier Analysis:

Applications to Algebraic Property Testing

Yuichi Yoshida

National Institute of Informatics, andPreferred Infrastructure, Inc

May 17, 2016

Yuichi Yoshida (NII and PFI) Applications to algebraic property testing May 17, 2016 1 / 37

Decision Problems

• Function f : Fn2 → {0, 1}.

• Function property P .(such as linearity: f (x) + f (y) ≡ f (x + y) mod 2 for all x , y .)

Q. How long does it take to decide f satisfies P?

A. Trivially, it takes 2n time.

Q. Can we do something in sublinear or even in constant time?

Decision Problems

• Function f : Fn2 → {0, 1}.

Q. How long does it take to decide f satisfies P?A. Trivially, it takes 2n time.

Decision Problems

• Function f : Fn2 → {0, 1}.

Q. How long does it take to decide f satisfies P?A. Trivially, it takes 2n time.

Property testing

Definition

f : {0, 1}n → {0, 1} is ε-far from P if,

dP(f ) := ming∈P

#{x ∈ {0, 1}n | f (x) 6= g(x)}2n

≥ ε.

Accept w.p. 2/3

Reject w.p. 2/3

ε-far

A tester for a property P :Given

• f : {0, 1}n → {0, 1}as a query access.

• proximity parameter ε > 0.

Property testing

Definition

f : {0, 1}n → {0, 1} is ε-far from P if,

dP(f ) := ming∈P

#{x ∈ {0, 1}n | f (x) 6= g(x)}2n

≥ ε.

Accept w.p. 2/3

Reject w.p. 2/3

ε-far

A tester for a property P :Given

• f : {0, 1}n → {0, 1}as a query access.

• proximity parameter ε > 0.

Testing f ≡ 1

Input: a function f : {0, 1}n → {0, 1} and ε > 0.Goal: f (x) = 1 for every x ∈ {0, 1}n?

1: for i = 1 to Θ(1/ε) do2: Sample x ∈ {0, 1}n uniformly at random.3: if f (x) = 0 then reject.4: Accept.

Theorem

• If f ≡ 1, always accepts. (one-sided error)

• If f is ε-far, accepts with probability (1− ε)Θ(1/ε) ≤ 2/3.

• Query complexity is O(1/ε)⇒ constant!

Testing f ≡ 1

Theorem

Testing f ≡ 1

Theorem

Testing f ≡ 1

Theorem

Testing f ≡ 1

Theorem

Backgrounds

Property testing was introduced by [RS96] for program checking.

Since then, various kinds of objects have been studied.Ex.: Functions, graphs, distributions, geometric objects, images.

Q. Why do we study property testing?A. Interested in

• ultra-efficient algorithms.

• connections to inapproximability, locally testable codes, andlearning.

• the relation between local view and global property.

Backgrounds

Q. Why do we study property testing?

A. Interested in

Backgrounds

Local testability of affine-Invariant properties

Definition

P is affine-invariant if a function f : Fn2 → {0, 1} satisfies P , then

f ◦ A satisfies P for any bijective affine transformation A : Fn2 → Fn

Definition

P is (locally) testable if there is a tester for P with q(ε) queries.

Definition

P is affine-invariant if a function f : Fn2 → {0, 1} satisfies P , then

f ◦ A satisfies P for any bijective affine transformation A : Fn2 → Fn

Definition

P is (locally) testable if there is a tester for P with q(ε) queries.

Some specific locally testable affine-invariant properties:

• Degree-d polynomials [AKK+05, BKS+10]

• Fourier sparsity [GOS+11]

• Odd-cycle-freeness: There exist no x1, . . . , x2k+1 ∈ Fn2 such that

f (x1) = · · · = f (x2k+1) = 1 and x1 + · · ·+ x2k+1 ≡ 0 [BGRS12].

The goal

Q. Can we characterize locally testable affine-invariantproperties? [KS08]

A. Yes.

In this talk, we review how we have attacked this question.

• One-sided error testable ≈ Affine-subspace hereditary

• Testable ⇔ Estimable

• Two-sided error testable ⇔ Regular-reducible

Higher order Fourier analysis has played a crucial role!

The goal

A. Yes.

The goal

A. Yes.

Fourier analysis

For S ⊆ [n], define χS : Fn2 → {−1, 1} as χS(x) = (−1)

∑i∈S xi .

{χS} forms an orthonormal basis for functions: Fn2 → R:

[χS(x)χT (x)] = Ex

[(−1)∑

i∈S4T xi ] =

{1 if S = T ,

0 otherwise.

⇒ A function f : Fn2 → R can be uniquely decomposed as

f (x) =∑S⊆[n]

f (S)χS(x),

where f (S) = Ex [f (x)χS(x)] measures the correlation of f with χS .(Fourier coefficients)

Fourier analysis

For S ⊆ [n], define χS : Fn2 → {−1, 1} as χS(x) = (−1)

∑i∈S xi .

{χS} forms an orthonormal basis for functions: Fn2 → R:

[χS(x)χT (x)] = Ex

[(−1)∑

i∈S4T xi ] =

{1 if S = T ,

0 otherwise.

⇒ A function f : Fn2 → R can be uniquely decomposed as

f (x) =∑S⊆[n]

f (S)χS(x),

where f (S) = Ex [f (x)χS(x)] measures the correlation of f with χS .(Fourier coefficients)

Linearity testing

Input: a function f : Fn2 → F2 and ε > 0.

Goal: f (x) + f (y) ≡ f (x + y) (mod 2) for every x , y ∈ Fn2?

1: for i = 1 to Θ(1/ε) do2: Sample x , y ∈ Fn

2 uniformly at random.3: if f (x) + f (y) 6≡ f (x + y) (mod 2) then reject.4: Accept.

Theorem ([BLR93])

• If f is linear, always accepts. (one-sided error)

• If f is ε-far, rejects with probability at least 2/3.

• Query complexity is O(1/ε)

Linearity testing

Theorem ([BLR93])

Linearity testing

Theorem ([BLR93])

Observation

For any S ⊆ [n], we have

ε ≤ dLIN(f ) = Pr[f (x) 6≡∑i∈S

xi ] = Pr[(−1)f (x) 6= χS(x)]

= E[1− (−1)f (x)χS(x)

1− g(S)

2(g := (−1)f ).

This fact can be used to analyze the rejection probability.

Fourier analysis is

• powerful enough to study specific properties.

• not powerful enough to obtain general results.

Observation

xi ] = Pr[(−1)f (x) 6= χS(x)]

= E[1− (−1)f (x)χS(x)

1− g(S)

2(g := (−1)f ).

Fourier analysis is

Observation

xi ] = Pr[(−1)f (x) 6= χS(x)]

= E[1− (−1)f (x)χS(x)

1− g(S)

2(g := (−1)f ).

Fourier analysis is

Higher order Fourier analysis

We look at correlations with polynomials instead of linear functions.

Main technical tools:

• Decomposition theoremA function can be decomposed into a structured part +pseudorandom part (with respect to low-degree polynomials)

• Equidistribution theorem“generic” polynomials look independently distributed.

Caveat: In this talk, we do not touch most of technical foundationssuch as Gowers norm, rank, and bias.

Oblivious tester

Definition

fAn oblivious tester works as follows:

• Take a restriction f |H .• H: random affine

subspace of dimension h(ε).

• Output based only on f |H .

Motivation: avoid “unnatural” properties such as f ∈ P ⇔ n is even.For natural properties, ∃ a tester ⇒ ∃ an oblivious tester [BGS10].

Decomposition theorem

µf ,h: the distribution of f |H .

Observation

A tester cannot distinguish f from g if µf ,h ≈ µg ,h.

Theorem (Decomposition theorem)

Any function can be decomposed as f = f1 + f2 + f3 for d = d(ε, h):

• f1 = Γ(P1, . . . ,PC ) for “generic” degree-d polynomials {Pi}.• f2: small L2 norm.

• f3: uncorrelated with degree-d polynomials.

The pseudorandom parts f2 and f3 do not affect µf ,h much.⇒ we can focus on the structured part f1.

Observation

Decomposition theorem for Fourier analysis

Any function f : F2 → {−1, 1} can be decomposed as:

f =∑

S⊆[n]:|f (S)|>ε

f (S)χS(x) +∑

S⊆[n]:|f (S)|≤ε

f (S)χS(x).

• By Perseval’s identity∑

S f (S)2 = 1⇒ the former depends only on O(1/ε2) many linear functions.

• The latter has negligible Fourier coefficients.

f =∑

S⊆[n]:|f (S)|>ε

f (S)χS(x) +∑

S⊆[n]:|f (S)|≤ε

f (S)χS(x).

S f (S)2 = 1

⇒ the former depends only on O(1/ε2) many linear functions.

f =∑

S⊆[n]:|f (S)|>ε

f (S)χS(x) +∑

S⊆[n]:|f (S)|≤ε

f (S)χS(x).

f =∑

S⊆[n]:|f (S)|>ε

f (S)χS(x) +∑

S⊆[n]:|f (S)|≤ε

f (S)χS(x).

One-sided error testable ≈Affine-subspace hereditary

Affine-subspace hereditary

Definition

A property P is affine-subspace hereditary iff ∈ P ⇒ f |H ∈ P for any affine subspace H .

• degree-d polynomials, Fourier sparsity, odd-cycle-freeness

• f = gh for some polynomials g , h of degree ≤ d − 1.

• f = g 2 for some polynomial g of degree ≤ d − 1.

Characterization of one-sided error testability

Conjecture ([BGS10])

P is testable with one-sided error by an oblivious tester⇔ P is (essentially) affine-subspace hereditary

⇒ is true [BGS10].

f |H 62 P1. Suppose f 2 P and

3. f is also rejected w.p.> 0, contradiction.

2. 9f |K , rejected

by the tester

Proof sketch:

2. 9f |K , rejected

by the tester

Proof sketch:

2. 9f |K , rejected

by the tester

Proof sketch:

Alternative formulation via linear forms

Think of affine-triangle-freeness:No x , y1, y2 ∈ Fn

2 s.t. f (x + y1) = f (x + y2) = f (x + y1 + y2) = 1.

⇔ No x , y1, y2 ∈ Fn2 s.t.

f (L1(x , y1, y2)) = σ1 for L1(x , y1, y2) = x + y1 and σ1 = 1,

f (L3(x , y1, y2)) = σ3 for L3(x , y1, y2) = x + y1 + y2 and σ3 = 1.

We call this (A = (L1, L2, L3), σ = (σ1, σ2, σ3))-freeness.

• A is called an affine system of linear forms.⇒ well studied in higher order Fourier analysis.

2 s.t. f (x + y1) = f (x + y2) = f (x + y1 + y2) = 1.

⇔ No x , y1, y2 ∈ Fn2 s.t.

2 s.t. f (x + y1) = f (x + y2) = f (x + y1 + y2) = 1.

⇔ No x , y1, y2 ∈ Fn2 s.t.

2 s.t. f (x + y1) = f (x + y2) = f (x + y1 + y2) = 1.

⇔ No x , y1, y2 ∈ Fn2 s.t.

Testability of subspace hereditary properties

Observation

The following are equivalent:

• P is affine-subspace hereditary.

• There exists a (possibly infinite) collection {(A1, σ1), . . .}s.t. f ∈ P ⇔ f is (Ai , σi)-free for each i .

Theorem ([BFH+13])

If each (Ai , σi) has bounded complexity, then the property is testablewith one-sided error.

Testability of subspace hereditary properties

Observation

The following are equivalent:

• P is affine-subspace hereditary.

• There exists a (possibly infinite) collection {(A1, σ1), . . .}s.t. f ∈ P ⇔ f is (Ai , σi)-free for each i .

Theorem ([BFH+13])

If each (Ai , σi) has bounded complexity, then the property is testablewith one-sided error.

Proof idea

Let’s focus on the case f = Γ(P1, . . . ,PC ) and P = affine 4-freeness.

f is ε-far from P⇒ There are x∗, y ∗1 , y

∗2 ∈ Fn

2 spanning an affine triangle.

Prx ,y1,y2

[f (x + y1) = f (x + y2) = f (x + y1 + y2) = 1]

≥ Prx ,y1,y2

[Pi(Lj(x , y1, y2)) = Pi(Lj(x∗, y ∗1 , y

∗2 )) ∀i ∈ [C ], j ∈ [3]],

which is non-negligibly high from the equidistribution theorem.⇒ Random sampling works.

Proof idea

∗2 ∈ Fn

Prx ,y1,y2

[f (x + y1) = f (x + y2) = f (x + y1 + y2) = 1]

≥ Prx ,y1,y2

[Pi(Lj(x , y1, y2)) = Pi(Lj(x∗, y ∗1 , y

∗2 )) ∀i ∈ [C ], j ∈ [3]],

Proof idea

∗2 ∈ Fn

Prx ,y1,y2

[f (x + y1) = f (x + y2) = f (x + y1 + y2) = 1]

≥ Prx ,y1,y2

[Pi(Lj(x , y1, y2)) = Pi(Lj(x∗, y ∗1 , y

∗2 )) ∀i ∈ [C ], j ∈ [3]],

Equidistribution theorem

The space Fn2 can be divided according to {Pi(Lj(x))}i∈[C ],j∈[3].

Theorem (Equidistribution theorem)

If each Pi is “generic” enough, then each cell has almost the samesize.

Almost the same size

Equidistribution theorem for Fourier analysis

Let P1, . . . ,PC : Fn2 → F2 be linear functions. If Pi(x) is of the form∑

j∈Si xj , and 1S1 , . . . , 1SC are linearly independent, then for anyσ1, . . . , σC ,

[Pi(x) = σi ∀i ∈ [C ]] =#{x ∈ Fn

2 |∑

j∈Si xj = σi ∀i ∈ [C ]}2n

= 2−|C |.

⇒ Equidistributed.

Testability ⇔ Estimability

Testability ⇐ Estimability

Definition

P is estimable if we can estimate dP(·) to within δ with q(δ) queriesfor any δ > 0.

Trivial direction: P is estimable ⇒ P is testable.

Theorem ([HL13])

P is testable ⇒ P is estimable.

Algorithm:

1: H ← a random affine subspace of a constant dimension.2: return Output dP(f |H).

Definition

Theorem ([HL13])

Algorithm:

Definition

Theorem ([HL13])

Algorithm:

Definition

Theorem ([HL13])

Algorithm:

Intuition behind the proof

Why can we hope dP(f ) ≈ dP(f |H)?

(Oversimplified argument)

• Since P is testable, dP(f ) is determined by the distribution µf ,h.

• If f = Γ(P1, . . . ,PC ), then µf ,h is determined by Γ and degreesof P1, . . . ,PC (rather than Pi ’s themselves).

• f = Γ(P1, . . . ,PC ) and fH = Γ(P1|H , . . . ,PC |H) share the sameΓ and degrees.⇒ µf ,h ≈ µf |H ,h.⇒ dP(f ) ≈ dP(f |H).

Two-sided error testability ⇔Regular-reducibility

Structured part

Recall that, for f = Γ(P1, . . . ,PC ) + f2 + f3,

µf ,h is determined by Γ and degrees of Pi ’s.

Let’s use them as a (constant-size) sketch of f .

Regularity-instance (simplified)

Definition

A regularity-instance I is a tuple of

• a structure function Γ : FC2 → [0, 1],

• a complexity parameter C ∈ N,

• a degree-bound parameter d ∈ N,

• a degree parameter d = (d1, . . . , dC ) ∈ NC with di < d ,

Satisfying a regularity-instance

Definition

Let I = (γ, Γ,C , d ,d) be a regularity-instance.f satisfies I if it is of the form

f (x) = Γ(P1(x), . . . ,PC (x)) + Υ(x),

• Pi is a polynomial of degree di ,

• P1, . . . ,PC are “generic” enough.

• Υ is uncorrelated with degree-(d − 1) polynomials.

Testing regularity-instances

Theorem ([Yos14a])

For any regularity-instance I , there is a tester for the property ofsatisfying I .

Algorithm:

1: H ← a random affine subspace of a constant dimension.2: if f |H is close to satisfying I then accept.3: else reject.

Regular-reducibility

A property P is regular-reducible if for any δ > 0, there exists a setR of constant number of regularity-instances such that:

f 2 P �

� ✏� � g : ✏-far from P� ✏� �

Characterization of two-sided error testability

Theorem

An affine-invariant property P is testablem

P is regular-reducible.

Proof sketch:

• Regular-reducible ⇒ testableRegularity-instances are testable, and testability impliesestimability [HL13]. Hence, we can estimate the distance to R.

• Testable ⇒ regular-reducibleThe behavior of a tester depends only on µf ,h. Since Γ and ddetermines the distribution, we can find R using the tester.

Characterization of two-sided error testability

Theorem

An affine-invariant property P is testablem

P is regular-reducible.

Proof sketch:

• Regular-reducible ⇒ testableRegularity-instances are testable, and testability impliesestimability [HL13]. Hence, we can estimate the distance to R.

• Testable ⇒ regular-reducibleThe behavior of a tester depends only on µf ,h. Since Γ and ddetermines the distribution, we can find R using the tester.

• We need to deal with “non-classical” polynomials instead ofpolynomials.

• Another characterization of testability was shown by introducing“functions limits” [Yos14b].

• Applications of the characterizations:• Low-degree polynomials.• Having a low spectral norm

∑S |f (S)|.

Summary

Higher order Fourier analysis is useful for studying property testing as

• we care about the distribution µf ,h for h = O(1),

• which is determined by the structured part given by thedecomposition theorem.

We are almost done, qualitatively .

• one-sided error testability ≈ affine-subspace hereditary (ofbounded complexity)

• two-sided error testability ⇔ regular-reducibility.

Summary

Higher order Fourier analysis is useful for studying property testing as

• we care about the distribution µf ,h for h = O(1),

• which is determined by the structured part given by thedecomposition theorem.

We are almost done, qualitatively .

• one-sided error testability ≈ affine-subspace hereditary (ofbounded complexity)

• two-sided error testability ⇔ regular-reducibility.

Future direction

Property Testing

• Other groups:• Abelian ⇒ higher order Fourier analysis exists [Sze12].• Non-Abelian ⇒ representation theory? [OY16]

• Why is affine invariance easier to deal with than permutationinvariance?

Other applications of higher order Fourier analysis.

• Coding theory [BG16, BL15a].

• Learning theory [BHT15].

• Complexity theory [BL15b].

• Algorithms for polynomials [Bha14].

Higher-Order Fourier Analysis: Applications to Algebraic...

Documents

Basic results on irregular varieties via Fourier--Mukai methodslibrary.msri.org/books/Book59/files/67pare.pdf · Current Developments in Algebraic Geometry MSRI Publications Volume

Fourier-Mukai Transforms in Algebraic Geometry

HANDBOOK OF MATHEMATICS - hdbom.com · Algebraic complex numbers over Q, 157 Algebraic curve, 204, 455 Algebraic dimension, 289 Algebraic dual, 324 Algebraic element, 719 Algebraic

An Algebraic Oak - Algebraic Poem

Fourier Series & Fourier Transform

2D Image Fourier Spectrum. Image Fourier spectrum Fourier Transform -- Examples

Algebraic Signal Processing Theorypueschel/teaching/18-799F-CMU-spring07/... · Fourier analysis/Fourier transforms on groups G (Beth, Rockmore, Clausen, Maslen, Healy, Terras, …)

Algebraic Numbers and Fourier Analysis - Salem

Higher-Order Fourier Analysis: Applications to Algebraic ...research.nii.ac.jp/~yyoshida/resources/Talks/2014-FOCS.pdf · Higher-Order Fourier Analysis: Applications to Algebraic

Appendix A : Fourier transformAppendix A : Fourier transformoptics.hanyang.ac.kr/~shsong/P4-Appendix-Fourier... · 2016-08-31 · 2D Fourier transform2D Fourier transform ... Heaviside

Higher-Order Fourier Analysis: Applications to Algebraic Property …research.nii.ac.jp/~yyoshida/resources/Talks/2016-CCC... · 2017. 7. 9. · Higher order Fourier analysis We look

Higher order fourier analysis as an algebraic theory I. · 2018-10-26 · Higher order fourier analysis as an algebraic theory I. BALAZS´ SZEGEDY October 25, 2018 Abstract Ergodic

中華大學資訊工程系 Fall 2002 Chap 5 Fourier Series. Page 2 Fourier Analysis Fourier Series Fourier Series Fourier Integral Fourier Integral Discrete Fourier Transform

FOURIER TRANSFORMS. JELMAAN FOURIER: Definition of the Fourier transforms Relationship between Laplace Transforms and Fourier Transforms Fourier transforms

Chapter 1 Algebraic numbers and algebraic integers...Chapter 1 Algebraic numbers and algebraic integers 1.1 Algebraic numbers Deﬁnition 1.1. The number α ∈ C is said to be algebraic

Fourier Transform and its Applications - Karl Rupp · 2012-10-21 · Making use of Fourier transform • Differential equations transform to algebraic equations that are often much

GettingStartedPartIII Rendering - Twilight Render › public_data › ... · TwilightExteli014eadyToRender Partll.skp - Sketch'_lp File Edit View Camera Draw Tools Window Plugins

Algebraic Fractions Algebraic Fractions Simplify Rational Expressions Multiply Rational Algebraic Expressions Add Rational Algebraic Expressions Solve

Fourier and Fourier Transform Why should we learn Fourier Transform?

ALGEBRAIC COMBINATORIAL FOURIER AND LEGENDRE TRANSFORMS ... › pdf › 1805.09812.pdf · ALGEBRAIC COMBINATORIAL FOURIER AND LEGENDRE TRANSFORMS WITH APPLICATIONS IN PERTURBATIVE