77
VC Dimension, VC Density, and the Sauer-Shelah Dichotomy Roland Walker University of Illinois at Chicago 2017 Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 1 / 72

VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

  • Upload
    others

  • View
    5

  • Download
    0

Embed Size (px)

Citation preview

Page 1: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

VC Dimension, VC Density, and the Sauer-ShelahDichotomy

Roland Walker

University of Illinois at Chicago

2017

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 1 / 72

Page 2: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

References

A. Aschenbrenner, A. Dolich, D. Haskell, H. D. Macpherson, and S.Starchenko,Vapnik-Chervonenkis density in some theories without theindependence property, I, Trans. Amer. Math. Soc. 368 (2016),5889-5949.

V. Guingona, On VC-density in VC-minimal theories, arXiv:1409.8060[math.LO].

P. Simon, A Guide to NIP Theories, Cambridge University Press (2015).

A. Chernikov and S. Starchenko, Definable regularity lemmas for NIPhypergraphs, arXiv:1607.07701 [math.LO].

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 2 / 72

Page 3: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Set Systems

Definition

Let X be a set and S ⊆ P(X ). We call the pair (X ,S) a set system.

Definition

Given A ⊆ X , define

S ∩ A = B ∩ A : B ∈ S.

We say A is shattered by S iff: S ∩ A = P(A).

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 3 / 72

Page 4: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

The Shatter Function and VC Dimension

Definition

The function πS : ω → ω given by

πS(n) = max|S ∩ A| : A ∈ [X ]n

is called the shatter function of S.

Definition

The Vapnik-Chervonenkis (VC) dimension of S is

VC(S) = supn < ω : S shatters some A ∈ [X ]n= supn < ω : πS(n) = 2n.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 4 / 72

Page 5: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: X = R, S = Half-Spaces

VC(S) ≥ 2:

VC(S) < 3:

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 5 / 72

Page 6: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: X = R2, S = Half-Spaces

VC(S) ≥ 3 :

VC(S) < 4 :

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 6 / 72

Page 7: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

The Sauer-Shelah Lemma

Let X be a set and S ⊆ P(X ).

Lemma (Sauer-Shelah)

If VC(S) = d < ω, then for all n ≥ d , we have

πS(n) ≤(n

0

)+ · · ·+

(n

d

)= O(nd).

Proof: Suppose VC(S) = d < ω, and fix n > d .

Let A ∈ [X ]n such that |S ∩A| = πS(n), and let (a1, . . . , an) enumerate A.

Inductively define S0, . . . ,Sn ⊆ P(A) as follows:

Let S0 = S ∩ A.

To construct Si+1, remove ai+1 where possible; i.e,

Si+1 = B : B ∈ Si and B\ai+1 ∈ Si∪ B\ai+1 : B ∈ Si and B\ai+1 /∈ Si.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 7 / 72

Page 8: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: S0 = S ∩ A.

A = a1 a2 a3 a4 a5

0 0 0 0 00 0 0 0 10 0 0 1 00 0 1 0 00 0 1 0 10 0 1 1 00 0 1 1 10 1 0 0 00 1 0 1 00 1 1 0 01 0 0 1 01 1 1 1 0

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 8 / 72

Page 9: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: Constructing S1.

A = a1 a2 a3 a4 a5

0 0 0 0 00 0 0 0 10 0 0 1 00 0 1 0 00 0 1 0 10 0 1 1 00 0 1 1 10 1 0 0 00 1 0 1 00 1 1 0 0

⇒ 1 0 0 1 0 ⇐1 1 1 1 0

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 9 / 72

Page 10: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: Constructing S1..

A = a1 a2 a3 a4 a5

0 0 0 0 00 0 0 0 1

→ 0 0 0 1 0 ←0 0 1 0 00 0 1 0 10 0 1 1 00 0 1 1 10 1 0 0 00 1 0 1 00 1 1 0 0

⇒ 1 0 0 1 0 ⇐1 1 1 1 0

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 10 / 72

Page 11: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: Constructing S1...

A = a1 a2 a3 a4 a5

0 0 0 0 00 0 0 0 1

→ 0 0 0 1 0 ←0 0 1 0 00 0 1 0 10 0 1 1 00 0 1 1 10 1 0 0 00 1 0 1 00 1 1 0 0

⇒ 1 0 0 1 0 ⇐1 1 1 1 0

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 11 / 72

Page 12: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: Constructing S1....

A = a1 a2 a3 a4 a5

0 0 0 0 00 0 0 0 10 0 0 1 00 0 1 0 00 0 1 0 10 0 1 1 00 0 1 1 10 1 0 0 00 1 0 1 00 1 1 0 01 0 0 1 0

⇒ 1 1 1 1 0 ⇐

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 12 / 72

Page 13: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: Constructing S1.....

A = a1 a2 a3 a4 a5

0 0 0 0 00 0 0 0 10 0 0 1 00 0 1 0 00 0 1 0 10 0 1 1 00 0 1 1 10 1 0 0 00 1 0 1 00 1 1 0 01 0 0 1 0

⇒ 0 1 1 1 0 ⇐

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 13 / 72

Page 14: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: Constructing S1......

A = a1 a2 a3 a4 a5

0 0 0 0 00 0 0 0 10 0 0 1 00 0 1 0 00 0 1 0 10 0 1 1 00 0 1 1 10 1 0 0 00 1 0 1 00 1 1 0 01 0 0 1 00 1 1 1 0

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 14 / 72

Page 15: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: Constructing S2.

A = a1 a2 a3 a4 a5

0 0 0 0 00 0 0 0 10 0 0 1 00 0 1 0 00 0 1 0 10 0 1 1 00 0 1 1 1

⇒ 0 1 0 0 0 ⇐0 1 0 1 00 1 1 0 01 0 0 1 00 1 1 1 0

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 15 / 72

Page 16: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: Constructing S2..

A = a1 a2 a3 a4 a5

→ 0 0 0 0 0 ←0 0 0 0 10 0 0 1 00 0 1 0 00 0 1 0 10 0 1 1 00 0 1 1 1

⇒ 0 1 0 0 0 ⇐0 1 0 1 00 1 1 0 01 0 0 1 00 1 1 1 0

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 16 / 72

Page 17: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: Constructing S2...

A = a1 a2 a3 a4 a5

→ 0 0 0 0 0 ←0 0 0 0 10 0 0 1 00 0 1 0 00 0 1 0 10 0 1 1 00 0 1 1 1

⇒ 0 1 0 0 0 ⇐0 1 0 1 00 1 1 0 01 0 0 1 00 1 1 1 0

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 17 / 72

Page 18: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: Constructing S2....

A = a1 a2 a3 a4 a5

0 0 0 0 00 0 0 0 10 0 0 1 00 0 1 0 00 0 1 0 10 0 1 1 00 0 1 1 10 1 0 0 00 1 0 1 00 1 1 0 01 0 0 1 00 1 1 1 0

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 18 / 72

Page 19: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: Constructing S3.

A = a1 a2 a3 a4 a5

0 0 0 0 00 0 0 0 10 0 0 1 00 0 1 0 00 0 1 0 10 0 1 1 0

⇒ 0 0 1 1 1 ⇐0 1 0 0 00 1 0 1 00 1 1 0 01 0 0 1 00 1 1 1 0

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 19 / 72

Page 20: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: Constructing S3..

A = a1 a2 a3 a4 a5

0 0 0 0 00 0 0 0 10 0 0 1 00 0 1 0 00 0 1 0 10 0 1 1 0

⇒ 0 0 0 1 1 ⇐0 1 0 0 00 1 0 1 00 1 1 0 01 0 0 1 00 1 1 1 0

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 20 / 72

Page 21: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: Constructing S3...

A = a1 a2 a3 a4 a5

0 0 0 0 00 0 0 0 10 0 0 1 00 0 1 0 00 0 1 0 10 0 1 1 00 0 0 1 10 1 0 0 00 1 0 1 00 1 1 0 01 0 0 1 00 1 1 1 0

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 21 / 72

Page 22: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: Constructing S4.

A = a1 a2 a3 a4 a5

0 0 0 0 00 0 0 0 10 0 0 1 00 0 1 0 00 0 1 0 10 0 1 1 00 0 0 1 10 1 0 0 00 1 0 1 00 1 1 0 01 0 0 0 00 1 1 1 0

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 22 / 72

Page 23: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: Constructing S5.

A = a1 a1 a3 a4 a5

0 0 0 0 00 0 0 0 10 0 0 1 00 0 1 0 00 0 1 0 10 0 1 1 00 0 0 1 10 1 0 0 00 1 0 1 00 1 1 0 01 0 0 0 00 1 1 1 0

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 23 / 72

Page 24: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Back to proof...

Lemma (Sauer-Shelah)

If VC(S) = d < ω, then for all n ≥ d , we have

πS(n) ≤(n

0

)+ · · ·+

(n

d

)= O(nd).

Proof (continued):

Notice that after stage i + 1, we have the following:

|Si+1| = |Si |.

Given A′ ⊆ A, if Si+1 shatters A′, then Si shatters A′.

Given B ∈ Si+1 and B ′ ⊆ B ∩ a1, . . . , ai+1,B ′ ∪ (B\a1, . . . , ai+1) ∈ Si+1.

Because of this, any member of Sn has cardinality at most d .

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 24 / 72

Page 25: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

VC Density

Definition

The VC density of S is

vc(S) = infr ∈ R>0 : πS(n) = O(nr )

= lim sup

n→ω

log π(n)

log n.

Lemma (Sauer-Shelah)

If VC(S) = d < ω, then for all n ≥ d , we have

πS(n) ≤(n

0

)+ · · ·+

(n

d

)= O(nd).

Corollary

vc(S) ≤ VC(S).

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 25 / 72

Page 26: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: When S is “uniform,” VC dimension and VCdensity agree.

Let X be an infinite set and S = [X ]≤d for some d < ω.

We have

πS(n) =

(n

0

)+ · · ·+

(n

d

),

so

VC(S) = vc(S) = d .

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 26 / 72

Page 27: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: VC dimension is more susceptible to localanomalies than VC density.

Let X = ω,m < ω, and S = P(m).

It follows that

πS(n) =

2n if n ≤ m

2m otherwise.

So

VC(S) = mand

vc(S) = lim supn→ω

log 2m

log n= 0.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 27 / 72

Page 28: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

The Dual Shatter Function

Definition

Given A1, ...,An ⊆ X , let S(A1, ...,An) denote the set of atoms in theBoolean algebra generated by A1, ...,An. That is

S(A1, · · · ,An) =

n⋂

i=1

Aσ(i)i : σ ∈ n2

\∅

where A1i = Ai and A0

i = X \ Ai .

Definition

The function π∗S : ω → ω given by

π∗S(n) = max|S(A1, ...,An)| : A1, ...An ∈ S

is called the dual shatter function of S.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 28 / 72

Page 29: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Dual VC Dimension and Dual VC Density

Definition

The dual VC dimension of S is

VC∗(S) = sup n < ω : π∗S(n) = 2n .

Definition

The dual VC density of S is

vc∗(S) = infr ∈ R>0 : π∗S(n) = O(nr )

.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 29 / 72

Page 30: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: X = R, S = Half-Spaces

VC∗(S) ≥ 1:

VC∗(S) < 2:

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 30 / 72

Page 31: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: X = R2, S = Half-Spaces

VC∗(S) ≥ 2 :

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 31 / 72

Page 32: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: X = R2, S = Half-Spaces

VC∗(S) < 3 :

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 32 / 72

Page 33: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Set Systems in a Model-Theoretic Context

Consider a sorted language L with sorts indexed by I .

Let M be an L-structure with domains (Mi : i ∈ I ).

Definition

Given an L-formula φ(x , y) where x = (x i11 , ..., xiss ) and y = (y j1

1 , ..., yjtt ),

define

Sφ = φ(X , b) : b ∈ Y

where X = Mi1 × · · · ×Mis and Y = Mj1 × · · · ×Mjt .

It follows that (X ,Sφ) is a set system. To ease notation, we let:

πφ denote πSφ , VC (φ) denote VC (Sφ), and vc(φ) denote vc(Sφ).

Similarly, we use π∗φ for π∗Sφ , VC∗(φ) for VC∗(Sφ), and vc∗(φ) for vc∗(Sφ).

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 33 / 72

Page 34: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

The dual shatter function of φ is really counting φ-types.

By definition, we have π∗φ(n) = max |S(φ(X , b) : b ∈ B)| : B ∈ [Y ]n.

Let B ∈ [Y ]n. Recall that

S(φ(X , b) : b ∈ B) =

⋂b∈B

φσ(b)(X , b) : σ ∈ B2

\∅.

There is a bijection

S(φ(X , b) : b ∈ B) −→

tpφ(a/B) : a ∈ X

= Sφ(B)

given by ⋂b∈B

φσ(b)(X , b) 7−→φσ(b)(x , b) : b ∈ B

.

It follows that|S(φ(X , b) : b ∈ B)| = |Sφ(B)|.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 34 / 72

Page 35: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

The Dual of a Formula

Definition

We call a formula φ(x ; y) a partitioned formula with object variable(s)x = (x1, ..., xs) and parameter variable(s) y = (y1, ..., yt).

Definition

We let φ∗(y ; x) denote the dual of φ(x ; y), meaning φ∗(y ; x) is φ(x ; y)but we view y as the object and x as the parameter.

It follows that

Sφ∗ = φ∗(Y , a) : a ∈ X= φ(a,Y ) : a ∈ X.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 35 / 72

Page 36: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

The shatter function of φ∗ is also counting φ-types.

By definition, we have πφ∗(n) = max |Sφ∗ ∩ B| : B ∈ [Y ]n .

Let B ∈ [Y ]n. It follows that

Sφ∗ ∩ B = φ∗(B, a) : a ∈ X= φ(a,B) : a ∈ X

There is a bijection

φ(a,B) : a ∈ X −→ tpφ(a/B) : a ∈ X = Sφ(B)

given byφ(a,B) 7−→ tpφ(a/B).

It follows that|Sφ∗ ∩ B| = |Sφ(B)|.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 36 / 72

Page 37: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Duality in a Model-Theoretic Context

Lemma

The dual shatter function of φ is the shatter function of φ∗.That is π∗φ = πφ∗ .

Proof: For all n < ω, we have

π∗φ(n) = max|S(φ(X , b) : b ∈ B)| : B ∈ [Y ]n= max|Sφ(B)| : B ∈ [Y ]n= max|Sφ∗ ∩ B| : B ∈ [Y ]n= πφ∗(n).

Corollary

VC∗(φ) = VC(φ∗) and vc∗(φ) = vc(φ∗).

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 37 / 72

Page 38: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

VC(φ) < ω ⇐⇒ VC∗(φ) < ω

Lemma

VC(φ) < 2VC∗(φ)+1.

Proof: Suppose VC(φ) ≥ 2n, there exists A ∈ [X ]2n

shattered by Sφ.

Let aJ : J ⊆ n enumerate A.

For all i < n, let bi ∈ Y such that M |= φ(aJ , bi )⇐⇒ i ∈ J.

Let B = bi : i < n.It follows that Sφ∗ shatters B, so VC(φ∗) ≥ n.

Corollary

VC∗(φ) < 2VC(φ)+1.

Corollary

VC(φ) < ω ⇐⇒ VC∗(φ) < ω.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 38 / 72

Page 39: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

VC Dimension, VC Density, and the Sauer-ShelahDichotomy

Roland Walker

University of Illinois at Chicago

2017

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 39 / 72

Page 40: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

The Dual Shatter Function

Definition

Given A1, ...,An ⊆ X , let S(A1, ...,An) denote the set of atoms in theBoolean algebra generated by A1, ...,An. That is

S(A1, · · · ,An) =

n⋂

i=1

Aσ(i)i : σ ∈ n2

\∅

where A1i = Ai and A0

i = X \ Ai .

Definition

The function π∗S : ω → ω given by

π∗S(n) = max|S(A1, ...,An)| : A1, ...An ∈ S

is called the dual shatter function of S.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 40 / 72

Page 41: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Dual VC Dimension and Dual VC Density

Definition

The dual VC dimension of S is

VC∗(S) = sup n < ω : π∗S(n) = 2n .

Definition

The dual VC density of S is

vc∗(S) = infr ∈ R>0 : π∗S(n) = O(nr )

.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 41 / 72

Page 42: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: X = R, S = Half-Spaces

VC∗(S) ≥ 1:

VC∗(S) < 2:

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 42 / 72

Page 43: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: X = R2, S = Half-Spaces

VC∗(S) ≥ 2 :

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 43 / 72

Page 44: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Example: X = R2, S = Half-Spaces

VC∗(S) < 3 :

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 44 / 72

Page 45: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Set Systems in a Model-Theoretic Context

Consider a sorted language L with sorts indexed by I .

Let M be an L-structure with domains (Mi : i ∈ I ).

Definition

Given an L-formula φ(x , y) where x = (x i11 , ..., xiss ) and y = (y j1

1 , ..., yjtt ),

define

Sφ = φ(X , b) : b ∈ Y

where X = Mi1 × · · · ×Mis and Y = Mj1 × · · · ×Mjt .

It follows that (X ,Sφ) is a set system. To ease notation, we let:

πφ denote πSφ , VC (φ) denote VC (Sφ), and vc(φ) denote vc(Sφ).

Similarly, we use π∗φ for π∗Sφ , VC∗(φ) for VC∗(Sφ), and vc∗(φ) for vc∗(Sφ).

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 45 / 72

Page 46: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Duality in a Model-Theoretic Context

Lemma

The dual shatter function of φ is the shatter function of φ∗.That is π∗φ = πφ∗ .

Proof: For all n < ω, we have

π∗φ(n) = max|S(φ(X , b) : b ∈ B)| : B ∈ [Y ]n= max|Sφ(B)| : B ∈ [Y ]n= max|Sφ∗ ∩ B| : B ∈ [Y ]n= πφ∗(n).

Corollary

VC∗(φ) = VC(φ∗) and vc∗(φ) = vc(φ∗).

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 46 / 72

Page 47: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

VC(φ) < ω ⇐⇒ VC∗(φ) < ω

Lemma

VC(φ) < 2VC∗(φ)+1.

Proof: Suppose VC(φ) ≥ 2n, there exists A ∈ [X ]2n

shattered by Sφ.

Let aJ : J ⊆ n enumerate A.

For all i < n, let bi ∈ Y such that M |= φ(aJ , bi )⇐⇒ i ∈ J.

Let B = bi : i < n.It follows that Sφ∗ shatters B, so VC(φ∗) ≥ n.

Corollary

VC∗(φ) < 2VC(φ)+1.

Corollary

VC(φ) < ω ⇐⇒ VC∗(φ) < ω.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 47 / 72

Page 48: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Key Point from Last Week

In the classical context...

π∗S(n) is counting atoms generated by n sets in S.

In the model-theoretic context...

π∗φ(n) is counting φ-types over n parameters.

So by duality...

πφ(n) is counting φ∗-types over n parameters.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 48 / 72

Page 49: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Duality in the Classical Context

Given (X ,S) a set system, let M = (X ,S,∈), and φ(x , y) be x ∈ y .

It follows that S = Sφ, so by definition, πS = πφ and π∗S = π∗φ.

Let X ∗ = S and

S∗ = B ∈ S : a ∈ B : a ∈ X= φ∗(S, a) : a ∈ X.

It follows that S∗ = Sφ∗ , so by definition, πS∗ = πφ∗ and π∗S∗ = π∗φ∗ .

Definition

We call (X ∗,S∗) the dual of (X ,S).

Lemma

π∗S = πS∗ and π∗S∗ = πS .

Proof: π∗S = π∗φ = πφ∗ = πS∗ and π∗S∗ = π∗φ∗ = πφ = πS .

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 49 / 72

Page 50: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Duality in the Classical Context

Corollary

VC∗(S) = VC(S∗) and vc∗(S) = vc(S∗).

Corollary

For any set system (X ,S), we have

VC(S) < 2VC∗(S)+1

andVC∗(S) < 2VC(S)+1.

Corollary

VC(S) < ω ⇐⇒ VC∗(S) < ω.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 50 / 72

Page 51: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Elementary Properties

Lemma

π∗φ is elementary (i.e., elementarily equivalent L-structures agree on π∗φ).

Proof: Given n < ω, let σ ∈ P(n)2. Consider the L-sentence

∃y1, ..., yn∧J⊆n

[∃x

n∧i=1

φ[i∈J](x , yi )

]σ(J)

.

Corollary

VC∗(φ) and vc∗(φ) are elementary.

Corollary

VC(φ) and vc(φ) are elementary.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 51 / 72

Page 52: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

NIP Formulae

Let T be a complete L-theory, and let φ(x , y) ∈ L.

Definition

We say φ has the independence property (IP) iff: for some M |= T , thereexists sequences (ai : i < ω) ⊆ M |x | and (bJ : J ⊆ ω) ⊆ M |y | such that

M |= φ(ai , bJ) ⇐⇒ i ∈ J.

If φ is not IP, we say φ is NIP.

Lemma

φ is IP ⇐⇒ VC(φ) = ω.

Proof: Compactness.

Corollary

φ is NIP ⇐⇒ VC(φ) < ω ⇐⇒ VC∗(φ) < ω.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 52 / 72

Page 53: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

NIP and vcT

Let T be a complete L-theory.

Definition

We say T is NIP iff: every partitioned L-formula is NIP.

Fact: It is sufficient to check all φ(x , y) with |y | = 1 (and |x | ≥ 1).

Open Question: Is it possible for vc(φ) to be irrational in an NIP theory?

Definition

The VC density of T is the function

vcT : ω −→ R≥0 ∪ ∞

defined by

vcT (n) = supvc(φ) : φ(x , y) ∈ L, |y | = n= supvc∗(φ) : φ(x , y) ∈ L, |x | = n.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 53 / 72

Page 54: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

NIP and vcT

Lemma

If vcT (n) <∞ for all n < ω, then T is NIP.

Note: Converse is not true in general; e.g., consider T eq where T is NIP.

Open Questions:

1 For every language L and every complete L-theory T , doesvcT (1) <∞ imply vcT (n) <∞ for all n < ω?

2 If so, is there some bounding function β, independent of L and T ,such that vcT (n) < β(vcT (1), n)?

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 54 / 72

Page 55: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Finite Types

Let ∆(x , y) be a finite set of L-formulae (with free variables x and y).

Definition

The set system generated by ∆ is

S∆ =φ(M |x |, b

): φ(x , y) ∈ ∆, b ∈ M |y |

.

The dual shatter function of ∆ is

π∗∆(n) = max|S∆(B)| : B ∈

[M |y |

]n.

The dual VC density of ∆ is

vc∗∆(n) = infr ∈ R>0 : π∗∆(n) = O(nr ).

Fact: π∗∆ and vc∗∆ are elementary.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 55 / 72

Page 56: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Defining Schemata

Let ∆(x , y) ⊆ L and B ⊆ M |y | both be finite. Let p ∈ S∆(B).

Definition

Given a schemad(y , z) = dφ(y , z) : φ ∈ ∆ ⊆ L

and a parameter c ∈ M |z|, we say that d(y , c) defines p iff:for every φ ∈ ∆ and b ∈ B, we have

φ(x , b) ∈ p ⇐⇒ M |= dφ(b, c).

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 56 / 72

Page 57: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

UDTFS and the VC n Property

Let ∆(x , y) ⊆ L be finite.

Definition

We say ∆ has uniform definability of types over finite sets (UDTFS) withn parameters iff: there is a finite family F of schemata each of the form

d(y , z1, ..., zn) = dφ(y , z1, ..., zn) : φ ∈ ∆

with |y | = |z1| = · · · = |zn| such that if B ⊆ M |y | is finite andp(x) ∈ S∆(B), then for some d ∈ F and b1, ..., bn ∈ B, d(y , b) defines p.

Definition

An L-structure has the VC n property iff:all finite ∆(x , y) ⊆ L with |x | = 1 have UDTFS with n parameters.

Fact: UDTFS with n parameters and VC n are elementary properties.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 57 / 72

Page 58: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Uniform Bounds on VC Density

Theorem (5.7)

If M has the VC n property, then every finite ∆(x , y) ⊆ L has UDTFSwith n|x | parameters.

Corollary (5.8a)

If M has the VC n property, then for every finite ∆(x , y) ⊆ L, we havevc∗(∆) ≤ n|x |.

Proof: Given ∆(x , y) finite, there exists finite F witnessing UDTFS withn|x | parameters. It follows that |S∆(B)| ≤ |F||B|n|x |.

Corollary (5.8b)

If T is complete and has the VC n property, then for all m < ω, we havevcT (m) ≤ nm.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 58 / 72

Page 59: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Weakly O-Minimal Theories are VC 1

Theorem (6.1)

If T is complete and weakly o-minimal, then T has the VC 1 property.

Proof: Let M |= T , and let ∆(x , y) ⊆ L be finite with |x | = 1.

By Compactness, there exists n < ω such that for all φ ∈ ∆ and b ∈ M |y |,

φ(M, b) has at most n maximal convex components.

For all φ ∈ ∆ and i < n, there exists φi (x , y) ∈ L such that for eachb ∈ M |y |,

φi (M, b) is the i thcomponent of φ(M, b).

It follows thatM |= φ(x , y) ↔

∨i<n

φi (x , y).

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 59 / 72

Page 60: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Proof of Theorem (cont.)

For each φ ∈ ∆ and i < n, let

φ≤i (x , y) be ∃x0 [φi (x0, y) ∧ x ≤ x0]

φ<i (x , y) be ∀x0 [φi (x0, y) → x < x0].

It follows that

M |= φi (x , y) ↔ φ≤i (x , y) ∧ ¬φ<i (x , y).

If we let Ψ = φ<i , φ≤i : φ ∈ ∆, i < n, each formula in ∆ is

T -equivalent to a boolean combination of 2n formulae in Ψ.

For each ψ ∈ Ψ and b ∈ M |y |, notice that ψ(M, b) is an initial segment ofM, so SΨ is directed.

Lemma 5.2 ⇒ Ψ has UDTFS with one parameter.

Lemma 5.5 ⇒ ∆ has UDTFS with one parameter.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 60 / 72

Page 61: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Application: RCVF

Let L = + ,− , · , 0 , 1 , < , |.

RCVF (with a proper convex valuation ring) where | is the divisibilitypredicate (i.e., a|b ⇔ v(a) ≤ v(b)) is a complete L-theory.

Cherlin and Dickmann showed RCVF has quantifier elimination and is,therefore, weakly o-minimal.

Corollary (6.2a)

In RCVF, if ∆(x , y) ⊆ L is finite, then vc∗(∆) ≤ |x |.

Corollary (6.2b)

vcRCVF(n) ≤ n for all n < ω.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 61 / 72

Page 62: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Application: ACVF(0,0)

Let L = + ,− , · , 0 , 1 , |.

ACVF(0,0) where | is the divisibility predicate is complete in L.

Let R |= RCVF (in L ∪ <).

Consider R(i) where i2 = −1 and

a + bi | c + di ⇔ a2 + b2 | c2 + d2.

It follows that R(i) |= ACVF(0,0) and is interpretable in R.

Corollary (6.3a)

In ACVF(0,0), if ∆(x , y) ⊆ L is finite, then vc∗(∆) ≤ 2|x |.

Corollary (6.3b)

vcACVF(0,0)(n) ≤ 2n, for all n < ω.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 62 / 72

Page 63: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Open Questions

1 For every language L and every complete L-theory T , doesvcT (1) <∞ imply vcT (n) <∞ for all n < ω?

RCVF : Yes ACVF(0,p) : ?

ACVF(0,0) : Yes ACVF(p,p) : ?

2 If so, is there some bounding function β, independent of L and T ,such that vcT (n) < β(vcT (1), n)?

RCVF : β(n) = n ACVF(0,p) : ?

ACVF(0,0) : β(n) = 2n ACVF(p,p) : ?

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 63 / 72

Page 64: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Counting TypesLet L be a language, M an L-structure, φ(x , y) ∈ L with |x | = 1, andB ⊆ M |y |.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 64 / 72

Page 65: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Regular Pairs

Let G = (V ,E ) be a finite graph. Fix ε, δ ∈ [0, 1].

Definition

Given A,B ⊆ V , we say the pair (A,B) is (ε, δ)-regular iff: there existsα ∈ [0, 1] such that for all nonempty sets A′ ⊆ A and B ′ ⊆ B with|A′| ≥ δ|A| and |B ′| ≥ δ|B|, we have |d(A′,B ′)− α| ≤ ε

2 .

d(A,B) = E(A,B)|A||B| = 2

200 = 0.01

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 65 / 72

Page 66: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Regular Pairs

Let G = (V ,E ) be a finite graph. Fix ε, δ ∈ [0, 1].

Definition

Given A,B ⊆ V , we say the pair (A,B) is (ε, δ)-regular iff: there existsα ∈ [0, 1] such that for all nonempty sets A′ ⊆ A and B ′ ⊆ B with|A′| ≥ δ|A| and |B ′| ≥ δ|B|, we have |d(A′,B ′)− α| ≤ ε

2 .

d(A,B) = E(A,B)|A||B| = 2

200 = 0.01

α = 0 δ = 0.5 ε = 0.04

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 65 / 72

Page 67: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Regular Pairs

Let G = (V ,E ) be a finite graph. Fix ε, δ ∈ [0, 1].

Definition

Given A,B ⊆ V , we say the pair (A,B) is (ε, δ)-regular iff: there existsα ∈ [0, 1] such that for all nonempty sets A′ ⊆ A and B ′ ⊆ B with|A′| ≥ δ|A| and |B ′| ≥ δ|B|, we have |d(A′,B ′)− α| ≤ ε

2 .

d(A,B) = E(A,B)|A||B| = 2

200 = 0.01

α = 0 δ = 0.2 ε = 0.25

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 65 / 72

Page 68: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Defect

Let P be a finite partition of V . Fix η ∈ [0, 1].

Definition

The defect of P is

defε,δ(P) := (A,B) ∈ P2 : (A,B) not (ε, δ)-regular,

and we say that P is (ε, δ, η)-regular iff:∑(A,B) ∈ defε,δ(P)

|A||B| ≤ η|V |2.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 66 / 72

Page 69: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Szemeredi Regularity Lemma (without Equipartition)

Lemma

For all ε, δ, η > 0, there exists M = M(ε, δ, η) such that any finite graphhas an (ε, δ, η)-regular partition with at most M parts.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 67 / 72

Page 70: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Szemeredi Regularity Lemma (without Equipartition)

Lemma

For all ε, δ, η > 0, there exists M = M(ε, δ, η) such that any finite graphhas an (ε, δ, η)-regular partition with at most M parts.

(Szemeredi 1976) M(ε, ε, ε) ≤ twr2(O(ε−5))

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 67 / 72

Page 71: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Szemeredi Regularity Lemma (without Equipartition)

Lemma

For all ε, δ, η > 0, there exists M = M(ε, δ, η) such that any finite graphhas an (ε, δ, η)-regular partition with at most M parts.

(Szemeredi 1976) M(ε, ε, ε) ≤ twr2(O(ε−5))

How fast does M grow as δ → 0?

(Gowers 1997) M(1− δ1/16, δ, 1− 20δ1/16) ≥ twr2(Ω(δ−1/16))

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 67 / 72

Page 72: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Szemeredi Regularity Lemma (without Equipartition)

Lemma

For all ε, δ, η > 0, there exists M = M(ε, δ, η) such that any finite graphhas an (ε, δ, η)-regular partition with at most M parts.

(Szemeredi 1976) M(ε, ε, ε) ≤ twr2(O(ε−5))

How fast does M grow as δ → 0?

(Gowers 1997) M(1− δ1/16, δ, 1− 20δ1/16) ≥ twr2(Ω(δ−1/16))

How fast does M grow as η → 0?

(Conlon-Fox 2012) ∃ ε, δ > 0 such that M(ε, δ, η) ≥ twr2(Ω(η−1))

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 67 / 72

Page 73: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Definable Regularity Lemma for NIP Relations

Let k ≥ 2 and d ∈ N.

Theorem

There is a constant c = c(k , d) such that IF

ε, δ, η > 0

E = φ(V ) for some φ(v1, . . . , vk) ∈ LM and structure MVC(E ) ≤ d

each µi is a Keisler measure on Vi which is fap on E

THEN there is an (ε, δ, η)-regular partition P of V with 0-1 densities suchthat

‖P‖ ≤ O(γ−c) where γ = minε, δ, ηfor each Pi , the parts of Pi are definable using a single formula ψi

which is a boolean combination of φ depending only on γ and φ.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 68 / 72

Page 74: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Stability

Let d ∈ N and R ⊆ V ×W .

Definition

We say R is d-stable iff: there is not a tree of parametersbτ : τ ∈ <d2 ⊆W along with a set of leaves aσ : σ ∈ d2 ⊆ V suchthat for any σ ∈ d2 and n < d , we have (aσ, bσn) ∈ R ⇐⇒ σ(n) = 1.

a(011) |= ¬R(x , b()) ∧ R(x , b(0)) ∧ R(x , b(01))

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 69 / 72

Page 75: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Definable Regularity Lemma for Stable Relations

Let k ≥ 2 and d ∈ N.

Theorem

There is a constant c = c(k , d) such that IF

ε, δ > 0 and η = 0

E = φ(V ) for some φ(v1, . . . , vk) ∈ LM and structure ME is d-stable

each µi is a Keisler measure on Vi which is fap on E

THEN there is an (ε, δ, η)-regular partition P of V with 0-1 densities suchthat

‖P‖ ≤ O(γ−c) where γ = minε, δfor each Pi , the parts of Pi are definable using a single formula ψi

which is a boolean combination of φ depending only on γ and φ.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 70 / 72

Page 76: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

DistalityLet T be a complete NIP theory and U a monster model for T .

Definition

We say T is distal iff: for all n ≥ 1, all indiscernible sequences I ⊆ Un, andall Dedekind cuts I = I1 + I2 + I3, if

I1 + a + I2 + I3 and I1 + I2 + b + I3

are both indiscernible, then

I1 + a + I2 + b + I3

is also indiscernible.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 71 / 72

Page 77: VC Dimension, VC Density, and the Sauer-Shelah Dichotomy

Definable Regularity Lemma for Distal NIP Structures

Let T be a complete distal NIP theory and M |= T .

Let k ≥ 2 and φ(v1, . . . , vk) ∈ LM .

Theorem

There is a constant c = c(M, φ) such that IF

ε = δ = 0 and η > 0

E = φ(V )

each µi is a Keisler measure on Vi which is fap on E

THEN there is an (ε, δ, η)-regular partition P of V with 0-1 densities suchthat

‖P‖ ≤ O(η−c)

for each Pi , the parts of Pi are definable using a single formula ψi

which is a boolean combination of φ depending only on φ.

Roland Walker (UIC) VC Dimension, VC Density, & Sauer-Shelah 2017 72 / 72