The Probabilistic Method

Joshua BrodyCS49/Math59

Fall 2015

Week 11: Applications, Randomized Algorithms

Reading Quiz

(A) memory space (hint: not the right answer)

(B) runtime

(C) power consumption

(D) approximation factor

(E) error guarantee

What resource is typically not an important factor in the analysis of streaming algorithms?

Reading Quiz

(A) memory space (hint: not the right answer)

(B) runtime

(C) power consumption

(D) approximation factor

(E) error guarantee

What resource is typically not an important factor in the analysis of streaming algorithms?

Definition: A streaming algorithm A is an (ɛ,𝜹)-approximation for f if for any stream S

Pr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-𝜹!

PAC Algorithms

F2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)

A(S)

Probably Approximately Correct (PAC) model:“most of the time we’re close enough”

Definition: A streaming algorithm A is an (ɛ,𝜹)-approximation for f if for any stream S

Pr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-𝜹!

Theorem: For any ɛ > 0, there is a streaming algorithm that (ɛ,1/3)-approximates for F2 in O([log(n) + log(m)]/ɛ2) space.

PAC Algorithms

F2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)

A(S)

Probably Approximately Correct (PAC) model:“most of the time we’re close enough”

Clicker Question

(A) no ε makes this work

(B) ε  > 1/3

(C) ε  > ∏/2

(D) ε  > √6

(E) None of the above

For what values of ε does the AMS Basic Estimator have error < 1/3?

Clicker Question

(A) no ε makes this work

(B) ε  > 1/3

(C) ε  > ∏/2

(D) ε  > √6

(E) None of the above

For what values of ε does the AMS Basic Estimator have error < 1/3?

Clicker Question

(A) no k makes this work

(B) k > 6/ε(C) k > 6/ε2

(D) k > √6

(E) None of the above

For arbitrary ε > 0, what values of k give error < 1/3?

Clicker Question

(A) no k makes this work

(B) k > 6/ε(C) k > 6/ε2

(D) k > √6

(E) None of the above

For arbitrary ε > 0, what values of k give error < 1/3?

Improved F2 EstimatorF2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)

X12X22 X32 X42 X52 X62X72X82 X92

AMS Improved Estimator Y:= (∑ Xi2)/k

Improved F2 EstimatorF2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)

X12X22 X32 X42 X52 X62X72X82 X92 Y

AMS Improved Estimator Y:= (∑ Xi2)/k

• k = Ω(1/ε2) : error < 1/3

Improved F2 EstimatorF2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)

AMS Improved Estimator Y:= (∑ Xi2)/k

• k = Ω(1/ε2) : error < 1/3• k = Ω(1/𝜹ε2) : error < 𝜹

X12X22 X32 X42 X52 X62X72X82 X92 X12X22 X32 X42 X52 X62X72X82 X92 X12X22 X32 X42 X52 X62X72X82 X92 X12X22 X32 X42 X52 X62X72X82 X92 X52X72X72X722X192Y

Improved F2 EstimatorF2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)

AMS Improved Estimator Y:= (∑ Xi2)/k

• k = Ω(1/ε2) : error < 1/3• k = Ω(1/𝜹ε2) : error < 𝜹

X12X22 X32 X42 X52 X62X72X82 X92 X12X22 X32 X42 X52 X62X72X82 X92 X12X22 X32 X42 X52 X62X72X82 X92 X12X22 X32 X42 X52 X62X72X82 X92 X52X72X72X722X192Y

Better error dependence possible with Chernoff Bounds

Final F2 EstimatorF2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)

AMS Improved Estimator Y:= (∑ Xi2)/k

• k = Ω(1/ε2) : error < 1/3

Y

Final F2 EstimatorF2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)

AMS Improved Estimator Y:= (∑ Xi2)/k

• k = Ω(1/ε2) : error < 1/3

Y

AMS Final Estimator median of k’ improved estimators

• k’ = Ω(log(1/𝜹)) : error < 𝜹

YY Y YY Y YY

Final F2 EstimatorF2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)

Y

AMS Improved Estimator Y:= (∑ Xi2)/k

• k = Ω(1/ε2) : error < 1/3

Y

AMS Final Estimator median of k’ improved estimators

• k’ = Ω(log(1/𝜹)) : error < 𝜹

YY Y YY Y Y

The Probabilistic Method