Computational Models - Lecture 111tau-cm2015.wdfiles.com/local--files/course-schedule/lecture11_B.pdf · Computational Models - Lecture 111 Handout Mode Guest LecturerL: Iftach Haitner

Computational Models - Lecture 111

Handout Mode

Guest LecturerL: Iftach Haitner

Tel Aviv University.

January 5, 2015

1Based on slides by Benny Chor, Tel Aviv University, modifying slides by MauriceHerlihy, Brown University.

Benny Chor (TAU) Computational Models Lecture 11 January 5, 2015 1 / 68

Talk Outline

Reminder – deterministic and nondeterministic time classesNP and verifiabilityAdditional NP languagesThe class co-NPP Verses NPNP-completenessSatisfiabilityReductions

Sipser’s book, 7.4–7.5


Reminder – Deterministic Time

Definition 1 (deterministic Time)

A deterministic TM M runs in time t , where t : N 7→ N, if for every input w , thenumber of steps that M(w) uses is at most t(|w |).

Definition 2 (DTIME)For t : N 7→ N, letDTIME(t) = {L ⊆ Σ∗ : L is decided by an O(t)-time single tape TM}

The bound on the running time is required to hold also for input not in thelanguage (i.e., L).

Definition 3 (P)

P =⋃

c≥0 DTIME(nc)


Reminder – Non-Deterministic Time

Definition 4 (nondeterministic Time)

A non-deterministic TM N runs in time t , where t : N 7→ N, if for every input w ,the maximum number of steps that N(w) uses on any branch of itscomputation tree, is at most t(|w |).

Notice that also non-accepting branches must reject within the required time.

Definition 5 (NTIME)

For t 7→ N 7→ N let NTIME(t) ={L ⊆ Σ∗ : L is decided by an O(t)-time single tape NTM}

Definition 6 (NP)

NP =⋃

c≥0 NTIME(nc)


Reminder – Non-Deterministic Time, cont.

Definition 7 (verifier)

Algorithm V is a verifier for a language L, if

x ∈ L =⇒ ∃c ∈ Σ∗ s.t. V(x , c) = 1.

x /∈ L =⇒ @c ∈ Σ∗ s.t. V(x , c) = 1.

A polynomial verifier runs in polynomial time in |x | (i.e., in the length of itsleft-hand-side input parameter).

Theorem 8A language is in NP iff it has a polynomial time verifier.


Section 1

NP and Polynomial Verifiability


Polynomial VerifiabilityDefinition 9 (verifier)

Algorithm V is a verifier for a language L, if

x ∈ L =⇒ ∃c ∈ {0,1}∗ s.t. V(x , c) = 1.

x /∈ L =⇒ @c ∈ Σ∗ s.t. V(x , c) = 1.

The verifier uses the additional information c to verify x ∈ L.

If V accepts (x , c) (i.e., outputs 1), the string c is called a certificate (alsoknown as, proof or witness) for x .

A polynomial verifier runs in polynomial time in |x | (i.e., in the length ofits left-hand-side input parameter).

A language L is polynomially verifiable, if it has a polynomial verifier.

A certificate for 〈G, s, t〉 ∈ HAMPATH is simply the Hamiltonian path froms to t .Easy to verify in time polynomial in |〈G, s, t〉| whether given path isHamiltonian.

Not all languages are known to be polynomially verifiable.Benny Chor (TAU) Computational Models Lecture 11 January 5, 2015 7 / 68

NP and Polynomial Verifiability

Theorem 10A language is in NP iff it has a polynomial time verifier.

Proof’s idea:

The NTM emulates the verifier by guessing the certificate.

Verifier emulates NTM by using accepting branch as certificate.


Polynomial Verifiability =⇒ NP

Claim 11If L has a poly-time verifier, then it is decided by some polynomial-time NTM.

Proof: Let V be poly-time verifier for L of running time p(n) for some p ∈ poly.

Algorithm 12 (N)

On input x ∈ {0,1}n:

1 Guess a string c of length at most p(n).

2 Emulate V on 〈x , c〉3 Accept if V accepts; Otherwise Reject.

♣

Why is it suffices to guess a string of length p(n)?


NP =⇒ Polynomial Verifiability

Claim 13If L is decided by a polynomial-time NTM N, then L has a poly-time verifier.

Proof: Assume for simplicity that at each step of N, the number of possiblenon-deterministic moves is at most two.

Algorithm 14 (V)

On input (x , c):

1 Emulate N(x), treating each symbol of c as a description of thenon-deterministic choice in each step of N.

2 Accept if this branch accepts; Otherwise Reject.

♣Without the simplifying assumption?


Section 2

Additional NP Languages


CLIQUE

A clique in a graph is a subgraph where every two nodes are connected by anedge.A k -clique is a clique of size k .

Question 15What is the largest k -clique in the figure?


CLIQUE cont.

CLIQUE = {〈G, k〉 : G is an undirected graph with a k -clique}

Theorem 16CLIQUE ∈ NP

Proof’s idea: The clique is the certificate.

Algorithm 17 (V)

On input (〈G, k〉, c)Accept if c is a k -clique subgraph of G;Otherwise Reject.


SUBSET-SUM

A collection of non-negative integers x1, . . . , xk

A target number t

Question 18does some sub-collection add up to t?

Example 19

〈{4,11,16,21,27},25〉 ∈ SUBSET-SUM because 4 + 21 = 25.

〈{4,11,16,21,27},26〉 /∈ SUBSET-SUM (?)


SUBSET-SUM cont.

SUBSET-SUM = {〈S = {x1, . . . , xk}, t〉 : ∃{y1, . . . , y`} ⊆ S :∑`

j=1 yj = t}

Collections are sets: repetitions not allowed.

Theorem 20SUBSET-SUM ∈ NP

Proof’s idea: The subset is the certificate.

Algorithm 21 (V)

On input (〈S = {x1, . . . , xk}, t〉, c):Accept if the following holds (otherwise Reject):

1 c is a collection of numbers summing to t .

2 c is a subset of S

Theorem 22There is a pseudo-polynomial algorithm for SUBSET-SUM


Graph characterization of SUBSET-SUM

Definition 23For 〈S = {x1, . . . , xk}, t〉, let G(S, t) = (V ,E)

V = V0 ·⋃

V1 . . . ·⋃

Vk , where Vi = {vi,0, . . . , vi,t}

E = E1 ·⋃. . . ·⋃

Ek , where Ei = {(vi−1,j , vi,j ), (vi−1,j , vi,j+xi ) : j ∈ {0, . . . , t}}

Claim 24vi,j is reachable from v0,0 in G(S, t), iff 〈{x1, . . . , xi}, j〉 ∈ SUBSET-SUM.

Proof?


Pseudo-polynomial algorithm for SUBSET-SUM

Algorithm 25

Input: 〈S, t〉.

Return TRUE iff vk,t is reachable from v0,0 in G(S, t).

Correctness

Size of G(S, t) is O(|S| · t), and therefore the resulting algorithm forSUBSET-SUM runs in time poly(|S| · t).

Not poly(log2 t) but poly(t) (i.e., pseudo-polynomial)


Independent Set

An independent set in a graph is a set of vertexes, no two of which are linkedby an edge.A k -IS is an independent set of size k .

Question 26What is the largest k -IS in the figure?


IND-SET cont.

IND-SET = {〈G, k〉 : G contains an independent set of size k}

Theorem 27IND-SET ∈ NP

Proof’s idea: The independent set is the certificate.

Algorithm 28 (V)

On input (〈G, k〉, c)Accept if c is a k -IS of G (no edges between nodes in c, and |c| = k );Otherwise Reject.


KNAPSACK

A collection of non-negative integers x1, . . . , xk (size)

A collection of non-negative integers y1, . . . , yk (benefit)

A capacity B

A target number t

Question: does some S ⊆ {1, . . . , k} has∑

i∈S xi ≤ B and∑

i∈S yi ≥ t?

Example 29

({4,11,16,21,27}, {7,3,9,5,35},B = 20, t = 16) ∈ KNAPSACKbecause S = {1,3}.


KNAPSACK cont.

KNAPSACK =

{〈{x1, . . . , xk}, {y1, . . . , yk},B, t〉 : ∃S ⊆ {1, . . . , k} :∑i∈S

xi ≤ B ∧∑i∈S

yi ≥ t}

Theorem 30KNAPSACK ∈ NP

Proof’s idea: The subset is the certificate.

Algorithm 31 (V)

On input (〈{x1, . . . , xk}, {y1, . . . , yk},B, t〉, c):Accept if the following holds (otherwise Reject):

1∑

i∈c xi ≤ B

2∑

i∈c yi ≥ t .

3 c is a subset of [1, . . . , k ]


Pseudo-polynomial algorithm for KNAPSACK

Theorem 32There is a pseudo-polynomial algorithm for KNAPSACK

Algorithm 33 (KNAPSACK)

Input: {x1, . . . , xk}, {y1, . . . , yk},B, t :

1 Set A(0,0) = 0 and A(0,p) =∞ for p 6= 0.

2 For i = 1 to k :

For p = 0 to t :Set A(i, p) = min{A(i − 1, p), xi + A(i − 1, p − yi)}.

3 Accept if t ≤ max{p : A(k ,p) ≤ B}.

Running time O(k · t)

Not poly(log2 t) but poly(t) (i.e., pseudo-polynomial)


Section 3

The Class coNP


The Class co-NP

CLIQUE = {〈G, k〉 : G is an undirected graph with no k -clique} seems not tobe member of NP.It is harder to efficiently verify that something does not exist than to efficientlyverify that something does exist.

Definition 34 (co-NP)

co-NP = {L : L ∈ NP}.

So far, no one knows if co-NP is distinct from NP.

Claim 35P ⊆ co-NP.

Proof?

L ∈ P =⇒ L ∈ P =⇒ L ∈ NP =⇒ L ∈ co-NP


Section 4

P vs. NP


P Vs. NP

NP

P P=NP

The question P ?= NP is one of the great unsolved mysteries in

contemporary mathematics.

Most computer scientists believe the two classes are not equal

Most bogus proofs show them equal (?)


P Vs. NP, cont.

If P differs from NP, then the distinction between P and NP \ P ismeaningful and important.

languages in P are tractable

languages in NP \ P are intractable

Until we can prove that P 6= NP, there is no hope of proving that a specificlanguage lies in NP \ P.

Nevertheless, we can prove statements of the form

If A ∈ NP \ P, then B ∈ NP \ P.


Section 5

NP Completeness


NP Completeness

P

NP

NP-complete

The class of NP-complete languages are

“hardest” languages in NP“least likely” to be in PIf any NP-complete L ∈ P, then NP = P.

Such languages, “carry on their backs” the burden of all of NP.

Question 36Are there NP-complete languages?


Polynomial-Time Computable Functions

Definition 37 (poly-time computable functions)

A function f : Σ∗ 7→ Σ∗ is polynomial-time computable, if there is a poly-timedeterministic TM that

starts with input w , and

halts with f (w) on tape.


Polynomial-Time Reducibility

Definition 38A language A is polynomial time mapping reducible to B, denoted A ≤P B, ifexists poly-time computable f such that

w ∈ A⇐⇒ f (w) ∈ B.

for every w ∈ Σ∗.The function f is called a polynomial-time reduction from A to B.

The mapping f efficiently converts questions about membership in A tomembership in B.


Reductions to PTheorem 39

If A ≤P B and B ∈ P then A ∈ P.

Proof:

Let f the reduction from A to B, computed by TM Mf .On input x , the TM Mf makes at most cf · |x |af steps.

Let MB be the poly-time decider for B.On input y , the TM MB makes at most cB · |y |aB steps.

Algorithm 40 (Decider MA for A)

On input x , return MB(f (x))

MA decides A

Running time of MB(x), is at mostcB · (cf · |x |af )aB = (cB · caB

f ) · |x |af ·aB ∈ poly(|x |)Hence, A ∈ P

♣ Benny Chor (TAU) Computational Models Lecture 11 January 5, 2015 32 / 68

What A ≤P B tells us about B?

Question 41Assume that {0n1n : n ≥ 0} ≤P L. Does it yield that L ∈ P?

Answer: No.

Let L = HTM and define f (x) =

{Mstop, x ∈ {0n1n : n ∈ N}Mno−stop, otherwise. .

A ≤P B does tell us that B is “at least as hard" as A.


NP Completeness, Formal Definition

Definition 42 (NP-complete)

A language B is NP-complete, if

B ∈ NP, and

Every A ∈ NP is poly-time reducible to B (i.e., A ≤P B)

We let NPC denote the class of all NP-complete languages.

Compare to

Definition 43 (RE-Complete)

A language B is RE-complete, if

B ∈ RE , and

Every A ∈ RE is mapping reducible to B.


Why NP Completeness?

Theorem 44

If B ∈ NPC and B ∈ P, then P = NP.

Proof: Immediately follows by Thm 39. ♣

To show P = NP (and make an instant fortune, seewww.claymath.org/millennium/P_vs_NP/), suffices to find a polynomial-timealgorithm for any NP-complete problem.

Question 45Is NPC empty?


NPC Is Not Empty

ANP = {〈M, x ,1n〉 : M is a TM ∧ ∃c ∈ Σ∗ s.t. M(x , c) accepts within n steps}.

Theorem 46ANP ∈ NPC

Proof:

Clearly ANP ∈ NP.(?)

Let L ∈ NP, let V be a verifier for L and let p ∈ poly be a bound on therunning time of V (i.e., V(x , ·) halts within p(|x |) steps, for every x ∈ Σ∗)).

Define f (x) = 〈V, x ,1p(|x|)〉.

Clearly f is poly-time computable and x ∈ L⇐⇒ f (x) ∈ ANP.

♣


Finding Additional NP-complete Languages

Theorem 47Assume that

1 B ∈ NP2 A ∈ NPC and A ≤P B

then B ∈ NPC.

Proof: Home exercise . . .♣

We would like to find L ∈ NPC that is “natural" and “easy" to reduce to.

The most useful such language is the language of satisfied formulas.


Section 6

Satisfiability


Boolean Variables

A Boolean variable assumes values

I TRUE (written 1), and FALSE (written 0).

Boolean operations:

I and: ∧I or: ∨I not: ¬

Examples:

0 ∧ 1 = 00 ∨ 1 = 1

0 = 1


Boolean Formulas and SATA Boolean formula is an expression involving Boolean variables andoperations.

φ = (x ∧ y) ∨ (x ∧ z)

Definition 48 (satisfiable formula)

A formula is satisfiable, if some Boolean assignment to its variables, makesthe formula evaluate to 1.

The formula φ = (x ∧ y) ∨ (x ∧ z) is satisfiable by the assignment

x = 0y = 1z = 0

The language of satisfied formulas:

SAT = {〈φ〉 : φ is a satisfiable Boolean formula}


SAT ∈ NPC

SAT = {〈φ〉 : φ is satisfiable Boolean formula}

Theorem 49 (Cook-Levin (early 70s))

SAT ∈ NPC.

The “most important" NP-complete language.

It is easy to see that SAT ∈ NP

For the proof of other part wait for next week . . .


The Language 3SATIt is useful to consider a special version of SAT

A literal is a variable or negated variable: x or x .

A clause is several literals joined by ∨s: (x1 ∨ x2 ∨ x3)

A Boolean formula is in conjunctive normal form (CNF) if it consists ofclauses, connected with ∧s.

For example: (x1 ∨ x2 ∨ x3 ∨ x4) ∧ (x3 ∨ x5 ∨ x6) ∧ (x3 ∨ x6)

Definition 50A Boolean formula is in k-CNF form, if it is a CNF formula, and all clauseshave k literals.

Example of 3CNF: (x1 ∨ x2 ∨ x3) ∧ (x3 ∨ x5 ∨ x6) ∧ (x3 ∨ x6 ∨ x4)

The language of satisfied 3CNF formulas:

3SAT = {〈φ〉 : φ is satisfiable 3CNF formula}


3SAT ∈ NPC

Clearly 3SAT ≤P SAT.

and 3SAT ∈ NP.

We show next class that SAT ≤P 3SAT, yielding that 3SAT ∈ NPC.

Since 3SAT has more structure than SAT, it is simpler to reduce it toother languages.

In the following we prove that several NP languages are NPC, byshowing a reduction from 3SAT to them.


1SAT,2SAT ∈ P

Question 51Is 1SAT ∈ P ?

Question 52Is 2SAT ∈ P ?


A Graph characterization for 2CNF

Definition 53For 2CNF formula φ with variables x1, . . . , xk , define a directed graphG(φ) = (V ,E) as

V = {x1, x1, . . . , xk , xk}

E = {(`,h), (h, `) : (` ∨ h) ∈ φ} (taking x = x).

Claim 54

G(φ) contains path from ` to h =⇒ in any satisfying assignment of φ with` = 1, it holds that h = 1.

Proof?

Claim 55

G(φ) contains path from ` to h =⇒ G(φ) contains path from h to `

Proof?


A Graph characterization for 2CNF , contDefinition 56A variable x in 2CNF φ is bad, if ∃ paths in G(φ) from x to x and from x to x .

Claim 57A 2CNF formula is satisfiable iff it contains no bad variables.

φ has bad variable =⇒ φ is unsatisfiable. Proof? by Claim 54

φ has no bad variables =⇒ φ is satisfiable. Proof?

Algorithm 58While φ has unassigned variables:

1 Pick unassigned literal ` ∈ V that has no path from ` to `2 Assign 1 to all (unassigned) literals reachable from `, and 0 to their

negations.

Can there be conflicts (i.e., x and x assigned the same value)?

` h h =⇒ ( Claim 55) h h ` =⇒ ` `.

Final assignment satisfies φ:

(h ∨ `) ∈ φ and h = 0 =⇒ (h, `) ∈ E and h = 1 =⇒ ` = 0.Benny Chor (TAU) Computational Models Lecture 11 January 5, 2015 46 / 68

Poly-time algorithm for 2SAT

Algorithm 59 (TwoSatSolver)

Input: 2CNF φReturn TRUE if there exist no bad variables in φ:

Efficiency?

Correctness?


Section 7

Clique


3SAT ≤P CLIQUE

CLIQUE = {〈G, k〉 : G is an undirected graph with a k -clique}

Claim 603SAT ≤P CLIQUE.

Since CLIQUE ∈ NP, it follows that CLIQUE ∈ NPC.

Proof’s idea: We’ll construct a poly-time reduction f that maps 3CNF formulaeφ to pairs 〈G, k〉 of graphs and numbers.

The function f will have the property that φ is satisfiable, iff G has a clique ofsize k .


Proving 3SAT ≤P CLIQUE

On input φ, the mapping reduction is defined as follows:

If φ is not a 3CNF, output some x /∈ CLIQUE.

Otherwise, let k be the number of clauses in φ.

We construct a graph G = G(φ), see below, and output (G, k).

The graph G is defined as follows:

Nodes in G are organized into triples t1, . . . , tk .

Each triple corresponds to a clause of φ

Each node in a triple corresponds to a literal in corresponding clause.

On going example:

(x1 ∨ x2 ∨ x3) ∧ (x3 ∨ x5 ∨ x6) ∧ (x3 ∨ x6 ∨ x4)


Nodes of G

φ =(x1 ∨ x2 ∨ x3) ∧ (x3 ∨ x5 ∨ x6) ∧ (x3 ∨ x4 ∨ x6)

Add a node per literal


Edges of GAdd edges between all vertex pairs, except

within same triple

between contradictory literals (e.g., x3 and x3)

φ = (x1 ∨ x2 ∨ x3) ∧ (x3 ∨ x5 ∨ x6) ∧ (x3 ∨ x4 ∨ x6)


φ ∈ 3SAT =⇒ 〈G, k〉 ∈ CLIQUE

Proof: Suppose φ is satisfiable by an assignment ψ.With respect to ψ:

At least one literal is assigned to 1 in every clause (?)

Select a 1-literal in every tuple

These literals can be joined by edges (?)

Yielding a k -clique in G.

♣Benny Chor (TAU) Computational Models Lecture 11 January 5, 2015 53 / 68

〈G, k〉 ∈ CLIQUE =⇒ φ ∈ 3SAT

Proof: Suppose G has a k -clique.

No two of the cliques nodes are in the same triple. (?)

G has k vertexes and k clauses, so each triple has exactly one cliquenode.

Assign 1 to each node in clique;

Assignment has no contradictions (?)

Yielding a satisfying assignment to φ.


Recap

We’ve constructed a poly-time computable function f .

We saw that f has the property that φ ∈ 3SAT iff f (φ) ∈ CLIQUE.

Therefore, f is a poly-time reduction from 3SAT to CLIQUE

=⇒ 3SAT ≤P CLIQUE. ♣


Section 8

Independent Set


Independent Set

Definition 61An independent set in a graph is a set of vertexes, no two of which are linkedby an edge.

The language of independent sets:

IND-SET = {〈G, k〉 : G contains an independent set of size k}

Clearly IND-SET ∈ NP. We show that CLIQUE ≤P IND-SET and deducethat IND-SET ∈ NPC


CLIQUE ≤P IND-SET

Proof:

Definition 62The complement of a graph G = (V ,E) is a graph Gc = (V ,Ec), whereEc = {(v1, v2) : v1, v2 ∈ V and (v1, v2)6∈E}.

Claim 63If U is an independent set in G, then U is a clique in Gc .

The reduction from CLIQUE to IND-SET simply compute the complement ofthe graph. ♣

Remark 64Same reduction shows that IND-SET ≤P CLIQUE


Section 9

Hamiltonian Paths and Cycles


Reminder – Hamiltonian Path

A Hamiltonian path in a directed G visits each node exactly once.

HAMPATH = {〈G, s, t〉 : G has Hamiltonian path from s to t}

Theorem 65HAMPATH ∈ NPC.

Not today . . .


Hamiltonian Cycle

A Hamiltonian Cycle in a graph is an Hamiltonian path that ends up where itstarts (i.e., s = t).

HAMCYCLE = {〈G〉 : G has Hamiltonian cycle}

Clearly HAMCYCLE ∈ NP. We will show that HAMPATH ≤P HAMCYCLE,

and deduce that HAMCYCLE ∈ NPC


HAMPATH ≤P HAMCYCLE.

Proof’s idea:

Hey, is the new vertex really needed? Why not just add an edge from t to s?


HAMPATH ≤P HAMCYCLE.

Why the new vertex really needed? Why not just add an edge from t to s?


HAMCYCLE ≤P HAMPATH.

Claim 66HAMCYCLE ≤P HAMPATH.

Left as an easy (recommended) exercise.


Section 10

Traveling Salesman


Traveling Salesman

10 9

36

9

a

b

c

d

5

Roger WilliamsZoo

Brown UniversityAl FornoRestaurant

StateCapitol

(not drawn to scale)

Input: set of cities C and set of directed inter-city distances D

Goal: find a an hamiltonian cycle (TS tour) of total distance at most k

TRAVELING-SALESMAN ={〈C,D, k〉 : (C,D) has a TS tour of total distance ≤ k}


HAMCYCLE ≤P TRAVELING-SALESMANProof: Given a directed graph G = (V ,E), construct traveling salesmaninstance.

The cities are identical to the nodes of the original graph, C = V .

The distance of going from v1 to v2 is 1 if (v1, v2) ∈ E , and 2 otherwise.

The bound on the total distance of a tour is k = |V |.

correctness: =⇒ Suppose G has a Hamiltonian cycle.I The distance assigned by reduction to all edges in this cycle is 1.I Thus in (C,D) there is a traveling salesman tour of total distance|V | = k (namely (C,D, k) ∈ TRAVELING-SALESMAN).

⇐= Suppose (C,D) has a traveling salesman tour of total distance|V | = k .

I Tour cannot contain any edge of distance 2.I Therefore it gives a Hamiltonian cycle in G.

Efficiency: reduction done in quadratic time (filling up distances for alledges of the complete graph).


Chains of Reductions: NPC Problems

SAT

IntegerProg 3SAT

Clique

IndepSet

VertexCover

SetCover

3ExactCover

Knapsack

Scheduling

3Color HamPath

HamCircuit

TRAVELING-SALESMAN


Documents

Computational Models - Lecture 111tau-cm2015.wdfiles.com/local--files/course-schedule/lecture11_B.pdf · Computational Models - Lecture 111 Handout Mode Guest LecturerL: Iftach Haitner