Prof. Mario Pavone Computazione Naturale CdL Magistrale in Informatica SDAI – DMI – UniCt [email protected] Teoria

Prof. Mario PavoneProf. Mario PavoneComputazione NaturaleComputazione Naturale

CdL Magistrale in InformaticaCdL Magistrale in InformaticaSDAI – DMI – UniCtSDAI – DMI – UniCt

[email protected]://www.dmi.unict.it/mpavone/

Teoria della Complessità:Teoria della Complessità:la classe NPla classe NP

Problem

A problem is a general question to be answered, usually possessing several parameters, or free variables, whose values are left unspecified.

A problem is described by giving:a general description of all its parameters, anda statement of what properties the answer, or

solution, is required to satisfyAn instance of a problem is obtained by

specifying particular values for all the problem parameters

Mario Pavone, AA 2010/2011 – Computazione Naturale, CdL Magistrale in Informatica, SDAI, DMI, UniCt

Search Space

Given a combinatorial problem PP, a search search spacespace associated to a mathematical formulation of PP is defined by a couple (S,(S,f f )) where SS is a finite set of configurations (or nodes

or points) andff a cost functioncost function which associates a real number

to each configurations of SS.For this structure two most common measures

are the minimum and the maximum coststhe minimum and the maximum costs. In this case we have the combinatorial combinatorial optimization problemsoptimization problems.


Example: K-SATAn instance of the K-SAT problem K-SAT problem consists of a

set V of variables, a collection C of clauses over V such that each clause c C has |c|= K

The problem is to find a satisfying truth assignment for C

The search space search space for the 2-SATfor the 2-SAT with |V|=2 is (S,(S,f f ) ) where S={ (T,T), (T,F), (F,T), (F,F) }S={ (T,T), (T,F), (F,T), (F,F) } andthe cost functionthe cost function for 2-SAT2-SAT computes only the

number of satisfied clauses

ffsatsat (s)= #SatisfiedClauses(F,s), s (s)= #SatisfiedClauses(F,s), s S S


Example: SEARCH SPACE of K-SAT

Let we consider F = (A B) ( A B)

A BA B ffsatsat(F,s)(F,s)

T T 1

T F 2

F T 1

F F 2


SEARCH LANDSCAPE

Given a search space (S,(S,ff)), a search landscapesearch landscape is defined by a triplet (S,n,f)(S,n,f) where nn is a neighborhood functionneighborhood function which verifies

n : S n : S 2 2SS - {0} - {0} This landscape, also called energy landscapeenergy landscape,

can be considered as a neutralneutral one since no search process is involved.

It can be conveniently viewed as weighted graphweighted graph G=(S,n,F)G=(S,n,F) where the weights are defined on the the weights are defined on the nodesnodes, not on the edges.


PROCESS LANDSCAPE

Given a search landscape (S,n,f)(S,n,f), a process process landscapelandscape is defined by a quadruplet (S,n,f,(S,n,f,)) where is a search processsearch process

The process landscape represents a particular view of the neutral landscape (S,n,f)(S,n,f) seen by a search algorithm.

Examples of search algorithmsExamples of search algorithms:Local Search AlgorithmsLocal Search AlgorithmsComplete Algorithms (e. g. Davis-Putnam algorithm)Complete Algorithms (e. g. Davis-Putnam algorithm)Evolutionary AlgorithmsEvolutionary Algorithms: Genetic Algorithms, Genetic Algorithms,

Genetic Programming, Evolution Strategies, Evolution Genetic Programming, Evolution Strategies, Evolution Programming, Immune AlgorithmsProgramming, Immune Algorithms


ALGORITHMS

Algorithms are general, step-by-step procedures for solving problems

For concreteness, we can think of them simply as being computer programs, written in some precise computer language.

An algorithm is said to An algorithm is said to solvesolve a problem a problem PP if that if that algorithm can be applied to algorithm can be applied to anyany instance instance II of of PP and is and is guaranteedguaranteed alwaysalways to produce a to produce a solution for that instance solution for that instance II

For example:An algorithm does not "solve" the TSP unless it

always constructs an ordering that gives a minimum length tour


BIBLIOGRAPHYBIBLIOGRAPHYM. R. Garey and D. S. Johnson,

Computers and Intractability: Computers and Intractability: A Guide to the Theory of NP-A Guide to the Theory of NP-CompletnessCompletness, W. H. Freeman, 1st ed. (1979).

OR: A compendium of NP A compendium of NP optimization problemsoptimization problems at the at the link:link:

http://www.csc.kth.se/~viggo/problemlist/


Problem: the start point

PROBLEMPROBLEM: to refer to a question such as: "Is a given graph G=(V,E) K-colorable?"

INSTANCEINSTANCE of a problem is a list of arguments, one argument for each parameter of the problem

a problem is a binary relationa problem is a binary relation on a set II of problem instancesproblem instances, and a set SS of problem problem solutionssolutions: Q: I → SQ: I → S

decision problemsdecision problems: Q: I → {0, 1}Q: I → {0, 1}in general, if we can solve an optimization in general, if we can solve an optimization

problem quickly, we can also solve its related problem quickly, we can also solve its related decision problem quicklydecision problem quickly


Problem: the start point

a problem is UNDECIDABLEUNDECIDABLE if there is no algorithm that takes as input an instance of the problem and determines whether the answer to that instance is "yes""yes" or "no""no"

problems that are solvable by polinomial-time solvable by polinomial-time algorithmsalgorithms as being TRACTABLETRACTABLE; ones that require superpolynomial timesuperpolynomial time as being INTRACTABLEINTRACTABLE


Problem: the start pointNP-complete problems are intractableNP-complete problems are intractable

If any single NP-complete problem can be If any single NP-complete problem can be solved in polynomial time then every NP-solved in polynomial time then every NP-complete problem has a polynomial-time complete problem has a polynomial-time algorithmalgorithm

If you can establish a problem as NP-complete you provide good evidence for its intractabilityits intractability

You would then do better spending your time developing an approximation algorithmdeveloping an approximation algorithm rather than searching for a fast algorithm that solves the problem exactly

Polynomial-time solvable problems are generally Polynomial-time solvable problems are generally regarded as tractableregarded as tractable


Complexity Class PAn algorithm solves a problem in time An algorithm solves a problem in time O(T(n))O(T(n)) if,

when it is provided a problem instance problem instance ii of of length length n=|i|n=|i|, the algorithm can produce the the algorithm can produce the solution in at most solution in at most O(T(n))O(T(n)) time time

A problem is polynomial-time solvableproblem is polynomial-time solvable if there exists an algorithm to solve it in time O(nan algorithm to solve it in time O(nkk)) for some constant k

COMPLEXITY CLASS PCOMPLEXITY CLASS P: the set of decision problems that are solvable in polynomial time

A function function ff is polynomial-time computable is polynomial-time computable if there exists a polynomial-time algorithm AA that, given any input x x I I, produces as output f(x)f(x)


Example: problem in P

Minimum Weight Spanning Tree.

Input: Graph G, integer k.

Decision problem: Does G have a spanning tree of Does G have a spanning tree of weight at most k? weight at most k?

If you are provided with a tree with weight at most k as part of the solution, the answer can be verified in O(n2) time.

Polynomial-Time Algorithms

Are some problems solvable in polynomial time?Of course: every algorithm we’ve studied (Algorithms

1) provides polynomial-time solution to some problemWe define P to be the class of problems solvable in

polynomial timeAre all problems solvable in polynomial time?

No: Turing’s “Halting Problem” is not solvable by any computer, no matter how much time is given

Such problems are clearly intractable, not in P

Complexity Class NP

Some problems are intractable: as they grow large, we are unable to solve them in reasonable time

What constitutes reasonable time? Standard working definition: polynomial timeOn an input of size n the worst-case running time

is O(nk) for some constant kPolynomial time: O(n2), O(n3), O(1), O(n lg n) Not in polynomial time: O(2n), O(nn), O(n!)


Complexity Class NPA decision problemdecision problem (yes/no question) is in the

class NPclass NP if it has a nondeterministic polynomial time algorithm

Informally, such an algorithm:

1.Guesses a solution (nondeterministically).

2.Checks deterministically in polynomial time that the answer is correct.

Or equivalently, when the answer is "yes", there is a certificate (a solution meeting the criteria) that can be verified in polynomial time (deterministically)

The Complexity Class NPComplexity Class NP is the class of problems that can be verified by a polynomial-time can be verified by a polynomial-time algorithmalgorithm

If Q Q PP then Q Q NPNP => P P NPNPOpen question is:Open question is: P = NP P = NP


Complexity Classes: P and NP

PP is set of problems that can be solved in polynomial time

NP (NNondeterministic PPolynomial time) is the set of problems that can be solved in polynomial time by a nondeterministic computerWhat the hell is that?


Nondeterminism

Think of a non-deterministic computer as a computer that magically “guesses” a solution, then has to verify that it is correctIf a solution exists, computer always guesses itOne way to imagine it: a parallel computer that

can freely spawn an infinite number of processesHave one processor work on each possible solutionAll processors attempt to verify that their solution

worksIf a processor finds it has a working solution

So: NP = problems verifiable in polynomial time


Review: P and NP

Summary so far:P = problems that can be solved in polynomial

timeNP = problems for which a solution can be

verified in polynomial timeUnknown whether P = NP (most suspect not)

Hamiltonian-cycle problem is in NP:Cannot solve in polynomial timeEasy to verify solution in polynomial time (How?)


Hamiltonian Cycle Problem

A hamiltonian cycle of an undirected graph G=(V,E) is a simple cycle that contains each vertex in V.

A graph that contains a hamiltonian cycle is said to be hamiltonianNOT ALL GRAPHS ARE HAMILTONIAN

Hamiltonian-cycle problem: “Does a graph G have a hamiltonian cycle?”



Hamiltonian Cycle Hamiltonian Cycle (input: a graph G)Does G have a Hamiltonian cycle?Does G have a Hamiltonian cycle?

Solution:Solution:

0, 1, 2, 11, 10, 9, 8, 7, 6, 5, 14, 0, 1, 2, 11, 10, 9, 8, 7, 6, 5, 14, 15, 6, 17, 18, 19, 12, 13, 3, 415, 6, 17, 18, 19, 12, 13, 3, 4



algorithm: lists all permutations of the vertices and check each permutation to see if it is a hamiltonian paththere are m! Possible permutations of the vertices

CHECK if the provided cycle is hamiltonian: if it is a permutation of the vertices of V and each consecutive edges along the cycle exists in the graph

This verification can be implemented to run in O(n2) time => a hamiltonian cycle exists in a graph can be verified in polynomial timeN is the lenght of the econding of G

NP-Complete Problems

The NP-Complete problems are an interesting class of problems whose status is unknown No polynomial-time algorithm has been

discovered for an NP-Complete problemNo suprapolynomial lower bound has been

proved for any NP-Complete problem, eitherWe call this the P = NP question

The biggest open problem in CS


NP Complete ProblemsThe class of problems in NP which are the problems in NP which are the

“hardest”“hardest” are called the NP-completeNP-complete problemsproblems

A problem QQ in NPNP is NP-completeNP-complete if the existence of a polynomial time algorithm for QQ implies the existence of a polynomial time existence of a polynomial time algorithm for all problems in algorithm for all problems in NPNP

Steve Cook in 1971 proved that SAT is NP-Steve Cook in 1971 proved that SAT is NP-complete. complete.

NP-Complete Problems

We will see that NP-Complete problems are the “hardest” problems in NP:If any one NP-Complete problem can be solved in

polynomial time……then every NP-Complete problem can be solved

in polynomial time……and in fact every problem in NP can be solved

in polynomial time (which would show P = NP)Thus: solve hamiltonian-cycle in O(n100) time,

you’ve proved that P = NP. Retire rich & famous.

Reduction

The crux of NP-Completeness is reducibilityInformally, a problem P can be reduced to

another problem Q if any instance of P can be “easily rephrased” as an instance of Q, the solution to which provides a solution to the instance of PWhat do you suppose “easily” means?This rephrasing is called transformation

Intuitively: If P reduces to Q, P is “no harder to solve” than Q

Reducibility

An example:P: Given a set of Booleans, is at least one TRUE?Q: Given a set of integers, is their sum positive?Transformation: (x1, x2, …, xn) = (y1, y2, …, yn)

where yi = 1 if xi = TRUE, yi = 0 if xi = FALSE

Another example: Solving linear equations is reducible to solving

quadratic equationsHow can we easily use a quadratic-equation solver to

solve linear equations?

Using Reductions

If P is polynomial-time reducible to Q, we denote this P p Q

Definition of NP-Complete: If P is NP-Complete, P NP and all problems R

are reducible to PFormally: R p P R NP

If P p Q and P is NP-Complete, Q is also NP-Complete

Ex. of some NP-complete problemsGiven one NP-Complete problem, we can

prove many interesting problems NP-CompleteGraph coloring (= register allocation)Hamiltonian cycleHamiltonian pathKnapsack problemTraveling salesmanJob scheduling with penalitiesMany, many more

An Aside: Terminology

What is the difference between a problem and an instance of that problem?

To formalize things, we will express instances of problems as stringsHow can we express a instance of the

hamiltonian cycle problem as a string?To simplify things, we will worry only about

decision problems with a yes/no answerMany problems are optimization problems, but

we can often re-cast those as decision problems

NP-Hard and NP-Complete

If P is polynomial-time reducible to Q, we denote this P p Q

Definition of NP-Hard and NP-Complete: If all problems R NP are reducible to P, then P

is NP-HardWe say P is NP-Complete if P is NP-Hard

and P NPIf P p Q and P is NP-Complete, Q is also

NP- Complete

Why Prove NP-Completeness?

Though nobody has proven that P != NP, if you prove a problem NP-Complete, most people accept that it is probably intractable

Therefore it can be important to prove that a problem is NP-CompleteDon’t need to come up with an efficient

algorithmCan instead work on approximation algorithms

Proving NP-Completeness

What steps do we have to take to prove a problem P is NP-Complete?Pick a known NP-Complete problem QReduce Q to P

Describe a transformation that maps instances of Q to instances of P, s.t. “yes” for P = “yes” for Q

Prove the transformation worksProve it runs in polynomial time

Oh yeah, prove P NP (What if you can’t?)

Review: P And NP Summary

P = set of problems that can be solved in polynomial time

NP = set of problems for which a solution can be verified in polynomial time

P NPOpen question: Does P = NP?

Review: Reduction

A problem P can be reduced to another problem Q if any instance of P can be rephrased to an instance of Q, the solution to which provides a solution to the instance of PThis rephrasing is called a transformation

Intuitively: If P reduces in polynomial time to Q, P is “no harder to solve” than Q

Review: NP-Hard and NP-CompleteIf P is polynomial-time reducible to Q, we

denote this P p Q

Definition of NP-Hard and NP-Complete: If all problems R NP are reducible to P, then P

is NP-HardWe say P is NP-Complete if P is NP-Hard

and P NP

If P p Q and P is NP-Complete, Q is alsoNP- Complete

Review: Proving NP-CompletenessWhat steps do we have to take to prove a

problem Q is NP-Complete?Pick a known NP-Complete problem PReduce P to Q

Describe a transformation that maps instances of P to instances of Q, s.t. “yes” for Q = “yes” for P

Prove the transformation worksProve it runs in polynomial time

Oh yeah, prove Q NP (What if you can’t?)

Directed Hamiltonian Cycle Undirected Hamiltonian CycleWhat was the hamiltonian cycle problem

again?For my next trick, I will reduce the directed

hamiltonian cycle problem to the undirected hamiltonian cycle problem before your eyesWhich variant am I proving NP-Complete?

Draw a directed example on the boardWhat transformation do I need to effect?

Transformation:Directed Undirected Ham. Cycle Transform graph G = (V, E) into G’ = (V’, E’):

Every vertex v in V transforms into 3 vertices v1, v2, v3 in V’ with edges (v1,v2) and (v2,v3) in E’

Every directed edge (v, w) in E transforms into the undirected edge (v3, w1) in E’ (draw it)

Can this be implemented in polynomial time?Argue that a directed hamiltonian cycle in G

implies an undirected hamiltonian cycle in G’Argue that an undirected hamiltonian cycle in G’

implies a directed hamiltonian cycle in G

Undirected Hamiltonian Cycle

Thus we can reduce the directed problem to the undirected problem

What’s left to prove the undirected hamiltonian cycle problem NP-Complete?

Argue that the problem is in NP

Hamiltonian Cycle TSP

The well-known traveling salesman problem:Optimization variant: a salesman must travel to n

cities, visiting each city exactly once and finishing where he begins. How to minimize travel time?

Model as complete graph with cost c(i,j) to go from city i to city j

How would we turn this into a decision problem?A: ask if a TSP with cost < k

Hamiltonian Cycle TSP

The steps to prove TSP is NP-Complete:Prove that TSP NP (Argue this)Reduce the undirected hamiltonian cycle

problem to the TSPSo if we had a TSP-solver, we could use it to solve the

hamilitonian cycle problem in polynomial timeHow can we transform an instance of the hamiltonian

cycle problem to an instance of the TSP?Can we do this in polynomial time?

The TSP

Random asides: TSPs (and variants) have enormous practical

importanceE.g., for shipping and freighting companiesLots of research into good approximation algorithms

Recently made famous as a DNA computing problem

Documents

Prof. Mario Pavone Computazione Naturale CdL Magistrale in Informatica SDAI – DMI – UniCt [email protected] Teoria