Upload
bulent-dogru
View
240
Download
0
Embed Size (px)
Citation preview
7/30/2019 NP Algoritm
1/23
Chapters 11-12P, NP, NP-Complete Problems
and
Approximation Algorithms
Dr. Natarajan MeghanathanAssociate Professor of Computer Science
Jackson State UniversityJackson, MS 39217
E-mail: [email protected]
7/30/2019 NP Algoritm
2/23
P, NP-Problems An algorithm is said to solve a problem in polynomial time if its worst-case time complexity is O(p(n)) where p(n) is a polynomial (includinglogarithmic time) of the problems input size n.
Problems that can be solved in polynomial time are called tractable,and those that cannot be solved in polynomial time are calledintractable.
A problem is a decision problem if it asks for an yes or no answer.
For example, the problem of determining whether a directedgraph is a DAG or not is a decision problem.
A problem is an optimization problem if it aims to find an optimal(minimum or maximum, depending on the context) solution.
For example, the minimum spanning tree problem, shortest pathproblem are all optimization problems.
7/30/2019 NP Algoritm
3/23
P, NP-Problems A decision problem can often be transformed to an optimizationproblem and vice-versa.
For example, the graph coloring problem that asks what is theminimum number of colors needed for a graph such that no twoadjacent vertices have the same color (an optimization problem).This can be transformed to a decision problem by asking whetherthe graph is 2-colorable, 3-colorable, etc.
Note that there are many decision or optimization problems for which
no polynomial-time algorithm has been found; but, neither theimpossibility of such an algorithm been proved.
Example: Traveling Salesman problem, Hamiltonian Circuit problem
Hamiltonian Circuit Problem: Determine whether a given graph has a
Hamiltonian Circuit a path that starts and ends at the same vertexand passes through all the other vertices exactly once.
Traveling Salesman Problem: Find the shortest tour through n citieswith known positive integer distances between them (find the
shortest Hamiltonian Circuit in a complete graph with positive integerweights).
7/30/2019 NP Algoritm
4/23
P, NP-Problems Class P (Polynomial) is a class of decision or optimization problemsthat can be solved in polynomial time by deterministic algorithms.
A deterministic algorithm is the one that produces the same outputfor a problem instance each time the algorithm is run.
A non-deterministic algorithm is a two-stage procedure that takes asits input an instance Iof a decision problem and does the following:
Stage 1: Non-deterministic (Guessing) Stage: An arbitrary string S isgenerated that can be thought of as a candidate solution to the given
instance I (could be sometimes, not a correct solution too). Stage 2: Deterministic (Verification) Stage: A deterministic algorithm
takes both I and S as its input and outputs yes if S represents a solutionto instance I. Note that we do not want a non-deterministic algorithm tooutput a yes answer on an instance for which the answer should be no.
A non-deterministic algorithm for the Traveling Salesman problemcould be the one that inputs a complete weighted graph of n vertices,then (stage 1): generates an arbitrary sequence of n vertices and(stage 2): verifies whether or not that each vertex, except the starting
vertex, is repeated only once in the sequence, and outputs a yes/noanswer accordingly.
7/30/2019 NP Algoritm
5/23
P, NP-Problems Class NP (Non-deterministic Polynomial) is the class of decisionproblems that can be solved by non-deterministic polynomialalgorithms.
Note that all problems in Class P are in Class NP: We can replacethe non-deterministic guessing of Stage 1 with the deterministicalgorithm for the decision problem, and then in Stage 2,deterministically verify the correctness of the solution of thedeterministic algorithm. In other words, P NP. However, it is notclear whether P = NP or P NP.
Polynomial reduction: A problem Iis said to be reducible to another
problem Jin polynomial time (denoted IP J), if every instance of Ican be reduced to an instance of Jin polynomial time. So, if apolynomial time algorithm exists to solve an instance of problem J,then we can say that there exists a polynomial time algorithm for I.
7/30/2019 NP Algoritm
6/23
NP-Complete Problems A problem Iis said to be NP-complete, if: Iis in NP
Every problem in class NP is polynomial-time reducible to I.
There are several decision problems that have been proven to beNP-complete.
If we could find a polynomial-time deterministic algorithm to solveany single NP-complete problem, then all problems in NP can be
considered to also be solvable deterministically in polynomial-time. Inthat case P = NP.
After proving that the problem they want to solve is an NP-completeproblem, researchers do not break their heads to find a polynomial-timedeterministic algorithm if they find one, they would surely win the
Turing award though!!! Instead, the focus will be on developing an approximate algorithm (called
a heuristic) that can yield a solution that will be bounded below or above
(depending on the context) by the optimal solution within a certain value,
that is as minimal as possible. That is, aim for an approximation ratio thatis closer to 1; but not equal to 1.
7/30/2019 NP Algoritm
7/23
NP-Complete Problems How would you prove that a problem H is NP-complete?
Let L be a problem that has been already provento be NP-complete.
We will show the following: (1) H is polynomial time reducible to L: HP L
(2) H is NP-hard: A problem H in NP is said to be
NP-hard, if a NP-complete problem is polynomialtime reducible to H. In this case, we show that LP H.
All NP-Problems
except H
P
L H
Proof of NP-hard: P
P
7/30/2019 NP Algoritm
8/23
Traveling Salesman Problem Given a complete weighted graph (i.e., weighted edges representingdistances in the Euclidean plane between any two vertices), find aminimum weight tour that starts from a particular vertex, goesthrough all the other vertices exactly once, and ends at the startingvertex. Possible Combinations and their Tour
Weights:
a b c d a 10
a b d c a 8 a c b d a 14
a c d b a 8
a d c b a 10
a d b c a 14 Similarly 18 more tours, can be written (6
each starting from b, c, or d).
The minimal tour weight is 8 and it could be
one of the tours of the correspondingweight , as listed above.
Traveling SalesmanProblem (TSP)
7/30/2019 NP Algoritm
9/23
Traveling Salesman Problem Both the Hamiltonian Circuit problem and the Traveling Salesmanproblems are NP-Complete problems.
Polynomial Reduction: We will see the polynomial-time reduction of a Hamiltonian Circuit
problem to a Traveling Salesman problem.
Recollect the Hamiltonian Circuit problem is to find a tour of a given
unweighted graph that simply starts at one vertex and goes throughall the other vertices and ends at the starting vertex.
Note that the input graph G to a Hamiltonian Circuit problem neednot be a complete graph connecting all vertices.
For the reduction, we will construct a complete graph G* such thatthere exists an edge of weight 1 in G* if the edge exists in G;otherwise, a weight of 2 is assigned to an edge between two verticesin G* if the edge does not exist in G.
7/30/2019 NP Algoritm
10/23
Traveling Salesman Problem This way, if we solve the complete graph G* for the TravelingSalesman problem, if there exists a Hamiltonian Circuit in the originalgraph G, the minimum weight tour in G* will involve only edges ofweight 1. All such edges in the minimal weight tour of G* will exist inan Hamiltonian Circuit of G.
The polynomial time reduction is involved in:
Given an weighted graph G for the Hamiltonian Circuit problem, theconstruction of the complete graph G* for the Traveling Salesman
problem:
For every edge (u, v) in G, include (u, v) in G* and weight (u, v) = 1
For every edge (u, v) not in G, include (u, v) in G* and weight(u, v) = 2
7/30/2019 NP Algoritm
11/23
Heuristic 1: Nearest Neighbor (NN)
Heuristic for the TSP Problem Start the tour with a particular
vertex, and include it to the tour.
For every iteration, a vertex (fromthe set of vertices that are not yetpart of the tour) that is closest to thelast added vertex to the tour is
selected, and included to the tour. The above procedure is repeated
until all vertices are part of the tour.
Time Complexity: It takes O(V)
times to choose a vertex among thecandidate vertices for inclusion asthe next vertex on the tour. Thisprocedure is repeated for V-1 times.Hence, the time complexity of theheuristic is O(V2).
7/30/2019 NP Algoritm
12/23
NN Heuristic Example (contd..)
7/30/2019 NP Algoritm
13/23
NN Heuristic Example (contd..)v1 v3 v2
v6v4v5
3 5
2
59
6
Improvement to the NN Heuristic using 2-Change HeuristicPick two non-overlapping edges (with no common end vertices) and see if we can
swap for them using edges that connect the end vertices so that the connectivity of thetour is maintained and the tour cost can be further reduced.
Strategy: Pick the costliest edge and a non-overlapping edge (i.e., no common endvertices) that is the next costliestIn the above example, we can pick v5 v4 (edge wt: 9) and the next costliest non-
overlapping edge v3 v2 (edge wt: 5) and replace them with edges v5 v2 (wt: 3) andv4 v3 (wt: 5). The revised tour is v1 v3 v4 v6 v2 v5 v1; tour weight: 24
v1 v3 v2
v6v4v5
3
2
5
6 35
7/30/2019 NP Algoritm
14/23
Heuristic # 2: Multi-fragment
Heuristic for the TSP Problem Step 1: Sort the edges in increasing
order of their weights (break the ties
arbitrarily). Initialize the set of touredges to be constructed to theempty set.
Step 2: Repeat this step |V| times,where |V| is the number of vertices
in the instance of the TSP problembeing solved:
Add the next edge on the sortededge list to the set of tour edges,
provided this addition does notcreate a vertex of degree 3 or acycle of length less than |V|;otherwise, skip the edge
Step 3: Return the set of tour edges
Complexity: O(E*logE) = O(V2*logV)
7/30/2019 NP Algoritm
15/23
Heuristic # 2: Multi-fragment
Heuristic for the TSP Problem
edge v2 v6 (wt: 2) OKedge v1 v3 (wt: 3) OK
edge v2 v5 (wt: 3) OKedge v1 v6 (wt: 4) OKedge v2 v4 (wt: 4) NOedge v1 v2 (wt: 5) NOedge v2 v3 (wt: 5) NO
edge v3 v4 (wt: 5) OKedge v4 v6 (wt: 5) NO
v2 v6 v4
v3v1v5edge v1 v5 (wt: 6) NOedge v3 v5 (wt: 6) NOedge v5 v6 (wt: 6) NOedge v3 v6 (wt: 7) NO
edge v1 v4 (wt: 8) NOedge v4 v5 (wt: 9) OK
Tour: v1 v3 v4 v5 v2 v6 v1
Tour Weight: 26
7/30/2019 NP Algoritm
16/23
Heuristic # 2: Multi-fragment
Heuristic for the TSP Problemv1 v3 v4
v5v2v6
Tour: v1 v3 v4 v6 v2 v5 v1 Tour Weight: 24
3 5
9
32
4
v1 v3 v4
v5v2v6
3 5
32
Tour from the Multi-fragment
Heuristic (prev. slide): wt 26
56
Improvement using 2-Change
v1 v3 v4
v6v2v5
3 5
5
32
6
7/30/2019 NP Algoritm
17/23
Heuristic # 3 for the TSP Problem
Twice-around-the Tree Algorithm Step 1: Construct a Minimum Spanning
Tree of the graph corresponding to agiven instance of the TSP problem
Step 2: Starting at an arbitrary vertex,perform a walk around the minimumspanning tree recording all the verticespassed by. This can be done by a DFS
traversal.
Step 3: Scan the vertex list obtained in
Step 2 and eliminate from it all repeatedoccurrences of the same vertex exceptthe starting one at the end of the list.
The vertices remaining on the list willform a Hamiltonian Circuit, which is theoutput of the algorithm.
v1 v2 v3
v4v5v6
Step 1: MST of the Graph
Note: We will use the principle of Triangle
Inequality for Euclidean plane:
The sum of the two sides of a triangle isgreater than the third side of the triangle
7/30/2019 NP Algoritm
18/23
Heuristic # 3 for the TSP Problem
Twice-around-the Tree Algorithm
v1 v2v3
v4v5v6
Step 2: DFS Traversal of the MST
MST (vertices rearranged) from Step 1
v1 v3 v1 v6 v2 v4 v2 v5 v2 v6 v1
v1 v2v3
v4v5v6
Step 3: Optimizing the DFS Walk
Tour from Step 2:
v1 v3 v1 v6 v2 v4 v2 v5 v2 v6 v1
Optimized Tour:v1 v3 v1 v6 v2 v4 v2 v5 v2 v6 v1
v1 v3 v6 v2 v4 v5 v1Tour Weight: 31
v1 v2v3
v4v5v6
7/30/2019 NP Algoritm
19/23
Heuristic # 3 for the TSP Problem
Twice-around-the Tree AlgorithmTSP Tour of Twice-around-the-Tree Algorithm
v1 v3 v6 v2 v4 v5 v1Tour Weight: 31
v1 v3 v6
v2v4v5
3 7
2
49
6
Improved Tour Weight: 26 Improvement using 2-Change
v1 v3 v6
v2v4v5
3
5 2
46
6
v1 v3 v4
v2v6v5
3 5
4
26
6
7/30/2019 NP Algoritm
20/23
Proof for the ApproximationRatio for Twice-around-the-Tree Let w(MST) be the weight of the MST generated from Step 1.
The weight of the DFS walk generated from Step 2 could be at most2*w(MST), as seen in the example.
In Step 3, we are trying to optimize the DFS walk and extract aHamiltonian Circuit of lower weight. Even if no optimization ispossible, the weight of the tour generated by the Twice-around-the-
Tree algorithm is at most twice the weight of the minimum spanningtree of the graph instance of the TSP problem.
Note that w(MST) of the graph has to be less than the weight of anoptimal tour, w(Optimal Tour). Otherwise, if w(Optimal Tour)
w(MST), then the so-called MST with V-1 edges is not a MST. W(Twice-around-the-Tree tour) 2*W(MST) < 2*w(Optimal Tour).
W(Twice-around-the-Tree tour) / W(Optimal Tour) < 2.
Hence, the approximation ratio of the Twice-around-the-Tree
algorithm is less than 2.
7/30/2019 NP Algoritm
21/23
Heuristic # 4 for the TSP Problem
Christofides Algorithm Step 1: Construct a Minimum Spanning
Tree (MST) of the graph correspondingto a given instance of the TSP problem
Step 2: Find the set of vertices thathave an odd degree in the MST of Step1. There will always be an even numberof vertices with an odd degree in a
connected graph. Form a complete subgraph of only the vertices with odddegree in the MST.
Step 3: Run the Edmonds-Karpalgorithm (a slight variation of the Ford-
Fulkerson algorithm with the augmentingpaths determined in increasing order ofthe hop count using BFS) on the
complete graph and determine the set ofedges that form a minimum-weight pair
matching (MWPM).
Step 4: Superimpose the edges formingthe MWPM of Step 3 with the edgesforming the MST of Step 1. On the
MST-MWPM graph, do a DFS traversaland construct a walk as in Step 2 of theTwice-around-the-Tree algorithm.
Step 5: Scan the vertex list obtained in
Step 4 and eliminate from it all repeatedoccurrences of the same vertex exceptthe starting one at the end of the list.The vertices remaining on the list will
form a Hamiltonian Circuit, which is theoutput of the algorithm.
Note that instead of using Edmonds-Karp algorithm, we will use a greedystrategy of picking the edges
in the increasing order of their edge
weights
7/30/2019 NP Algoritm
22/23
Heuristic # 4 for the TSP Problem
Christofides Algorithm
v1 v2 v3
v4v5v6
Step 1: MST of the Graph
Step 2: Complete Graph of theVertices with Odd Degree in the MST
v2, v3, v4 and v5 are the four vertices
with odd degree in the MST of Step 1.We now form a complete graph involving
these vertices and their edges
v2 v3
v4v5
3
5
5
9
46
Step 3: Min. Wt. Pair Matching (MWPM)
v2 v3
v4v5
3 5
7/30/2019 NP Algoritm
23/23
Heuristic # 4 for the TSP Problem
Christofides Algorithm
v1 v2 v3
v4v5v6
Step 4: Superimposition ofthe MST and MWPM
Comparison of Tour Weights ofHeuristics before and after theApplication of 2-Change
Before AfterNearest Neighbor 30 24Multi-fragment 26 24
Twice-around-the-Tree 31 26
Christofides algorithm 25 25
Step 5: Optimizing the DFS Walk
List of vertices on a DFS Traversal
v1 v3 v4 v2 v5 v2 v6 v1
List of vertices on a Hamiltonian Circuit
v1 v3 v4 v2 v5 v6 v1
v1 v3 v4
v2v5v6
3 5
4
36
4
Tour Weight: 25No further improvement with 2-change
is possible.