A Taste of Complexity

For this section it will be assumed that you have familiarity withthe basic concepts of graph theory.

Two Fundamental ProblemsRecall that a walk W in a graph G is a sequencev1e1v2e2 . . . vn−1en−1vn with the property that, fori ∈ {1, . . . , n − 1} the edge vi is incident with the vertices vi andvi+1. Walks may contain repeated edges and vertices. If there areno repeated edges or vertices, the walk is a path.

I An Eulerian Walk is one in which every edge of the graph isused exactly once.

I A Hamiltonian Path is a walk in which each vertex of thegraph is used exactly once. Note that such a walk isnecessarily a path as, if an edge appears twice, then so do itsincident vertices.

Here are two decision problems for graphs.

EULERIAN WALKInput: A graph GQuestion: Does G contain an Eulerian Walk?

HAMILTONIAN PATHInput: A graph GQuestion: Does G contain a Hamiltonian Path?

We’ll refer to these problems as we discuss the basic ideas ofcomplexity theory.

Warning

A problem closely related to hamiltonian path is hamiltoniancycle which askes for a cycle containing all the vertices. I may attimes muddle the two. It’s not a big deal, as the complexitytheoretic issues for both problems are the same.

In complexity theory we seek to classify problems according to thefollowing criteria:

I Is a problem intrinsically easy or difficult to solve?

I Given two problems, P1 and P2, which is easier to solve?

For us an algorithm will be a method of solving a problem that issufficiently precise that we can program a computer to do it. Analgorithm proceeds by performing a sequence of operations orsteps.We will treat these notions informally. To make them preciserequires the theory of Turing machines and is beyond the scope ofthis course. Enrol in MATH 435 Computability and Complexitynext year.

Superficially both eulerian walk and hamiltonian path seempretty straightforward. An Eulerian Walk is a permutation of theedges of G . Just check all permutations. How hard is that? It iseasy to see that a computer could be programmed to do it, so thatwe have an algorithm. How many operations do we need? Thatdepends on the size of the graph.

An instance of eulerian walk is obtained by choosing a fixedgraph G .

I Thus an algorithm is a method for solving the problem thatwill work for all possible instances.

I Clearly the “bigger” the graph, the more operations will beneeded by an algorithm to solve the problem.

I We need a measure of the size of an instance. Most problemshave natural measures.

Measuring the size of a graph

For graph problems we could use the number of vertices, or thenumber of edges, or the sum of the numbers of vertices and edges.It turns out that for many graph problems it doesn’t make thatmuch difference which of these we use. We’ll use the number ofvertices as a measure of the size of a graph.

The running time of an algorithm A on an instance is the numberof steps it takes to complete the computation. The timecomplexity TA(n) for instances of size n is the maximum over allinstances of size n of the running time of A.

I The time complexity given here is worst case. It may be thatfor an average case the running time is much better. Butwhat an average case is depends on how instances are chosen.

I Note that the time complexity is a function whose domain isthe natural numbers.

A powerful computer

Imagine a very powerful computer. This computer makes use ofevery atom in the Earth in such a way that each atom couldperform 1010 operations per second and it started running at thebeginning of the earth. How many operations would it haveperformed?

Apparently the Earth has been around for about 4.1× 1017

seconds and there are about 3.6× 1051 atoms in the Earth, so ourcomputer would have done about 1.5× 1079 operations by now. Itseems fair to say that no human-built computer will be likely to domore than 1080 operations, either now, or in the foreseeable future.

Our algorithm for testing whether a graph has an Eulerian Walkhas time complexity at least n!, as a graph with n vertices canhave at least n edges and in the worst case we need to check allpermutations of the edges. According to my calculator65! ∼= 8.2× 1090, so that a graph with 65 edges is well beyond afeasible calculations.

We need a better algorithm. To get that, we need a theorem.Recall that the degree of a vertex in a graph is the number ofedges incident with it.

TheoremA graph has an Eulerian Walk if and only if it has either zero ortwo vertices of odd degree.

Here is a new algorithm for solving eulerian walk. Check thedegrees of each vertex. It is straightforward to see that thecomplexity of this algorithm is at most n2. The problem can besolved for a graph with 1, 000 vertices in at most 1, 000, 000 steps.No problems. We have found a polynomial-time algorithm andthat is good.

What about hamiltonian path? Perhaps surprisingly, there is nopolynomial-time algorithm known for this problem. Indeed, there isno known algorithm that has better complexity than 2n. How badis this? Well 2300 ∼= 2× 1090, so even our super computer couldnever solve the problem for graphs with 300 vertices.

Order notationWe slipped past a few subtleties in the above. One is that it isquite rare to be precise about exact values of complexity functions.Let f : N→ N and g : N→ N be functions. We say that f is oforder g and write f (n) = O(g(n)) if there are natural number kand l such that f (n) ≤ k(g(n)) for all n ≥ l .

Thus, for example, if f (n) = n2 + 4n + 7 then f (n) = O(n2).

More precisely we should have said that the algorithm foreulerian walk has O(n2). I’m not sure what the running timeof the current best algorithm for this problems is, but the crucialthing is that it has polynomial complexity. On the other hand I liedsomewhat about the running time of hamiltonian path. It maybe that the best known algorithm for this problem has O((1.7)n).(You should look it up!) The crucial point is that is has exponentialcomplexity. The blowout for running time for exponentialcomplexity versus polynomial complexity illustrated in the nexttable happens eventually so long as the exponent is greater than 1.

n n2 2n

10 100 1024100 10 000 1.26× 1030

1000 106 1.07× 10301

In general we regard a polynomial running time as good and anexponential one as bad. Of course there are issues.

The Class PA problem lies in P, or is polynomial time solvable if there is analgorithm that solves the problem that has complexity O(p(n)) forsome polynomial p(n). Strictly speaking the class P only applies todecision problems, but we will also extend it to other types ofproblems.

We know that eulerian walk lies in P. What abouthamiltonian path? We don’t have a polynomial-time algorithm;but, on the other hand we have no proof that it is not in P.

Many familiar problems lie in P. A natural way to measure the sizeof a number for complexity purposes is to use the number of digitsin a decimal or binary representation. It doesn’t matter which wechoose. Why?

With the above measure of size, addition and multiplication ofnumbers is in P. What about exponentiation mod q?

The Class NPThe Delphic Oracle was all seeing and gave sage advice to theGreeks of ancient times. But the Delphic Oracle was mysteriousand even a little untrustworthy so her advice had to be treatedwith great care. Instead of trying to find a Hamiltonian Path in ourgraph with 300 vertices using an algorithm, we visit the Oracleinstead. The Oracle inspects the graph for ten seconds or so andthen she reports to us that the graph indeed has a HamiltonianPath. Our suspicions are raised so we—very politely—ask for theOracle to prove it to us. The oracle then rolls her eyes, sighs, andcarefully shows us a Hamiltonian Path. The crucial thing is thatshe can describe this path to us in polynomial time.

It may take exponential time to find a Hamiltonian Path, but onceit is found, it can be proved in polynomial time that the graph isindeed Hamiltonian. Problems with this property are in the classNP.

I What if the Oracle decides that the graph is not Hamiltonian?How does the Oracle prove this to us in polynomial time?

Consider the following problem.

PRIMEInput: A numberQuestion: Is the number prime?

Is this problem in NP? The Oracle decides that the number isprime, but how does she quickly prove to us that she is right?

On the other hand we could consider the alternative problem.

COMPOSITEInput: A numberQuestion: Is the number composite?

In this case the problem is indeed in NP as if the Oracle finds thatthe number n factorises as pq she can prove to us in polynomialtime that this is indeed the case as there is an efficient algorithmfor multiplication.

The crucial point is that to show that a problem is in NP we onlyhave to be able to prove a polynomial time certificate for a yesinstance.

NP-complete problems

Let P1 and P2 be problems in NP. Loosely speaking we say thatP1 is (polynomial-time) reducible to P2 if there exists apolynomial-time algorithm that converts an instance of P1 to aninstance of P2.

I Observe that if P1 is reducible to P2, then given apolynomial-time algorithm for solving P2, there is also apolynomial-time algorithm for solving P1.

Consider the following problem. Recall from MATH161 that aBoolean formula is satisfiable if there is an assignment of truthvalues to the variables of the formula that makes the formula true.Thus p ∨ q is satisfiable but p ∧ q ∧ ¬q is not satisfiable.

SATISFIABILITYInput: A Boolean formulaQuestion: Is the formula satisfiable?

In 1971 Cook proved the following remarkable theorem.

Theorem (Cook’s Theorem)

Every problem in NP is reducible to satisfiability.

In other words: if there exists a polynomial-time algorithm forsolving satisfiability, then every problem in NP has apolynomial-time algorithm. That’s an amazing theorem. But waitthere’s more!

A problem is NP-complete if every problem in NP is reducible to it.Cook’s Theorem shows that satisfiability is NP-complete. In1972 Richard Karp gave a list of 21 NP-complete problems; one ofwhich is hamiltonian path. Now there are over 3000 suchproblems. See http://en.wikipedia.org/wiki/List_of_NP-complete_problems

Exercise 1. Go to the above web address, choose your favouriteNP-complete problem and find a polynomial-time algorithm for it.

If you succeed, you will have shown that every problem in NP is inP. In other words, you have proved that P = NP.

How likely are you to succeed? It seems to good to be true thatP = NP. It’s hard to even contemplate the possibility thatsatisfiability has an algorithm much better than trying allpossibilities. In this case you should try

Exercise 2 Choose your favourite NP-complete problem and provethat it does not have a polynomial-time algorithm.

If you succeed, you will have shown that P 6= NP.

The amazing thing is, that, despite the best attempts ofmany—apparently bright—people, this issue is still not resolved. Itis not known whether or not P = NP. This is one of the mostimportant open problems in mathematics. I don’t know anymathematician who believes that P = NP. How could it be true.But . . .

Hard problems can have easy instances

Just because a problem is thought to be hard does not mean thata given instance of the problem is hard. If, for example, the graphyou are given is a tree, then it is certainly easy to decide if it has aHamiltonian path. Remember this when we get back tocryptography.