Computability and computational complexityn Master the main computational complexity classes, their underlying models of computation, and relationships. n Understand the concept of

May 21, 2018 1

Computability and computational complexity

Fall 2015http://users.abo.fi/ipetre/computability/

Lecture 1: Introduction

Ion PetreComputer Science, Åbo Akademi University

http://users.abo.fi/ipetre/computability/


What is computer science?

n Hugely successful discipline, best seen from the impact of computers and computer technologies in everyday lifeq computers control aircrafts, nuclear powerplants, chemical plantsq computers help run companies, guide investmentsq many medical procedures cannot be executed without computers;

intelligent body implants interacting with the organismq computers execute tasks with great precision: intricate simulations, data

mining, robot-aided manufacturing, integrated circuit designq handle huge amounts of data, process them, extract informationq help in educationq provide entertainmentq help us stay connected

May 21, 2018 http://users.abo.fi/ipetre/computability/ 2

What is computer science

n Profound influence on many other disciplinesq computational biologyq computational chemistryq computational physicsq computational mathematicsq computational linguisticsq computational and algorithmic musicq computational intelligenceq computational philosophyq computational neuroscienceq computational economicsq …


Computer Science as a scientific discipline

n While answering to challenges from technology and from other sciences, Computer Science also developed as a fundamental disciplineq Developed its own language and tools to reason about computingq Independently of the physical implementation of computersq Identified several great challenges stemming from within Computer

Science itself; some received great recognition from the scientific community

n P=NP? one of the jewels of Computer Scienceq Clay Mathematics Institute:

http://www.claymath.org/millennium/P_vs_NP/q Great company, see e.g. the Riemann hypothesis (1859)q A problem about the inherent limits of computing


http://www.claymath.org/millennium/P_vs_NP/

Algorithms

n Long studied in mathematics

q The oldest known algorithm is Euclid’s

algorithm to compute the greatest

common divisor – more than 2000 years

old

q Term “algorithm” comes from the Persian

mathematician Abu Abd-allah

Mohammad ibn Musa al-Khwarizmi (ca.

780 – ca. 850; Latin version of his name

Algoritmi) n developed the discrete algorithmic rules

(still used today) to compute additions,

subtractions, multiplications and divisions

of decimal numbers


Source: Wikipedia

Theory of algorithms: three directions

n Computability theoryq focuses on the study of undecidabilityq started with Gödel’s proof that there are algorithmically undecidable

problemsn destroyed David Hilbert’s (1862-1943) dream that all “reasonably defined”

problems are in principle algorithmically decidable

n Design and analysis of algorithms (not covered in this course)q grew in connection with the developing of computer technologies q focuses on developing fast algorithms, methods for analyzing them

n Computational complexityq focuses on how many resources (e.g., time and space) are needed to

solve a given problem (not an algorithm)q complexity classes


Focus of the course

n Central concept in the course: computational problemsq Seen from the point of view of applications: solve them (algorithms)q Seen from the point of view of computability: study their properties

n Reduce them to another computational problemn Establish their complexity classn Degrees of complexity

n Focus in this course on the inherent limitations of the world of computingq Things that computers really cannot doq Things that computers really cannot do efficientlyq What (not) to do in practice with a problem of high computational

complexity


May 21, 2018 8

Content of the course

n Problems and algorithmsn Turing machinesn Computabilityn Boolean logicn Relations between complexity

classesn Relations and completenessn NP-complete problems

n coNPn Randomized computationn Approximabilityn On P vs NPn Parallel computationn Polynomial spacen Beyond Turing


May 21, 2018 9

Learning objectives

n Master the main computational complexity classes, their underlying

models of computation, and relationships.

n Understand the concept of reductions and its role in classifying problems

by their computational complexity.

n Understand the concept of “unsolvable/undecidable”

n Understand the concept of universal machines

n Understand the meaning, the significance and the implications of the P

vs. NP problem

n Become familiar with the concepts of randomized, approximation, and

parallel algorithms and become aware of the related complexity classes.


May 21, 2018 10

Administrative details

n Webpage: http://users.abo.fi/ipetre/computability/n Email address: [email protected] Teaching assistant: Diana-Elena Gratie [email protected]

n Lectures: Mondays 10.15-11.45 and Wednesdays 10.15-11.45 (Catbert, B3028, ICT House)

n Exercise sessions: Mondays 13.30-15.00 (Catbert, B3028, ICT House)n Exam: 30 points for maximum mark, 15 to pass

q Exam dates: 18.12.2015, 8.1.2016

n Course book: Christos H. Papadimitriou, Computational complexity. Addison-Wesley, 1994.

n Other useful books:q O. Goldreich. Computational complexity: a conceptual perspective. Cambridge University Press,

2008q M. Sipser. Introduction to the theory of computation. PWS Pub co, 1996.q O. Goldreich. P, NP, and NP-completeness. Cambridge University Press, 2010.q D. Harel. Computers ltd. What they really can't do. Oxford University Press, 2000.

n Prerequisites: formal languages and automata, algorithmics



mailto:[email protected]

mailto:[email protected]

Problems vs. algorithms

n We focus in this course on the computational complexity of problems, not of algorithms

n Question: what is a “problem”?n Introduce here three simple examples

q REACHABILITYq MAX FLOWq TSP

q through them introduce a few key definitions and concepts


Example 1: REACHABILITY

n REACHABILITYq Instance: a finite directed graph G=(V,E) and two nodes 1,n ∈ Vq Question: Is there a path in G from 1 to n?

q Example:


1

5

2 3

4

Algorithm for REACHABILITY

S={1}; mark node 1while S≠Ø do

choose a node i from S and remove it from S;for all edges (i,j)∈E do

if j is unmarked, then mark j and add it to Sendif

endforendwhileif n is marked, then return “yes: there is a path from 1 to n”else return “no: there is no path from 1 to n”endif


REACHABILITY

n The algorithm is correctq prove by induction on the number of nodes of the graph that there is a

path from 1 to n if and only if n is marked at the end of the algorithm

n Unclear points in the algorithmq how is the graph represented in the algorithm?

n we discuss later that this depends on our model of an algorithmn assume here we have the adjacency matrix

q how do we choose element i from Sn different strategies will lead to different graph traversal strategies

q if S is a queue: breadth-first traversalq if S is a stack: depth-first traversal

q how efficient is the algorithmn note that each edge is considered oncen In terms of edges O(|E|) stepsn in terms of nodes: O(|V|2) steps


Measure of algorithm efficiency

n Typically measured by the rate of growth of the running time (or memory consumption)q how does the running time grow when the input size grows

n The O-notation most used hereq multiplicative and additive constants ignored


O-notation

Let f,g:N→Nn f(n)=O(g(n)) if f(n) ≤ C·g(n) for all n≥n0 for two constants n0 and C

n f(n)=Ω(g(n)) if g(n)=O(f(n))q Equivalently: f(n) ≥ C·g(n) for all n≥n0 for two constants n0 and C

n f(n)=Q(g(n)) if f(n)=O(g(n)) and g(n)=O(f(n))q k1·g(n)≤f(n) ≤ k2·g(n), for all n≥n0, for some constants k1, k2, n0

n Example: Let p(n) a polynomial of degree k q p(n)= Q(nk)q for any constant c>1, p(n)=O(cn), but p(n)≠Ω(cn)

n In other words: any polynomial grows strictly slower than any exponential

q Consequence: logkn=O(n)


Efficiently solvable vs. intractable problems

n Throughout this course we consider that:q a problem is efficiently solvable if there is an algorithm solving the

problem such that the rate of growth of its solution time is polynomial, i.e., it is O(nd), for some d

q a problem is intractable when it has no polynomial time algorithm


Discussion

n Polynomial time bounds as a definition for efficiently solvable problemsq there exist efficient computations that are not polynomial

n for example O(2n/100)q there exist polynomial computations that are not efficient

n for example O(n100)q Argument: any polynomial rate will eventually be overcome by any exponential one; the

latter rate is only better for a finite number of instancesn Our definitions only consider worst-case complexity

q the exponential worst-case performance of an algorithm may be due to a statistically insignificant (or uninteresting) fraction of the inputs; might perform for most inputs

q more informative would be the expected performance of an algorithm, rather than its worst-case one

q Argument: the input distribution of a problem is rarely available in practice

“Adopting polynomial time worst-case performance as our criterion of efficiency results in an elegant and useful theory that says something meaningful about

practical computation, and would be impossible without this simplification”Papadimitriou, 1994


Example 2: MAX FLOW

MAX FLOW

n Instance: Network N=(V,E,s,t,c), where q (V,E) is a directed graph, s,tÎV (source/sink)

q no incoming edges into s, no outgoing edges from tq for each edge (i,j)∈E, c(i,j) is a non-negative integer (the capacity of the

edge)

q flow of N is a function over non-negative integers f(i,j)≤c(i,j) such that for each node other than s and t, the sum of flows of its incoming edges is equal to the sum of the flow of its outgoing edges

q the value of the flow is the sum of the flows of the outgoing edges of s (the same as the sum of the flows of the incoming edges in t)

n Question: What is the largest possible value for a flow in N?


MAX FLOW – an example


ts

2

2

1

13 3

4

1/1

0/1

ts

0/2

2/2

3/3 1/3

2/4

Network N

A flow of value 3 Increasing the flow to 4

ts

1/2

2/2

3/3 1/3

3/4

1/1

1/1

Discussion

n MAX FLOWq this is not a decision problem as REACHABILITYq it is an optimization problemq it can be made into a decision problem: is there a flow of value at least

K, for some constant Kn Historical details

q proposed in 1954 by T.E.Harris as a simplified model of Soviet railway traffic flow

q First solution (exponential time): Ford-Fulkerson algorithm, 1955q First polynomial algorithm: Edmonds, Karp, 1972

n Nice example of a problem where, once the exponential barrier was broken, better and better polynomial time algorithms were developedq O(n5), …, O(n3), …

n Note: there is a positive flow iff t is reachable from s


MAX FLOW

n A polynomial-time algorithm for deciding whether a given flow f is optimal or not

q For simplicity, we assume that between no two vertices are there edges of opposing orientation

q Assume there is a bigger flow f’ and define ∆f(i,j)=f’(i,j)-f(i,j)

q Because of the flow equalities in each node, for both f and f’, we get that ∆f also satisfies the equalities, i.e., it is a flow as well

n some of the values ∆f(i,j) might be negative; they can be thought of as a flow in the reverse direction; in this case |∆f(i,j)|≤f(i,j) (because f’(i,j)≥0)

n for the values ∆f(i,j) that are positive, we have that ∆f(i,j)≤c(i,j)-f(i,j) (because f’(i,j)≤c(i,j))


MAX FLOW

n A polynomial-time algorithm for deciding whether a given flow f is optimal or not (continued from previous slide)

q Based on this idea, for a given flow f we build the following derived network N(f)=(V,E’,s,t,c’), where

n E’=E U {(i,j): (j,i)∈E, f(j,i)>0}

n c’(i,j)=c(i,j)-f(i,j) if (i,j)∈E

n c’(i,j)=f(j,i) if (i,j)∈E’-E

n Key idea: f is optimum for N if and only if there is no positive flow in N(f)q Note: s is not a source in N(f); define the flow of N(f) as the sum of the flows on the

outgoing edges from s minus the sum of the incoming edges into s

n Note: there is a positive flow if and only of t is reachable from s q à REACHABILITY


MAX FLOW – an example


ts

2

2

1

13 3

4

ts

0/2

2/2

0/1

1/1

3/3 1/3

2/4

ts

2

2

1

13

1

2

22

Network N

A flow of value 3 Network N(f)

MAX FLOW - a polynomial algorithm

n Start with the everywhere-zero flow in Nn Repeatedly build N(f) and find a path from s to t in N(f)

q If there is one, find the smallest capacity of any edge on the path and add this number to the value of f on all edges of this path

q If there is no path then f is maximum and we stop

n Complexity of the algorithm:q Each iteration takes O(n2) – solving REACHABILITYq There can only be n·C stages, where C is the maximum capacity of any

edge in the networkn each iteration increases the flow with at least 1 (work over integers)n maximum flow cannot be more than n·C because there are at most n edges

going out of the sourceq Complexity: O(n3·C)


Example 2.1: Bipartite matching

MATCHINGn A bipartite graph is a triple B=(U,V,E), where

U={u1, u2,…, un} is a set of nodes called boys and V={v1, v2,…, vn} is a set of girls. EÍUxV is the set of edges

n A perfect matching is a set MÍE of n edges such that for any two edges (u,v),(u’,v’)ÎM we have u¹u’ and v¹v’q no two edges are adjacent to the same boy,

no two edges are adjacent to the same girl

n Question: Given a bipartite graph, does it have a perfect matching?


A key tool: problem reductions

n A reduction is an algorithm that solves problem A by transforming any instance x of A into an equivalent instance R(x) of a previously solved problem B

n An efficient algorithm for B plus an efficient reduction of A to B provides an efficient algorithm for A


Reduction R

Algorithm for B

R(x)

Algorithm for A:

Input x Answer

Reduction of MATCHING to MAX FLOW

n Build a network from the given bipartite graph as suggested above, where all capacities are equal to 1

n The bipartite graph has a perfect matching if and only if its associated network has a flow of value nq our polynomial solution for MAX FLOW yields a polynomial solution for

MATCHING


s t

Example 3: Traveling salesman problem

TSP

n Instance: Given n cities 1,2,…,n and a nonnegative integer distance di,j between any two cities i and j (with di,j=dj,i)

n Question: what is the shortest tour of the cities?

q In other words: find a permutation p of the cities such that Si dp(i),p(i+1) is minimal (where p(n+1)=1)

n The formulation above: optimization problem

n TSP as a decision problem: is there a tour of length at most B (budget)?


Discussion

n Naïve solution for TSP: enumerate all possible tours (permutations), compute their cost, pick the bestq Impractical: O(n!) tours

n There are no polynomial-time algorithms known for TSPq Not known whether there are polynomial algorithms for TSPq Closely related to the P¹NP? problem


Learning objectives for this lecture

n Growth rate, O-notationn The concepts of decision problem, optimization problemn Problem reductions


Documents

Computability and computational complexityn Master the main computational complexity classes, their underlying models of computation, and relationships. n Understand the concept of