CS 448 Foundation of CS 448 Foundation of ComputingComputing

Soundararajan EzekielSoundararajan Ezekiel

Department of Computer ScienceOhio Northern University

ADA, Ohio 45810e-mail: s-ezekiel@onu.edu

http://www.onu.edu/user/FS/sezekiel

04/22/23 2

Today’s PlanToday’s Plan What are the fundamental capabilities What are the fundamental capabilities

and limitation of computersand limitation of computers Three major topics: What is a computer? - Automata Theory What can (cannot) be computed? - Computability Theory What can (cannot) be computed efficiently?--

Complexity Theory

04/22/23 3

Why Study Theory?

Pragmatic reasonsPragmatic reasons• apply efficient algorithms to tractable problemsapply efficient algorithms to tractable problems• avoid intractable or impossible problemsavoid intractable or impossible problems• learn to tell the differencelearn to tell the differenceMathematical EducationMathematical Education• automata theoryautomata theory• complexity classescomplexity classes• decidability theorydecidability theoryPhilosophyPhilosophy• what is computation?what is computation?• what is provable?what is provable?

04/22/23 4

Topics Automata Theory: What is a computer? (Review) Computability Theory: What can computers do?(Semi-review) Complexity Theory: What makes some problems computationally hard

and others easy? Evading intractability: Approximation Randomization Parallelism ... Lower bounds: Circuit complexity.

04/22/23 5

Complexity Theory Key notion: Key notion: tractable tractable vs. vs. intractable intractable problems.problems. A A problem problem is a general question:is a general question:

– description of parametersdescription of parameters– description of solutiondescription of solution

An An algorithm algorithm is a step-by-step procedureis a step-by-step procedure– a recipea recipe– a computer programa computer program– a mathematical objecta mathematical object

We want the most We want the most efficient efficient algorithmalgorithm– fastest (mostly)fastest (mostly)– most economical with memory (sometimes)most economical with memory (sometimes)– expressed as a function of problem sizeexpressed as a function of problem size

Example: Traveling Salesman Problem

a

b

c

d

10

5

6

9

3

9

(not drawn to scale)

ONU

CS dept

Park

Dave’s market

Parameters:• set of places• set of inter- places distances

Example: Traveling Salesman Problem

a

b

c

d

10

5

6

9

3

9

(not drawn to scale)Solution: Want the shortest tour through the places Example: a; b; d; c; a has length 27.

ONU

CS dept

Park

Dave’s market

04/22/23 8

Problem Size What is appropriate measure of problem size?

– m nodes?– m(m+1)/2distances?

Use an encoding of the problem– alphabet of symbols– strings: abcd//10/5/9//6/9//3.

Measures– Problem Size: length of encoding– Time Complexity: how long an algorithm takes, as function of problem

size.

04/22/23 9

Time Complexity

What is tractable? A function f(n) is O(g(n)) whenever|f(n)| c*|

g(n)| for n >some constant. A polynomial-time algorithm is one whose time

complexity is O(p(n)) for some polynomial p(n). An exponential-time algorithm is one whose time

complexity cannot be bounded by a polynomial (e.g., n log n).

04/22/23 10

TractabilityBasic distinction:• polynomial time = tractable exponential time = intractable

Execution time 10 20 30 40 50 60n .00001 .00002 .00003 .00004 .00005 .00006 second second second second second secondn2 .00001 .00004 .00009 .00016 .00025 .00036 second second second second second secondn3 .00001 .00008 .027 .064 .125 .216 second second second second second secondn5 .1 3.2 24.3 1.7 5.2 13.0 second seconds seconds minute minutes minutes2n .001 1.0 17.9 12.7 35.7 366 second second minutes days years centuries3n .059 58 6.5 3855 2*10^8 1.3*10^13 second minutes years centuries centuries centuries

11 04/22/23

Effect of Speed-UpsLet's wait for faster hardware!Consider maximum problem size you can solve in an hour. present 100 times faster 1000 times faster

n N1 100 N1 1000 N1

n2 N2 10 N2 31.6 N2

n3 N3 4.64 N3 10 N3

n5 N4 2.5 N4 3.98 N4

2n N5 N5+6.64 N5+9.97

3n N6 N6 +4.19 N6+6.29

04/22/23 12

Your boss says:Your boss says:Get me an efficient traveling-salesmanGet me an efficient traveling-salesmanalgorithm, or else.”algorithm, or else.”What are you going to do?What are you going to do?

Response“Yes Ma'am, expect it this afternoon!"Problem is• All known algorithms (essentially) check all possible paths.• Exhaustive checking is exponential.• Good luck!

04/22/23 13

AnswerAnswer “Hah!, I will prove that no such algorithm is possible" Problem is, proving intractability is very hard. Many important problems have no known tractable

algorithms -- no known proof of intractability. “I can't find an efficient algorithm, I guess I'm just a

pathetic loser. " Bad for job security. “The problem is NP-complete. I can't find an efficient

algorithm, but neither can any of these famous people . . ." Advantage is: The problem is “just as hard" as other

problems ---smart people can't solve efficiently.----So it would do no good to fire you and hire a Dartmouth graduate.

04/22/23 14

Further ReadingsFurther Readings http://www.claymath.org/prizeproblems/milliondollarminesweeper.htm http://www.claymath.org/prizeproblems/pvsnp.htm

Response “Would you settle for a pretty good algorithm?" Intractability isn't everything.

– find an approximate solution– use randomization (“probably pretty good")– parallelism can also help

Approximation, randomization, etc. are (arguably) the hottest areas in complexity theory today.

04/22/23 15

Computability In the first half of the 20th century, mathematicians such as In the first half of the 20th century, mathematicians such as

Kurt Goedel, Alan Turing, and Alonzo Church discovered Kurt Goedel, Alan Turing, and Alonzo Church discovered that some problems do not have a computable solution.that some problems do not have a computable solution.

Example: is a mathematical statement true or false?Example: is a mathematical statement true or false? What could be more amenable to automation?What could be more amenable to automation? Results needed theoretical models for computers.Results needed theoretical models for computers. These theoretical models helped lead to the construction of These theoretical models helped lead to the construction of

real computersreal computers..

04/22/23 16

Computability Hilbert's tenth problem (1900): Find an finite procedure

(algorithm) that decides whether a polynomial has an integer root.

A polynomial is a sum of terms, each term is a product of variables and a constant (called coefficient).

A root of a polynomial is an assignment of values to the variables such that the value of the polynomial is 0.

x = 2; y = 1is a root of the polynomial 5x +15y= 25 There is no such algorithm (1970)!

04/22/23 17

Some Other Undecidable Some Other Undecidable ProblemsProblems Does a program run forever?Does a program run forever? Is a program correct?Is a program correct? Are two programs equivalent?Are two programs equivalent? Is a program optimal?Is a program optimal? Is a finitely-presented group Is a finitely-presented group

trivial?trivial?

04/22/23 18

Sets---order doesn't matter finite:{2; 4; 6} infinite:{2, 4,6,...},{x/ x is even} subset: UV , UV union: UV--- intersection: UV Sequences--order matters (1, 3, 5, ...) finite sequence called a tuple-2 elements is pair k elements is k-tuple Powerset of U is set of all subsets Cartesian product of U x V

Mathematical ToolsMathematical Tools

04/22/23 19

Strings and Languages an alphabet is a finite set of symbols a string over an alphabet is a finite sequence of symbols from that

alphabet. the length of a string is the number of symbols the empty string reverse: abcd is dcba. substring: xyz in xyzzy. concatenation of xyz and zy is xyzzy. xk and x, ... x, k times. a language is a set of strings.

04/22/23 20

ProofsProofs We will use the following basic kinds of proofs. by constructionby construction by contradictionby contradiction by inductionby induction by reductionby reduction

Sometimes we mix them.Sometimes we mix them.

04/22/23 21

Proof by Construction Graph:- G = (V;E), where V is set of nodes or vertices,

and E is set of edges --degree of a vertex is number of edges A graph is k-regular if every node has degree k. Theorem: for every even n >2,there exists a 3-regular graph with

n nodes

Note: picture is helpful, but not a proof!

04/22/23 22

Theorem: 2 is irrational. Proof: Suppose not. Then 2 = m/n ,where m and n are relatively relatively

primeprime. n 2 = m 2n2 = m2

So m2 is even, and so is m = 2k. 2n2 = (2k)2= 4k2

n2 = 2k2 So n2 is even, and so is n. Thus both m and n are even, Thus both m and n are even, not relatively prime!not relatively prime!

Proof by ContradictionProof by Contradiction

04/22/23 23

Induction Prove properties of elements of an infinite set.N={1,

2, 3,....} To prove that holds for each element, show: base step: show that (1) is true. induction step: show that if (i) is true (the

induction hypothesis), then so is (i +1).

04/22/23 24

Theorem: All cows are the same color. Base step A single-cow set is definitely the same color. Induction Step: Assume all sets of ii cows are the same

color. Divide the set{1..., i+1} into U ={1,…, i}, and V {2,…, i+1}.

all cows in U are the same color by the induction hypothesis. all cows in V are the same color by the induction hypothesis. all cows in UV are the same color by the induction hypothesis.

Ergo, all cows are the same color.

Induction Example

04/22/23 25

Back to Language alphabet example ={x} L1={ x^n/ n=1,2,..} L2={x^(2n+1)/ n=0,1,2...} ={ a b} L=PALINDROME={, set of all string x/

reverse(x)=x}={ a b aa bb aaa ....} KLEENE CLOSURE (Kleene star/ Kleene

operator) Stpehern Kleene http://burks.brighton.ac.uk/burks/foldoc/55/109.htm

given an alphabet -- define language = + any string of letter from and denoted by *-- language called closure

04/22/23 26

Example ={x} then *={ x^n, n=1,2..} ={0 1} then *={ 0 1 00 01 ..} extend it to set of words-- S= set of words then S*={ plus

set of all finite strings formed by concatenating words} example: S={ a ab} then S*={ b a aa bb ... abaab.. .} Factoring:- abaab= (ab)(a)(ab) these are in S Factoring is not unique--- example S={xx xxx} S*={x^n,

n=0,2,3,..}-- x^6= x^3 x^3 or x^2 x^2 x^2

04/22/23 27

Note parentheses () are not letters --- in case if you use () are letters

of alphabet -- ={x ( )} length(xxxxx)=5 but length( (xx) (xxxx))=10 Proof by constructive algorithm closure( closure )= closure ---- (S*)*= S**= S* Proof: ==> S*S** Every word in S** is made up of factors from

S*. Every factors from S* is made up of factors from S--- every word in S** is made up from S. therefore every word from S** is also in S*

<=== S** S* we know that S S* apply closure both side we have S* S**

Together S*=S**

04/22/23 28

Homework Problems

Problem # 5, 7, 9. 14,