44
Remaining Topics Decidability Concept 4.1 The Halting Problem 4.2 P vs. NP 7.2 and 7.3 NP-completeness & Cook-Levin Theorem 7.4

Remaining Topics Decidability Concept 4.1 The Halting Problem 4.2 P vs. NP 7.2 and 7.3 NP-completeness & Cook-Levin Theorem 7.4

Embed Size (px)

Citation preview

Page 1: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Remaining Topics

Decidability Concept 4.1 The Halting Problem 4.2 P vs. NP 7.2 and 7.3 NP-completeness &

Cook-Levin Theorem 7.4

Page 2: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Review: Turing Machines in a nutshell

Church-Turing Thesis Turing Machine equal Notion of an

Algorithm

Turing Machine Most simple machine possible Computational power of

modern computer and high-level language

Not particularly efficient in a practical sense

Page 3: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Review: Turing Machines in a nutshell

Most simply model possible… Adding another tape can improve

efficiency (time or computational “speed”) but not computational “power”

(ability to solve a problem). Every multi-tape TM has an equivalent single-

tape TM (Theorem 3.13, p.149) Similar to an Automata

Non-determinism does NOT add any“computational power” (Theorem

3.15, p.150)

Page 4: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Review: Turing Machines in a nutshell

Computational power of modern computer and high-level language

Every operation and statement in a high level language can be implemented with a Turing Machine (TM)

Just as statements can be combinedSo can TMs (HW5 illustrates this)

Page 5: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Decidability 4.1

Is there an algorithm that can decide if An item is in a set. A string is in a language A formula is a member of a theory

These are all variations of the same concept, i.e., the concept of decidability

Page 6: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Decidability in Languages

We will concentrate on this: Algorithms for deciding if a string is

in a language But, the strings and languages are going

to represent deeper problems

ADFA = {<B,w> | B is a DFA that accepts input string w}

Page 7: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

ADFA

ADFA = {<B,w> | B is a DFA that accepts input string w}

B is the encoding of a DFA Remember that you can encode a DFA as

follows: B = (Q,Σ, δ, qstart, F)

We are literally encoding the machine and the input (w) as a string “<({1,2,3},{a,b},{(1,2,a),(1,3,b)},1,{3}),abc>”

Page 8: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

ADFA

ADFA = {<B,w> | B is a DFA that accepts input string w}

Testing whether a DFA accepts an input w is the same as the problem of testing whether the string <B,w> is a member of the language ADFA

Just as A = {w | w = (11)*} would accept the set {ε, 11, 1111, 111111, …}

ADFA would enumerate all the <B,w>’s such that w is accepted by the encoded B.

Page 9: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

ADFA is decidable

ADFA = {<B,w> | B is a DFA that accepts input string w}

What does this mean in plain English?

How can we prove it?

Page 10: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

ADFA is decidable

ADFA = {<B,w> | B is a DFA that accepts input string w}

What does this mean in plain English?

“An algorithm exists that can accept strings that adhere to the definition of ADFA and reject string that don’t”

Page 11: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

ADFA is decidable

ADFA = {<B,w> | B is a DFA that accepts input string w}

How can we prove it? Proof is on p.167

Page 12: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

ANFA is decidable

ANFA = {<B,w> | B is a NFA that accepts input string w}

How do we know this to be true? Hint: How are NFAs and DFAs

different?

Page 13: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

AREX is decidable

AREX = {<R,w> | R is a Regular Expression that generates the string w}

How do we know this to be true? Hint: How are Regular Expressions

and DFAs related?

Page 14: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

EDFA is decidable

EDFA = {<A> | A is a DFA and L(A) is empty}

Prove it Hint: Just as Turing Machine can

“simulate” a DFA it can also determine if a state is unreachable.

Page 15: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

EQDFA is decidable

EQDFA = {<A,B> | A and B are DFAs and L(A) = L(B)}

How do we know this to be true? Hint: Symmetric Difference formula

Page 16: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Decidability of Regular Languages Deciding if

a language is Regular or not If given DFA, NFA or REX

a Regular language is empty two Regular languages are equal

Page 17: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Decidability of Context Free

Deciding if a language is Context Free or not

(Theorem 4.7) If given CFG

a Context Free language is empty (Theorem 4.8)

two Context Free languages are equal

Page 18: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Classes of languages

Turing-recognized

Decidable

Context-Free (ACFG)

Regular (ADFA)

Page 19: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

The Halting Problem

Will an algorithm halt on a given input. Intuition:

Can you ever be sure that a loop is infinite? It might just terminate in a few minutes,

hours, years, millenniums, etc. Sometimes you can make such a

determination: while (x > 0) {x=1;} But is it always possible to make such a

determination?

Page 20: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Infinite Looping

DFA: by definition, upon consuming the input, the machine rejects

unless it is in an accept state. Looping is simply not an option by definition.

PDA: very, very hard to make deterministic PDA’, but it can be done. Once the input is consumed, empty transitions can move to a

reject/accept state. Every CF language has a PDA that will halt (not loop).

TM: Just like a high-level language TMs can loop forever. Intuition: you don’t consume the input, you can move on the

tape infinitely, and the states can have a loop with no accept or reject.

Page 21: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

ATM

ATM= {<M,w> | M is a TM and M accepts w} U = “on input <M,w> simulate M on w”

If M accepts, U accepts If M rejects, U rejects

Simple intuition: M could be a Turing Machine that loops

forever on certain input. If M loops forever, U cannot be a decider for

ATM

Page 22: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Is ATM decidable?

ATM= {<M,w> | M is a TM and M accepts w} U = “on input <M,w> simulate M on w”

If M accepts, U accepts If M rejects, U rejects

BUT! Perhaps there is a way to implement M such that we can detect the infinite loop?

Upon infinite loop detection, U rejects. U could still be a decider for ATM

Page 23: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

The Halting Problem

H(<M,w>) = if M accepts w acceptif M rejects w reject

Page 24: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

The Halting Problem is Undecidable Proof: First, consider the machine/algorithm D:

D = “on input <M>, where M is a TM: Run H on input <M,<M>> Output the opposite of what H outputs; that is; if

H accepts, reject and if H rejects, accept.”

Recall H: H(<M,w>) = if M accepts w accept

if M rejects w reject

Page 25: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

D is a crazy Decider Algorithm D is implemented with a Turing Machine D(<M>) = if D does not accept <M>, accept

if D accepts <M>, reject

What happens if we run D with its own Turing Machine description?

D(<D>) = if D does not accept <D>, acceptif D accepts <D>, reject

Page 26: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

A paradox emerges

D(<D>) = if D does not accept <D>, acceptif D accepts <D>, reject

If D accepts, how can D(<D>) reject?

We assumed that H could decide ATM because it could ‘somehow detect an infinite loop” Think of H as a deterministic decider if a Turing Machine

loops Then, we use H to build D (the crazy decider)

Here we assume H can stop D from looping infinitely Then, we run D on its own encoding, which creates a

paradox.

Page 27: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Paradox resolved

Either H or D cannot exist. Which one? D is a TM machine that can simulate another

Turing Machine, which has been elegantly proven.

Intuition: Consider a program that can take another program and simulate its execution. Program, Algorithm, and Turing Machine are all

synonymous (Church-Turing Thesis) Compilers Virtual Machines

Page 28: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Significance of Turing Machines Turing Machines are the “tool” we

used to prove that the Halting Problem is un-decidable. In other words, no algorithm exists to

determine if a general algorithm will halt or not.

Note: There are some algorithms where its easy to show/prove that it will halt, but we are interested in the general case (any/all algorithms).

Page 29: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Un-decidable Languages…

…there are many, but this is the interesting one: ATM= {<M,w> | M is a TM and M accepts w}

Obviously, this language can’t be generated by a REX or CFG. So, a NFA, DFA, and PDA can’t be used as a decider to

accept/reject strings But, even a Turing machine cannot act as a decider.

It may be able to decide some input on some machines, but not all.

There are strings in ATM that will cause the decider to loop infinitely. Specifically <D,<D>> and likely other strings.

Page 30: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Significance of ATM

A formal language that cannot be decided by Turing Machine. We can define this language’s concept But we cannot create an algorithm (TM) to

determine if a string is in this language or not. ATM

Turing-Decidable

Context-Free (ACFG)

Regular (ADFA)

Page 31: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Decidable vs. Recognizable

Turing Decidable Languages

Language such that some TM will accept all of its strings

And, reject strings in the language’s compliment

Halts on all input

Turing Recognizable Languages

Language such that some TM will accept all of its strings

But, might not halt on strings in the language’s compliment

Its it looping infinitely or will it accept? We don’t know.

Page 32: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

ATM is Turing Recognizable

ATM= {<M,w> | M is a TM and M accepts w} U = “on input <M,w> simulate M on w”

If M accepts, U accepts If M rejects, U rejects

U will always halt if M halts. If M doesn’t halt on w than M doesn’t accept w, so <M,w> isn’t in the language. By its very definition U will always halt on strings in ATM. The un-decidability is when U has been looping for 10 million

years, we really don’t know

Is it eventually going to be an accepted w or an infinite loop caused by a rejected w. This is why infinity is

trouble.

Page 33: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Time Complexity

TM’s are a formal way to describe algorithms

Some problems don’t have algorithms that will always halt, i.e., determining if a string is in ATM.

Algorithms that do halt can still take a long time.

How long is long?

Page 34: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

General Time Unit

With Turing Machines we can define a unit of time to be the execution time of one TM transition.

With more practical machines, a time unit could be a CPU clock cycle, which might execute one machine-level instruction.

Some machines can execute 1 billion instructions per second, so the time unit would be 1/100000000 seconds.

Page 35: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Time as a function of input size N is the size of the input f(N) is the number of time unit to solve

the problem. The running time of algorithms can be

expressed as functions: f(N) = 2N + 5;

Two loops of size N and 5 setup instructions Or, on loop of size N with two instruction

inside and 5 instructions outside the loop.

Page 36: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Constants don’t matter

For really big problems, constants don’t matter f(N) = 2N is the same as g(N) = 100N

While 100 days seems like forever compared to 2 days, parallel computation and faster computers can eventually make up the difference (we hope).

For big problems, f(N) = N4 is much different than g(N) = N2

A faster computer may not help, why?

Page 37: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

1.15 Days vs. 11 billion years

N N^2 days N^4 years

1 1 1.15741E-14 1 3.17E-17

2 4 4.62963E-14 16 5.07E-16

3 9 1.04167E-13 81 2.57E-15

4 16 1.85185E-13 256 8.12E-15

10 100 1.15741E-12 10000 3.17E-13

20 400 4.62963E-12 160000 5.07E-12

50 2500 2.89352E-11 6250000 1.98E-10

1000 1000000 1.15741E-08 1E+12 3.17E-05

10000 100000000 1.15741E-06 1E+16 3.17E-01

100000 10000000000 0.000115741 1E+20 3.17E+03

1000000 1E+12 0.011574074 1E+24 3.17E+07

10000000 1E+14 1.157407407 1E+28 3.17E+11

Page 38: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Big-O

Review Constants don’t matter Only the leading exponent matters Why?

Page 39: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

1.157407 days vs. 1.157419 days

N N^2 days N^2 + 100N days

1 1 1.15741E-14 101 1.16898E-12

2 4 4.62963E-14 204 2.36111E-12

3 9 1.04167E-13 309 3.57639E-12

4 16 1.85185E-13 416 4.81481E-12

10 100 1.15741E-12 1100 1.27315E-11

20 400 4.62963E-12 2400 2.77778E-11

50 2500 2.89352E-11 7500 8.68056E-11

1000 1000000 1.15741E-08 1100000 1.27315E-08

10000 100000000 1.15741E-06 101000000 1.16898E-06

100000 10000000000 0.000115741 10010000000 0.000115856

1000000 1E+12 0.011574074 1.0001E+12 0.011575231

10000000 1E+14 1.157407407 1.00001E+14 1.157418981

Page 40: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Why do we only care about big N’s

Same reason I would worry about a $10,000 bill in my wallet but not a penny.

Same reason I would worry about a trip to Mars but not a trip to Menands.

Page 41: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Real Algorithm TM Decider

Prepare for “hand-waving magic:” Any algorithm that can be programmed

can be reduced into a language problem. A = {<p,i,o> | p is the

encoding/description of a problem, i is the input, and o is the correct output.}

Deciding if a string is in L is the same thing as solving the problem.

The TM that decides A solves problem p.

Page 42: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

The class P

The class of languages that can be decided in polynomial time.

Corresponds, the set of problems that can be solved in polynomial time.

Polynomial is O(nk) What are some problem in P that you

have studied?

Page 43: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

Did you know?

Every context free language is in P

Page 44: Remaining Topics  Decidability Concept 4.1  The Halting Problem 4.2  P vs. NP 7.2 and 7.3  NP-completeness & Cook-Levin Theorem 7.4

The class NP

Non-deterministically Polynomial. One way to think of this is NOT

Polynomial. Or, exponential Or N! But that is not the whole story.