Theory of Computation CS3102 Spring 2014 - Computer …njb2b/theory/Theory_lecture20_web.pdf ·...

Nathan Brunelle

Department of

Computer Science

University of Virginia

www.cs.virginia.edu/~njb2b/theory

Theory of Computation CS3102 – Spring 2014

A tale of computers, math, problem solving, life, love and tragic death

Rice’s Theorem

• Any nontrivial property about the language of a Turing machine is undecidable.

• A property is trivial if it either applies to ALL languages of Turing Machines, or NONE of them.

Henry Gordon Rice, 1951

Machine R: 1) Build (but don’t run!) machine 𝑀’

2) Return B( 𝑀′ )

M

w

Machine 𝑀′: 1) Simulate M(w)

2) Check if x is a TM with property P

x

Let ∅ have property P, otherwise replace P with 𝑃

Which of these are Undecidable?

• Does TM M accept any strings?

• Does TM M accept “Hello”?

• Does TM M take more than 1000 steps to process input w?

• Does TM M accept a finite number of strings?

• Does TM M accept a Decidable Language?

• Does TM M accept a Recognizable Language?

Decidable

Undecidable

Undecidable

Undecidable

Undecidable

Decidable

Warning: Rice’s theorem applies to properties (i.e., sets of languages), not (directly to) TM’s or other object types!

Undecidability + Rice + Church-Turing

Undecidability: undecidable languages that cannot be decided by any Turing Machine

Rice’s Theorem: all nontrivial properties about the language of a TM are undecidable

Church-Turing Thesis: any mechanical computation can be done by some TM

Conclusion: any nontrivial property about general mechanical computations cannot be decided!

Language and Problems: any problem can be restated as a language recognition problem

Some “useful” undecidable Problems

• Malware detection • Virus Scanning • Bug detection • Program equivalence • Type checking • Solutions to polynomials • N-body problems • Fluid dynamics • Compressibility • Enscheidungsproblem

Machine R: 1) Build (but don’t run!) machine 𝑀’

2) Return Malware( 𝑀′ )

M

w

Machine 𝑀′: 1) Simulate M(w) “safely”

2) Infect machine!

x

Recognizing vs. Deciding

Turing-decidable: A language L is “decidable” if there exists a TM M such that for all strings w: If w L, M enters qaccept. If w L, M enters qreject. Turing-recognizable: A language L is “Turing-recognizable” if there exists a TM M such that for all strings w: If w L eventually M enters qaccept If w L either M enters qreject or M never terminates

“Computable”

“Non-Computable”

Recognizable Languages

Revised Hierarchy

Decidable Languages

Regular Languages

finite Languages

Deterministic CFLs

Example of a Recognizable Language

The Halting Problem, or ATM

Algorithm: Universal Turing Machine.

Universal Turing

Machine

M

w

Output Tape for running TM-M

starting on tape w

Closure Properties of Recognizable Languages

Theorem: The recognizable languages are closed under .

1. Copy input

2. Run machine 1 on copy of input (never cross $)

1. If reject then move into reject state

3. Erase everything to right of $

4. Run machine 2

a b b a $ a b b a

input copy

M1 M2

Closure Properties of Recognizable Languages

Theorem: The recognizable languages are closed under union.

1. Copy input

2. Run machine 1 on copy of input (never cross $)

1. If accept then move into accept state

3. Erase everything to right of $

4. Run machine 2

Problem: What if the machine runs forever?

Closure Properties of Recognizable Languages

Theorem: The recognizable languages are closed under union.

1. Copy input to second tape

2. Run machine 1 on tape 1, machine 2 on tape 2 “in parallel”, “time-multiplexed”

1. If either accept then move into accept state

M1 M1 M1

M2 M2 M2 Time

Closure Properties of Recognizable Languages

Theorem: The recognizable languages are closed under concat.

Using non-determinism:

Tape 1: input

Tape 2: tape for Machine 1

Tape 3: tape for Machine 2

Copy some characters from the first tape to tape 2

Nondeterminstically decide when to copy the rest onto tape 3

a b b a

a b

b a

“choose” Non-deterministically!

Closure Properties of Recognizable Languages

Theorem: The recognizable languages are closed under Kleene star.

Similar to concatenation, but now we also try all possible numbers of

concatenations.

Non-deterministically insert $’s into the input. The machine must accept each of

these substrings.

a $ b b $ a

Inserted Non-deterministically!

Each substring then used as input to the machine

Closure Properties of Recognizable Languages

Theorem: The recognizable languages are not closed under complement.

Hint: reduction from halting problem.

This one takes some new tools

Co-Recognizable Languages

Turing-recognizable: A language L is “Turing-recognizable” if there exists a TM M such that for all strings w: If w L eventually M enters qaccept If w L either M enters qreject or M never terminates

Co-Turing-recognizable: A language L is “Co-Turing-recognizable” if there exists a TM M such that for all strings w: If w L eventually M enters qaccept If w L either M enters qreject or M never terminates

A language 𝐿 is Co-Recognizable if 𝐿 is Recognizable.

Closure Properties of Recognizable Languages

Theorem: If a language is both Recognizable and Co-Recognizable then it is

decidable.

Proof: Let M1 be a recognizer for L, M2 be a Co-Recognizer for L.

We build MD, a Decider for L:

Simulate M1 and M2 “in parallel” (time multiplexed)on input w.

If M1 accepts then accept, else if M2 accepts, then reject.

Every string will be accepted by exactly one machine in finite time.

M1 M1 M1

M2 M2 M2 Time

Closure Properties of Recognizable Languages

Theorem: If a language is both Recognizable and Co-Recognizable then it is

decidable.

Corollary 1: 𝐴𝑇𝑀 is unrecognizable since 𝐴𝑇𝑀 is recognizable and undecidable.

Corollary 2: The recognizable languages are not closed under complement.

Oracles

• Originated in Turing’s Ph.D. thesis

• Named after the “Oracle of Apollo”

at Delphi, ancient Greece

• Black-box subroutine / language

• Can compute arbitrary functions

• Instant computations “for free”

• Can greatly increase computation power of basic TMs

E.g., oracle for halting problem

The “Oracle of Omaha”

The “Oracle” of the Matrix

• A special case of “hyper-computation”

• Allows “what if” analysis: assumes certain undecidable languages can be recognized

• An oracle can profoundly impact the

decidability & tractability of a language

• Any language / problem can be

“relativized” WRT an arbitrary oracle

• Undecidability / intractability exists even

for oracle machines!

Turing Machines with Oracles

Theorem [Turing]: Some problems are still not computable, even by Turing machines with an oracle for the halting problem!

Reaching a Contradiction

What happens if we run D on its own description, <D>?

Define D (<M>) = Construct a TM that: Outputs the opposite of the result of simulating H on input (M, <M>)

Assume there exists some TM H that decides ATM.

If M accepts its own description <M>, D(<M>) rejects. If M rejects its own description <M>, D(<M>) accepts.

If D accepts its own description <D>, D(<D>) rejects. If D rejects its own description <D>, D(<D>) accepts. substituting

D for M…

M* is Relativized to O H* is Relativized to O D* is Relativized to O * *

* *

* *

* * *

* *

* *

* * *

* * *

*

*

Ø

• Turing (1937); studied by Post (1944) and Kleene (1954)

• Quantifies the non-computability (i.e., algorithmic unsolvability) of (decision) problems and languages

• Some problems are “more unsolvable” than others!

Turing Degrees

Emil Post 1897-1954

Alan Turing 1912-1954

Stephen Kleene 1909-1994

Students of Alonzo Church:

H H

H*

Turing degree 0 Turing degree 1 Turing degree 2

• Defines computation “relative” to an oracle.

• “Relativized computation” - an infinite hierarchy!

• A “relativity theory of computation”!

Georg Cantor 1845-1918

• Turing degree of a set X is the set of all Turing-equivalent

(i.e., mutually-reducible) sets: an equivalence class [X]

• Turing degrees form a partial order / join-semilattice

• [0]: the unique Turing degree containing all computable sets

• For set X, the “Turing jump” operator X’ is the set of indices of oracle TMs which halt when using X as an oracle

• [0’]: Turing degree of the halting problem H; [0’’]: Turing degree of the halting problem H* for TMs with oracle H.

Turing Degrees

Emil Post 1897-1954

Alan Turing 1912-1954

Stephen Kleene 1909-1994

Students of Alonzo Church:

Turing jump

Turing jump

Turing Degrees

Emil Post 1897-1954

Alan Turing 1912-1954

Stephen Kleene 1909-1994

Students of Alonzo Church:

Turing jump

Turing jump

• Each Turing degree is countably infinite (has exactly 0 sets)

• There are uncountably many (20) Turing degrees

• A Turing degree X is strictly smaller than its Turing jump X’

• For a Turing degree X, the set of degrees smaller than X is countable; set of degrees larger than X is uncountable (20)

• For every Turing degree X there is an incomparable degree

(i.e., neither X Y nor Y X holds).

• There are 20 pairwise incomparable Turing degrees

• For every degree X, there is a degree D strictly between X and X’ so that X < D < X’ (there are actually 0 of them)

The structure of the Turing degrees semilattice is extremely complex!

PSP

AC

E-co

mp

lete

QB

F

The Extended Chomsky Hierarchy

Finite {a,b} Regular a* Det. CF anbn

Context-free wwR P anbncn

NP

PSPACE

EXPSPACE

Rec

ogn

izab

le

No

t R

eco

gniz

able

H H

Decidable Presburger arithmetic

NP

-co

mp

lete

SAT

No

t fi

nit

ely

des

crib

able

?

2S*

EXPTIME

EXP

TIM

E-co

mp

lete

Go

EXP

SPA

CE-

com

ple

te =

RE

Turing Context sensitive LBA

H*

degrees