Embed Size (px)
Citation preview
Theory of Computation
Trivial & Non-Trivial Program Properties, The Halting Problem,
Index Sets, Rice's TheoremVladimir Kulyukin
www.vkedco.blogspot.com
Outline
● STP Theorem● Trivial and Non-Trivial Program Properties● Halting Problem● Example of Self-Reference Argument● Index Sets● Statement of Rice's Theorem
STP Theorem
Theorem 3.2 (Ch. 4): STP Theorem
( )recursive. primitive is
,,,..., predicate the,0For 1)( tyxxSTPn n
n>
The Big Picture
GÖDEL Coder
GÖDEL Decoder
L Program Number Universal Program
Number
???
GÖDEL Coder = CompilerGÖDEL Decoder = Reverse CompilerUniversal Program = Operating System/VM/Interpreter
Universality Theorem
Simulator
Step-Counter Theorem
Interesting Implications
• We do not have to be tied to a specific universal program (operating system) to run our programs– Example: We do not need the L operating system to run L
programs. We can implement an L simulator on Linux in C and run L programs
• We can simulate one universal program (operating system) on another universal program (operating system)– Example: Virtual Box is can simulate Linux on Windows
Trivial & Non-Trivial Program Properties&
The Halting Problem
Trivial & Non-Trivial Program Properties&
The Halting Problem
Program Properties
● Can we determine non-trivial properties of programs?
● Examples:– Does a program halt on a given input?– Does a program compute a specific
function?– Does a program plagiarize another program?
The Halting Problem
.number input on
halts number program),(
x
yyxHALT ⇔
The Halting Problem
( )
( )( )
( )( )
↑
↓=
↑↓
=
=
xΨ
xΨyxHALT
yxΦ
yxΦyxHALT
yPP
P
P
)1(
)1(
if 0
if 1),(
, if 0
, if 1),(
.# such that program a be Let
Theorem 2.1 (Ch. 4)
computablenot is ),( yxHALT
Proof
● Suppose that HALT(x,y) is computable● Consider the following L program P:
[A1] IF HALT(X1,X1) GOTO A1
Y Y + 1←
Proof: What Does P Do?
● P is a valid L program● It considers X1 to be a program number, i.e., a Gödel
number● If X1 halts on its own Gödel number, P goes into an
infinite loop, i.e., its output is undefined● If X1 does not halt on its own Gödel number, P goes to
line 2 and sets Y to 1
Proof
( )
( )
( )
¬↑
=
==↑
=
),( if 1
),( if
:
0),( if 1
1),( if
? isWhat
)1(
)1(
)1(
xxHALT
xxHALTxΨ
Or
xxHALT
xxHALTxΨ
xΨ
P
P
P
Proof
( )( )
( ) ),( iff ),(
:
0),( if 1),( .2
1),( if 0),( 1.
Then .#Let
.# computecan weprogram, L valida is Since
0
0
0
0
xxHALTyxHALTx
Or
xxHALTyxHALT
xxHALTyxHALT
yP
PP
¬∀
====
=
Proof
( )
).,( iff ),(
Then .Let
).,( iff ),( that know We
?),( isWhat
0000
0
0
00
yyHALTyyHALT
yx
xxHALTyxHALTx
yyHALT
¬=
¬∀
Important Implication
.on halt
eventually willnot or whether determinecan
, input an and language)other any (or in
program agiven that,algorithm no is There
x
P
xLP
Example of the Self-Reference Argument
● Suppose that some security expert comes to you and says that he/she has developed a program that detects all viruses of type Z
● Let us assume, for the sake of simplicity, that Z viruses are programs of one argument (e.g., a file path)
● Suppose the expert's program implements the predicate isVirusZ(x,y), where y is the compiled source code of a suspect program and x is an input to y (e.g. x is a file path)
● Question: Is isVirusZ(x,y) really computable?
Example of the Self-Reference Argument
● Suppose that isVirus(X1, X2) is computable● Suppose that we have implemented ApplyVirusZ(X1)
that applies virus Z to X1● Consider this program P of two arguments X1 and X2:
Y 0←
IF isVirusZ(X1, X2) GOTO A1
ApplyVirusZ(X1) GOTO E
[A1] Y Y + 1←
Example of the Self-Reference Argument
( ) ( ) ( )( ) ( ) ( )
( )( ) ( ) ( )
( ) 1, thus Z,Virusapply does
and 0, then ,0, If
0, thus Z,Virusapply not does
and 1, then ,1, If
?, is What . Suppose
00
00002
P
00
00002
P
002
P0
===
===
=
yyisVirusZP
yyisVirusZyyΨ
yyisVirusZP
yyisVirusZyyΨ
yyΨyP#
Index Sets
Definition: Index Set
{ }.|
variable.one of functions computablepartially of collection a is
Γ∈Φ∈=Γ
Γ tNtR
Examples● Example 1
– Γ = primitive recursive functions of one variable– RΓ = L-program numbers that compute primitive recursive functions of one
variable● Example 2
– Γ = functions that map a specific program to a plagiarized version of that program
– RΓ = L-program numbers that compute these plagiarizing functions of one variable
● Example 3– Γ = sorting functions of one variable– RΓ = L-program numbers that compute sorting functions of one variable
Rice's Theorem
Properties of Programs
.in function a
computes program particular a on whether
depending FALSEor TRUE returns and
input as programs accepts that algorithman is
thereifonly and if decidable is in Membership
Γ
ΓR
Rice’s Theorem
● The theorem is named after Henry Gordon Rice, a U.S. mathematician
● Rice’s Theorem (Informal Statement): For any non-trivial property of partially computable functions, there is no general and effective method to decide whether an algorithm computes a partially computable function with that property
● A property is trivial if it holds for all partially computable functions or for none
Rice’s Theorem
( ) ( )
( ) ( ) recursive.not is Then . ,
such that functions computablepartially
be ,Let variable.one of functions
computablepartially of collection a be Let
ΓΓ∉Γ∈
Γ
Rxgxf
xgxf
Rice’s Theorem: Original Formulation
{ }
variable.one of
functions computablepartially all includes
or ifonly and if recursive
is |Then variable.one of
functions p.c. of collection a be Let
CC
Cx
C
x
∅=∈Φ
Proof Sketch
● Suppose we have Γ, a non-trivial collection of partially computable functions
● Suppose that there exists an algorithm (an L-program) that decides if another algorithm computes a function in Γ. To put it differently, Γ has a computable characteristic function
● Let us call this algorithm isInGamma(x), where x is a Gödel number, i.e. a compiled L-program
● We now proceed to show that we can use this algorithm to solve the Halting Problem
Proof Sketch: Reduction
● We will use the so-called reduction argument● Suppose that we have a problem A that we know to be
undecidable, e.g. the Halting Problem● Suppose that we are given another problem B ● If we want to show that B is undecidable, we can assume that B
is decidable and then conclude that the decidability of B leads to the decidability of A
● This is a contradiction, of course, because we know that A is undecidable
● In this case, A is the Halting Problem and B is Rice’s theorem
Proof Sketch
● Since Γ is non-trivial, let f(x) be a partially computable function in Γ and let F be a corresponding L-program
● Since Γ is non-trivial, let g(x) be a partially computable function not in Γ and let G be a corresponding L-program
● For any two values of z and w, we can construct the following program P:
IF HALT(z,w) GOTO A1 Y G(X1)←
GOTO E[A1] Y F(X1)←
Proof Sketch● Since isInGamma(x) is an algorithm that computes a
characteristic function of Γ, it must return 1 or 0 for any natural number x
● Since P is a valid L program, it has a unique Gödel number #(P)
● Let x = #(P). Then isInGamma(x) = 1 if and only if HALT(z, w) = 1
● Since we can do this construction for any numbers of z and w, the existence of isInGamma shows that the Halting Problem is computable (or decidable), which is a contradiction
Brief & Gentle Introduction to
Turing Machines
Hilbert’s Lecture
David HilbertMathematical Problems
Lecture at the International Congress of Mathematicians, Paris 1900
Hilbert’s 10th Problem
Given a Diophantine equation with any number of unknown quantities and with rational integral numerical coefficients: To devise a process according to which it can be determined by a finite number of operations whether the equation is solvable in rational integers.
Diophantine Problem
A Diophantine equation is an equation in which only integers solutions are allowed
ax + by = c, where a, b, c are integers
Turing Machines
● Alan Turing discovered this device in 1936-37● A Turing Machine (TM) is a device with internal states● At any particular instant, the device is scanning one
square on a linear tape which is infinite in both directions
● The combination of the current symbol and the state determines what the device does next
Turing Machines
● A TM is a theoretical symbol manipulating device● A TM can simulate any computer algorithm (this is a
simple formulation of what came to be known as the Church-Turing thesis, a version of Church's thesis)
● TMs are useful for simulating and understanding how computer CPUs work
Alan Turing's Description
...an infinite memory capacity obtained in the form of an infinite tape marked out into squares, on each of which a symbol could be printed. At any moment there is one symbol in the machine; it is called the scanned symbol. The machine can alter the scanned symbol and its behavior is in part determined by that symbol, but the symbols on the tape elsewhere do not affect the behavior of the machine. However, the tape can be moved back and forth through the machine, this being one of the elementary operations of the machine. Any symbol on the tape may therefore eventually have an innings.
Alan Turing, “Intelligent Machinery,” 1948
Turing Machines
● The actions available to the device are:– Print a symbol– Move right– Move left– Enter a new state
Turing Machines: Notation
blank. i.e. ,
symbols. printablerepresent ,...,, Symbols
states.represent ...,, Symbols
0
210
321
Bs
sss
qqq
=
Turing Machines: A Perspective
● A Turing Machine (TM) is another formalism that captures the intuitive notion of computation
● We will confine ourselves with the formal definition of TM and a brief review of several TM theorems
● The most important results are 1) Any partially computable function can be computed by a TM and 2) The Universality Theorem can be proved with Turing Machines (there exists a Universal Turing Machine)
Turing Machines: Alphabet
{ }
alphabet. theofpart
not is that , symbol,blank special a is There
.,...,, TM a ofalphabet The
0
21
Bs
sss m
==
Action Quadruples
. into go andleft square one move
,s scanning and statein When
s .3
. into go andright square one move
,s scanning and statein When
s .2
. into go and print
,s scanning and statein When
s 1.
l
ji
lji
l
ji
lji
lk
ji
lkji
q
q
qLq
q
q
qRq
qs
q
qsq
Turing Machine: Definition
A Turing Machine is a finite set of action quadruples
Deterministic Turing Machine
A deterministic Turing Machine is a
finite set of quadruples no two of which
begin with the same pair qisj
Non-deterministic Turing Machine
A non-deterministic Turing Machine is a
finite set of quadruples some of which
may begin with the same pair qisj
Turing Machines: Computation
tape.on theleft
symbolsblank -non theis TM a ofoutput The .3
. start with that quadruples no are thereand
scanning statein isit when halts TMA .2
. statein starts always TMA 1. 1
ji
ji
sq
sq
q
Example
13
33
32
22
21
1 .5
1 .4
1 .3
1 .2
.1
:TM following heConsider t
qBq
Rqq
qBq
Rqq
BRqq
Example: Input 1
13
33
32
22
21
1 .5
1 .4
1 .3
1 .2
.1
qBq
Rqq
qBq
Rqq
BRqqB 1 B B B B
q1
MOVE 1: Action quadruple 1 applies. The TM moves rights and goes into q2.
Example: Input 1
13
33
32
22
21
1 .5
1 .4
1 .3
1 .2
.1
qBq
Rqq
qBq
Rqq
BRqqB 1 B B B B
q2
MOVE 2: Action quadruple 2 applies. The TM moves rights and stays in q2.
Example: Input 1
13
33
32
22
21
1 .5
1 .4
1 .3
1 .2
.1
qBq
Rqq
qBq
Rqq
BRqqB 1 B B B B
q2
MOVE 3: Action quadruple 3 applies. The TM prints 1 over B and goes into q3.
Example: Input 1
13
33
32
22
21
1 .5
1 .4
1 .3
1 .2
.1
qBq
Rqq
qBq
Rqq
BRqqB 1 1 B B B
q3
MOVE 4: Action quadruple 4 applies. The TM moves right and stays in q3.
Example: Input 1
13
33
32
22
21
1 .5
1 .4
1 .3
1 .2
.1
qBq
Rqq
qBq
Rqq
BRqqB 1 1 B B B
q3
MOVE 5: Action quadruple 5 applies. The TM prints 1 over B and goes into q1.
Example: Input 1
13
33
32
22
21
1 .5
1 .4
1 .3
1 .2
.1
qBq
Rqq
qBq
Rqq
BRqqB 1 1 1 B B
q1
No action quadruple applies so the TM halts with theoutput 111.
Example: Input 2
13
33
32
22
21
1 .5
1 .4
1 .3
1 .2
.1
qBq
Rqq
qBq
Rqq
BRqqB 1 1 B B B
q1
MOVE 1: Action quadruple 1 applies. The TM moves rights and goes into q2.
Example: Input 2
13
33
32
22
21
1 .5
1 .4
1 .3
1 .2
.1
qBq
Rqq
qBq
Rqq
BRqqB 1 1 B B B
q2
MOVE 2: Action quadruple 2 applies. The TM moves rights and stays in q2.
Example: Input 2
13
33
32
22
21
1 .5
1 .4
1 .3
1 .2
.1
qBq
Rqq
qBq
Rqq
BRqqB 1 1 B B B
q2
MOVE 3: Action quadruple 2 applies. The TM moves rights and stays in q2.
Example: Input 2
13
33
32
22
21
1 .5
1 .4
1 .3
1 .2
.1
qBq
Rqq
qBq
Rqq
BRqqB 1 1 B B B
q2
MOVE 4: Action quadruple 3 applies. The TM prints 1 over B and goes in q3.
Example: Input 2
13
33
32
22
21
1 .5
1 .4
1 .3
1 .2
.1
qBq
Rqq
qBq
Rqq
BRqqB 1 1 1 B B
q3
MOVE 5: Action quadruple 4 applies. The TM moves right and stays in q3.
Example: Input 2
13
33
32
22
21
1 .5
1 .4
1 .3
1 .2
.1
qBq
Rqq
qBq
Rqq
BRqqB 1 1 1 B B
q3
MOVE 6: Action quadruple 5 applies. The TM prints 1 and goes into q1.
Example: Input 2
13
33
32
22
21
1 .5
1 .4
1 .3
1 .2
.1
qBq
Rqq
qBq
Rqq
BRqqB 1 1 1 1 B
q1
NO MOVE: No action quadruple is applicable. The TM halts with the output 1111.
Question
What does the TM in the previous example compute?
Answer
f(x) = x + 2
Two Fundamental Theoremson Turing Machines
Theorem 1
. computes that TM a is there
Then .alphabet somefor ,on function
computablepartially ary -man be Let *
f
AA
f
Theorem 2
There exists a universal TM, i.e., a TM that takes a code (description) of an arbitrary TM and its input and proceeds to execute that TM on that input.
Theorem 3: Proof Outline
( )
( )
( )
( ). computes that TM a ofn descriptio thecalled is context, TM In the
:)on s terminateprogram the(if tapeon the of ueoutput val the
leaves andion configurat following in the starts .it call number, a havemust
programThat it. computes that in program a is therefunction, partial a is if
Thus, . and 1 symbolsonly usingit computes that TM a is thereTheorem),
ity Universalin thein proved (wefunction computable partial a is , Since
0
1
0
0
xgz
q
BxBz
xxg
Mz
Lxg
BM
zxΦ
↑
Reading Suggestions
● Ch. 4, Computability, Complexity, and Languages, 2nd Edition, by Davis, Weyuker, Sigal
