Upload
quinto
View
63
Download
0
Embed Size (px)
DESCRIPTION
Theory of Computation 計算理論. Instructor: 顏嗣鈞 E-mail: [email protected] Web: http://www.ee.ntu.edu.tw/~yen Time: 2:20-5:10 PM, Tuesday Place: BL 112 Office hours: by appointment Class web page: http://www.ee.ntu.edu.tw/~yen/courses/TOC-2006.htm. textbook. : - PowerPoint PPT Presentation
Citation preview
1
Theory of Computation
計算理論
2
Instructor: 顏嗣鈞E-mail: [email protected]
Web: http://www.ee.ntu.edu.tw/~yen
Time: 2:20-5:10 PM, Tuesday
Place: BL 112
Office hours: by appointment
Class web page: http://www.ee.ntu.edu.tw/~yen/courses/TOC-2006.htm
3
• :
Introduction to Automata Theory, Languages, and Computation
John E. Hopcroft, Rajeev Motwani,
Jeffrey D. Ullman,
(2nd Ed. Addison-Wesley, 2001)
textbook
4
1st Edition
Introduction to Automata Theory, Languages, and Computation John E. Hopcroft, Jeffrey D. Ullman,
(Addison-Wesley, 1979)
5
Grading HW : 0-20%
Midterm exam.: 35-45%
Final exam.: 45-55%
6
Why Study Automata Theory?
Finite automata are a useful model for important kinds of hardware and software:
• Software for designing and checking digital circuits.
• Lexical analyzer of compilers. • Finding words and patterns in large bodies of
text, e.g. in web pages.• Verification of systems with finite number of
states, e.g. communication protocols.
7
Why Study Automata Theory? (2)The study of Finite Automata and Formal
Languages are intimately connected. Methods for specifying formal languages are very important in many areas of CS, e.g.:
• Context Free Grammars are very useful when designing software that processes data with recursive structure, like the parser in a compiler.
• Regular Expressions are very useful for specifying lexical aspects of programming languages and search patterns.
8
Why Study Automata Theory? (3)
Automata are essential for the study of the limits of computation. Two issues:
• What can a computer do at all? (Decidability)
• What can a computer do efficiently? (Intractability)
9
Applications
Theoretical Computer ScienceAutomata Theory, Formal Languages,
Computability, Complexity …
Compile
r
Prog. languages
Comm
. protocols
circuits
Pattern recognition
Supervisory control
Quantum
computing
Computer-
Aided Verification ...
10
Aims of the Course• To familiarize you with key Computer Science
concepts in central areas like- Automata Theory- Formal Languages- Models of Computation- Complexity Theory
• To equip you with tools with wide applicability in the fields of CS and EE, e.g. for- Complier Construction- Text Processing- XML
11
Fundamental Theme
• What are the capabilities and limitations of computers and computer programs?– What can we do with
computers/programs?– Are there things we cannot do with
computers/programs?
12
Studying the Theme
• How do we prove something CAN be done by SOME program?
• How do we prove something CANNOT be done by ANY program?
13
Example: The Halting Problem (1)
Consider the following program. Does it terminate for all values of n 1?
while (n > 1) {if even(n) {
n = n / 2;} else {
n = n * 3 + 1;}
}
14
Example: The Halting Problem (2)Not as easy to answer as it might first seem.Say we start with n = 7, for example:
7, 22, 11, 34, 17, 52, 26, 13, 40, 20, 10, 5,16, 8, 4, 2, 1
In fact, for all numbers that have been tried(a lot!), it does terminate . . .
. . . but in general?
15
Example: The Halting Problem (3)Then the following important undecidability
result should perhaps not come as a total surprise:
It is impossible to write a program that decides if another, arbitrary, program terminates (halts) or not.
What might be surprising is that it is possible to prove such a result. This was first done by the British mathematician Alan Turing.
16
Our focus
Automata Computability
Complexity
17
Topics1. Finite automata, Regular languages, Regular
grammars: deterministic vs. nondeterministic, one-way vs. two-way finite automata, minimization, pumping lemma for regular sets, closure properties.
2. Pushdown automata, Context-free languages, Context-free grammars: deterministic vs. nondeterministic, one-way vs. two-way PDAs, reversal bounded PDAs, linear grammars, counter machines, pumping lemma for CFLs, Chomsky normal form, Greibach normal form, closure properties.
3.
18
Topics (cont’d)3. Linear bounded automata, Context-
sensitive languages, Context-sensitive grammars.
4. Turing machines, Recursively enumerable sets, Type 0 grammars: variants of Turing machines, halting problem, undecidability, Post correspondence problem, valid and invalid computations of TMs.
19
Topics (cont’d)5. Basic recursive function theory 6. Basic complexity theory: Various
resource bounded complexity classes, including NLOGSPACE, P, NP, PSPACE, EXPTIME, and many more. reducibility, completeness.
7. Advanced topics: Tree Automata, quantum automata, probabilistic automata, interactive proof systems, oracle computations, cryptography.
20
Who should take this course?
YOU
21
Languages
The terms language and word are used in a strict technical sense in this course:
A language is a set of words. A word is a sequence (or string) of
symbols. (or ) denotes the empty word, the
sequence of zero symbols.
22
Symbols and Alphabets• What is a symbol, then?• Anything, but it has to come from an
alphabet which is a finite set.• A common (and important) instance is
= {0, 1}. , the empty word, is never an symbol
of an alphabet.
23
Computation
CPU memory
24
CPUinput memory
output memory
Program memory
temporary memory
25
CPUinput memory
output memoryProgram memory
temporary memory
3)( xxf
compute xx
compute xx 2
Example:
26
CPU
input memory
output memoryProgram memory
temporary memory
3)( xxf
compute xx
compute xx 2
2x
27
CPU
input memory
output memoryProgram memory
temporary memory 3)( xxf
compute xx
compute xx 2
2x
42*2 z82*)( zxf
28
CPU
input memory
output memoryProgram memory
temporary memory 3)( xxf
compute xx
compute xx 2
2x
42*2 z82*)( zxf
8)( xf
29
Automaton
CPUinput memory
output memory
Program memory
temporary memory
Automaton
30
Different Kinds of AutomataAutomata are distinguished by the temporary memory
• Finite Automata: no temporary memory
• Pushdown Automata: stack
• Turing Machines: random access memory
31
input memory
output memory
temporary memory
FiniteAutomaton
Finite Automaton
Example: Vending Machines (small computing power)
32
input memory
output memory
Stack
PushdownAutomaton
Pushdown Automaton
Example: Compilers for Programming Languages (medium computing power)
Push, Pop
33
input memory
output memory
Random Access Memory
TuringMachine
Turing Machine
Examples: Any Algorithm (highest computing power)
34
Finite Automata
PushdownAutomata
TuringMachine
Power of Automata
Less power More powerSolve more computational problems
35
Mathematical Preliminaries
36
Mathematical Preliminaries
• Sets • Functions• Relations• Graphs• Proof Techniques
37
}3,2,1{AA set is a collection of elements
SETS
},,,{ airplanebicyclebustrainB
We writeA1
Bship
38
Set Representations
C = { a, b, c, d, e, f, g, h, i, j, k }
C = { a, b, …, k }
S = { 2, 4, 6, … }
S = { j : j > 0, and j = 2k for some k>0 }
S = { j : j is nonnegative and even }
finite set
infinite set
39
A = { 1, 2, 3, 4, 5 }
Universal Set: all possible elements U = { 1 , … , 10 }
1 2 34 5
AU
67
8910
40
Set OperationsA = { 1, 2, 3 } B = { 2, 3, 4, 5}
• Union A U B = { 1, 2, 3, 4, 5 } • Intersection
A B = { 2, 3 }• Difference
A - B = { 1 }B - A = { 4, 5 }
U
A B
A-B
41
A
• ComplementUniversal set = {1, …, 7} A = { 1, 2, 3 } A = { 4, 5, 6, 7}
1 2 34
56
7A
A = A
42
024
61
35
7even
{ even integers } = { odd integers }
odd
Integers
43
DeMorgan’s Laws
A U B = A BU
A B = A U BU
44
Empty, Null Set:= { }
S U = S
S =
S - = S
- S =
U = Universal Set
45
SubsetA = { 1, 2, 3} B = { 1, 2, 3, 4, 5 }
A B
U
Proper Subset: A B
UAB
46
Disjoint SetsA = { 1, 2, 3 } B = { 5, 6}
A B = UA B
47
Set Cardinality• For finite setsA = { 2, 5, 7 }
|A| = 3
48
PowersetsA powerset is a set of sets
Powerset of S = the set of all the subsets of S
S = { a, b, c }
2S = { , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} }
Observation: | 2S | = 2|S| ( 8 = 23 )
49
Cartesian ProductA = { 2, 4 } B = { 2, 3, 5 }
A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) }
|A X B| = |A| |B|
Generalizes to more than two sets
A X B X … X Z
50
FUNCTIONSdomain
123
ab
c
range
f : A -> B
A B
If A = domain then f is a total function otherwise f is a partial function
f(1) = a
51
RELATIONS R = {(x1, y1), (x2, y2), (x3, y3), …}
xi R yi
e. g. if R = ‘>’: 2 > 1, 3 > 2, 3 > 1
In relations xi can be repeated
52
Equivalence Relations• Reflexive: x R x• Symmetric: x R y y R x• Transitive: x R y and y R z x R z
Example: R = ‘=‘ • x = x• x = y y = x• x = y and y = z x = z
53
Equivalence ClassesFor equivalence relation R equivalence class of x = {y : x R y}
Example: R = { (1, 1), (2, 2), (1, 2), (2, 1), (3, 3), (4, 4), (3, 4), (4, 3) }
Equivalence class of 1 = {1, 2}Equivalence class of 3 = {3, 4}
54
GRAPHSA directed graph
• Nodes (Vertices) V = { a, b, c, d, e }• Edges E = { (a,b), (b,c), (b,e),(c,a), (c,e), (d,c), (e,b), (e,d) }
node
edge
ab
c
d
e
55
Labeled Graph
ab
c
d
e
1 35 6
262
56
Walk
ab
c
d
e
Walk is a sequence of adjacent edges (e, d), (d, c), (c, a)
57
Path
ab
c
d
e
Path is a walk where no edge is repeated
Simple path: no node is repeated
58
Cycle
ab
c
d
e
123
Cycle: a walk from a node (base) to itself
Simple cycle: only the base node is repeated
base
59
Euler Tour
ab
c
d
e 123
4 5 678 base
A cycle that contains each edge once
60
Hamiltonian Cycle
ab
c
d
e 123
45 base
A simple cycle that contains all nodes
61
Treesroot
leaf
parent
child
Trees have no cycles
62
root
leaf
Level 0
Level 1
Level 2
Level 3
Height 3
63
Binary Trees
64
PROOF TECHNIQUES
• Proof by induction
• Proof by contradiction
65
Induction
We have statements P1, P2, P3, …
If we know• for some b that P1, P2, …, Pb are true• for any k >= b that
P1, P2, …, Pk imply Pk+1
Then Every Pi is true
66
Proof by Induction• Inductive basis
Find P1, P2, …, Pb which are true
• Inductive hypothesisLet’s assume P1, P2, …, Pk are true, for any k >= b
• Inductive stepShow that Pk+1 is true
67
ExampleTheorem: A binary tree of height n has at most 2n leaves.Proof by induction: let L(i) be the number of leaves at level i
L(0) = 1L(1) = 2L(2) = 4
L(3) = 8
68
We want to show: L(i) <= 2i
• Inductive basis L(0) = 1 (the root node)
• Inductive hypothesisLet’s assume L(i) <= 2i for all i = 0, 1, …, k
• Induction stepwe need to show that L(k + 1) <= 2k+1
69
Induction Step
From Inductive hypothesis: L(k) <= 2k
Levelk
k+1
70
L(k) <= 2kLevel
k
k+1
L(k+1) <= 2 * L(k) <= 2 * 2k = 2k+1
Induction Step
71
RemarkRecursion is another thing
Example of recursive function:
f(n) = f(n-1) + f(n-2)
f(0) = 1, f(1) = 1
72
Proof by Contradiction
We want to prove that a statement P is true
• we assume that P is false• then we arrive at an incorrect conclusion• therefore, statement P must be true
73
ExampleTheorem: is not rational
Proof:Assume by contradiction that it is rational = n/m n and m have no common factors
We will show that this is impossible
2
2
74
= n/m 2 m2 = n2
Therefore, n2 is even n is evenn = 2 k
2 m2 = 4k2 m2 = 2k2m is evenm = 2 p
Thus, m and n have common factor 2
Contradiction!
2