Upload
arannya-monzur
View
240
Download
0
Embed Size (px)
Citation preview
8/12/2019 Sr1 Toc Intro
1/56
CSE 211: Introduction to the Theory of Computation
Md. Shaifur Rahman
Dept. of CSEBangladesh University of Engineering & Technology
Week 1 Class 1
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 1 / 29
8/12/2019 Sr1 Toc Intro
2/56
Outline
1 Necessity of CS Theory Courses
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 2 / 29
8/12/2019 Sr1 Toc Intro
3/56
Outline
1 Necessity of CS Theory Courses
2 What is Theory of Computation?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 2 / 29
8/12/2019 Sr1 Toc Intro
4/56
Outline
1 Necessity of CS Theory Courses
2 What is Theory of Computation?
3 Course Outline
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 2 / 29
8/12/2019 Sr1 Toc Intro
5/56
Why Theory? (again!)
We dont like theory, because:One has to memorize a lot of theorems, lemmas, proofs
We like non-theory courses,
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 3 / 29
8/12/2019 Sr1 Toc Intro
6/56
Why Theory? (again!)
We dont like theory, because:One has to memorize a lot of theorems, lemmas, proofs
It is inherently boring!
We like non-theory courses,
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 3 / 29
8/12/2019 Sr1 Toc Intro
7/56
Why Theory? (again!)
We dont like theory, because:One has to memorize a lot of theorems, lemmas, proofs
It is inherently boring!
Where is the output?
We like non-theory courses,
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 3 / 29
8/12/2019 Sr1 Toc Intro
8/56
Why Theory? (again!)
We dont like theory, because:One has to memorize a lot of theorems, lemmas, proofs
It is inherently boring!
Where is the output?
How is it related to our career-path?We like non-theory courses,
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 3 / 29
8/12/2019 Sr1 Toc Intro
9/56
Why Theory? (again!)
We dont like theory, because:One has to memorize a lot of theorems, lemmas, proofs
It is inherently boring!
Where is the output?
How is it related to our career-path?We like non-theory courses,
We like programming in C, C++, JAVA, Python, Android etc.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 3 / 29
8/12/2019 Sr1 Toc Intro
10/56
Why Theory? (again!)
We dont like theory, because:One has to memorize a lot of theorems, lemmas, proofs
It is inherently boring!
Where is the output?
How is it related to our career-path?We like non-theory courses,
We like programming in C, C++, JAVA, Python, Android etc.
The outcome is visible, projects done in these courses brighten our
resume
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 3 / 29
8/12/2019 Sr1 Toc Intro
11/56
8/12/2019 Sr1 Toc Intro
12/56
Why Theory? (again!)
We dont like theory, because:One has to memorize a lot of theorems, lemmas, proofs
It is inherently boring!
Where is the output?
How is it related to our career-path?We like non-theory courses,
We like programming in C, C++, JAVA, Python, Android etc.
The outcome is visible, projects done in these courses brighten our
resumeThe skills are necessary for the career-path of a developer
Achieving grades are comparatively easy :)
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 3 / 29
8/12/2019 Sr1 Toc Intro
13/56
Theory vs. Non-theory
Difficult to separate theories from all courses!
Course Theory-bias
CSE 103 (Discrete Math) 90%
CSE 105 (Structured Programming Language) 30%
CSE 201 (Object Oriented Programming Language) 20%
CSE 203 (Data Structure) 80%CSE 205 (Digital Logic Design) 60%
In this semester:
Course Theory-bias
CSE 207 (Algorithms) 95%CSE 209 (Digital Electronics and Pulse Technique) 40%
CSE 211 (Theory of Computation) 100%
CSE207 is thefraternal brotherof CSE211
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 4 / 29
8/12/2019 Sr1 Toc Intro
14/56
The Power of Theory
Write a C program that finds all the factors of a 512-bit integer
For input 24, Output is: 1, 2, 3, 4, 6, 8, 12 and 24 , since 24 = 1 23 3
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 5 / 29
8/12/2019 Sr1 Toc Intro
15/56
The Power of Theory
Write a C program that finds all the factors of a 512-bit integer
For input 24, Output is: 1, 2, 3, 4, 6, 8, 12 and 24 , since 24 = 1 23 3But wait! 512-bit number!So the input ranges from 1 to 2512 1thats roughly 1.3 10154
The highest input can be of155-digits!
Input: 12345676449393993. . .
930049922(155 digits)
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 5 / 29
8/12/2019 Sr1 Toc Intro
16/56
The Power of Theory
Write a C program that finds all the factors of a 512-bit integer
For input 24, Output is: 1, 2, 3, 4, 6, 8, 12 and 24 , since 24 = 1 23 3But wait! 512-bit number!So the input ranges from 1 to 2512 1thats roughly 1.3 10154
The highest input can be of155-digits!
Input: 12345676449393993. . .
930049922(155 digits)
3 hoursgone, your C-program is not coming up with any answer?Your program worked perfectly for 1, 2, 10, 20 even 50 digit inputs!
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 5 / 29
Th P f Th
8/12/2019 Sr1 Toc Intro
17/56
The Power of Theory
Write a C program that finds all the factors of a 512-bit integer
For input 24, Output is: 1, 2, 3, 4, 6, 8, 12 and 24 , since 24 = 1 23 3But wait! 512-bit number!So the input ranges from 1 to 2512 1thats roughly 1.3 10154
The highest input can be of155-digits!
Input: 12345676449393993. . .
930049922(155 digits)
3 hoursgone, your C-program is not coming up with any answer?Your program worked perfectly for 1, 2, 10, 20 even 50 digit inputs!
So whats wrong??Is your program erroneous?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 5 / 29
Th P f Th
8/12/2019 Sr1 Toc Intro
18/56
The Power of Theory
Write a C program that finds all the factors of a 512-bit integer
For input 24, Output is: 1, 2, 3, 4, 6, 8, 12 and 24 , since 24 = 1 23 3But wait! 512-bit number!So the input ranges from 1 to 2512 1thats roughly 1.3 10154
The highest input can be of155-digits!
Input: 12345676449393993. . .
930049922(155 digits)
3 hoursgone, your C-program is not coming up with any answer?Your program worked perfectly for 1, 2, 10, 20 even 50 digit inputs!
So whats wrong??Is your program erroneous?
Is the computation power of the machine insufficient?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 5 / 29
Th P f Th
8/12/2019 Sr1 Toc Intro
19/56
The Power of Theory
Write a C program that finds all the factors of a 512-bit integer
For input 24, Output is: 1, 2, 3, 4, 6, 8, 12 and 24 , since 24 = 1 23 3But wait! 512-bit number!So the input ranges from 1 to 2512 1thats roughly 1.3 10154
The highest input can be of155-digits!
Input: 12345676449393993. . .
930049922(155 digits)
3 hoursgone, your C-program is not coming up with any answer?Your program worked perfectly for 1, 2, 10, 20 even 50 digit inputs!
So whats wrong??Is your program erroneous?
Is the computation power of the machine insufficient?
Is there a probem in the nature of the input?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 5 / 29
Th P f Th (C td )
8/12/2019 Sr1 Toc Intro
20/56
The Power of Theory (Contd.)
Case: Your program is erroneousHow do you know? The program is not even stopping!It may indeed stop after 10 hours and show the correct output!
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 6 / 29
Th P f Th (C td )
8/12/2019 Sr1 Toc Intro
21/56
The Power of Theory (Contd.)
Case: Your program is erroneousHow do you know? The program is not even stopping!It may indeed stop after 10 hours and show the correct output!
Case: Computation Power of the Machine is insufficient
We may use multi-core processors clocked at a very high speedlike: 10-core processor each core clocked at 3GHz
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 6 / 29
The Power of Theory (Contd )
8/12/2019 Sr1 Toc Intro
22/56
The Power of Theory (Contd.)
Case: Your program is erroneousHow do you know? The program is not even stopping!It may indeed stop after 10 hours and show the correct output!
Case: Computation Power of the Machine is insufficient
We may use multi-core processors clocked at a very high speedlike: 10-core processor each core clocked at 3GHzCan we afford that machine?What if the number is even bigger? Can the machine scale up to the
input?Caveat: Even with todays fastest computer, the computation for a500-digit input will take 5 billion years!
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 6 / 29
The Power of Theory (Contd )
8/12/2019 Sr1 Toc Intro
23/56
The Power of Theory (Contd.)
Case: Nature of InputSome numbers are too big or too smallAre you crazy? 500-digit number? Thats pretty big!
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 7 / 29
The Power of Theory (Contd )
8/12/2019 Sr1 Toc Intro
24/56
The Power of Theory (Contd.)
Case: Nature of InputSome numbers are too big or too smallAre you crazy? 500-digit number? Thats pretty big!So is:
the number of stars in the universe
So, data can be very large or too small? You never can tell
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 7 / 29
The Power of Theory (Contd )
8/12/2019 Sr1 Toc Intro
25/56
The Power of Theory (Contd.)
Case: Nature of InputSome numbers are too big or too smallAre you crazy? 500-digit number? Thats pretty big!So is:
the number of stars in the universethe number of electrons in your body
So, data can be very large or too small? You never can tell
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 7 / 29
The Power of Theory (Contd )
8/12/2019 Sr1 Toc Intro
26/56
The Power of Theory (Contd.)
Case: Nature of InputSome numbers are too big or too smallAre you crazy? 500-digit number? Thats pretty big!So is:
the number of stars in the universethe number of electrons in your body
the energy in electron volt when two stars collide
So, data can be very large or too small? You never can tell
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 7 / 29
The Power of Theory (Contd )
8/12/2019 Sr1 Toc Intro
27/56
The Power of Theory (Contd.)
Case: Nature of InputSome numbers are too big or too smallAre you crazy? 500-digit number? Thats pretty big!So is:
the number of stars in the universethe number of electrons in your body
the energy in electron volt when two stars collide
the number of combinatorial choices of pairings of almost half-million
genes in our cellSo, data can be very large or too small? You never can tell
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 7 / 29
The Power of Theory (Contd )
8/12/2019 Sr1 Toc Intro
28/56
The Power of Theory (Contd.)
Case: Nature of Computation
Can we break down the processing into multiple blocks and distribute
those among multiple computers?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 8 / 29
The Power of Theory (Contd )
8/12/2019 Sr1 Toc Intro
29/56
The Power of Theory (Contd.)
Case: Nature of Computation
Can we break down the processing into multiple blocks and distributethose among multiple computers?
Can we predict the time required by analyzing the nature of input(input-domain) and the method of computation (algorithm)?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 8 / 29
The Power of Theory (Contd.)
8/12/2019 Sr1 Toc Intro
30/56
The Power of Theory (Contd.)
Case: Nature of Computation
Can we break down the processing into multiple blocks and distributethose among multiple computers?
Can we predict the time required by analyzing the nature of input(input-domain) and the method of computation (algorithm)?
Can we identify certain problems asreally hardproblems?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 8 / 29
Really Hard Problems
8/12/2019 Sr1 Toc Intro
31/56
Really Hard Problems
General Tile Fitting Problem
Given a finite set of tiles and an area on a plane, is it possible to cover thecomplete area using each type of tiles at least once?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 9 / 29
Really Hard Problems (Contd.)
8/12/2019 Sr1 Toc Intro
32/56
y ( )
Sometimes, there is a solutionSometimes, there is no solutionSometimes, we cannot tell for sure- if there is a solution! (undecidable)
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 10 / 29
Really Hard Problems (Contd.)
8/12/2019 Sr1 Toc Intro
33/56
y ( )
There are many problems that cannot be guaranteed to be solved by anycomputer! These are calledundecidable problems
David Hilbert
Entscheidungsproblem
Give an algorithmAthat takes as inputa hypothesis Hand answersTrueorFalsebased on a finite set of axioms SA.In other word, AlgorithmAdecideswhether H logically follows from SA.
Hypothesis: A machine works withoutconsuming any energy.Decision: False (according to the set ofaxioms of Physics)
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 11 / 29
Really Hard Problems (Contd.)
8/12/2019 Sr1 Toc Intro
34/56
y ( )
Alonzo Church & Alan Turing
Alonzo Church and Alan Turing independently proved respectively in 1936
and in 1937 that
Hilberts Entscheidungsproblem isundecidable
This is known asChurch-Turing Thesis.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 12 / 29
Really Hard Problems (Contd.)
8/12/2019 Sr1 Toc Intro
35/56
y ( )
More undecided problems:
Mortal Matrix ProblemGiven a finite set ofn n matrices with integer entries, determinewhether they can be multiplied in some order, possibly with repetition, toyield the zero matrix.
Conjugacy problem
Given two words xand y of a Group G, whether there exists a z such that:y=zxz1
*A group is an algebraic structure consisting of a set with one or moreoperators such that the result of applying the operators on one or moreelements of the is also an element in the set. For example, the set ofinteger I is a group.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 13 / 29
Really Hard Problems (Contd.)
8/12/2019 Sr1 Toc Intro
36/56
( )
Some problems are hard but still solvable i.e. NOT undecidable
(Euclidian) Travelling Salesman Problem
Given a set of vertices and their coordinates, find a tour or cycle whichvisits each vertex exactly once and is of the minimum length.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 14 / 29
Really Hard Problems (Contd.)
8/12/2019 Sr1 Toc Intro
37/56
Some problems are hard but still solvable i.e. NOT undecidable
Chromatic Number or Graph Coloring ProblemGiven a graph G= (V,E), find the minimum number of different colorsneeded for this graph such that two adjacent vertices are not given thesame color.
Total three different colors are needed
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 15 / 29
8/12/2019 Sr1 Toc Intro
38/56
Really Hard Problems (Contd.)
8/12/2019 Sr1 Toc Intro
39/56
There is relationships among the hard solvable problems, we can comparetheir hardness!
Circuit satisfiability problem(Circuit-SAT) can bereducedtoTravelling Salesman
Problem (TSP)If there is agoodsolution toCircuit-SAT, there is also agoodsolution to TSP
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 17 / 29
Really Hard Problems (Contd.)
8/12/2019 Sr1 Toc Intro
40/56
Moral of the story
If the given problem is proved to be undecidable or unsolvable, dontinvest your valuable effort to write program to solve it.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 18 / 29
Really Hard Problems (Contd.)
8/12/2019 Sr1 Toc Intro
41/56
Moral of the story
If the given problem is proved to be undecidable or unsolvable, dontinvest your valuable effort to write program to solve it.
If the given problem is solvable but still hard, it is not your programsfault that the computation is taking too much time.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 18 / 29
Really Hard Problems (Contd.)
8/12/2019 Sr1 Toc Intro
42/56
Moral of the story
If the given problem is proved to be undecidable or unsolvable, dontinvest your valuable effort to write program to solve it.
If the given problem is solvable but still hard, it is not your programsfault that the computation is taking too much time.
If the given problem is solvable andnot hard, and your program failsto solve it in the reasonable amount of time, only theninvest effortfor improvement of your program!
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 18 / 29
Theory of Computation
8/12/2019 Sr1 Toc Intro
43/56
Theory of computation
The branch that deals with whether and how efficiently problems can besolved on amodel of computation, using an algorithm
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 19 / 29
Theory of Computation
8/12/2019 Sr1 Toc Intro
44/56
Theory of computation
The branch that deals with whether and how efficiently problems can besolved on amodel of computation, using an algorithm
The field is divided into three major branches:
Automata Theory
Computability Theory and
Computational Complexity Theory
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 19 / 29
Theory of Computation
8/12/2019 Sr1 Toc Intro
45/56
Theory of computation
The branch that deals with whether and how efficiently problems can besolved on amodel of computation, using an algorithm
The field is divided into three major branches:
Automata Theory
Computability Theory and
Computational Complexity Theory
There are many models of computation:
Turing Machine
Combinatory logic
Markov Algorithm etc.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 19 / 29
8/12/2019 Sr1 Toc Intro
46/56
Branches of Theory of Computation
8/12/2019 Sr1 Toc Intro
47/56
Automata Theory
the study ofabstract machines( in our case, Turing Machine) andthe problems that can be solved using these machines
If the abstract machine cannot solve the problem, no computer on
earth, no matter how powerful it is, can ever solve the problem
Topics include: Finite Automaton (FA), Push-down Automaton(PDA), Turing Machine (TM) etc.
studied in the theory of computation, compiler design, verification
systems, robotics etc.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 20 / 29
Branches of Theory of Computation (Contd.)
8/12/2019 Sr1 Toc Intro
48/56
Computability Theory
deals with the question of the extent to which a problem is solvableon a computer
Topics include: Turing Degree (level of unsolvabiltiy), Reducibility(the ability to transform one hard problem to another) etc.
studied in mathematical logic theories like recursion theory, artificialintelligence, machine learning etc.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 21 / 29
Branches of Theory of Computation (Contd.)
8/12/2019 Sr1 Toc Intro
49/56
Computational Complexity Theory
deals with classifying computational problems according to theirinherent difficulty
Two major aspects are considered: time complexity (how many stepsthe algorithm takes) and space complexity (how much memory isrequired by the program)
studies complexity classes: P, NP, NPCand relationships amongthese classes
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 22 / 29
Course Outline
8/12/2019 Sr1 Toc Intro
50/56
Course Name: CSE 211 (Theory of Computation)Credit Hour: 2.0Number of Class Tests: 3
Course Teachers:Md. Shaifur Rahman(first seven weeks before mid-term)and
Sukarna Barua(after mid-term)
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 23 / 29
Class Outline
8/12/2019 Sr1 Toc Intro
51/56
Week Day Topics
1
1 Introduction to TOC
2 Deterministic Finite Automaton (DFA)
2 1 DFA (Contd.)
2 Operators of Finite Automaton
3 1 Non-deterministic Finite Automaton (NFA)
2 Equivalence of DFA and NFA
4 1 Regular Expressions
2 Class Test-1
5 1 Regular Expressions (Contd.)
2 Pumping Lemma
6 1 Context-free Grammar (CFG)2 CFG (Contd.)
7 1 Overall Discussion
2 Class Test-2
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 24 / 29
8/12/2019 Sr1 Toc Intro
52/56
Textbooks (Contd.)
8/12/2019 Sr1 Toc Intro
53/56
Supporting textbooks
Introduction to Automata Theory,Languages & Computation
(3rd Edition)by
Hopcroft et. al.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 26 / 29
Textbooks (Contd.)
8/12/2019 Sr1 Toc Intro
54/56
Supporting textbooks
Elements of the Theory ofComputation(2nd Edition)
by
Lewis et. al.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 27 / 29
Course Website
8/12/2019 Sr1 Toc Intro
55/56
http://sites.google.com/site/cse211february2013
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 28 / 29
http://sites.google.com/site/cse211february2013http://sites.google.com/site/cse211february20138/12/2019 Sr1 Toc Intro
56/56
Question?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 29 / 29