Automata and Formal Languages - CM0081 … J. E., Motwani, R. and Ullman, J. D. (2007). Introduction to Automata theory, Languages, and Computation. 3rd ed. Pearson Education. Hutton,

Automata and Formal Languages - CM0081Introduction

Andrés Sicard-Ramírez

Universidad EAFIT

Semester 2018-1

Administrative Information

Course CoordinatorAndrés Sicard Ramírez

Head of the Department of Mathematical SciencesCarlos Mario Vélez Sánchez

Course web pagehttp://www1.eafit.edu.co/asr/courses/automata-CM0081/

Exams, programming labs, etc.See course web page.

ConventionThe numbers assigned to examples, exercises or theorems correspond to thenumbers in the textbook (Hopcroft, Motwani and Ullman 2007).

Introduction 2/28

http://www1.eafit.edu.co/asr/courses/automata-CM0081/

Course Outline

Language Machine Other modelsRegular DFA • Regular expressions

• NFA• 𝜀-NFA

Context-free Pushdownautomata

Recursive Halting TMs • 𝜆-calculus• Total recursive functions

Recursivelyenumerable

TMs • 𝜆-calculus• Partial recursive functions

DFA: Deterministic finite automataNFA: Non-deterministic finite automata𝜀-NFA: Non-deterministic finite automata with 𝜀-transitionsTM: Turing machine

Introduction 3/28

Computability (Decidability)

QuestionWhat can a computer do at all?

DefinitionA computable (or decidable) problem is a problem than can be solved by acomputer (informal definition).

Introduction 4/28


QuestionWhat can a computer do at all?

DefinitionA computable (or decidable) problem is a problem than can be solved by acomputer (informal definition).

Introduction 5/28


Example (The halting problem: An undecidable problem)Given an program P and an input I, to decide if the program will halt orwill run forever.

The halting algorithm

P Yes

I No

Introduction 6/28

Algorithmic Complexity (Tractability)

Comparison of several time complexity functions

𝑓(𝑛) 10 50 100log 𝑛 2.3 sec 3.9 sec 4.6 sec𝑛 10 sec 50 sec 1.7 min𝑛2 1.7 min 41.7 min 2.8 h2𝑛 17.1 min 358.001 c 4 × 1020 c3𝑛 16.4 h 2.3 × 1014 c 1.6 × 1038 c𝑛! 42 d 9.7 × 1054 c 3 × 10148 c

Introduction 7/28


QuestionWhat can a computer do efficiently?

DefinitionA tractable problem is a problem than can be solved by a computer algorithmthat run in polynomial time.

Introduction 8/28


QuestionWhat can a computer do efficiently?

DefinitionA tractable problem is a problem than can be solved by a computer algorithmthat run in polynomial time.

Introduction 9/28


DefinitionA literal is an atomic formula (propositional variable) or the negation of anatomic formula.

DefinitionA (propositional logic) formula 𝐹 is said to be in conjunctive normal form,if and only if,

𝐹 has the form 𝐹1 ∧ ⋯ ∧ 𝐹𝑛, 𝑛 ≥ 1,where each 𝐹1, … , 𝐹𝑛 is a disjunction of literals.

Introduction 10/28


Example (3-SAT: An intractable problem)To determine the satisfiability of a propositional formula in conjunctivenormal form where each disjunction of literals is limited to at most threeliterals.The problem was proposed in Karp’s 21 NP-complete problems (Karp1972).

Introduction 11/28


3-SAT better deterministic algorithmic complexity

𝑂(1.3303𝑛) Makino, Tamaki and Yamamoto (2011)𝑂(1.33334𝑛) Moser and Scheder (2011)𝑂(1.439𝑛) Kutzkov and Scheder (2010)𝑂(1.465𝑛) Scheder (2008)𝑂(1.473𝑛) Brueggemann and Kern (2004)𝑂(1.481𝑛) Dantsin, Goerdt, Hirsch, Kannan, Kleinberg, Papadimitriou,

Raghavan and Schöning (2002)𝑂(1.497𝑛) Schiermeyer (1996)𝑂(1.505𝑛) Kullmann (1999)𝑂(1.681𝑛) Monien and Speckenmeyer (1979, 1985)

Introduction 12/28


SupercomputersMachines from: www.top500.orgPetaflops (PFs): 1015 floating-point operations per second

Date Machine PFs2016-06 Sunway TaihuLight 93.012013-06 Tianhe-2 33.862012-06 Blue Gene/Q 16.322011-06 K computer 8.16

(Last updated: TOP500 List - November 2017)

Introduction 13/28

www.top500.org


3-SAT simulation with 𝑂(1.3303𝑛)PFs 150 200 400

93.01 0.7 min 2.1 y 1.3 × 1023 c33.86 1.9 min 5.8 y 3.6 × 1023 c16.32 4.0 min 12.0 y 7.4 × 1023 c8.16 8.0 min 24.0 y 1.5 × 1024 c

Introduction 14/28

Computability and Algorithmic Complexity

Problem(TM-Computability)

⎧{{{{{{{{{⎨{{{{{{{{{⎩

Computable(decidable)problem(AlgorithmComplexity)

⎧{{{{⎨{{{{⎩

Tractableproblem

Intractableproblem

Non-computable(undecidable)problem(Hypercomputation)

Introduction 15/28

Some Paradigms of Programming

Imperative: Describe computation in terms of state-transformingoperations such as assignment. Programming is done with statements.Logic: Predicate calculus as a programming language. Programming isdone with sentences.Functional: Describe computation in terms of (mathematical) functions.Programming is done with expressions.

Examples

Imperative⎧{⎨{⎩

CC++Java

Logic {CLP(R)Prolog

Functional

⎧{{{⎨{{{⎩

Standard MLErlang

Pure⎧{⎨{⎩

CleanHaskellIdris

Introduction 16/28

Pure Functional Programming

Side effects“A side effect introduces a dependency between the global state of thesystem and the behaviour of a function... Side effects are essentially invisibleinputs to, or outputs from, functions.” (O’Sullivan, Goerzen and Stewart2008, p. 27)

Introduction 17/28


Pure functions“A function may be described as pure if both these statements aboutthe function hold:”†

1. “The function always evaluates the same result value given thesame argument value(s). The function result value cannotdepend on any... state that may change as program executionproceeds or between different executions of the program, nor canit depend on any external input from I/O devices.”

2. “Evaluation of the result does not cause any semanticallyobservable side effect or output, such as mutation of mutableobjects or output to I/O devices.”

†Wikipedia: Pure function (July 28, 2014).Introduction 18/28


Pure functions“A function may be described as pure if both these statements aboutthe function hold:”†

1. “The function always evaluates the same result value given thesame argument value(s). The function result value cannotdepend on any... state that may change as program executionproceeds or between different executions of the program, nor canit depend on any external input from I/O devices.”

2. “Evaluation of the result does not cause any semanticallyobservable side effect or output, such as mutation of mutableobjects or output to I/O devices.”

†Wikipedia: Pure function (July 28, 2014).Introduction 19/28


Pure functions“...take all their input as explicit arguments, and produce all theiroutput as explicit results.” (Hutton 2016, § 10.1)

Introduction 20/28


Referential transparencyEquals can be replaced by equals

“By definition, a function in Haskell defines a fixed relation betweeninputs and output: whenever a function 𝑓 is applied to the argumentvalue 𝑎𝑟𝑔 it will produce the same output no matter what the overallstate of the computation is. Haskell, like any other pure functionallanguage, is said to be ‘referentially transparent’ or ‘side-effect free’.This property does not hold for imperative languages...” (Grune, Bal,Jacobs and Langendoen 2003, pp. 544–545)

Introduction 21/28


Referential transparencyEquals can be replaced by equals“By definition, a function in Haskell defines a fixed relation betweeninputs and output: whenever a function 𝑓 is applied to the argumentvalue 𝑎𝑟𝑔 it will produce the same output no matter what the overallstate of the computation is. Haskell, like any other pure functionallanguage, is said to be ‘referentially transparent’ or ‘side-effect free’.This property does not hold for imperative languages...” (Grune, Bal,Jacobs and Langendoen 2003, pp. 544–545)

Introduction 22/28


Reasoning about (pure) functional programsEquational reasoning + induction + co-induction + …

Introduction 23/28

Reading

HomeworkTo read from the textbook the following sections:§ 1.1. Why Study Automata Theory?§ 1.2. Introduction to Formal Proofs§ 1.3. Additional Forms of Proofs

Introduction 24/28

References

Brueggemann, T. and Kern, W. (2004). An Improved Deterministic LocalSearch Algorithm for 3-SAT. Theoretical Computer Science 329.1–3,pp. 303–313. doi: 10.1016/j.tcs.2004.08.002 (cit. on p. 12).Dantsin, E. et al. (2002). A Deterministic (2 − 2/(𝑘 + 1))𝑛 Algorithm for𝑘-SAT Based on Local Search. Theoretical Computer Science 289.1,pp. 69–83. doi: 10.1016/S0304-3975(01)00174-8 (cit. on p. 12).Grune, D., Bal, H. E., Jacobs, C. J. H. and Langendoen, K. G. (2003).Modern Compiler Desing. John Wiley & Sons, LTD (cit. on pp. 21, 22).Hopcroft, J. E., Motwani, R. and Ullman, J. D. (2007). Introduction toAutomata theory, Languages, and Computation. 3rd ed. Pearson Education(cit. on p. 2).Hutton, G. (2016). Programming in Haskell. 2nd ed. Cambridge UniversityPress (cit. on p. 20).

Introduction 25/28

https://doi.org/10.1016/j.tcs.2004.08.002

https://doi.org/10.1016/S0304-3975(01)00174-8

References

Karp, R. M. (1972). Reducibility Among Combinatorial Problems. In:Complexity of Computer Computations. Ed. by Miller, R. E. andThatcher, J. W. Plenum Press, pp. 85–103. doi:10.1007/978-1-4684-2001-2_9 (cit. on p. 11).Kullmann, O. (1999). New Methods for 3-SAT Decision and Worst-CaseAnalysis. Theoretical Computer Science 223.1–2, pp. 1–72. doi:10.1016/S0304-3975(98)00017-6 (cit. on p. 12).Kutzkov, K. and Scheder, D. (2010). Using CSP to Improve Deterministic3-SAT. CoRR abs/1007.1166. url: https://arxiv.org/abs/1007.1166(cit. on p. 12).Makino, K., Tamaki, S. and Yamamoto, M. (2011). Derandomizing HSSWAlgorithm for 3-SAT. CoRR. url: https://arxiv.org/abs/1102.3766(cit. on p. 12).Monien, B. and Speckenmeyer, E. (1979). 3-Satisfiability is Testable in𝑂(1.62𝑟) Steps. Tech. rep. 3/1979. Reihe Theoretische Informatik,Universität-Gesamthochschule-Paderborn (cit. on p. 12).

Introduction 26/28

https://doi.org/10.1007/978-1-4684-2001-2_9

https://doi.org/10.1016/S0304-3975(98)00017-6

https://arxiv.org/abs/1007.1166

https://arxiv.org/abs/1102.3766

References

Monien, B. and Speckenmeyer, E. (1985). Solving Satisfiability in less than2𝑛 Steps. Discrete Applied Mathematics 10.3, pp. 287–295. doi:10.1016/0166-218X(85)90050-2 (cit. on p. 12).Moser, R. A. and Scheder, D. (2011). A Full Derandomization ofSchöning’s 𝑘-SAT Algorithm. In: Proceedings of the Forty-third AnnualACM Symposium on Theory of Computing (STOC 2011), pp. 245–252.doi: 10.1145/1993636.1993670 (cit. on p. 12).O’Sullivan, B., Goerzen, J. and Stewart, D. (2008). Real World Haskell.O’REILLY (cit. on p. 17).Scheder, D. (2008). Guided Search and a Faster Deterministic Algorithm for3-SAT. In: Proc. of the 8th Latin American Symposium on TheoreticalInformatic (LATIN 2008). Ed. by Laber, E. S., Bornstein, C.,Nogueira, T. L. and Faria, L. Vol. 4957. Lecture Notes in Computer Science.Springer, pp. 60–71. doi: 10.1007/978-3-540-78773-0_6 (cit. on p. 12).

Introduction 27/28

https://doi.org/10.1016/0166-218X(85)90050-2

https://doi.org/10.1145/1993636.1993670

https://doi.org/10.1007/978-3-540-78773-0_6

References

Schiermeyer, I. (1996). Pure Literal Look Ahead: An 𝑂(1.497𝑛)3-Satisfability Algorithm (Extended Abstract). Workshop on theSatisfability Problem, Siena 1996. url:http://gauss.ececs.uc.edu/franco_files/SAT96/sat-workshop-abstracts.html (cit. on p. 12).

Introduction 28/28

http://gauss.ececs.uc.edu/franco_files/SAT96/sat-workshop-abstracts.html

http://gauss.ececs.uc.edu/franco_files/SAT96/sat-workshop-abstracts.html

Automata and Formal Languages - CM0081Formal Proofs


Universidad EAFIT

Semester 2018-1

Proofs by Contradiction and Proofs of Negations

Proof by contradiction(or reductio ad absurdum)

[¬𝛽]⋮

⊥𝛽

Proof of negation (Bauer 2017)

[𝛽]⋮⊥¬𝛽

Justifications

[¬𝛽]⋮

⊥ (conditional proof)¬𝛽 → ⊥(¬𝛼 def= 𝛼 → ⊥)¬¬𝛽 (⊢ ¬¬𝛼 → 𝛼)𝛽

[𝛽]⋮

⊥ (conditional proof)𝛽 → ⊥(¬𝛼 def= 𝛼 → ⊥)¬𝛽

Formal Proofs 2/13

Proofs by Contradiction and Proofs of Negations

Proof by contradiction(or reductio ad absurdum)

[¬𝛽]⋮

⊥𝛽

Proof of negation (Bauer 2017)

[𝛽]⋮⊥¬𝛽

Justifications

[¬𝛽]⋮

⊥ (conditional proof)¬𝛽 → ⊥(¬𝛼 def= 𝛼 → ⊥)¬¬𝛽 (⊢ ¬¬𝛼 → 𝛼)𝛽

[𝛽]⋮

⊥ (conditional proof)𝛽 → ⊥(¬𝛼 def= 𝛼 → ⊥)¬𝛽

Formal Proofs 3/13

Inductive Proofs: Mathematical Induction

The induction principleLet 𝑆(𝑛) be a property about the natural numbers. If

we prove 𝑆(𝑖) (basis step) andwe prove that for all natural number 𝑛 ≥ 𝑖, 𝑆(𝑛) implies 𝑆(𝑛 + 1)(inductive step)

then we may conclude 𝑆(𝑛) for all 𝑛 ≥ 𝑖.

Formal Proofs 4/13

Inductive Proofs: Structural Induction

The structural induction principleLet 𝑆(𝑋) be a property about the structures 𝑋 that are defined by somerecursive/inductive definition. If

we prove 𝑆(𝑋) for the basis structure(s) of 𝑋 (basis step) andgiven a structure 𝑋 whose recursive/inductive definition says it isformed from 𝑌1, … , 𝑌𝑘, we prove 𝑆(𝑋) assuming that the properties𝑆(𝑌1), … , 𝑆(𝑌𝑘) hold (inductive step)

then 𝑆(𝑋) is true for all 𝑋.

Formal Proofs 5/13

Inductive Proofs: Mutual Induction

Example𝑓(0) = 0 𝑔(0) = 1 ℎ(0) = 0

𝑓(𝑛 + 1) = 𝑔(𝑛) 𝑔(𝑛 + 1) = 𝑓(𝑛) ℎ(𝑛 + 1) = 1 − ℎ(𝑛)

1. To prove 𝑆(𝑛): ℎ(𝑛) = 𝑓(𝑛). Impossible!

2. To prove 𝑇 (𝑛): a) ℎ(𝑛) = 𝑓(𝑛) and b) ℎ(𝑛) = 1 − 𝑔(𝑛).

Formal Proofs 6/13


Example𝑓(0) = 0 𝑔(0) = 1 ℎ(0) = 0

𝑓(𝑛 + 1) = 𝑔(𝑛) 𝑔(𝑛 + 1) = 𝑓(𝑛) ℎ(𝑛 + 1) = 1 − ℎ(𝑛)

1. To prove 𝑆(𝑛): ℎ(𝑛) = 𝑓(𝑛). Impossible!2. To prove 𝑇 (𝑛): a) ℎ(𝑛) = 𝑓(𝑛) and b) ℎ(𝑛) = 1 − 𝑔(𝑛).

Formal Proofs 7/13


Example (cont.)

𝑓(0) = 0 𝑔(0) = 1 ℎ(0) = 0𝑓(𝑛 + 1) = 𝑔(𝑛) 𝑔(𝑛 + 1) = 𝑓(𝑛) ℎ(𝑛 + 1) = 1 − ℎ(𝑛)

Proof of 𝑇 (𝑛): a) ℎ(𝑛) = 𝑓(𝑛) and b) ℎ(𝑛) = 1 − 𝑔(𝑛).Basis step 𝑇 (0). Easy.Induction step 𝑇 (𝑛) ⇒ 𝑇 (𝑛 + 1):

a) ℎ(𝑛 + 1) = 1 − ℎ(𝑛) (def. of ℎ)= 𝑔(𝑛) (IH b)= 𝑓(𝑛 + 1) (def. of 𝑓)

b) ℎ(𝑛 + 1) = 1 − ℎ(𝑛) (def. of ℎ)= 1 − 𝑓(𝑛) (IH a)= 1 − 𝑔(𝑛 + 1) (def. of 𝑔)

Formal Proofs 8/13


Example (cont.)


Proof of 𝑇 (𝑛): a) ℎ(𝑛) = 𝑓(𝑛) and b) ℎ(𝑛) = 1 − 𝑔(𝑛).

Basis step 𝑇 (0). Easy.Induction step 𝑇 (𝑛) ⇒ 𝑇 (𝑛 + 1):



Formal Proofs 9/13


Example (cont.)


Proof of 𝑇 (𝑛): a) ℎ(𝑛) = 𝑓(𝑛) and b) ℎ(𝑛) = 1 − 𝑔(𝑛).Basis step 𝑇 (0). Easy.

Induction step 𝑇 (𝑛) ⇒ 𝑇 (𝑛 + 1):a) ℎ(𝑛 + 1) = 1 − ℎ(𝑛) (def. of ℎ)

= 𝑔(𝑛) (IH b)= 𝑓(𝑛 + 1) (def. of 𝑓)


Formal Proofs 10/13


Example (cont.)





Formal Proofs 11/13


Example (cont.)





Formal Proofs 12/13

References

Bauer, A. (2017). Five States of Accepting Constructive Mathematics. Bull.Amer. Math. Soc. 54.3, pp. 481–498. doi: 10.1090/bull/1556 (cit. onpp. 2, 3).

Formal Proofs 13/13

https://doi.org/10.1090/bull/1556

Automata and Formal Languages - CM0081The Central Concepts of Automata Theory


Universidad EAFIT

Semester 2018-1

Alphabets and Words

DefinitionAn alphabet is a finite, non-empty set of symbols.

ExamplesΣ1 = {0, 1},Σ2 = {𝑎, 𝑏, … , 𝑧},Σ3 = {𝑥 ∣ 𝑥 is a Unicode codepoint }.

The Central Concepts of Automata Theory 2/34

Alphabets and Words

DefinitionAn alphabet is a finite, non-empty set of symbols.

ExamplesΣ1 = {0, 1},Σ2 = {𝑎, 𝑏, … , 𝑧},Σ3 = {𝑥 ∣ 𝑥 is a Unicode codepoint }.


Alphabets and Words

DefinitionA string (or word or event) is a finite sequence of symbols of an alphabet.Inductive definition:Let Σ be an alphabet.Basis step: The empty word (denoted 𝜀) is a word over Σ.Inductive step: If 𝑥 is a word over Σ and 𝑎 ∈ Σ, then 𝑥𝑎 is a word

over Σ.

ConventionsAlphabets: Σ, Γ, …Symbols: 𝑎, 𝑏, 𝑐, …Words: 𝑤, 𝑥, 𝑦, 𝑧, …


Alphabets and Words

DefinitionA string (or word or event) is a finite sequence of symbols of an alphabet.Inductive definition:Let Σ be an alphabet.Basis step: The empty word (denoted 𝜀) is a word over Σ.Inductive step: If 𝑥 is a word over Σ and 𝑎 ∈ Σ, then 𝑥𝑎 is a word

over Σ.

ConventionsAlphabets: Σ, Γ, …Symbols: 𝑎, 𝑏, 𝑐, …Words: 𝑤, 𝑥, 𝑦, 𝑧, …


Operations on Words

DefinitionLet Σ be an alphabet. The length of a word is recursively defined by

|_| ∶ Σ∗ → ℕ|𝜀| = 0

|𝑥𝑎| = |𝑥| + 1


Operations on Words

DefinitionLet Σ be an alphabet. The concatenation of words is recursively defined by

_ · _ ∶ Σ∗ × Σ∗ → Σ∗

𝑥 · 𝜀 = 𝑥𝑥 · 𝑦𝑎 = (𝑥 · 𝑦)𝑎

Note: 𝑥 · 𝜀 = 𝜀 · 𝑥 = 𝑥 (that implies an algebraic structure)

NotationWe remove the dot in the concatenation.


Operations on Words


_ · _ ∶ Σ∗ × Σ∗ → Σ∗

𝑥 · 𝜀 = 𝑥𝑥 · 𝑦𝑎 = (𝑥 · 𝑦)𝑎




Operations on Words


_ · _ ∶ Σ∗ × Σ∗ → Σ∗

𝑥 · 𝜀 = 𝑥𝑥 · 𝑦𝑎 = (𝑥 · 𝑦)𝑎




Operations on Words

ExampleLet Σ be an alphabet and let 𝑥 and 𝑦 be words over Σ. Prove that|𝑥𝑦| = |𝑥| + |𝑦|.


Operations on Words

Proof.By structural induction on 𝑦.

(or by mathematical induction on |𝑦|).

Basis step (𝑦 = 𝜀):

(or |𝑦| = 0, then 𝑦 = 𝜀):

|𝑥𝜀| = |𝑥| (def. of concatenation)= |𝑥| + |𝜀| (def. of length)

Induction step (𝑦 = 𝑤𝑎):

(or |𝑦| = 𝑛 + 1, then 𝑦 = 𝑤𝑎 where|𝑤| = 𝑛):

|𝑥(𝑤𝑎)| = |𝑥𝑤(𝑎)| (def. of concatenation)= |𝑥𝑤| + 1 (def. of length)= |𝑥| + |𝑤| + 1 (IH)= |𝑥| + |𝑤𝑎| (def. of length)


Operations on Words

Proof.By structural induction on 𝑦 (or by mathematical induction on |𝑦|).

Basis step (𝑦 = 𝜀) (or |𝑦| = 0, then 𝑦 = 𝜀):

|𝑥𝜀| = |𝑥| (def. of concatenation)= |𝑥| + |𝜀| (def. of length)

Induction step (𝑦 = 𝑤𝑎) (or |𝑦| = 𝑛 + 1, then 𝑦 = 𝑤𝑎 where |𝑤| = 𝑛):

|𝑥(𝑤𝑎)| = |𝑥𝑤(𝑎)| (def. of concatenation)= |𝑥𝑤| + 1 (def. of length)= |𝑥| + |𝑤| + 1 (IH)= |𝑥| + |𝑤𝑎| (def. of length)


Operations on Words

Strings, length and concatenation in Haskelldata List a = Nil | Cons a (List a)data RList a = Lin | Snoc (RList a) a

length' :: RList a -> Intlength' Lin = 0 -- Eq. 1length' (Snoc xs x) = 1 + length' xs -- Eq. 2

(+++) :: RList a -> RList a -> RList a(+++) xs Lin = xs -- Eq. 1(+++) xs (Snoc ys y) = Snoc (xs +++ ys) y -- Eq. 2


Operations on Words





Operations on Words





Operations on Words

ExampleProve that length' (xs +++ ys) = length' xs + length' ys.

Proof by structural recursion on ys

Basis step (ys is Lin):

length' (xs +++ Lin)

= length' xs (Eq. 1 of (+++))= length' xs +++ length' Lin (Eq. 1 of length')


Operations on Words

ExampleProve that length' (xs +++ ys) = length' xs + length' ys.

Proof by structural recursion on ys

Basis step (ys is Lin):

length' (xs +++ Lin)

= length' xs (Eq. 1 of (+++))= length' xs +++ length' Lin (Eq. 1 of length')


Operations on Words

Induction step (ys is Snoc ys' y'):

length' (xs +++ (Snoc ys' y'))

= length' (Snoc (xs +++ ys') y')) (Eq. 2 of (+++))= 1 + length' (xs +++ ys') (Eq. 2 of length')= 1 + (length' xs + length' ys') (IH)= length' xs + (1 + length' ys') (arithmetic)= length' xs + length' (Snoc ys' y) (Eq. 2 of length')


Operations on Words and Alphabets

DefinitionLet Σ be an alphabet. The power of words is recursively defined by

__ ∶ Σ∗ × ℕ → Σ∗

𝑥0 = 𝜀𝑥𝑛+1 = 𝑥𝑛𝑥


Operations on Words and Alphabets

DefinitionThe 𝑘-power of an alphabet Σ, denoted Σ𝑘, is the set of strings of length 𝑘over Σ.

ExamplesGiven Σ = {0, 1} then

Σ0 = {𝜀},Σ1 = {0, 1},Σ2 = {00, 01, 10, 11},Σ3 = {000, 001, 010, 011, 100, 101, 110, 111}.


Universal Language

DefinitionLet Σ be an alphabet. The universal language over Σ, denoted by Σ∗, is theset of all the strings over Σ.


Languages

DefinitionIf Σ is an alphabet and 𝐿 ⊆ Σ∗ then 𝐿 is a language over Σ.

Examples∅ and Σ∗ are languages over any alphabetThe set of string of 0’s and 1’s with equal number of each

{𝜀, 01, 10, 0011, 0110, 1001, 1100, … }

{0𝑛1𝑛 ∣ 𝑛 ≥ 1}{𝜀} ≠ ∅The set of binary numbers whose value is a primeThe set of legal C programsThe set of legal English words


Languages


Examples∅ and Σ∗ are languages over any alphabet

The set of string of 0’s and 1’s with equal number of each

{𝜀, 01, 10, 0011, 0110, 1001, 1100, … }



Languages



{𝜀, 01, 10, 0011, 0110, 1001, 1100, … }



Languages



{𝜀, 01, 10, 0011, 0110, 1001, 1100, … }

{0𝑛1𝑛 ∣ 𝑛 ≥ 1}

{𝜀} ≠ ∅The set of binary numbers whose value is a primeThe set of legal C programsThe set of legal English words


Languages



{𝜀, 01, 10, 0011, 0110, 1001, 1100, … }

{0𝑛1𝑛 ∣ 𝑛 ≥ 1}{𝜀} ≠ ∅

The set of binary numbers whose value is a primeThe set of legal C programsThe set of legal English words


Languages



{𝜀, 01, 10, 0011, 0110, 1001, 1100, … }

{0𝑛1𝑛 ∣ 𝑛 ≥ 1}{𝜀} ≠ ∅The set of binary numbers whose value is a prime

The set of legal C programsThe set of legal English words


Languages



{𝜀, 01, 10, 0011, 0110, 1001, 1100, … }

{0𝑛1𝑛 ∣ 𝑛 ≥ 1}{𝜀} ≠ ∅The set of binary numbers whose value is a primeThe set of legal C programs

The set of legal English words


Languages



{𝜀, 01, 10, 0011, 0110, 1001, 1100, … }



Problems

DefinitionLet 𝐿 ⊆ Σ∗ be a language and let 𝑤 ∈ Σ∗ be word. To decide whether ornot 𝑤 ∈ 𝐿 is the decision problem on the language 𝐿.

Problem = decision problem?It is a language or a problem?Is the problem decidable or undecidable?Is the problem tractable or intractable?


Problems


Problem = decision problem?

It is a language or a problem?Is the problem decidable or undecidable?Is the problem tractable or intractable?


Problems


Problem = decision problem?It is a language or a problem?

Is the problem decidable or undecidable?Is the problem tractable or intractable?


Problems


Problem = decision problem?It is a language or a problem?Is the problem decidable or undecidable?

Is the problem tractable or intractable?


Problems


Problem = decision problem?It is a language or a problem?Is the problem decidable or undecidable?Is the problem tractable or intractable?


Automata and Formal Languages - CM0081Determinist Finite Automata


Universidad EAFIT

Semester 2018-1

Formal Languages: Origins

Source areas (Greibach 1981, p. 14)Logic and recursive-function theorySwitching circuit theory and logic designModelling of biological systems (brain activity)Mathematical and computation linguisticsComputer programming and the design of Algol and otherproblem-oriented languages

Determinist Finite Automata 2/32

Finite Automata

Example (Modeling an on/off switch)

onstart off

push

push

Example (Recognising the word ‘then’)

start t th the thent h e n


Finite Automata

Example (Modeling an on/off switch)

onstart off

push

push

Example (Recognising the word ‘then’)

start t th the thent h e n


The Wolf, the Goat and the Cabbage Problem“A man with a wolf, goat, and cabbage is onthe left bank of a river. There is a boat largeenough to carry the man and only one of theother three. The man and his entouragewish to cross to the right bank, and the mancan ferry each across, one at a time.However, if the man leaves the wolf andgoat unattended on either shore, the wolfwill surely eat the goat. Similarly, if the goatand cabbage are left unattended, the goatwill eat the cabbage. Is it possible to crossthe river without the goat or cabbage beingeaten?” (Hopcroft and Ullman 1979, p. 14)†

†The illustration is from the cover of (Levitin 2006).Determinist Finite Automata 5/32

The Wolf, the Goat and the Cabbage Problem

MWGC/∅start WC/MG MWC/G

C/MWG W/MGC

MGC/W MWG/C

G/MWCMG/WC∅/MWGC

g

g

mm

w

g

c w

g

c

c

g

wc

g

w

g

g

m

m


Finite Automata

Description“The finite automaton is a mathematical model of a system, with discreteinputs and outputs. The system can be in any one of a finite number ofinternal configurations or “states”. The state of the system summarizes theinformation concerning past inputs that is needed to determine the beha-viour the system on subsequent inputs.” (Hopcroft and Ullman 1979, p. 123)


Deterministic Finite Automata

DefinitionA deterministic finite automata (DFA) is a quintuple (𝑄, Σ, 𝛿, 𝑞0, 𝐹 ), where

𝑄: A finite set of stateΣ: An alphabet of input symbols

𝛿 ∶ 𝑄 × Σ → 𝑄: A transition function𝑞0 ∈ 𝑄: A start state𝐹 ⊆ 𝑄: A set of accepting (or final) states


DFA Representations

Transition diagramTransition tableDetailed description


Transition Diagram

ExampleThe following transition diagram represents a DFA that accepts thelanguage 𝐿 = {𝑥01𝑦 ∈ Σ∗ ∣ 𝑥, 𝑦 ∈ Σ∗}, where Σ = {0, 1}.

𝑞0start 𝑞1 𝑞20

1

1

0

0, 1

States description:𝑞0: The automaton has never seen 01, but its last input was eithernonexistent or it last saw a 1.𝑞1: The automaton has never seen 01, but its most recent input was 0.𝑞2: The automaton has already seen 01.


Transition Diagram



1

1

0

0, 1



Transition Diagram



1

1

0

0, 1

Processing the input 0101: 𝛿(𝑞0, 0) = …



Transition Diagram



1

1

0

0, 1



Transition Tables and Detailed Descriptions

Example


1

1

0

0, 1

Transition table:0 1

→ 𝑞0 𝑞1 𝑞0𝑞1 𝑞1 𝑞2

∗𝑞2 𝑞2 𝑞2

Detailed descriptions:𝑄 = {𝑞0, 𝑞1, 𝑞2}, Σ = {0, 1}, 𝑞0 (start state),{𝑞2} (set of accepting states) and 𝛿 ∶ 𝑄 × Σ → 𝑄where

𝛿(𝑞0, 0) = 𝑞1, 𝛿(𝑞0, 1) = 𝑞0,𝛿(𝑞1, 0) = 𝑞1, 𝛿(𝑞1, 1) = 𝑞2,𝛿(𝑞2, 0) = 𝑞2, 𝛿(𝑞2, 1) = 𝑞2.


Transition Tables and Detailed Descriptions

Example


1

1

0

0, 1

Transition table:0 1

→ 𝑞0 𝑞1 𝑞0𝑞1 𝑞1 𝑞2

∗𝑞2 𝑞2 𝑞2

Detailed descriptions:𝑄 = {𝑞0, 𝑞1, 𝑞2}, Σ = {0, 1}, 𝑞0 (start state),{𝑞2} (set of accepting states) and 𝛿 ∶ 𝑄 × Σ → 𝑄where

𝛿(𝑞0, 0) = 𝑞1, 𝛿(𝑞0, 1) = 𝑞0,𝛿(𝑞1, 0) = 𝑞1, 𝛿(𝑞1, 1) = 𝑞2,𝛿(𝑞2, 0) = 𝑞2, 𝛿(𝑞2, 1) = 𝑞2.


Extension of the Transition Function for DFAs

Let 𝐷 = (𝑄, Σ, 𝛿, 𝑞0, 𝐹 ) be a DFA. The extension of the transitionfunction, denoted by 𝛿, is recursively defined by

𝛿 ∶ 𝑄 × Σ∗ → 𝑄𝛿(𝑞, 𝜀) = 𝑞,

𝛿(𝑞, 𝑥𝑎) = 𝛿( 𝛿(𝑞, 𝑥), 𝑎).



Example


1

1

0

0, 1

𝛿(𝑞0, 0101) = 𝛿( 𝛿(𝑞0, 010), 1)= 𝛿(𝛿( 𝛿(𝑞0, 01), 0), 1)= 𝛿(𝛿(𝛿( 𝛿(𝑞0, 0), 1), 0), 1)= 𝛿(𝛿(𝛿(𝛿( 𝛿(𝑞0, 𝜀), 0), 1), 0), 1)= 𝛿(𝛿(𝛿(𝛿(𝑞0, 0), 1), 0), 1)= 𝛿(𝛿(𝛿(𝑞1, 1), 0), 1)= 𝛿(𝛿(𝑞2, 0), 1)= 𝛿(𝑞2, 1) = 𝑞2



Exercise (2.2.2)

Prove that 𝛿(𝑞, 𝑥𝑦) = 𝛿( 𝛿(𝑞, 𝑥), 𝑦) for any state 𝑞 and strings 𝑥 and 𝑦.(Hint: Perform induction on 𝑦).

Proof by induction on 𝑦.Basis step (𝑦 = 𝜀)

𝛿( 𝛿(𝑞, 𝑥), 𝜀) = 𝛿(𝑞, 𝑥) (def. of 𝛿)= 𝛿(𝑞, 𝑥𝜀) (def. of concatenation)

Inductive step (𝑦 = 𝑤𝑎)

𝛿(𝑞, 𝑥 · 𝑤𝑎) = 𝛿( 𝛿(𝑞, 𝑥𝑤), 𝑎) (def. of 𝛿 and concatenation)= 𝛿( 𝛿( 𝛿(𝑞, 𝑥), 𝑤), 𝑎) (IH)= 𝛿( 𝛿(𝑞, 𝑥), 𝑤𝑎) (def. of 𝛿)



Exercise (2.2.2)








Exercise (2.2.2)








Exercise (2.2.7)Let 𝐷 be a DFA and 𝑞 a particular state of 𝐷, such that 𝛿(𝑞, 𝑎) = 𝑞 for allinput symbols 𝑎. Show by induction on the input that for all inputstrings 𝑤, 𝛿(𝑞, 𝑤) = 𝑞.

Proof by induction on 𝑤.Basis step (𝑤 = 𝜀)

𝛿(𝑞, 𝜀) = 𝑞 (def. of 𝛿)Inductive step (𝑤 = 𝑥𝑎)

𝛿(𝑞, 𝑥𝑎) = 𝛿( 𝛿(𝑞, 𝑥), 𝑎) (def. of 𝛿)= 𝛿(𝑞, 𝑎) (IH)= 𝑞 (hypothesis)





𝛿(𝑞, 𝜀) = 𝑞 (def. of 𝛿)

Inductive step (𝑤 = 𝑥𝑎)






𝛿(𝑞, 𝜀) = 𝑞 (def. of 𝛿)Inductive step (𝑤 = 𝑥𝑎)



Regular Languages

DefinitionLet 𝐷 = (𝑄, Σ, 𝛿, 𝑞0, 𝐹 ) be a DFA. The language accepted by 𝐷 is

𝐿(𝐷) = {𝑤 ∈ Σ∗ ∣ 𝛿(𝑞0, 𝑤) ∈ 𝐹}.

DefinitionA language 𝐿 is regular if exists a DFA 𝐷 such 𝐿 = 𝐿(𝐷).

Examples∅ is a regular language.Let Σ be an alphabet. Σ∗ is a regular language.


Regular Languages


𝐿(𝐷) = {𝑤 ∈ Σ∗ ∣ 𝛿(𝑞0, 𝑤) ∈ 𝐹}.




Regular Languages


𝐿(𝐷) = {𝑤 ∈ Σ∗ ∣ 𝛿(𝑞0, 𝑤) ∈ 𝐹}.


Examples∅ is a regular language.

Let Σ be an alphabet. Σ∗ is a regular language.


Regular Languages


𝐿(𝐷) = {𝑤 ∈ Σ∗ ∣ 𝛿(𝑞0, 𝑤) ∈ 𝐹}.




Regular Languages

ExampleLet 𝐿 be the set of words with both an even number of 0’s and an evennumber of 1’s. 𝐿 is a regular language.

𝑞0: Both the number of 0’s seenso far and the number of 1’sseen so far are even.𝑞1: The number of 0’s seen sofar is even, but the numberof 1’s seen so far is odd.𝑞2: The number of 1’s seen sofar is even, but the numberof 0’s seen so far is odd.𝑞3: Both the number of 0’s seenso far and the number of 1’sseen so far are odd.

𝑞0

start

𝑞1

𝑞2 𝑞3

1

01

001

0

1


Regular Languages

ExampleLet 𝐿 be the set of words with both an even number of 0’s and an evennumber of 1’s. 𝐿 is a regular language.

𝑞0: Both the number of 0’s seenso far and the number of 1’sseen so far are even.𝑞1: The number of 0’s seen sofar is even, but the numberof 1’s seen so far is odd.𝑞2: The number of 1’s seen sofar is even, but the numberof 0’s seen so far is odd.𝑞3: Both the number of 0’s seenso far and the number of 1’sseen so far are odd.

𝑞0

start

𝑞1

𝑞2 𝑞3

1

01

001

0

1Determinist Finite Automata 29/32

Functional Program View of Finite Automata

(Adapted from (Keller 2001))Each state of the automaton is identified with a function from Σ∗ toa truth value.The initial state is identified with the overall function of theautomaton.

ExampleSee the implementation for the representation functional of a DFA in thecourse homepage.


Functional Program View of Finite Automata

(Adapted from (Keller 2001))Each state of the automaton is identified with a function from Σ∗ toa truth value.The initial state is identified with the overall function of theautomaton.

ExampleSee the implementation for the representation functional of a DFA in thecourse homepage.


References

Greibach, S. A. (1981). Formal Languages: Origins and Directions. Annalsof History of Computing 3.1, pp. 14–41. doi: 10.1109/MAHC.1981.10006(cit. on p. 2).Hopcroft, J. E. and Ullman, J. D. (1979). Introduction to AutomataTheory, Languages, and Computation. Addison-Wesley Publishing Company(cit. on pp. 5, 7).Keller, R. M. (2001). Computer Science: Abstraction to Implementation.url: www.cs.hmc.edu/~keller/cs60book/ (visited on 07/02/2018) (cit. onpp. 30, 31).Levitin, A. (2006). Introduction to the Design and Analysis of Algorithms.2nd ed. Addison Wesley (cit. on p. 5).


https://doi.org/10.1109/MAHC.1981.10006

www.cs.hmc.edu/~keller/cs60book/

Automata and Formal Languages - CM0081Non-Deterministic Finite Automata


Universidad EAFIT

Semester 2018-1

Non-Deterministic Finite Automata (NFA)

Introduction

𝑞𝑖

𝑞𝑗

𝑞𝑘

⋮

a

a

a

The automaton can be in several states at onceFacilitates the design of automata

Question: Are the NFAs more powerful than the DFAs?

Non-Deterministic Finite Automata 2/38

Non-Deterministic Finite Automata (NFA)

Example (A NFA accepting all the binary strings that end in 01)


0, 1

0 1

𝑞0: The automaton “guess” that the final 01 has not begun.𝑞1: The automaton “guess” that the final 01 has begun.𝑞2: The word ends in 01.


Definition

DefinitionA non-deterministic finite automata (NFA) is a quintuple (𝑄, Σ, 𝛿, 𝑞0, 𝐹 ),where

𝑄: A finite set of statesΣ: An alphabet of input symbols

𝛿 ∶ 𝑄 × Σ → 𝒫(𝑄): A transition function𝑞0 ∈ 𝑄: A start state𝐹 ⊆ 𝑄: A set of accepting (or final) states


Extension of the Transition Function for NFAs

Let 𝑁 = (𝑄, Σ, 𝛿, 𝑞0, 𝐹 ) be a NFA. The extension of the transitionfunction, denoted by 𝛿, is recursively defined by

𝛿 ∶ 𝑄 × Σ∗ → 𝒫(𝑄)𝛿(𝑞, 𝜀) = {𝑞},

𝛿(𝑞, 𝑥𝑎) = ⋃𝑝∈ 𝛿(𝑞,𝑥)

𝛿(𝑝, 𝑎).


Language Accepted by a NFA

RecallLet 𝐷 = (𝑄, Σ, 𝛿, 𝑞0, 𝐹 ) be a DFA. Recall that the language acceptedby 𝐷 is

𝐿(𝐷) = {𝑤 ∈ Σ∗ ∣ 𝛿(𝑞0, 𝑤) ∈ 𝐹}.

DefinitionLet 𝑁 = (𝑄, Σ, 𝛿, 𝑞0, 𝐹 ) be a NFA. The language accepted by 𝑁 is

𝐿(𝑁) = {𝑤 ∈ Σ∗ ∣ 𝛿(𝑞0, 𝑤) ∩ 𝐹 ≠ ∅}.ReadingAn application: Text search (Hopcroft, Motwani and Ullman 2007, § 2.4).




𝐿(𝐷) = {𝑤 ∈ Σ∗ ∣ 𝛿(𝑞0, 𝑤) ∈ 𝐹}.


𝐿(𝑁) = {𝑤 ∈ Σ∗ ∣ 𝛿(𝑞0, 𝑤) ∩ 𝐹 ≠ ∅}.

ReadingAn application: Text search (Hopcroft, Motwani and Ullman 2007, § 2.4).




𝐿(𝐷) = {𝑤 ∈ Σ∗ ∣ 𝛿(𝑞0, 𝑤) ∈ 𝐹}.


𝐿(𝑁) = {𝑤 ∈ Σ∗ ∣ 𝛿(𝑞0, 𝑤) ∩ 𝐹 ≠ ∅}.ReadingAn application: Text search (Hopcroft, Motwani and Ullman 2007, § 2.4).



Example (2.9)For the NFA of the figure, 𝐿(𝑁) = {𝑤 ∈ {0, 1}∗ ∣ 𝑤 ends in 01}.


0, 1

0 1

Sketch of proofMutual induction on the following propositions:𝑆0(𝑤): 𝑞0 ∈ 𝛿(𝑞0, 𝑤) for all 𝑤 ∈ Σ∗

𝑆1(𝑤): 𝑞1 ∈ 𝛿(𝑞0, 𝑤) ⇔ 𝑤 ends in 0𝑆2(𝑤): 𝑞2 ∈ 𝛿(𝑞0, 𝑤) ⇔ 𝑤 ends in 01From 𝑆2(𝑤) and 𝐹 = {𝑞2} then the theorem follows.



Example (2.9)For the NFA of the figure, 𝐿(𝑁) = {𝑤 ∈ {0, 1}∗ ∣ 𝑤 ends in 01}.


0, 1

0 1

Sketch of proofMutual induction on the following propositions:𝑆0(𝑤): 𝑞0 ∈ 𝛿(𝑞0, 𝑤) for all 𝑤 ∈ Σ∗

𝑆1(𝑤): 𝑞1 ∈ 𝛿(𝑞0, 𝑤) ⇔ 𝑤 ends in 0𝑆2(𝑤): 𝑞2 ∈ 𝛿(𝑞0, 𝑤) ⇔ 𝑤 ends in 01From 𝑆2(𝑤) and 𝐹 = {𝑞2} then the theorem follows.



Example (Exercise 2.3.4.a)NFA accepting the set of strings over Σ = {1, 2, 3} such that the finaldigit has appeared before.

𝑖

start

𝑞2

𝑞1

𝑞3

𝑓

Σ 1

2

3

Σ − {1}

1Σ − {2}

2

Σ − {3} 3

𝑞𝑖: Theautomaton“guess” that therepeated digitis 𝑖.



Example (Exercise 2.3.4.a)NFA accepting the set of strings over Σ = {1, 2, 3} such that the finaldigit has appeared before.

𝑖

start

𝑞2

𝑞1

𝑞3

𝑓

Σ 1

2

3

Σ − {1}

1Σ − {2}

2

Σ − {3} 3

𝑞𝑖: Theautomaton“guess” that therepeated digitis 𝑖.



Example (Exercise 2.3.4.b)NFA accepting the set of strings over Σ = {0, 1, 2} such that the finaldigit has not appeared before.

𝑞0

𝑞1

𝑞2

𝑞𝑖

start

𝑞𝑓Σ − {0}

Σ − {1}

Σ − {2}

Σ

Σ − {0}

0

Σ − {1}1

Σ − {2}2



Example (Exercise 2.3.4.b)NFA accepting the set of strings over Σ = {0, 1, 2} such that the finaldigit has not appeared before.

𝑞0

𝑞1

𝑞2

𝑞𝑖

start

𝑞𝑓Σ − {0}

Σ − {1}

Σ − {2}

Σ

Σ − {0}

0

Σ − {1}1

Σ − {2}2



Example (Exercise 2.3.4.c)NFA accepting the set of strings over Σ = {0, 1} such that there aretwo 0’s separated by a number of positions that is multiple of 2. Notethat 0 is an allowable multiple of 2.

𝑞0

start

𝑞1 𝑞2

𝑞3 𝑞4 𝑞5 𝑞6

Σ

0

0

0

Σ

Σ Σ 0

1 Σ



Example (Exercise 2.3.4.c)NFA accepting the set of strings over Σ = {0, 1} such that there aretwo 0’s separated by a number of positions that is multiple of 2. Notethat 0 is an allowable multiple of 2.

𝑞0

start

𝑞1 𝑞2

𝑞3 𝑞4 𝑞5 𝑞6

Σ

0

0

0

Σ

Σ Σ 0

1 Σ



Example (Exercise 2.5.3.b)NFA accepting the set of strings that consist of either 01 repeated one ormore times or 010 repeated one or more times.

𝑞0

start

𝑞1 𝑞2

𝑞3 𝑞4 𝑞5

0

0

1

0

1 0

0



Example (Exercise 2.5.3.b)NFA accepting the set of strings that consist of either 01 repeated one ormore times or 010 repeated one or more times.

𝑞0

start

𝑞1 𝑞2

𝑞3 𝑞4 𝑞5

0

0

1

0

1 0

0Non-Deterministic Finite Automata 18/38

The Subset Construction

Input: A NFA 𝑁 = (𝑄𝑁 , Σ, 𝛿𝑁 , 𝑞0, 𝐹𝑁)Output: A DFA 𝐷 = (𝑄𝐷, Σ, 𝛿𝐷, {𝑞0}, 𝐹𝐷) where

𝑄𝐷 = 𝒫(𝑄𝑁),

𝐹𝐷 = {𝑆 ∈ 𝒫(𝑄𝑁) ∣ 𝑆 ∩ 𝐹𝑁 ≠ ∅},

𝛿𝐷(𝑆, 𝑎) = ⋃𝑝∈𝑆

𝛿𝑁(𝑝, 𝑎), for each 𝑆 ∈ 𝒫(𝑄𝑁) and 𝑎 ∈ Σ.



Input: A NFA 𝑁 = (𝑄𝑁 , Σ, 𝛿𝑁 , 𝑞0, 𝐹𝑁)Output: A DFA 𝐷 = (𝑄𝐷, Σ, 𝛿𝐷, {𝑞0}, 𝐹𝐷) where

𝑄𝐷 = 𝒫(𝑄𝑁),

𝐹𝐷 = {𝑆 ∈ 𝒫(𝑄𝑁) ∣ 𝑆 ∩ 𝐹𝑁 ≠ ∅},

𝛿𝐷(𝑆, 𝑎) = ⋃𝑝∈𝑆

𝛿𝑁(𝑝, 𝑎), for each 𝑆 ∈ 𝒫(𝑄𝑁) and 𝑎 ∈ Σ.



ExampleGiven the following NFA to build the DFA given by the subset construction.


0, 1

0 1



Example (cont.)

{𝑞0}

start

{𝑞0, 𝑞1} {𝑞0, 𝑞2}

{𝑞0, 𝑞1, 𝑞2} ∅ {𝑞1, 𝑞2}

{𝑞1} {𝑞2}

1

0

0

1

01

01

0, 1

0

1

0, 1

0

1



Example (cont.)

{𝑞0}

start

{𝑞0, 𝑞1} {𝑞0, 𝑞2}

{𝑞0, 𝑞1, 𝑞2} ∅ {𝑞1, 𝑞2}

{𝑞1} {𝑞2}

1

0

0

1

01

01

0, 1

0

1

0, 1

0

1



Example (cont.)

{𝑞0}start {𝑞0, 𝑞1} {𝑞0, 𝑞2}

1

0

0

1

01


Equivalence of DFAs and NFAs

Example (special case)A NFA recognising the word ‘then’:

𝑞0start t th the thent h e n

The DFA given by the subset construction:

{𝑞0}start {t} {th} {the} {then}

∅

t

Σ − {t}

h

Σ − {h}

e

Σ − {e}

n

Σ − {n}

Σ

Σ



Example (special case)A NFA recognising the word ‘then’:

𝑞0start t th the thent h e n

The DFA given by the subset construction:

{𝑞0}start {t} {th} {the} {then}

∅

t

Σ − {t}

h

Σ − {h}

e

Σ − {e}

n

Σ − {n}

Σ

ΣNon-Deterministic Finite Automata 26/38


Let 𝐷 = (𝑄𝐷, Σ, 𝛿𝐷, {𝑞0}, 𝐹𝐷) be the DFA constructed from a NFA𝑁 = (𝑄𝑁 , Σ, 𝛿𝑁 , 𝑞0, 𝐹𝑁) by the subset construction.

Theorem(Proof by structural induction on 𝑤)

𝛿𝐷({𝑞0}, 𝑤) = 𝛿𝑁(𝑞0, 𝑤).

Theorem (2.11)

𝐿(𝐷) = 𝐿(𝑁).



Let 𝐷 = (𝑄𝐷, Σ, 𝛿𝐷, {𝑞0}, 𝐹𝐷) be the DFA constructed from a NFA𝑁 = (𝑄𝑁 , Σ, 𝛿𝑁 , 𝑞0, 𝐹𝑁) by the subset construction.

Theorem(Proof by structural induction on 𝑤)

𝛿𝐷({𝑞0}, 𝑤) = 𝛿𝑁(𝑞0, 𝑤).

Theorem (2.11)

𝐿(𝐷) = 𝐿(𝑁).



Theorem (Exercise 2.3.5)Let 𝐷 = (𝑄, Σ, 𝛿𝐷, 𝑞0, 𝐹 ) be a DFA and let 𝑁 = (𝑄, Σ, 𝛿𝑁 , 𝑞0, 𝐹 ) be aNFA, where 𝛿𝑁 is defined by:

if 𝛿𝐷(𝑞, 𝑎) = 𝑝, then 𝛿𝑁(𝑞, 𝑎) = {𝑝}. (1)

Hence, if 𝛿𝐷(𝑞0, 𝑤) = 𝑝, then 𝛿𝑁(𝑞0, 𝑤) = {𝑝}.(Proof in the next slides)



Proof (by structural induction on 𝑤)

1. 𝑃(𝑤): If 𝛿𝐷(𝑞0, 𝑤) = 𝑝, then 𝛿𝑁(𝑞0, 𝑤) = {𝑝}.

2. Basis step 𝑃(𝜀):Since 𝛿𝐷(𝑞0, 𝜀) = 𝑞0 (by def. of 𝛿𝐷) and 𝛿𝑁(𝑞0, 𝜀) = {𝑞0} (by def.of 𝛿𝑁) then 𝑃(𝜀).



Proof (by structural induction on 𝑤)

1. 𝑃(𝑤): If 𝛿𝐷(𝑞0, 𝑤) = 𝑝, then 𝛿𝑁(𝑞0, 𝑤) = {𝑝}.

2. Basis step 𝑃(𝜀):Since 𝛿𝐷(𝑞0, 𝜀) = 𝑞0 (by def. of 𝛿𝐷) and 𝛿𝑁(𝑞0, 𝜀) = {𝑞0} (by def.of 𝛿𝑁) then 𝑃(𝜀).



Proof (cont.)3. Inductive step: ∀𝑥 ∈ Σ∗.∀𝑎 ∈ Σ. 𝑃(𝑥) ⇒ 𝑃(𝑥𝑎)

Inductive hypothesis 𝑃(𝑥): If 𝛿𝐷(𝑞0, 𝑥) = 𝑝, then 𝛿𝑁(𝑞0, 𝑥) = {𝑝}.

Let 𝛿𝐷(𝑞0, 𝑥) = 𝑝 and 𝛿𝐷(𝑝, 𝑎) = 𝑞, that is, 𝛿𝐷(𝑞0, 𝑥𝑎) = 𝑞.Hence

𝛿𝑁(𝑞0, 𝑥𝑎) = ⋃𝑝∈ 𝛿𝑁(𝑞0,𝑥)

𝛿𝑁(𝑝, 𝑎) (by def. of 𝛿𝑁)

= ⋃𝑝∈{𝑝}

𝛿𝑁(𝑝, 𝑎) (by IH)

= {𝑞} (by (1))

Therefore, if 𝛿𝐷(𝑞0, 𝑥𝑎) = 𝑞, then 𝛿𝑁(𝑞0, 𝑥𝑎) = {𝑞}.








= ⋃𝑝∈{𝑝}


= {𝑞} (by (1))






Let 𝛿𝐷(𝑞0, 𝑥) = 𝑝 and 𝛿𝐷(𝑝, 𝑎) = 𝑞, that is, 𝛿𝐷(𝑞0, 𝑥𝑎) = 𝑞.

Hence



= ⋃𝑝∈{𝑝}


= {𝑞} (by (1))









= ⋃𝑝∈{𝑝}


= {𝑞} (by (1))




Theorem (2.12)A language 𝐿 is accepted by some DFA if and only if 𝐿 is accepted by someNFA.

DefinitionA language 𝐿 is regular if exists a finite automaton 𝐴 (DFA or NFA) such𝐿 = 𝐿(𝐴).



Theorem (2.12)A language 𝐿 is accepted by some DFA if and only if 𝐿 is accepted by someNFA.

DefinitionA language 𝐿 is regular if exists a finite automaton 𝐴 (DFA or NFA) such𝐿 = 𝐿(𝐴).


References

Hopcroft, J. E., Motwani, R. and Ullman, J. D. (2007). Introduction toAutomata theory, Languages, and Computation. 3rd ed. Pearson Education(cit. on pp. 6–8).


Automata and Formal Languages - CM0081Finite Automata with 𝜀-Transitions


Universidad EAFIT

Semester 2018-1

Finite Automata with 𝜀-Transitions (𝜀-NFA)

FeaturesTo allow a transition with 𝜀, the empty stringFacilitate the design of the automata

QuestionAre the 𝜀-NFAs more powerful than the [D|N]FAs?

Finite Automata with 𝜀-Transitions 2/16

Finite Automata with 𝜀-Transitions (𝜀-NFA)

FeaturesTo allow a transition with 𝜀, the empty stringFacilitate the design of the automata

QuestionAre the 𝜀-NFAs more powerful than the [D|N]FAs?


𝜀-NFA

ExampleAn 𝜀-NFA that accepts decimal numbers consisting of:

i) an optional + or − sign,ii) a string of digits,iii) a decimal point andiv) another string of digits. Either this string of digits, or the string (ii)

can be empty, but at least one of the two strings must be non-empty.

Figure: Next slide


𝜀-NFA

Example (cont.)

𝑞0

start

𝑞1 𝑞2 𝑞3

𝑞4

𝑞5𝜀, +, −

0, 1, … , 9

.

0, 1, … , 9

0, 1, … , 9

0, 1, … , 9

𝜀

.


Definition

DefinitionA finite automata with 𝜀-transitions (𝜀-NFA) is a quintuple (𝑄, Σ, 𝛿, 𝑞0, 𝐹 ),where

𝑄: A finite set of statesΣ: An alphabet of input symbols

𝛿 → 𝑄 × Σ ∪ {𝜀} → 𝒫(𝑄): A transition function𝑞0 ∈ 𝑄: A start state𝐹 ⊆ 𝑄: A set of accepting (or final) states


Epsilon-Closures

𝜀-close a state 𝑞: Find all states reachable from 𝑞 by a sequence 𝜀 ⋯ 𝜀.

DefinitionThe ECLOSE ∶ 𝑄 → 𝒫(𝑄) function is inductively defined by:

Basis step: 𝑞 ∈ ECLOSE(𝑞), for all 𝑞 ∈ 𝑄.Inductive step: If 𝑝 ∈ ECLOSE(𝑞) and 𝛿(𝑝, 𝜀) = 𝑟, then𝑟 ∈ ECLOSE(𝑞).

Example

1

2 3

4 5

6

7

𝜀

𝜀

𝜀 𝜀

a

b

𝜀


Epsilon-Closures




Example

1

2 3

4 5

6

7

𝜀

𝜀

𝜀 𝜀

a

b

𝜀


Epsilon-Closures




Example

1

2 3

4 5

6

7

𝜀

𝜀

𝜀 𝜀

a

b

𝜀


Epsilon-Closures

DefinitionLet 𝑆 be a set of states. The 𝜀-closure of 𝑆 is defined by

ECLOSE(𝑆) = ⋃𝑞∈𝑆

ECLOSE(𝑞).


Extension of the Transition Function for 𝜀-NFAs

Let 𝐸 = (𝑄, Σ, 𝛿, 𝑞0, 𝐹 ) be an 𝜀-NFA. The extension of the transitionfunction, 𝛿 ∶ 𝑄 × Σ∗ → 𝒫(𝑄) is recursively defined by

𝛿(𝑞, 𝜀) = ECLOSE(𝑞),𝛿(𝑞, 𝑥𝑎) = ECLOSE({𝑟1, 𝑟2, … , 𝑟𝑚}),

where

𝛿(𝑞, 𝑥) = {𝑝1, 𝑝2, … , 𝑝𝑘},𝑘

⋃𝑖=1

𝛿(𝑝𝑖, 𝑎) = {𝑟1, 𝑟2, … , 𝑟𝑚}.


Language Accepted by an 𝜀-NFA

RecallLet 𝐷 = (𝑄, Σ, 𝛿, 𝑞0, 𝐹 ) and 𝑁 = (𝑄, Σ, 𝛿, 𝑞0, 𝐹 ) be a DFA and a NFA,respectively. Recall that the languages accepted by 𝐷 and 𝑁 are

𝐿(𝐷) = {𝑤 ∈ Σ∗ ∣ 𝛿(𝑞0, 𝑤) ∈ 𝐹},𝐿(𝑁) = {𝑤 ∈ Σ∗ ∣ 𝛿(𝑞0, 𝑤) ∩ 𝐹 ≠ ∅}.

DefinitionLet 𝐸 = (𝑄, Σ, 𝛿, 𝑞0, 𝐹 ) be an 𝜀-NFA. The language accepted by 𝐸 is

𝐿(𝐸) = {𝑤 ∈ Σ∗ ∣ 𝛿(𝑞0, 𝑤) ∩ 𝐹 ≠ ∅}.


Language Accepted by an 𝜀-NFA

RecallLet 𝐷 = (𝑄, Σ, 𝛿, 𝑞0, 𝐹 ) and 𝑁 = (𝑄, Σ, 𝛿, 𝑞0, 𝐹 ) be a DFA and a NFA,respectively. Recall that the languages accepted by 𝐷 and 𝑁 are

𝐿(𝐷) = {𝑤 ∈ Σ∗ ∣ 𝛿(𝑞0, 𝑤) ∈ 𝐹},𝐿(𝑁) = {𝑤 ∈ Σ∗ ∣ 𝛿(𝑞0, 𝑤) ∩ 𝐹 ≠ ∅}.

DefinitionLet 𝐸 = (𝑄, Σ, 𝛿, 𝑞0, 𝐹 ) be an 𝜀-NFA. The language accepted by 𝐸 is

𝐿(𝐸) = {𝑤 ∈ Σ∗ ∣ 𝛿(𝑞0, 𝑤) ∩ 𝐹 ≠ ∅}.


Equivalence of DFAs, NFAs and 𝜀-NFAs

DFA NFA

𝜀-NFA


Regular Languages

DefinitionA language 𝐿 is regular if exists a finite automaton 𝐴 (DFA, NFA or 𝜀-NFA)such 𝐿 = 𝐿(𝐴).

Exercise (2.5.3.a)Let 𝐿 be the set of strings consisting of zero or more 𝑎’s followed by zeroor more 𝑏’s, followed by zero or more 𝑐’s. Prove that 𝐿 is a regularlanguage.


Regular Languages

DefinitionA language 𝐿 is regular if exists a finite automaton 𝐴 (DFA, NFA or 𝜀-NFA)such 𝐿 = 𝐿(𝐴).

Exercise (2.5.3.a)Let 𝐿 be the set of strings consisting of zero or more 𝑎’s followed by zeroor more 𝑏’s, followed by zero or more 𝑐’s. Prove that 𝐿 is a regularlanguage.


Automata and Formal Languages - CM0081Regular Expressions


Universidad EAFIT

Semester 2018-1

Introduction

Finite automataMachine-like descriptions of regular languages.

Regular expressionsAlgebraic description of regular languagesDeclarative (“user-friendly”) way to express the strings that belong tothe languageExample: (01)(01)∗ + (010)(010)∗Uses

Search commands (e.g. Grep)Lexical-analyzer generators (e.g. Lex and Alex)Domain specific languages (DSLs)

Regular Expressions 2/40

Introduction


Regular expressionsAlgebraic description of regular languages

Declarative (“user-friendly”) way to express the strings that belong tothe languageExample: (01)(01)∗ + (010)(010)∗Uses



Introduction


Regular expressionsAlgebraic description of regular languagesDeclarative (“user-friendly”) way to express the strings that belong tothe language

Example: (01)(01)∗ + (010)(010)∗Uses



Introduction


Regular expressionsAlgebraic description of regular languagesDeclarative (“user-friendly”) way to express the strings that belong tothe languageExample: (01)(01)∗ + (010)(010)∗

UsesSearch commands (e.g. Grep)Lexical-analyzer generators (e.g. Lex and Alex)Domain specific languages (DSLs)


Introduction



Search commands (e.g. Grep)

Lexical-analyzer generators (e.g. Lex and Alex)Domain specific languages (DSLs)


Introduction



Search commands (e.g. Grep)Lexical-analyzer generators (e.g. Lex and Alex)

Domain specific languages (DSLs)


Introduction





Operations on Languages

Let 𝐿 and 𝑀 be languages:

Union 𝐿 ∪𝑀 = {𝑤 ∣ 𝑤 ∈ 𝐿 or 𝑤 ∈ 𝑀}

Concatenation 𝐿.𝑀 = {𝑤 ∣ 𝑤 = 𝑥𝑦, 𝑥 ∈ 𝐿 and 𝑦 ∈ 𝑀}

Powers 𝐿0 = {𝜀}𝐿1 = 𝐿

𝐿𝑘+1 = 𝐿.𝐿𝑘

Kleene closure 𝐿∗ =∞⋃𝑖=0

𝐿𝑖



ExamplesIf 𝐿 = {0, 1}, then 𝐿∗ consists of all strings of 0’s and 1’s and theempty word.

If 𝐿 = {0𝑛 ∣ 𝑛 ≥ 1}, then 𝐿∗ = 𝐿 ∪ {𝜀}.If 𝐿 = {0, 11}, then 𝐿∗ consists of the empty word and those stringsof 0’s and 1’s such that the 1’s come in pairs.∅0 = {𝜀}∅𝑖 = ∅, for 𝑖 ≥ 1∅∗ = {𝜀}



ExamplesIf 𝐿 = {0, 1}, then 𝐿∗ consists of all strings of 0’s and 1’s and theempty word.If 𝐿 = {0𝑛 ∣ 𝑛 ≥ 1}, then 𝐿∗ = 𝐿 ∪ {𝜀}.

If 𝐿 = {0, 11}, then 𝐿∗ consists of the empty word and those stringsof 0’s and 1’s such that the 1’s come in pairs.∅0 = {𝜀}∅𝑖 = ∅, for 𝑖 ≥ 1∅∗ = {𝜀}



ExamplesIf 𝐿 = {0, 1}, then 𝐿∗ consists of all strings of 0’s and 1’s and theempty word.If 𝐿 = {0𝑛 ∣ 𝑛 ≥ 1}, then 𝐿∗ = 𝐿 ∪ {𝜀}.If 𝐿 = {0, 11}, then 𝐿∗ consists of the empty word and those stringsof 0’s and 1’s such that the 1’s come in pairs.

∅0 = {𝜀}∅𝑖 = ∅, for 𝑖 ≥ 1∅∗ = {𝜀}



ExamplesIf 𝐿 = {0, 1}, then 𝐿∗ consists of all strings of 0’s and 1’s and theempty word.If 𝐿 = {0𝑛 ∣ 𝑛 ≥ 1}, then 𝐿∗ = 𝐿 ∪ {𝜀}.If 𝐿 = {0, 11}, then 𝐿∗ consists of the empty word and those stringsof 0’s and 1’s such that the 1’s come in pairs.∅0 = {𝜀}∅𝑖 = ∅, for 𝑖 ≥ 1∅∗ = {𝜀}


Syntax: What the Regular Expressions Are

DefinitionLet Σ be an alphabet. The regular expressions on Σ are inductivelydefined by:

Basis step: 𝜀 and ∅ are regular expressions. If 𝑎 ∈ Σ then a is aregular expression.Inductive step: Let 𝐸 and 𝐹 be regular expressions. Then 𝐸 + 𝐹 ,𝐸𝐹 , 𝐸∗ and (𝐸) are regular expressions.




Basis step: 𝜀 and ∅ are regular expressions. If 𝑎 ∈ Σ then a is aregular expression.

Inductive step: Let 𝐸 and 𝐹 be regular expressions. Then 𝐸 + 𝐹 ,𝐸𝐹 , 𝐸∗ and (𝐸) are regular expressions.




Basis step: 𝜀 and ∅ are regular expressions. If 𝑎 ∈ Σ then a is aregular expression.Inductive step: Let 𝐸 and 𝐹 be regular expressions. Then 𝐸 + 𝐹 ,𝐸𝐹 , 𝐸∗ and (𝐸) are regular expressions.


Semantics: What Language Denotes a Regular Expression

DefinitionLet 𝐸 be a regular expression. The language denoted by 𝐸, denotedby 𝐿(𝐷), is inductively defined by:

Basis step:𝐿(𝜀) = {𝜀},𝐿(∅) = ∅,𝐿(a) = {𝑎}.

Inductive step: Let 𝐿(𝐸) and 𝐿(𝐹) be the languages denoted by theregular expressions 𝐸 and 𝐹 , then

𝐿(𝐸 + 𝐹) = 𝐿(𝐸) ∪ 𝐿(𝐹),𝐿(𝐸𝐹) = 𝐿(𝐸)𝐿(𝐹),𝐿(𝐸∗) = (𝐿(𝐸))∗,𝐿((𝐸)) = 𝐿(𝐸).














Precedence of Operators

Order of precedence, from highest to lowest: (), ∗, . and +.The operators . and + are left-associative.

Example01∗ + 1 = (0(1∗)) + 1

≠ (01)∗ + 1≠ 0(1∗ + 1)


Precedence of Operators

Order of precedence, from highest to lowest: (), ∗, . and +.The operators . and + are left-associative.

Example01∗ + 1 = (0(1∗)) + 1

≠ (01)∗ + 1≠ 0(1∗ + 1)


Languages Denoted by Regular Expressions

𝐸 𝐿(𝐸)a + b 𝐿(a) ∪ 𝐿(b) = {𝑎} ∪ {𝑏} = {𝑎, 𝑏}

a∗ {𝜀, 𝑎, 𝑎𝑎, 𝑎𝑎𝑎,…}

(a + b)(a + b) 𝐿(a + b).𝐿(a + b) ={𝑎, 𝑏}.{𝑎, 𝑏} = {𝑎𝑎, 𝑎𝑏, 𝑏𝑎, 𝑏𝑏}

a + (ab)∗ {𝑎, 𝜀, 𝑎𝑏, 𝑎𝑏𝑎𝑏, 𝑎𝑏𝑎𝑏𝑎𝑏,…}

(0 + 1)∗01(0 + 1)∗ {𝑥01𝑦 ∣ 𝑥, 𝑦 ∈ {0, 1}∗}

ai(a1 + a2 +⋯+ an)∗ {𝑤 ∈ Σ∗ ∣ 𝑤 starts by 𝑎𝑖}



𝐸 𝐿(𝐸)a + b 𝐿(a) ∪ 𝐿(b) = {𝑎} ∪ {𝑏} = {𝑎, 𝑏}




(0 + 1)∗01(0 + 1)∗ {𝑥01𝑦 ∣ 𝑥, 𝑦 ∈ {0, 1}∗}




𝐸 𝐿(𝐸)a + b 𝐿(a) ∪ 𝐿(b) = {𝑎} ∪ {𝑏} = {𝑎, 𝑏}




(0 + 1)∗01(0 + 1)∗ {𝑥01𝑦 ∣ 𝑥, 𝑦 ∈ {0, 1}∗}




𝐸 𝐿(𝐸)a + b 𝐿(a) ∪ 𝐿(b) = {𝑎} ∪ {𝑏} = {𝑎, 𝑏}




(0 + 1)∗01(0 + 1)∗ {𝑥01𝑦 ∣ 𝑥, 𝑦 ∈ {0, 1}∗}




𝐸 𝐿(𝐸)a + b 𝐿(a) ∪ 𝐿(b) = {𝑎} ∪ {𝑏} = {𝑎, 𝑏}




(0 + 1)∗01(0 + 1)∗ {𝑥01𝑦 ∣ 𝑥, 𝑦 ∈ {0, 1}∗}




𝐸 𝐿(𝐸)a + b 𝐿(a) ∪ 𝐿(b) = {𝑎} ∪ {𝑏} = {𝑎, 𝑏}




(0 + 1)∗01(0 + 1)∗ {𝑥01𝑦 ∣ 𝑥, 𝑦 ∈ {0, 1}∗}

ai(a1 + a2 +⋯+ an)∗ {𝑤 ∈ Σ∗ ∣ 𝑤 starts by 𝑎𝑖}Regular Expressions 27/40


ExampleWrite a regular expression for the language 𝐿 defined by

𝐿 = {𝑤 ∈ {0, 1}∗ ∣ 0 and 1 alternate in 𝑤}.

Solution: (01)∗ + (10)∗ + 0(10)∗ + 1(01)∗

Other solution: (𝜀 + 1)(01)∗(𝜀 + 0)

ExampleThe regular expression

(10 + 0)∗(𝜀 + 1)denotes the set of strings of 0’s and 1’s that have no two adjacent 1’s.





Solution: (01)∗ + (10)∗ + 0(10)∗ + 1(01)∗








Solution: (01)∗ + (10)∗ + 0(10)∗ + 1(01)∗








Solution: (01)∗ + (10)∗ + 0(10)∗ + 1(01)∗






ExampleWrite a regular expression for denoting the set of strings over Σ = {0, 1}not ending in 01.

Solution: 𝜀 + 0 + 1 + (0 + 1)∗(00 + 10 + 11)



ExampleWrite a regular expression for denoting the set of strings over Σ = {0, 1}not ending in 01.Solution: 𝜀 + 0 + 1 + (0 + 1)∗(00 + 10 + 11)


Libraries

Note: Theoretical regular expressions ≠ practical regular expressions.

Some programming languages with support to regular expressions.NET, C, Haskell, Java, Mathematica, MATLAB and Perl.


Libraries

Note: Theoretical regular expressions ≠ practical regular expressions.

Some programming languages with support to regular expressions.NET, C, Haskell, Java, Mathematica, MATLAB and Perl.


Algorithms

AlgorithmsSee the Haskell implementation of some algorithms on regular expressionsin the course homepage.


Applications

Some programs that use regular expressionsGrep: print lines matching a patternAwk: pattern scanning and processing languageSed: stream editor for filtering and transforming textAlex, Flex and Lex: lexical-analyzer generatorsEmacs and Vim: test editorsMySQL, Oracle and PostgreSQL: databases


Applications

ReadingApplications of Regular Expressions (Hopcroft, Motwani and Ullman 2007,§ 3.3).

𝐿+ def= 𝐿𝐿∗ (one or many times operator)

𝐿? def= 𝜀 + 𝐿 (zero or one time operator)


An Implementation: A Regular Expression Matcher

“Rob’s implementation itself is asuperb example of beautiful code:compact, elegant, efficient, anduseful. It’s one of the best examplesof recursion that I have ever seen.”

Brian Kernighan, p. 3.


References

Hopcroft, J. E., Motwani, R. and Ullman, J. D. (2007). Introduction toAutomata theory, Languages, and Computation. 3rd ed. Pearson Education(cit. on p. 38).


Automata and Formal Languages - CM0081Finite Automata and Regular Expressions


Universidad EAFIT

Semester 2018-1

Introduction

Equivalences

DFA NFA

𝜀-NFA RE

Finite Automata and Regular Expressions 2/40

From Finite Automata to Regular Expressions

Theorem (3.4)If 𝐿 = 𝐿(𝐷) for some DFA 𝐷, then there is a regular expression 𝑅 suchthat 𝐿 = 𝐿(𝑅).



Inductive proof on the number of states of the automatonLet the states of 𝐷 be {1, 2, … , 𝑛} with 1 the start state.

𝑅𝑘𝑖𝑗: Regular expression describing the set of labels of all paths in 𝐷

from state 𝑖 to state 𝑗 such that the path has no intermediate nodewhose number is greater than 𝑘.†

†Figure from Hopcroft, Motwani and Ullman (2007, Fig. 3.2).Finite Automata and Regular Expressions 4/40


Inductive proof on the number of states of the automatonLet the states of 𝐷 be {1, 2, … , 𝑛} with 1 the start state.𝑅𝑘

𝑖𝑗: Regular expression describing the set of labels of all paths in 𝐷from state 𝑖 to state 𝑗 such that the path has no intermediate nodewhose number is greater than 𝑘.†



Basis step: Proof for 𝑘 = 0 (i.e. no intermediate states) and 𝑖 ≠ 𝑗

No transition from state 𝑖 to state 𝑗

𝑖 𝑗 𝑅0𝑖𝑗 = ∅

One transition from state 𝑖 to state 𝑗

𝑖 𝑗a 𝑅0𝑖𝑗 = a

Various transitions from state 𝑖 to state 𝑗

𝑖 𝑗a1…

a𝑛

𝑅0𝑖𝑗 = a1 + ⋯ + an



Basis step: Proof for 𝑘 = 0 (i.e. no intermediate states) and 𝑖 ≠ 𝑗No transition from state 𝑖 to state 𝑗





𝑖 𝑗a1…

a𝑛

𝑅0𝑖𝑗 = a1 + ⋯ + an








𝑖 𝑗a1…

a𝑛

𝑅0𝑖𝑗 = a1 + ⋯ + an








𝑖 𝑗a1…

a𝑛

𝑅0𝑖𝑗 = a1 + ⋯ + an



Basis step: Proof for 𝑘 = 0 (i.e. no intermediate states) and 𝑖 = 𝑗

No loops

𝑖 𝑅0𝑖𝑖 = 𝜀

One loop

𝑖a 𝑅0𝑖𝑖 = 𝜀 + a

Question: Why do not to define 𝑅0𝑖𝑖 = a∗ ?

Various loops

𝑖a1

…

a𝑛𝑅0

𝑖𝑖 = 𝜀 + a1 + ⋯ + an



Basis step: Proof for 𝑘 = 0 (i.e. no intermediate states) and 𝑖 = 𝑗No loops


One loop



Various loops

𝑖a1

…

a𝑛𝑅0

𝑖𝑖 = 𝜀 + a1 + ⋯ + an





One loop



Various loops

𝑖a1

…

a𝑛𝑅0

𝑖𝑖 = 𝜀 + a1 + ⋯ + an





One loop



Various loops

𝑖a1

…

a𝑛𝑅0

𝑖𝑖 = 𝜀 + a1 + ⋯ + an





One loop



Various loops

𝑖a1

…

a𝑛𝑅0

𝑖𝑖 = 𝜀 + a1 + ⋯ + an



Inductive step: Proof for 𝑘Inductive hypothesis 𝑅𝑘−1

𝑖𝑗 : path from state 𝑖 to state 𝑗 that goes throughno state higher than 𝑘 − 1.

The path does not go through state 𝑘 at all

𝑖 𝑘 𝑗 𝑅𝑘𝑖𝑗 = 𝑅𝑘−1

𝑖𝑗

The path goes through state 𝑘 at least once

𝑖 𝑘 𝑘 𝑘 𝑗𝑅𝑘

𝑖𝑗 = 𝑅𝑘−1𝑖𝑘 (𝑅𝑘−1

𝑘𝑘 )∗𝑅𝑘−1𝑘𝑗

From the previous cases:

𝑅𝑘𝑖𝑗 = 𝑅𝑘−1

𝑖𝑗 + 𝑅𝑘−1𝑖𝑘 (𝑅𝑘−1








𝑖𝑗















𝑖𝑗















𝑖𝑗











Given that the states of 𝐷 are {1, 2, … , 𝑛} with 1 the start state, then

𝐿(𝐷) = 𝐿(𝑅𝑛1𝑓1

+ ⋯ + 𝑅𝑛1𝑓𝑚

) with 𝑓𝑖 ∈ 𝐹.



ExampleTo convert the DFA 𝐷 to a regular expression.

1start 2

10 0, 1

𝐿(𝐷) = {𝑤 ∈ {0, 1}∗ ∣ 𝑤 has at least one 0}.



Example (cont.)

1start 2

10 0, 1

𝑅0𝑖𝑗

𝑅0𝑖𝑗 Regexp

𝑅011 𝜀 + 1

𝑅012 0

𝑅021 ∅

𝑅022 𝜀 + 0 + 1



Example (cont.)𝑅1

𝑖𝑗 = 𝑅0𝑖𝑗 + 𝑅0

𝑖1(𝑅011)∗𝑅0

1𝑗


𝑅011 𝜀 + 1

𝑅012 0

𝑅021 ∅

𝑅022 𝜀 + 0 + 1


𝑅111 (𝜀 + 1) + (𝜀 + 1)(𝜀 + 1)∗(𝜀 + 1)

𝑅112 0 + (𝜀 + 1)(𝜀 + 1)∗0

𝑅121 ∅ + ∅(𝜀 + 1)∗(𝜀 + 1)

𝑅122 𝜀 + 0 + 1 + ∅(𝜀 + 1)∗0



Example (cont.)Some simplifications for regular expressionsLet 𝐿 and 𝑀 be regular expression variables.

(𝜀 + 𝐿)∗ = 𝐿∗

(𝜀 + 𝐿)𝐿∗ = 𝐿∗

𝐿 + 𝑀∗𝐿 = 𝑀∗𝐿𝐿∅ = ∅𝐿 = ∅ (∅ is the annihilator for concatenation)

𝐿 + ∅ = ∅ + 𝐿 = 𝐿 (∅ is the identity for union)





𝑖1(𝑅011)∗𝑅0

1𝑗

𝑅1𝑖𝑗 Regexp Simplified

𝑅111 (𝜀 + 1) + (𝜀 + 1)(𝜀 + 1)∗(𝜀 + 1) 1∗

𝑅112 0 + (𝜀 + 1)(𝜀 + 1)∗0 1∗0

𝑅121 ∅ + ∅(𝜀 + 1)∗(𝜀 + 1) ∅

𝑅122 𝜀 + 0 + 1 + ∅(𝜀 + 1)∗0 𝜀 + 0 + 1





𝑖2(𝑅122)∗𝑅1

2𝑗


𝑅111 1∗

𝑅112 1∗0

𝑅121 ∅

𝑅122 𝜀 + 0 + 1


𝑅211 1∗ + 1∗0(𝜀 + 0 + 1)∗∅

𝑅212 1∗0 + 1∗0(𝜀 + 0 + 1)∗(𝜀 + 0 + 1)

𝑅221 ∅ + (𝜀 + 0 + 1)(𝜀 + 0 + 1)∗∅

𝑅222 𝜀 + 0 + 1 + (𝜀 + 0 + 1)(𝜀 + 0 + 1)∗(𝜀 + 0 + 1)





𝑖2(𝑅122)∗𝑅1

2𝑗

𝑅2𝑖𝑗 Regexp Simplified

𝑅211 1∗ + 1∗0(𝜀 + 0 + 1)∗∅ 1∗

𝑅212 1∗0 + 1∗0(𝜀 + 0 + 1)∗(𝜀 + 0 + 1) 1∗0(0 + 1)∗

𝑅221 ∅ + (𝜀 + 0 + 1)(𝜀 + 0 + 1)∗∅ ∅

𝑅222 𝜀 + 0 + 1 + (𝜀 + 0 + 1)(𝜀 + 0 + 1)∗(𝜀 + 0 + 1) (0 + 1)∗



Example (cont.)The regular expression equivalent to the automaton 𝐷 is

𝑅212 = 1∗0(0 + 1)∗.


From Regular Expressions to Finite Automata

Theorem (3.7)

Every language defined by a regular expression is also defined by a finiteautomaton.

Inductive proofFor each regular expression 𝑅 we construct an 𝜀-NFA 𝐸 such that 𝐿(𝑅) =𝐿(𝐸) with:

1. exactly one accepting state,2. no arcs into the initial state and3. no arcs out of the accepting state.



Theorem (3.7)

Every language defined by a regular expression is also defined by a finiteautomaton.

Inductive proofFor each regular expression 𝑅 we construct an 𝜀-NFA 𝐸 such that 𝐿(𝑅) =𝐿(𝐸) with:

1. exactly one accepting state,2. no arcs into the initial state and3. no arcs out of the accepting state.



Basis step: Automata for 𝜀 (a), ∅ (b) and a (c).†

From regex’s to ε-NFA’s

Theorem 3.7: For every regex R we can con-

struct and ε-NFA A, s.t. L(A) = L(R).

Proof: By structural induction:

Basis: Automata for ε, ∅, and a.

ε

a

(a)

(b)

(c)

73



Inductive step: Automaton for 𝑅 + 𝑆.†�� FINITE AUTOMATA AND REGULAR EXPRESSIONS ��

(a)

(b)

(c)

R

S

R S

R

ε ε

εε

ε

ε

ε

ε ε

Figure �� The inductive step in the regularexpressionto�NFA construction

automaton of Fig� ��c�� That automaton allows us to go either�

�a� Directly from the start state to the accepting state along a pathlabeled �� That path lets us accept �� which is in L�R�� no matterwhat expression R is�

�b� To the start state of the automaton for R� through that automatonone or more times� and then to the accepting state� This set of pathsallows us to accept strings in L�R�� L�R�L�R�� L�R�L�R�L�R�� andso on� thus covering all strings in L�R�� except perhaps �� which wascovered by the direct arc to the accepting state mentioned in ��a��

�� The expression is �R� for some smaller expression R� The automatonfor R also serves as the automaton for �R�� since the parentheses do notchange the language de ned by the expression�

It is a simple observation that the constructed automata satisfy the three conditions given in the inductive hypothesis � one accepting state� with no arcsinto the initial state or out of the accepting state� �



Inductive step: Automaton for 𝑅𝑆.†

�� FINITE AUTOMATA AND REGULAR EXPRESSIONS ��

(a)

(b)

(c)

R

S

R S

R

ε ε

εε

ε

ε

ε

ε ε









Inductive step: Automaton for 𝑅∗.†

�� FINITE AUTOMATA AND REGULAR EXPRESSIONS ��

(a)

(b)

(c)

R

S

R S

R

ε ε

εε

ε

ε

ε

ε ε









ExampleConvert the regular expression 𝑅 = (0 + 1)∗1(0 + 1) to an 𝜀-NFA 𝐸 such𝐿(𝑅) = 𝐿(𝐸).

1. 𝜀-NFA for 0 + 1:

start𝜀

𝜀

0 𝜀

1𝜀



ExampleConvert the regular expression 𝑅 = (0 + 1)∗1(0 + 1) to an 𝜀-NFA 𝐸 such𝐿(𝑅) = 𝐿(𝐸).

1. 𝜀-NFA for 0 + 1:

start𝜀

𝜀

0 𝜀

1𝜀



Example (cont.)2. 𝜀-NFA for (0 + 1)∗:

start 𝜀 𝜀

𝜀

0 𝜀

1𝜀

𝜀

𝜀

𝜀



Example (cont.)3. 𝜀-NFA for (0 + 1)∗1:

start 𝜀 𝜀

𝜀

0 𝜀

1𝜀

𝜀

𝜀

𝜀

𝜀 1



Example (cont.)4. 𝜀-NFA for (0 + 1)∗1(0 + 1):

start 𝜀 𝜀

𝜀

0 𝜀

1𝜀

𝜀

𝜀

𝜀

𝜀 1 𝜀 𝜀

𝜀

0 𝜀

1𝜀



Exercise (3.2.7)There are some simplifications to the constructions of Theorem 3.7, where weconverted a regular expression to an 𝜀-NFA. Here are three:

1. For the union operator, instead of creating new start and accepting states,merge the two start states into one state with all the transitions of bothstart states. Likewise, merge the two accepting states, having all transitionsto either go to the merged state instead.

2. For the concatenation operator, merge the accepting state of the first auto-maton with the start state of the second.

3. For the closure operator, simply add 𝜀-transitions from the accepting state tothe start state and vice-versa.

Each of these simplifications, by themselves, still yield a correct construction; thatis, the resulting 𝜀-NFA for any regular expression accepts the language of the expres-sion. Which subsets of changes (1), (2) and (3) may be made to the constructiontogether, while still yielding a correct automaton for every regular expression?


References

Hopcroft, J. E., Motwani, R. and Ullman, J. D. (2007). Introduction toAutomata theory, Languages, and Computation. 3rd ed. Pearson Education(cit. on pp. 4, 5, 30–33).


Automata and Formal Languages - CM0081Algebraic Laws for Regular Expressions


Universidad EAFIT

Semester 2018-1

Algebraic Laws for Regular Expressions

DefinitionTwo regular expressions with variables are equivalent if whatever languageswe substitute for the variables, the results of the two expressions are thesame language.

NotationLet 𝐿, 𝑀 and 𝑁 be regular expression variables.

Sugar syntax

𝐿+ def= 𝐿𝐿∗,𝐿? def= 𝜀 + 𝐿.

Algebraic Laws for Regular Expressions 2/33




Sugar syntax






Sugar syntax




Some laws for union

(𝐿 +𝑀) +𝑁 = 𝐿+ (𝑀 +𝑁) (associativity)𝐿 + ∅ = ∅ + 𝐿 = 𝐿 (identity)

𝐿 +𝑀 = 𝑀 +𝐿 (commutativity)𝐿 + 𝐿 = 𝐿 (idempotence)

RemarkThere is no inverse for union.



Some laws for concatenation

(𝐿𝑀)𝑁 = 𝐿(𝑀𝑁) (associativity)𝐿𝜀 = 𝜀𝐿 = 𝐿 (identity)

𝐿𝑀 ≠ 𝑀𝐿 (non-commutativity)𝐿∅ = ∅𝐿 = ∅ (∅ is the annihilator for concatenation)

RemarkThere is no inverse for concatenation.



Some laws for union and concatenation

𝐿(𝑀 +𝑁) = 𝐿𝑀 +𝐿𝑁 (distributive)(𝐿 +𝑀)𝑁 = 𝐿𝑁 + 𝐿𝑀 (distributive)



Some laws for closure

(𝐿∗)∗ = 𝐿∗ (idempotence)∅∗ = 𝜀𝜀∗ = 𝜀

(𝜀 + 𝐿)∗ = 𝐿∗

𝐿∗ = 𝐿+ + 𝜀


Simplification of Regular Expressions

Example

0 + (𝜀 + 1)(𝜀 + 1)∗0 = 0 + (𝜀 + 1)1∗0 ((𝜀 + 𝐿)∗ = 𝐿∗)= 0 + (𝜀1∗ + 11∗)0 (distributive)= 0 + (1∗ + 11∗)0 (identity)= 0 + (1∗ + 1+)0 (def. 𝐿+)= 0 + 1∗0 (equivalence)= 1∗0 (equivalence)


Discovering Laws for Regular Expressions

MethodLet 𝐸 and 𝐹 be two regular expressions with the same set of variables{𝐿1,… , 𝐿𝑛}.To test if 𝐸 = 𝐹 :

1. Convert 𝐸 and 𝐹 to concrete regular expressions 𝐶 and 𝐷, replacingeach 𝐿𝑖 by a different symbol a𝑖, for 𝑖 = 1, 2,… , 𝑛.

2. Test whether 𝐿(𝐶) = 𝐿(𝐷). If so, then 𝐸 = 𝐹 , and if not 𝐸 ≠ 𝐹 .

ObservationWe are proving by example!



MethodLet 𝐸 and 𝐹 be two regular expressions with the same set of variables{𝐿1,… , 𝐿𝑛}.To test if 𝐸 = 𝐹 :

1. Convert 𝐸 and 𝐹 to concrete regular expressions 𝐶 and 𝐷, replacingeach 𝐿𝑖 by a different symbol a𝑖, for 𝑖 = 1, 2,… , 𝑛.

2. Test whether 𝐿(𝐶) = 𝐿(𝐷). If so, then 𝐸 = 𝐹 , and if not 𝐸 ≠ 𝐹 .

ObservationWe are proving by example!



ExampleProve or disprove that 𝐿∗ = 𝐿∗𝐿∗.

1. We replace the variable 𝐿 by the concrete regular expression a.2. a∗ ≟ a∗a∗.3. Because 𝐿(a∗) = 𝐿(a∗a∗), we conclude that 𝐿∗ = 𝐿∗𝐿∗.




1. We replace the variable 𝐿 by the concrete regular expression a.

2. a∗ ≟ a∗a∗.3. Because 𝐿(a∗) = 𝐿(a∗a∗), we conclude that 𝐿∗ = 𝐿∗𝐿∗.




1. We replace the variable 𝐿 by the concrete regular expression a.2. a∗ ≟ a∗a∗.

3. Because 𝐿(a∗) = 𝐿(a∗a∗), we conclude that 𝐿∗ = 𝐿∗𝐿∗.




1. We replace the variable 𝐿 by the concrete regular expression a.2. a∗ ≟ a∗a∗.3. Because 𝐿(a∗) = 𝐿(a∗a∗), we conclude that 𝐿∗ = 𝐿∗𝐿∗.



ExampleProve or disprove that 𝐿 +𝑀𝐿 = (𝐿 +𝑀)𝐿.

1. We replace the variables 𝐿 and 𝑀 by the concrete regularexpressions a and b respectively.

2. a + ba ≟ (a + b)a.3. 𝑎𝑎 ∉ 𝐿(a + ba) and 𝑎𝑎 ∈ 𝐿((a + b)a)

⇒ 𝐿(a + ba) ≠ 𝐿((a + b)a)⇒ 𝐿 +𝑀𝐿 ≠ (𝐿 +𝑀)𝐿






⇒ 𝐿(a + ba) ≠ 𝐿((a + b)a)⇒ 𝐿 +𝑀𝐿 ≠ (𝐿 +𝑀)𝐿





2. a + ba ≟ (a + b)a.

3. 𝑎𝑎 ∉ 𝐿(a + ba) and 𝑎𝑎 ∈ 𝐿((a + b)a)⇒ 𝐿(a + ba) ≠ 𝐿((a + b)a)⇒ 𝐿 +𝑀𝐿 ≠ (𝐿 +𝑀)𝐿






⇒ 𝐿(a + ba) ≠ 𝐿((a + b)a)⇒ 𝐿 +𝑀𝐿 ≠ (𝐿 +𝑀)𝐿



Example (Exercise 3.4.2.d)Prove or disprove that (𝐿 +𝑀)∗𝑀 = (𝐿∗𝑀)∗.


2. (a + b)∗b ≟ (a∗b)∗.3. Since 𝜀 ∉ (a + b)∗b and 𝜀 ∈ (a∗b)∗

⇒ (𝐿 +𝑀)∗𝑀 ≠ (𝐿∗𝑀)∗






⇒ (𝐿 +𝑀)∗𝑀 ≠ (𝐿∗𝑀)∗





2. (a + b)∗b ≟ (a∗b)∗.

3. Since 𝜀 ∉ (a + b)∗b and 𝜀 ∈ (a∗b)∗

⇒ (𝐿 +𝑀)∗𝑀 ≠ (𝐿∗𝑀)∗






⇒ (𝐿 +𝑀)∗𝑀 ≠ (𝐿∗𝑀)∗



Example (counter-example)Extensions of the previous test beyond regular expressions may fail.

1. Add ∩ to the regular expression operators.2. Test if 𝐿 ∩𝑀 ∩𝑁 = 𝐿 ∩𝑀 .3. From 𝐿 = a, 𝑀 = b and 𝑁 = c and since

{𝑎} ∩ {𝑏} ∩ {𝑐} = ∅ = {𝑎} ∩ {𝑏},

we should conclude that the “property” is true.4. But, the “property” is false. For example, if 𝐿 = 𝑀 = a and 𝑁 = ∅

then𝐿 ∩𝑀 ∩𝑁 ≠ 𝐿 ∩𝑀.

5. Therefore, the test is not valid!





{𝑎} ∩ {𝑏} ∩ {𝑐} = ∅ = {𝑎} ∩ {𝑏},

we should conclude that the “property” is true.

4. But, the “property” is false. For example, if 𝐿 = 𝑀 = a and 𝑁 = ∅then

𝐿 ∩𝑀 ∩𝑁 ≠ 𝐿 ∩𝑀.5. Therefore, the test is not valid!





{𝑎} ∩ {𝑏} ∩ {𝑐} = ∅ = {𝑎} ∩ {𝑏},

we should conclude that the “property” is true.4. But, the “property” is false. For example, if 𝐿 = 𝑀 = a and 𝑁 = ∅

then𝐿 ∩𝑀 ∩𝑁 ≠ 𝐿 ∩𝑀.

5. Therefore, the test is not valid!


Derivative of a Regular Expression

From (Brzozowski 1964).

DefinitionLet 𝐿 ⊆ Σ∗ be a language and 𝑎 ∈ Σ a symbol. We define the derivativeof 𝐿 by 𝑎, denoted by 𝑎\𝐿, by

𝑎\𝐿 = {𝑥 ∈ Σ∗ ∣ 𝑎𝑥 ∈ 𝐿}.

Examples

𝑎\{𝑎𝑏𝑎𝑏, 𝑎𝑏𝑏𝑎} = {𝑏𝑎𝑏, 𝑏𝑏𝑎},𝑎\𝐿(ab∗) = 𝐿(b∗),𝑏\𝐿(ab∗) = ∅.



From (Brzozowski 1964).

DefinitionLet 𝐿 ⊆ Σ∗ be a language and 𝑎 ∈ Σ a symbol. We define the derivativeof 𝐿 by 𝑎, denoted by 𝑎\𝐿, by

𝑎\𝐿 = {𝑥 ∈ Σ∗ ∣ 𝑎𝑥 ∈ 𝐿}.

Examples

𝑎\{𝑎𝑏𝑎𝑏, 𝑎𝑏𝑏𝑎} = {𝑏𝑎𝑏, 𝑏𝑏𝑎},𝑎\𝐿(ab∗) = 𝐿(b∗),𝑏\𝐿(ab∗) = ∅.



DefinitionLet 𝐸 be a regular expression and 𝑎 ∈ Σ a symbol. We define recursivelythe derivative of 𝐸 by 𝑎, denoted 𝑎\𝐸, by

𝑎\∅ = ∅,𝑎\𝜀 = ∅,𝑎\a = 𝜀,𝑎\b = ∅ for 𝑎 ≠ 𝑏,

𝑎\(𝐸 + 𝐹) = 𝑎\𝐸 + 𝑎\𝐹 ,

𝑎\(𝐸𝐹) = {(𝑎\𝐸)𝐹 + 𝑎\𝐹 , if 𝜀 ∈ 𝐿(𝐸),(𝑎\𝐸)𝐹 , otherwise,

𝑎\(𝐸∗) = (𝑎\𝐸)𝐸∗.



DefinitionLet 𝐸 be a regular expression and 𝑎 ∈ Σ a symbol. We define recursivelythe derivative of 𝐸 by 𝑎, denoted 𝑎\𝐸, by

𝑎\∅ = ∅,𝑎\𝜀 = ∅,𝑎\a = 𝜀,𝑎\b = ∅ for 𝑎 ≠ 𝑏,

𝑎\(𝐸 + 𝐹) = 𝑎\𝐸 + 𝑎\𝐹 ,

𝑎\(𝐸𝐹) = {(𝑎\𝐸)𝐹 + 𝑎\𝐹 , if 𝜀 ∈ 𝐿(𝐸),(𝑎\𝐸)𝐹 , otherwise,

𝑎\(𝐸∗) = (𝑎\𝐸)𝐸∗.



DefinitionLet 𝐸 be a regular expression and 𝑤 ∈ Σ∗ a string. We define recursivelythe derivative of 𝐸 by 𝑤, denoted 𝑤\𝐸, by

𝜀\𝐸 = 𝐸,𝑎𝑥\𝐸 = 𝑎\(𝑥\𝐸).

Theorem (Brzozowski (1964), Theorem 4.2)

𝑤 ∈ 𝐿(𝐸) ⇔ 𝜀 ∈ 𝐿(𝑤\𝐸).



DefinitionLet 𝐸 be a regular expression and 𝑤 ∈ Σ∗ a string. We define recursivelythe derivative of 𝐸 by 𝑤, denoted 𝑤\𝐸, by

𝜀\𝐸 = 𝐸,𝑎𝑥\𝐸 = 𝑎\(𝑥\𝐸).

Theorem (Brzozowski (1964), Theorem 4.2)

𝑤 ∈ 𝐿(𝐸) ⇔ 𝜀 ∈ 𝐿(𝑤\𝐸).


References

Brzozowski, J. A. (1964). Derivates of Regular Expressions. Journal of theACM 11.4, pp. 481–494. doi: 10.1145/321239.321249 (cit. on pp. 27, 28,31, 32).


https://doi.org/10.1145/321239.321249

Automata and Formal Languages - CM0081Proving Languages Not to Be Regular


Universidad EAFIT

Semester 2018-1

Properties of Regular Languages

Proving languages not to be regularClosure propertiesDecision propertiesEquivalence and minimization of automata

Proving Languages Not to Be Regular 2/36

The Pumping Lemma

IntroductionIs 𝐿 = {0𝑚1𝑛 ∣ 𝑚, 𝑛 ≥ 0} a regular language?

Yes! 𝐿 = 𝐿(0∗1∗).Is 𝐿 = {0𝑚1𝑛 ∣ 𝑚, 𝑛 ≥ 1} a regular language?Yes! 𝐿 = 𝐿(0+1+).Is 𝐿 = {0𝑚1𝑛 ∣ 𝑚 ≥ 2, 𝑛 ≥ 4} a regular language?Yes! 𝐿 = 𝐿(000∗11111∗).Is 𝐿01 = {0𝑛1𝑛 ∣ 𝑛 ≥ 1} a regular language?No! Informal proof (whiteboard).


The Pumping Lemma

IntroductionIs 𝐿 = {0𝑚1𝑛 ∣ 𝑚, 𝑛 ≥ 0} a regular language?Yes! 𝐿 = 𝐿(0∗1∗).

Is 𝐿 = {0𝑚1𝑛 ∣ 𝑚, 𝑛 ≥ 1} a regular language?Yes! 𝐿 = 𝐿(0+1+).Is 𝐿 = {0𝑚1𝑛 ∣ 𝑚 ≥ 2, 𝑛 ≥ 4} a regular language?Yes! 𝐿 = 𝐿(000∗11111∗).Is 𝐿01 = {0𝑛1𝑛 ∣ 𝑛 ≥ 1} a regular language?No! Informal proof (whiteboard).


The Pumping Lemma

IntroductionIs 𝐿 = {0𝑚1𝑛 ∣ 𝑚, 𝑛 ≥ 0} a regular language?Yes! 𝐿 = 𝐿(0∗1∗).Is 𝐿 = {0𝑚1𝑛 ∣ 𝑚, 𝑛 ≥ 1} a regular language?

Yes! 𝐿 = 𝐿(0+1+).Is 𝐿 = {0𝑚1𝑛 ∣ 𝑚 ≥ 2, 𝑛 ≥ 4} a regular language?Yes! 𝐿 = 𝐿(000∗11111∗).Is 𝐿01 = {0𝑛1𝑛 ∣ 𝑛 ≥ 1} a regular language?No! Informal proof (whiteboard).


The Pumping Lemma

IntroductionIs 𝐿 = {0𝑚1𝑛 ∣ 𝑚, 𝑛 ≥ 0} a regular language?Yes! 𝐿 = 𝐿(0∗1∗).Is 𝐿 = {0𝑚1𝑛 ∣ 𝑚, 𝑛 ≥ 1} a regular language?Yes! 𝐿 = 𝐿(0+1+).

Is 𝐿 = {0𝑚1𝑛 ∣ 𝑚 ≥ 2, 𝑛 ≥ 4} a regular language?Yes! 𝐿 = 𝐿(000∗11111∗).Is 𝐿01 = {0𝑛1𝑛 ∣ 𝑛 ≥ 1} a regular language?No! Informal proof (whiteboard).


The Pumping Lemma

IntroductionIs 𝐿 = {0𝑚1𝑛 ∣ 𝑚, 𝑛 ≥ 0} a regular language?Yes! 𝐿 = 𝐿(0∗1∗).Is 𝐿 = {0𝑚1𝑛 ∣ 𝑚, 𝑛 ≥ 1} a regular language?Yes! 𝐿 = 𝐿(0+1+).Is 𝐿 = {0𝑚1𝑛 ∣ 𝑚 ≥ 2, 𝑛 ≥ 4} a regular language?

Yes! 𝐿 = 𝐿(000∗11111∗).Is 𝐿01 = {0𝑛1𝑛 ∣ 𝑛 ≥ 1} a regular language?No! Informal proof (whiteboard).


The Pumping Lemma

IntroductionIs 𝐿 = {0𝑚1𝑛 ∣ 𝑚, 𝑛 ≥ 0} a regular language?Yes! 𝐿 = 𝐿(0∗1∗).Is 𝐿 = {0𝑚1𝑛 ∣ 𝑚, 𝑛 ≥ 1} a regular language?Yes! 𝐿 = 𝐿(0+1+).Is 𝐿 = {0𝑚1𝑛 ∣ 𝑚 ≥ 2, 𝑛 ≥ 4} a regular language?Yes! 𝐿 = 𝐿(000∗11111∗).

Is 𝐿01 = {0𝑛1𝑛 ∣ 𝑛 ≥ 1} a regular language?No! Informal proof (whiteboard).


The Pumping Lemma

IntroductionIs 𝐿 = {0𝑚1𝑛 ∣ 𝑚, 𝑛 ≥ 0} a regular language?Yes! 𝐿 = 𝐿(0∗1∗).Is 𝐿 = {0𝑚1𝑛 ∣ 𝑚, 𝑛 ≥ 1} a regular language?Yes! 𝐿 = 𝐿(0+1+).Is 𝐿 = {0𝑚1𝑛 ∣ 𝑚 ≥ 2, 𝑛 ≥ 4} a regular language?Yes! 𝐿 = 𝐿(000∗11111∗).Is 𝐿01 = {0𝑛1𝑛 ∣ 𝑛 ≥ 1} a regular language?

No! Informal proof (whiteboard).


The Pumping Lemma

IntroductionIs 𝐿 = {0𝑚1𝑛 ∣ 𝑚, 𝑛 ≥ 0} a regular language?Yes! 𝐿 = 𝐿(0∗1∗).Is 𝐿 = {0𝑚1𝑛 ∣ 𝑚, 𝑛 ≥ 1} a regular language?Yes! 𝐿 = 𝐿(0+1+).Is 𝐿 = {0𝑚1𝑛 ∣ 𝑚 ≥ 2, 𝑛 ≥ 4} a regular language?Yes! 𝐿 = 𝐿(000∗11111∗).Is 𝐿01 = {0𝑛1𝑛 ∣ 𝑛 ≥ 1} a regular language?No! Informal proof (whiteboard).


The Pumping Lemma

Theorem (Pumping Lemma for regular languages, 4.1)Let 𝐿 be a regular language, then

(∃𝑛 ∈ ℤ+)(∀𝑤 ∈ 𝐿) (|𝑤| ≥ 𝑛 ⇒ (∃𝑥 𝑦 𝑧) 𝑤 = 𝑥𝑦𝑧)

such that1. 𝑦 ≠ 𝜀,2. |𝑥𝑦| ≤ 𝑛 and3. (∀𝑘 ≥ 0) 𝑥𝑦𝑘𝑧 ∈ 𝐿.


The Pumping Lemma

Proof.1. Suppose 𝐿 is a regular language. Exist a DFA 𝐴 = (𝑄, Σ, 𝛿, 𝑞0, 𝐹 )

with 𝑛 states such 𝐿(𝐴) = 𝐿.

2. Let 𝑤 = 𝑎1 ⋯ 𝑎𝑚 ∈ 𝐿, 𝑚 ≥ 𝑛 and 𝑞𝑖 = 𝛿(𝑞0, 𝑎1 ⋯ 𝑎𝑖).3. By the pigeonhole principle, exists 𝑖 and 𝑗, with 0 ≤ 𝑖 < 𝑗 ≤ 𝑛 such

𝑞𝑖 = 𝑞𝑗.4. Let 𝑤 = 𝑥𝑦𝑧 where

𝑞0start 𝑞𝑖𝑥 = 𝑎1 ⋯ 𝑎𝑖

𝑦 = 𝑎𝑖+1 ⋯ 𝑎𝑗

𝑧 = 𝑎𝑗+1 ⋯ 𝑎𝑚

5. Then (∀𝑘 ≥ 0) 𝑥𝑦𝑘𝑧 ∈ 𝐿.


The Pumping Lemma


with 𝑛 states such 𝐿(𝐴) = 𝐿.2. Let 𝑤 = 𝑎1 ⋯ 𝑎𝑚 ∈ 𝐿, 𝑚 ≥ 𝑛 and 𝑞𝑖 = 𝛿(𝑞0, 𝑎1 ⋯ 𝑎𝑖).

3. By the pigeonhole principle, exists 𝑖 and 𝑗, with 0 ≤ 𝑖 < 𝑗 ≤ 𝑛 such𝑞𝑖 = 𝑞𝑗.

4. Let 𝑤 = 𝑥𝑦𝑧 where


𝑦 = 𝑎𝑖+1 ⋯ 𝑎𝑗

𝑧 = 𝑎𝑗+1 ⋯ 𝑎𝑚

5. Then (∀𝑘 ≥ 0) 𝑥𝑦𝑘𝑧 ∈ 𝐿.


The Pumping Lemma


with 𝑛 states such 𝐿(𝐴) = 𝐿.2. Let 𝑤 = 𝑎1 ⋯ 𝑎𝑚 ∈ 𝐿, 𝑚 ≥ 𝑛 and 𝑞𝑖 = 𝛿(𝑞0, 𝑎1 ⋯ 𝑎𝑖).3. By the pigeonhole principle, exists 𝑖 and 𝑗, with 0 ≤ 𝑖 < 𝑗 ≤ 𝑛 such

𝑞𝑖 = 𝑞𝑗.

4. Let 𝑤 = 𝑥𝑦𝑧 where


𝑦 = 𝑎𝑖+1 ⋯ 𝑎𝑗

𝑧 = 𝑎𝑗+1 ⋯ 𝑎𝑚

5. Then (∀𝑘 ≥ 0) 𝑥𝑦𝑘𝑧 ∈ 𝐿.


The Pumping Lemma





𝑦 = 𝑎𝑖+1 ⋯ 𝑎𝑗

𝑧 = 𝑎𝑗+1 ⋯ 𝑎𝑚

5. Then (∀𝑘 ≥ 0) 𝑥𝑦𝑘𝑧 ∈ 𝐿.


The Pumping Lemma





𝑦 = 𝑎𝑖+1 ⋯ 𝑎𝑗

𝑧 = 𝑎𝑗+1 ⋯ 𝑎𝑚

5. Then (∀𝑘 ≥ 0) 𝑥𝑦𝑘𝑧 ∈ 𝐿.


Application of the Pumping Lemma: Proving LanguagesNot to Be Regular

MethodWhiteboard

Exercise (4.1.2.e)Let Σ = {0, 1} be an alphabet and let 𝐿 = {𝑤𝑤 ∣ 𝑤 ∈ Σ∗} be theso-called copy language. Prove that 𝐿 is not regular.



MethodWhiteboard

Exercise (4.1.2.e)Let Σ = {0, 1} be an alphabet and let 𝐿 = {𝑤𝑤 ∣ 𝑤 ∈ Σ∗} be theso-called copy language. Prove that 𝐿 is not regular.



Exercise (cont.)

Proof.1. Suppose 𝐿 is regular.

2. Let 𝑛 ∈ ℤ+ be a constant (according to the Pumping Lemma).3. Let 𝑤 = 0𝑛10𝑛1 ∈ 𝐿 and |𝑤| ≥ 𝑛.4. For the Pumping Lemma exists 𝑥, 𝑦 and 𝑧 such that 𝑤 = 𝑥𝑦𝑧,

|𝑥𝑦| ≤ 𝑛, 𝑦 ≠ 𝜀 and ∀𝑘 ≥ 0. 𝑥𝑦𝑘𝑧 ∈ 𝐿.5. Then 𝑦 = 0𝑚, 0 < 𝑚 ≤ 𝑛.6. But, 𝑥𝑦0𝑧 ∉ 𝐿 which is a contradiction by the Pumping Lemma.7. Therefore, 𝐿 is not regular.



Exercise (cont.)

Proof.1. Suppose 𝐿 is regular.2. Let 𝑛 ∈ ℤ+ be a constant (according to the Pumping Lemma).

3. Let 𝑤 = 0𝑛10𝑛1 ∈ 𝐿 and |𝑤| ≥ 𝑛.4. For the Pumping Lemma exists 𝑥, 𝑦 and 𝑧 such that 𝑤 = 𝑥𝑦𝑧,




Exercise (cont.)

Proof.1. Suppose 𝐿 is regular.2. Let 𝑛 ∈ ℤ+ be a constant (according to the Pumping Lemma).3. Let 𝑤 = 0𝑛10𝑛1 ∈ 𝐿 and |𝑤| ≥ 𝑛.

4. For the Pumping Lemma exists 𝑥, 𝑦 and 𝑧 such that 𝑤 = 𝑥𝑦𝑧,|𝑥𝑦| ≤ 𝑛, 𝑦 ≠ 𝜀 and ∀𝑘 ≥ 0. 𝑥𝑦𝑘𝑧 ∈ 𝐿.

5. Then 𝑦 = 0𝑚, 0 < 𝑚 ≤ 𝑛.6. But, 𝑥𝑦0𝑧 ∉ 𝐿 which is a contradiction by the Pumping Lemma.7. Therefore, 𝐿 is not regular.



Exercise (cont.)

Proof.1. Suppose 𝐿 is regular.2. Let 𝑛 ∈ ℤ+ be a constant (according to the Pumping Lemma).3. Let 𝑤 = 0𝑛10𝑛1 ∈ 𝐿 and |𝑤| ≥ 𝑛.4. For the Pumping Lemma exists 𝑥, 𝑦 and 𝑧 such that 𝑤 = 𝑥𝑦𝑧,

|𝑥𝑦| ≤ 𝑛, 𝑦 ≠ 𝜀 and ∀𝑘 ≥ 0. 𝑥𝑦𝑘𝑧 ∈ 𝐿.

5. Then 𝑦 = 0𝑚, 0 < 𝑚 ≤ 𝑛.6. But, 𝑥𝑦0𝑧 ∉ 𝐿 which is a contradiction by the Pumping Lemma.7. Therefore, 𝐿 is not regular.



Exercise (cont.)


|𝑥𝑦| ≤ 𝑛, 𝑦 ≠ 𝜀 and ∀𝑘 ≥ 0. 𝑥𝑦𝑘𝑧 ∈ 𝐿.5. Then 𝑦 = 0𝑚, 0 < 𝑚 ≤ 𝑛.

6. But, 𝑥𝑦0𝑧 ∉ 𝐿 which is a contradiction by the Pumping Lemma.7. Therefore, 𝐿 is not regular.



Exercise (cont.)


|𝑥𝑦| ≤ 𝑛, 𝑦 ≠ 𝜀 and ∀𝑘 ≥ 0. 𝑥𝑦𝑘𝑧 ∈ 𝐿.5. Then 𝑦 = 0𝑚, 0 < 𝑚 ≤ 𝑛.6. But, 𝑥𝑦0𝑧 ∉ 𝐿 which is a contradiction by the Pumping Lemma.

7. Therefore, 𝐿 is not regular.



Exercise (cont.)





Exercise (4.1.2.a)Let 𝐿 be the language

𝐿 = {0𝑛 ∣ 𝑛 is a perfect square}.

Prove that 𝐿 is not regular.



Exercise (cont.)Proof.

1. Suppose 𝐿 is regular.

2. Let 𝑛 ∈ ℤ+ be a constant (according to the Pumping Lemma).3. Let 𝑤 = 0𝑛2 ∈ 𝐿 and |𝑤| ≥ 𝑛.4. For the Pumping Lemma exists 𝑥, 𝑦 and 𝑧 such that 𝑤 = 𝑥𝑦𝑧,

|𝑥𝑦| ≤ 𝑛, 𝑦 ≠ 𝜀 and ∀𝑘 ≥ 0. 𝑥𝑦𝑘𝑧 ∈ 𝐿.5. Then 𝑦 = 0𝑚, 0 < 𝑚 ≤ 𝑛 and 𝑛2 + 1 ≤ |𝑥𝑦𝑦𝑧| ≤ 𝑛2 + 𝑛.6. Since the next perfect square after 𝑛2 is (𝑛 + 1)2 = 𝑛2 + 2𝑛 + 1, we

know that 𝑥𝑦𝑦𝑧 ∉ 𝐿 (|𝑥𝑦𝑦𝑧| is strictly between the consecutiveperfect squares 𝑛2 and (𝑛 + 1)2).

7. This a contradiction by the Pumping Lemma.8. Therefore, 𝐿 is not regular.




1. Suppose 𝐿 is regular.2. Let 𝑛 ∈ ℤ+ be a constant (according to the Pumping Lemma).

3. Let 𝑤 = 0𝑛2 ∈ 𝐿 and |𝑤| ≥ 𝑛.4. For the Pumping Lemma exists 𝑥, 𝑦 and 𝑧 such that 𝑤 = 𝑥𝑦𝑧,







1. Suppose 𝐿 is regular.2. Let 𝑛 ∈ ℤ+ be a constant (according to the Pumping Lemma).3. Let 𝑤 = 0𝑛2 ∈ 𝐿 and |𝑤| ≥ 𝑛.

4. For the Pumping Lemma exists 𝑥, 𝑦 and 𝑧 such that 𝑤 = 𝑥𝑦𝑧,|𝑥𝑦| ≤ 𝑛, 𝑦 ≠ 𝜀 and ∀𝑘 ≥ 0. 𝑥𝑦𝑘𝑧 ∈ 𝐿.

5. Then 𝑦 = 0𝑚, 0 < 𝑚 ≤ 𝑛 and 𝑛2 + 1 ≤ |𝑥𝑦𝑦𝑧| ≤ 𝑛2 + 𝑛.6. Since the next perfect square after 𝑛2 is (𝑛 + 1)2 = 𝑛2 + 2𝑛 + 1, we






1. Suppose 𝐿 is regular.2. Let 𝑛 ∈ ℤ+ be a constant (according to the Pumping Lemma).3. Let 𝑤 = 0𝑛2 ∈ 𝐿 and |𝑤| ≥ 𝑛.4. For the Pumping Lemma exists 𝑥, 𝑦 and 𝑧 such that 𝑤 = 𝑥𝑦𝑧,

|𝑥𝑦| ≤ 𝑛, 𝑦 ≠ 𝜀 and ∀𝑘 ≥ 0. 𝑥𝑦𝑘𝑧 ∈ 𝐿.

5. Then 𝑦 = 0𝑚, 0 < 𝑚 ≤ 𝑛 and 𝑛2 + 1 ≤ |𝑥𝑦𝑦𝑧| ≤ 𝑛2 + 𝑛.6. Since the next perfect square after 𝑛2 is (𝑛 + 1)2 = 𝑛2 + 2𝑛 + 1, we







|𝑥𝑦| ≤ 𝑛, 𝑦 ≠ 𝜀 and ∀𝑘 ≥ 0. 𝑥𝑦𝑘𝑧 ∈ 𝐿.5. Then 𝑦 = 0𝑚, 0 < 𝑚 ≤ 𝑛 and 𝑛2 + 1 ≤ |𝑥𝑦𝑦𝑧| ≤ 𝑛2 + 𝑛.

6. Since the next perfect square after 𝑛2 is (𝑛 + 1)2 = 𝑛2 + 2𝑛 + 1, weknow that 𝑥𝑦𝑦𝑧 ∉ 𝐿 (|𝑥𝑦𝑦𝑧| is strictly between the consecutiveperfect squares 𝑛2 and (𝑛 + 1)2).















7. This a contradiction by the Pumping Lemma.

8. Therefore, 𝐿 is not regular.









Proving Languages Not to Be Regular

RemarkFrishberg and Gasarch (2018) show other methods and some openproblems when proving that a language is not regular.


References

Frishberg, D. and Gasarch, W. (2018). Open Problems Column. DifferentWays to Prove a Language is Not Regular. SIGACT News 49.1, pp. 40–54.doi: 10.1145/3197406.3197413 (cit. on p. 35).


https://doi.org/10.1145/3197406.3197413

Automata and Formal Languages - CM0081Closure Properties of Regular Languages


Universidad EAFIT

Semester 2018-1

Introduction

Let 𝐿 and 𝑀 be regular languages. The following languages are regular:

Union: 𝐿 ∪ 𝑀Intersection: 𝐿 ∩ 𝑀Complement: 𝐿Difference: 𝐿 − 𝑀Reversal: 𝐿𝑅 = {𝑤𝑟 ∈ Σ∗ ∣ 𝑤 ∈ 𝐿}Closure: 𝐿∗

Concatenation: 𝐿𝑀Homomorphism: ℎ(𝐿) = {ℎ(𝑤) ∣ 𝑤 ∈ 𝐿 and ℎ is a homomorphism}Inverse homomorphism:ℎ−1(𝐿) = {𝑤 ∈ Σ ∣ ℎ(𝑤) ∈ 𝐿 and ℎ is a homomorphism}

Closure Properties of Regular Languages 2/67

Introduction

Let 𝐿 and 𝑀 be regular languages. The following languages are regular:Union: 𝐿 ∪ 𝑀

Intersection: 𝐿 ∩ 𝑀Complement: 𝐿Difference: 𝐿 − 𝑀Reversal: 𝐿𝑅 = {𝑤𝑟 ∈ Σ∗ ∣ 𝑤 ∈ 𝐿}Closure: 𝐿∗



Introduction

Let 𝐿 and 𝑀 be regular languages. The following languages are regular:Union: 𝐿 ∪ 𝑀Intersection: 𝐿 ∩ 𝑀

Complement: 𝐿Difference: 𝐿 − 𝑀Reversal: 𝐿𝑅 = {𝑤𝑟 ∈ Σ∗ ∣ 𝑤 ∈ 𝐿}Closure: 𝐿∗



Introduction

Let 𝐿 and 𝑀 be regular languages. The following languages are regular:Union: 𝐿 ∪ 𝑀Intersection: 𝐿 ∩ 𝑀Complement: 𝐿

Difference: 𝐿 − 𝑀Reversal: 𝐿𝑅 = {𝑤𝑟 ∈ Σ∗ ∣ 𝑤 ∈ 𝐿}Closure: 𝐿∗



Introduction

Let 𝐿 and 𝑀 be regular languages. The following languages are regular:Union: 𝐿 ∪ 𝑀Intersection: 𝐿 ∩ 𝑀Complement: 𝐿Difference: 𝐿 − 𝑀

Reversal: 𝐿𝑅 = {𝑤𝑟 ∈ Σ∗ ∣ 𝑤 ∈ 𝐿}Closure: 𝐿∗



Introduction

Let 𝐿 and 𝑀 be regular languages. The following languages are regular:Union: 𝐿 ∪ 𝑀Intersection: 𝐿 ∩ 𝑀Complement: 𝐿Difference: 𝐿 − 𝑀Reversal: 𝐿𝑅 = {𝑤𝑟 ∈ Σ∗ ∣ 𝑤 ∈ 𝐿}

Closure: 𝐿∗



Introduction

Let 𝐿 and 𝑀 be regular languages. The following languages are regular:Union: 𝐿 ∪ 𝑀Intersection: 𝐿 ∩ 𝑀Complement: 𝐿Difference: 𝐿 − 𝑀Reversal: 𝐿𝑅 = {𝑤𝑟 ∈ Σ∗ ∣ 𝑤 ∈ 𝐿}Closure: 𝐿∗



Introduction


Concatenation: 𝐿𝑀

Homomorphism: ℎ(𝐿) = {ℎ(𝑤) ∣ 𝑤 ∈ 𝐿 and ℎ is a homomorphism}Inverse homomorphism:ℎ−1(𝐿) = {𝑤 ∈ Σ ∣ ℎ(𝑤) ∈ 𝐿 and ℎ is a homomorphism}


Introduction


Concatenation: 𝐿𝑀Homomorphism: ℎ(𝐿) = {ℎ(𝑤) ∣ 𝑤 ∈ 𝐿 and ℎ is a homomorphism}

Inverse homomorphism:ℎ−1(𝐿) = {𝑤 ∈ Σ ∣ ℎ(𝑤) ∈ 𝐿 and ℎ is a homomorphism}


Introduction




Closure Under Union and Complementation

Theorem (4.4)If 𝐿 and 𝑀 are regular languages, then so is 𝐿 ∪ 𝑀 .

Proof(Using regular expressions)



Theorem (4.4)If 𝐿 and 𝑀 are regular languages, then so is 𝐿 ∪ 𝑀 .

Proof(Using regular expressions)



Theorem (4.5)Let 𝐿 = Σ∗ − 𝐿 be the complement of a language 𝐿. If 𝐿 is a regularlanguage, then so is 𝐿.

Proof.Let

𝐴 = (𝑄, Σ, 𝛿, 𝑞0, 𝐹 )

be a DFA such that 𝐿(𝐴) = 𝐿. Then

𝐵 = (𝑄, Σ, 𝛿, 𝑞0, 𝑄 − 𝐹)

is a DFA such 𝐿(𝐵) = 𝐿.



Theorem (4.5)Let 𝐿 = Σ∗ − 𝐿 be the complement of a language 𝐿. If 𝐿 is a regularlanguage, then so is 𝐿.

Proof.Let

𝐴 = (𝑄, Σ, 𝛿, 𝑞0, 𝐹 )

be a DFA such that 𝐿(𝐴) = 𝐿. Then

𝐵 = (𝑄, Σ, 𝛿, 𝑞0, 𝑄 − 𝐹)

is a DFA such 𝐿(𝐵) = 𝐿.



Question: “Do you see how to take a regular expression and change it intoone that defines the complement language?” (Hopcroft, Motwani andUllman 2007, p. 136)



Using the closure properties we can prove that a language is not regular.

ExampleGiven that

𝐿= = {𝑤 ∈ {0, 1}∗ ∣ 𝑤 has an equal numbers of 0’s and 1’s}

is a language not regular. Prove that

𝐿≠ = {𝑤 ∈ {0, 1}∗ ∣ 𝑤 has an unequal numbers of 0’s and 1’s}

is a language not regular.

ProofWhiteboard.




ExampleGiven that





ProofWhiteboard.




ExampleGiven that





ProofWhiteboard.


Product Construction

Let 𝐴𝐿, 𝐴𝑀 and 𝐴 be DFAs given by

𝐴𝐿 = (𝑄𝐿, Σ, 𝛿𝐿, 𝑞𝐿, 𝐹𝐿),𝐴𝑀 = (𝑄𝑀 , Σ, 𝛿𝑀 , 𝑞𝑀 , 𝐹𝑀),

𝐴 = (𝑄𝐿 × 𝑄𝑀 , Σ, 𝛿, (𝑞𝐿, 𝑞𝑀), 𝐹𝐿 × 𝐹𝑀),

where

𝛿 ∶ (𝑄𝐿 × 𝑄𝑀) × Σ → 𝑄𝐿 × 𝑄𝑀𝛿((𝑝, 𝑞), 𝑎) = (𝛿𝐿(𝑝, 𝑎), 𝛿𝑀(𝑞, 𝑎)).

Theorem (Exercise 4.2.15)For all 𝑤 ∈ Σ∗,

𝛿((𝑞𝐿, 𝑞𝑀), 𝑤) = ( 𝛿𝐿(𝑞𝐿, 𝑤), 𝛿𝑀(𝑞𝑀 , 𝑤)).



Let 𝐴𝐿, 𝐴𝑀 and 𝐴 be DFAs given by

𝐴𝐿 = (𝑄𝐿, Σ, 𝛿𝐿, 𝑞𝐿, 𝐹𝐿),𝐴𝑀 = (𝑄𝑀 , Σ, 𝛿𝑀 , 𝑞𝑀 , 𝐹𝑀),

𝐴 = (𝑄𝐿 × 𝑄𝑀 , Σ, 𝛿, (𝑞𝐿, 𝑞𝑀), 𝐹𝐿 × 𝐹𝑀),

where

𝛿 ∶ (𝑄𝐿 × 𝑄𝑀) × Σ → 𝑄𝐿 × 𝑄𝑀𝛿((𝑝, 𝑞), 𝑎) = (𝛿𝐿(𝑝, 𝑎), 𝛿𝑀(𝑞, 𝑎)).

Theorem (Exercise 4.2.15)For all 𝑤 ∈ Σ∗,

𝛿((𝑞𝐿, 𝑞𝑀), 𝑤) = ( 𝛿𝐿(𝑞𝐿, 𝑤), 𝛿𝑀(𝑞𝑀 , 𝑤)).



Proof by induction on 𝑤.1. Basis step

𝛿((𝑞𝐿, 𝑞𝑀), 𝜀) = (𝑞𝐿, 𝑞𝑀) (def. of 𝛿)= ( 𝛿𝐿(𝑞𝐿, 𝜀), 𝛿𝑀(𝑞𝑀 , 𝜀)) (def. of 𝛿𝐿 and 𝛿𝑀)

2. Inductive step

𝛿((𝑞𝐿, 𝑞𝑀), 𝑥𝑎)= 𝛿( 𝛿((𝑞𝐿, 𝑞𝑀), 𝑥), 𝑎) (def. of 𝛿)= 𝛿(( 𝛿𝐿(𝑞𝐿, 𝑥), 𝛿𝑀(𝑞𝑀 , 𝑥)), 𝑎) (by IH)= (𝛿𝐿( 𝛿𝐿(𝑞𝐿, 𝑥), 𝑎), 𝛿𝑀( 𝛿𝑀(𝑞𝑀 , 𝑥), 𝑎)) (def. of 𝛿)= ( 𝛿𝐿(𝑞𝐿, 𝑥𝑎), 𝛿𝑀(𝑞𝑀 , 𝑥𝑎)) (def. of 𝛿𝐿 and 𝛿𝐿)



Proof by induction on 𝑤.1. Basis step

𝛿((𝑞𝐿, 𝑞𝑀), 𝜀) = (𝑞𝐿, 𝑞𝑀) (def. of 𝛿)= ( 𝛿𝐿(𝑞𝐿, 𝜀), 𝛿𝑀(𝑞𝑀 , 𝜀)) (def. of 𝛿𝐿 and 𝛿𝑀)

2. Inductive step

𝛿((𝑞𝐿, 𝑞𝑀), 𝑥𝑎)= 𝛿( 𝛿((𝑞𝐿, 𝑞𝑀), 𝑥), 𝑎) (def. of 𝛿)= 𝛿(( 𝛿𝐿(𝑞𝐿, 𝑥), 𝛿𝑀(𝑞𝑀 , 𝑥)), 𝑎) (by IH)= (𝛿𝐿( 𝛿𝐿(𝑞𝐿, 𝑥), 𝑎), 𝛿𝑀( 𝛿𝑀(𝑞𝑀 , 𝑥), 𝑎)) (def. of 𝛿)= ( 𝛿𝐿(𝑞𝐿, 𝑥𝑎), 𝛿𝑀(𝑞𝑀 , 𝑥𝑎)) (def. of 𝛿𝐿 and 𝛿𝐿)


Closure Under Intersection

Theorem (4.8)If 𝐿 and 𝑀 are regular languages, then so is 𝐿 ∩ 𝑀 .

Proof.Let 𝐴𝐿 and 𝐴𝑀 be DFAs accepting 𝐿 and 𝑀 . The product constructionof 𝐴𝐿 and 𝐴𝑀 accepts 𝐿 ∩ 𝑀 .

Different proof.The regular languages are closure under union and complement, and

𝐿 ∩ 𝑀 = 𝐿 ∪ 𝑀.






𝐿 ∩ 𝑀 = 𝐿 ∪ 𝑀.






𝐿 ∩ 𝑀 = 𝐿 ∪ 𝑀.


Closure Under Reversal

DefinitionLet 𝑤 = 𝑎1𝑎2 ⋯ 𝑎𝑛 be a word. The reversal of 𝑤 is defined by

𝑤𝑅 = 𝑎𝑛𝑎𝑛−1 ⋯ 𝑎1.

DefinitionLet 𝐿 be a language on alphabet Σ. The reversal of 𝐿 is defined by

𝐿𝑅 = {𝑤𝑅 ∈ Σ∗ ∣ 𝑤 ∈ 𝐿}.

Theorem (4.11)If 𝐿 is regular language, then so is 𝐿𝑅 (proof using automata or regularexpressions)






𝐿𝑅 = {𝑤𝑅 ∈ Σ∗ ∣ 𝑤 ∈ 𝐿}.







𝐿𝑅 = {𝑤𝑅 ∈ Σ∗ ∣ 𝑤 ∈ 𝐿}.




Proof using automata.Let 𝐿 be recognized by a finite automaton 𝐴. Turn 𝐴 into finite automatonfor 𝐿𝑅, by

1. Reversing all arcs.2. Make the start state of 𝐴 be the only accepting state.3. Create a new start state 𝑝0 with transitions 𝛿(𝑝0, 𝜀) = 𝑓 ,

where 𝑓 ∈ 𝐹 are the accepting states of 𝐴.



ExampleA NFA accepting all the binary strings that end in 01.


0, 1

0 1

A NFA accepting all the binary strings that start with 10.

𝑞0 𝑞1 𝑞2 𝑝0 start

0, 1

0 1 𝜀



ExampleA NFA accepting all the binary strings that end in 01.


0, 1

0 1

A NFA accepting all the binary strings that start with 10.

𝑞0 𝑞1 𝑞2 𝑝0 start

0, 1

0 1 𝜀


Homomorphisms

DefinitionAn algebraic structure on a set 𝐴 ≠ ∅ is essentially a collection of 𝑛-aryoperations on 𝐴 (Birkhoff 1946, 1987).

Example (Semigroup)A semigroup (𝑆, ∗) is a set 𝑆 with an associative binary operation∗ ∶ 𝑆 × 𝑆 → 𝑆.

Example (Monoid)A monoid (𝑀, ∗, 𝜀) is a semigroup (𝑀, ∗) with an element 𝜀 ∈ 𝑀 which isan unit for ∗, i.e. ∀𝑥. 𝑥 ∗ 𝜀 = 𝜀 ∗ 𝑥 = 𝑥.


Homomorphisms





Homomorphisms





Homomorphisms

DefinitionA homomorphism is a structure-preserving map between two algebraic struc-tures.

ExampleA homomorphism between two semigroups (𝑆, ∗) and (𝑆′, ∗′) is a function𝜑 ∶ 𝑆 → 𝑆′ such that:

∀𝑥 𝑦. 𝜑(𝑥 ∗ 𝑦) = 𝜑(𝑥) ∗′ 𝜑(𝑦).

Graphically (see whiteboard).


Homomorphisms

DefinitionA homomorphism is a structure-preserving map between two algebraic struc-tures.

ExampleA homomorphism between two semigroups (𝑆, ∗) and (𝑆′, ∗′) is a function𝜑 ∶ 𝑆 → 𝑆′ such that:

∀𝑥 𝑦. 𝜑(𝑥 ∗ 𝑦) = 𝜑(𝑥) ∗′ 𝜑(𝑦).

Graphically (see whiteboard).


Homomorphisms

ExampleA homomorphism between two monoids (𝑀, ∗, 𝜀) and (𝑀 ′, ∗′, 𝜀′) is afunction 𝜑 ∶ 𝑀 → 𝑀 ′ such that:

∀𝑥 𝑦. 𝜑(𝑥 ∗ 𝑦) = 𝜑(𝑥) ∗′ 𝜑(𝑦),𝜑(𝜀) = 𝜀′.


Homomorphisms

DefinitionA homomorphism 𝜑 between two algebraic structures is (Cohn 1981):

a monomorphism if 𝜑 is an injection,an epimorphism if 𝜑 is a surjection,an endomorphism if 𝜑 is from an algebraic structure to itself,an isomorphism if 𝜑 is a bijection,an automorphism if 𝜑 is a bijective endomorphism.


Closure Under Homomorphism

DefinitionLet Σ and Γ be two alphabets. A homomorphism between (Σ∗, ·, 𝜀) and(Γ∗, ·, 𝜀) is a function

ℎ ∶ Σ∗ → Γ∗

𝑎1𝑎2 ⋯ 𝑎𝑛 ↦ ℎ(𝑎1)ℎ(𝑎2) ⋯ ℎ(𝑎𝑛)𝜀 ↦ 𝜀

Note: For this reason the textbook talk about a homomorphism ℎ ∶ Σ → Γ∗.



ExampleLet ℎ ∶ {0, 1}∗ → {𝑎, 𝑏}∗ be a homomorphism defined by

ℎ(0) = 𝑎𝑏,ℎ(1) = 𝜀.

Then

ℎ(0011) = ℎ(0)ℎ(0)ℎ(1)ℎ(1)= 𝑎𝑏𝑎𝑏.



DefinitionLet 𝐿 be a language over an alphabet Σ and let ℎ be a homomorphismon Σ. The application of ℎ to 𝐿, denoted ℎ(𝐿), is defined by†

ℎ(𝐿) = {ℎ(𝑤) ∣ 𝑤 ∈ 𝐿}.

Inverse Homomorphism

Let h : Σ∗ → Θ∗ be a homom. Let L ⊆ Θ∗,and define

h−1(L) = {w ∈ Σ∗ : h(w) ∈ L}

L h(L)

Lh-1 (L)

(a)

(b)

h

h

110

†Figure from Hopcroft, Motwani and Ullman (2007, Fig. 4.5a).Closure Properties of Regular Languages 42/67


ExampleLet ℎ ∶ {0, 1}∗ → {𝑎, 𝑏}∗ be a homomorphism defined by

ℎ(0) = 𝑎𝑏,ℎ(1) = 𝜀.

If 𝐿 = 𝐿(10∗1), then ℎ(𝐿) = 𝐿((ab)∗).



Theorem (4.14)If 𝐿 is a regular language over the alphabet Σ and ℎ is a homomorphismon Σ, then ℎ(𝐿) is also regular.

Proof planLet 𝐸 be a regular expression such 𝐿 = 𝐿(𝐸).Let ℎ(𝐸) be the regular expression replacing each symbol 𝑎 ∈ Σby ℎ(𝑎) in the regular expression 𝐸.We need to prove that 𝐿(ℎ(𝐸)) = ℎ(𝐿(𝐸)).



Theorem (4.14)If 𝐿 is a regular language over the alphabet Σ and ℎ is a homomorphismon Σ, then ℎ(𝐿) is also regular.

Proof planLet 𝐸 be a regular expression such 𝐿 = 𝐿(𝐸).Let ℎ(𝐸) be the regular expression replacing each symbol 𝑎 ∈ Σby ℎ(𝑎) in the regular expression 𝐸.We need to prove that 𝐿(ℎ(𝐸)) = ℎ(𝐿(𝐸)).



Proving 𝐿(ℎ(𝐸)) = ℎ(𝐿(𝐸)).Basis step

𝐸 is 𝜀 or ∅.1. ℎ(𝐸) = 𝐸 (ℎ does not affect 𝐸)2. ℎ(𝐿(𝐸)) = 𝐿(𝐸) (𝐿(𝐸) is empty or only contains 𝜀)3. 𝐿(ℎ(𝐸)) = 𝐿(𝐸) = ℎ(𝐿(𝐸)) (by 1 and 2)

𝐸 = a1. 𝐿(𝐸) = {𝑎}2. ℎ(𝐿(𝐸)) = {ℎ(𝑎)}3. ℎ(𝐸) is the regular expression that is the string of symbols ℎ(𝑎)4. 𝐿(ℎ(𝐸)) = {ℎ(𝑎)}5. 𝐿(ℎ(𝐸)) = ℎ(𝐿(𝐸)) (by transitivity between 2 and 4)



Proving 𝐿(ℎ(𝐸)) = ℎ(𝐿(𝐸)).Basis step

𝐸 is 𝜀 or ∅.1. ℎ(𝐸) = 𝐸 (ℎ does not affect 𝐸)2. ℎ(𝐿(𝐸)) = 𝐿(𝐸) (𝐿(𝐸) is empty or only contains 𝜀)3. 𝐿(ℎ(𝐸)) = 𝐿(𝐸) = ℎ(𝐿(𝐸)) (by 1 and 2)

𝐸 = a1. 𝐿(𝐸) = {𝑎}2. ℎ(𝐿(𝐸)) = {ℎ(𝑎)}3. ℎ(𝐸) is the regular expression that is the string of symbols ℎ(𝑎)4. 𝐿(ℎ(𝐸)) = {ℎ(𝑎)}5. 𝐿(ℎ(𝐸)) = ℎ(𝐿(𝐸)) (by transitivity between 2 and 4)



Proving 𝐿(ℎ(𝐸)) = ℎ(𝐿(𝐸)).Inductive step

𝐸 = 𝐹 + 𝐺1. 𝐿(𝐸) = 𝐿(𝐹) ∪ 𝐿(𝐺) (def. of +)2. ℎ(𝐸) = ℎ(𝐹 + 𝐺) = ℎ(𝐹) + ℎ(𝐺) (def. of ℎ(𝐸))3. 𝐿(ℎ(𝐸)) = 𝐿(ℎ(𝐹) + ℎ(𝐺)) = 𝐿(ℎ(𝐹)) ∪ 𝐿(ℎ(𝐺)) (def. of +)4. ℎ(𝐿(𝐸)) = ℎ(𝐿(𝐹) ∪ 𝐿(𝐺)) = ℎ(𝐿(𝐹)) ∪ ℎ(𝐿(𝐺)) (ℎ is applied

to a language by application to each of its strings)5. 𝐿(ℎ(𝐹)) = ℎ(𝐿(𝐹) and 𝐿(ℎ(𝐺)) = ℎ(𝐿(𝐺) (IH)6. 𝐿(ℎ(𝐸)) = ℎ(𝐿(𝐸))



Proving 𝐿(ℎ(𝐸)) = ℎ(𝐿(𝐸)).Inductive step (cont.)

𝐸 = 𝐹𝐺 (similar to the previous case)

𝐸 = 𝐹 ∗ (similar to the previous case)1. 𝐿(𝐸) = (𝐿(𝐹))∗ (def. of ∗)2. ℎ(𝐸) = ℎ(𝐹 ∗) = (ℎ(𝐹))∗ (def. of ℎ(𝐸))3. 𝐿(ℎ(𝐸)) = 𝐿((ℎ(𝐹))∗) = (𝐿(ℎ(𝐹)))∗ (def. of ∗)4. ℎ(𝐿(𝐸)) = ℎ((𝐿(𝐹))∗) = (ℎ(𝐿(𝐹)))∗ (ℎ is applied to a language

by application to each of its strings)5. 𝐿(ℎ(𝐹)) = ℎ(𝐿(𝐹)) (IH)6. 𝐿(ℎ(𝐸)) = ℎ(𝐿(𝐸))



Proving 𝐿(ℎ(𝐸)) = ℎ(𝐿(𝐸)).Inductive step (cont.)

𝐸 = 𝐹𝐺 (similar to the previous case)𝐸 = 𝐹 ∗ (similar to the previous case)

1. 𝐿(𝐸) = (𝐿(𝐹))∗ (def. of ∗)2. ℎ(𝐸) = ℎ(𝐹 ∗) = (ℎ(𝐹))∗ (def. of ℎ(𝐸))3. 𝐿(ℎ(𝐸)) = 𝐿((ℎ(𝐹))∗) = (𝐿(ℎ(𝐹)))∗ (def. of ∗)4. ℎ(𝐿(𝐸)) = ℎ((𝐿(𝐹))∗) = (ℎ(𝐿(𝐹)))∗ (ℎ is applied to a language

by application to each of its strings)5. 𝐿(ℎ(𝐹)) = ℎ(𝐿(𝐹)) (IH)6. 𝐿(ℎ(𝐸)) = ℎ(𝐿(𝐸))



ExampleProve that

𝐿 = {0𝑖1𝑗2𝑘 ∣ 𝑖, 𝑗, 𝑘 ∈ ℤ+ and 𝑖 ≠ 𝑗 ≠ 𝑘}is a language not regular.

Proof.1. We define the homomorphism

ℎ(0) = 0, ℎ(1) = 1, ℎ(2) = 𝜀.

2. The homomorphism ℎ removes the 2𝑘s, so

ℎ(𝐿) = {0𝑖1𝑗 ∣ 𝑖, 𝑗 ∈ ℤ+ and 𝑖 ≠ 𝑗}.

3. We know that ℎ(𝐿) is not regular, so 𝐿 is not regular.



ExampleProve that



ℎ(0) = 0, ℎ(1) = 1, ℎ(2) = 𝜀.


ℎ(𝐿) = {0𝑖1𝑗 ∣ 𝑖, 𝑗 ∈ ℤ+ and 𝑖 ≠ 𝑗}.




ExampleProve that



ℎ(0) = 0, ℎ(1) = 1, ℎ(2) = 𝜀.


ℎ(𝐿) = {0𝑖1𝑗 ∣ 𝑖, 𝑗 ∈ ℤ+ and 𝑖 ≠ 𝑗}.




ExampleProve that



ℎ(0) = 0, ℎ(1) = 1, ℎ(2) = 𝜀.


ℎ(𝐿) = {0𝑖1𝑗 ∣ 𝑖, 𝑗 ∈ ℤ+ and 𝑖 ≠ 𝑗}.




ExampleLet 𝐿 be a regular language and ℎ a homomorphism on 𝐿. Define ℎ∗(𝐿) by

ℎ∗(𝐿) = 𝐿 ∪ ℎ(𝐿) ∪ ℎ(ℎ(𝐿)) ∪ ℎ(ℎ(ℎ(𝐿))) ∪ …

Is ℎ∗(𝐿) necessarily regular?

No. Let 𝐿 = {01} and ℎ defined as ℎ(0) = 00 and ℎ(1) = 11. Then

ℎ∗(𝐿) = {01, 0011, 00001111, … }= {0𝑛1𝑛 ∣ 𝑛 = 2𝑘 for 𝑘 ≥ 0},

which is a language not regular.†

†From somewhere in Internet (I don’t remember).Closure Properties of Regular Languages 55/67


ExampleLet 𝐿 be a regular language and ℎ a homomorphism on 𝐿. Define ℎ∗(𝐿) by

ℎ∗(𝐿) = 𝐿 ∪ ℎ(𝐿) ∪ ℎ(ℎ(𝐿)) ∪ ℎ(ℎ(ℎ(𝐿))) ∪ …

Is ℎ∗(𝐿) necessarily regular?No. Let 𝐿 = {01} and ℎ defined as ℎ(0) = 00 and ℎ(1) = 11. Then

ℎ∗(𝐿) = {01, 0011, 00001111, … }= {0𝑛1𝑛 ∣ 𝑛 = 2𝑘 for 𝑘 ≥ 0},

which is a language not regular.†

†From somewhere in Internet (I don’t remember).Closure Properties of Regular Languages 56/67

Closure Under Inverse Homomorphism

DefinitionLet ℎ ∶ Σ∗ → Γ∗ be a homomorphism and 𝐿 ⊆ Γ∗ a language. Theapplication of ℎ−1 to 𝐿, denoted ℎ−1(𝐿), is defined by†

ℎ−1(𝐿) = {𝑤 ∈ Σ∗ ∣ ℎ(𝑤) ∈ 𝐿}.

Inverse Homomorphism

Let h : Σ∗ → Θ∗ be a homom. Let L ⊆ Θ∗,and define

h−1(L) = {w ∈ Σ∗ : h(w) ∈ L}

L h(L)

Lh-1 (L)

(a)

(b)

h

h

110RemarkNote that ℎ−1 is a relation but it is not necessarily a function.

†Figure from Hopcroft, Motwani and Ullman (2007, Fig. 4.5b).Closure Properties of Regular Languages 57/67


ExampleLet ℎ ∶ {𝑎, 𝑏} → {0, 1}∗ a homomorphism defined by

ℎ(𝑎) = 01,ℎ(𝑏) = 10.

If 𝐿 = 𝐿((00 + 1)∗), then ℎ−1(𝐿) = 𝐿((ba)∗)Note that ℎ−1 is not a function, but a relation.(It is necessary to prove ℎ(𝑤) ∈ 𝐿 ⇔ 𝑤 = 𝑏𝑎𝑏𝑎 ⋯ 𝑏𝑎).



Theorem (4.16)Let ℎ ∶ Σ∗ → Γ∗ be a homomorphism and 𝐿 ⊆ Γ∗ a regular language. Thenℎ−1(𝐿) is regular (proof using automata).



ExampleProve that 𝐿 = {0𝑛12𝑛 ∣ 𝑛 ≥ 0} is a language not regular.

Proof.1. Given the homomorphism

ℎ(0) = 0,ℎ(1) = 11,

then ℎ−1(𝐿) = {0𝑛1𝑛 ∣ 𝑛 ≥ 0}.2. Since ℎ−1(𝐿) is not regular, then 𝐿 is not regular.





ℎ(0) = 0,ℎ(1) = 11,

then ℎ−1(𝐿) = {0𝑛1𝑛 ∣ 𝑛 ≥ 0}.

2. Since ℎ−1(𝐿) is not regular, then 𝐿 is not regular.





ℎ(0) = 0,ℎ(1) = 11,

then ℎ−1(𝐿) = {0𝑛1𝑛 ∣ 𝑛 ≥ 0}.2. Since ℎ−1(𝐿) is not regular, then 𝐿 is not regular.


Closure Properties

Exercise (4.2.2)If 𝐿 is a language, and 𝑎 is a symbol, then 𝐿/𝑎, the quotient of 𝐿 and 𝑎, isthe set of strings 𝑤 such that 𝑤𝑎 is in 𝐿. For example, if 𝐿 = {𝑎, 𝑎𝑎𝑏, 𝑏𝑎𝑎},then 𝐿/𝑎 = {𝜀, 𝑏𝑎}. Prove that if 𝐿 is regular, so is 𝐿/𝑎. Hint: Start witha DFA for 𝐿 and consider the set of accepting states.

Proof (Hopcroft, Motwani and Ullman (2007) solution).Start with a DFA 𝐴 for 𝐿. Construct a new DFA 𝐵, that is exactly the sameas 𝐴, except that state 𝑞 is an accepting state of 𝐵 if and only if 𝛿(𝑞, 𝑎)is an accepting state of 𝐴. Then 𝐵 accepts input string 𝑤 if and only if 𝐴accepts 𝑤𝑎; that is, 𝐿(𝐵) = 𝐿/𝑎.


Closure Properties

Exercise (4.2.2)If 𝐿 is a language, and 𝑎 is a symbol, then 𝐿/𝑎, the quotient of 𝐿 and 𝑎, isthe set of strings 𝑤 such that 𝑤𝑎 is in 𝐿. For example, if 𝐿 = {𝑎, 𝑎𝑎𝑏, 𝑏𝑎𝑎},then 𝐿/𝑎 = {𝜀, 𝑏𝑎}. Prove that if 𝐿 is regular, so is 𝐿/𝑎. Hint: Start witha DFA for 𝐿 and consider the set of accepting states.

Proof (Hopcroft, Motwani and Ullman (2007) solution).Start with a DFA 𝐴 for 𝐿. Construct a new DFA 𝐵, that is exactly the sameas 𝐴, except that state 𝑞 is an accepting state of 𝐵 if and only if 𝛿(𝑞, 𝑎)is an accepting state of 𝐴. Then 𝐵 accepts input string 𝑤 if and only if 𝐴accepts 𝑤𝑎; that is, 𝐿(𝐵) = 𝐿/𝑎.


Closure Properties

Exercise (4.2.3)If 𝐿 is a language, and 𝑎 is a symbol, then 𝑎\𝐿 is the set of strings 𝑤 suchthat 𝑎𝑤 is in 𝐿. For example, if 𝐿 = {𝑎, 𝑎𝑎𝑏, 𝑏𝑎𝑎}, then 𝑎\𝐿 = {𝜀, 𝑎𝑏}.Prove that if 𝐿 is regular, so is 𝑎\𝐿. Hint: Start with a DFA for 𝐿 andconsider its start state.

Proof (Hopcroft, Motwani and Ullman (2007) solution).Start with a DFA 𝐴 for 𝐿. Construct a new DFA 𝐵, that is exactly thesame as 𝐴, except that its start state is 𝛿(𝑞0, 𝑎) where 𝑞0 is the start stateof 𝐴. Then 𝐵 accepts input string 𝑤 if and only if 𝐴 accepts 𝑎𝑤; that is,𝐿(𝐵) = 𝐿\𝑎.


Closure Properties

Exercise (4.2.3)If 𝐿 is a language, and 𝑎 is a symbol, then 𝑎\𝐿 is the set of strings 𝑤 suchthat 𝑎𝑤 is in 𝐿. For example, if 𝐿 = {𝑎, 𝑎𝑎𝑏, 𝑏𝑎𝑎}, then 𝑎\𝐿 = {𝜀, 𝑎𝑏}.Prove that if 𝐿 is regular, so is 𝑎\𝐿. Hint: Start with a DFA for 𝐿 andconsider its start state.Proof (Hopcroft, Motwani and Ullman (2007) solution).Start with a DFA 𝐴 for 𝐿. Construct a new DFA 𝐵, that is exactly thesame as 𝐴, except that its start state is 𝛿(𝑞0, 𝑎) where 𝑞0 is the start stateof 𝐴. Then 𝐵 accepts input string 𝑤 if and only if 𝐴 accepts 𝑎𝑤; that is,𝐿(𝐵) = 𝐿\𝑎.


References

Birkhoff, G. (1946). Universal Algebra. In: Comptes Rendus du PremierCongrès Canadien de Mathématiques. University of Toronto Press,pp. 310–326 (cit. on pp. 33–35).— (1987). Universal Algebra. In: Selected Papers on Algebra andTopology by Garrett Birkhoff. Ed. by Rota, G.-C. and Oliveira, J. S.Birkhäuser, pp. 146–162 (cit. on pp. 33–35).Cohn, P. M. (1981). Universal Algebra. Revised edition. D. ReidelPublishing Company (cit. on p. 39).Hopcroft, J. E., Motwani, R. and Ullman, J. D. (2007). Introduction toAutomata theory, Languages, and Computation. 3rd ed. Pearson Education(cit. on pp. 16, 42, 57, 63–66).


Automata and Formal Languages - CM0081Problems That Computers Cannot Solve


Universidad EAFIT

Semester 2018-1

Undecidable Problems

RecallGiven 𝐿 ⊆ Σ∗ and a word 𝑤 ∈ Σ∗, to recall that to decide whether or not𝑤 ∈ 𝐿 is a (decision) problem on 𝐿.

Definition (informally)A problem is undecidable if no program can solve it.

Are there undecidable problems?Yes! (by cardinality)

1. {𝐿 ∣ 𝐿 ⊆ Σ∗} is not enumerable.2. {𝑃 ∣ 𝑃 is a program} is enumerable (lexicographical order).3. There are more languages than programs.4. Therefore, there must be undecidable problems.

Problems That Computers Cannot Solve 2/38




Are there undecidable problems?

Yes! (by cardinality)1. {𝐿 ∣ 𝐿 ⊆ Σ∗} is not enumerable.2. {𝑃 ∣ 𝑃 is a program} is enumerable (lexicographical order).3. There are more languages than programs.4. Therefore, there must be undecidable problems.





Are there undecidable problems?Yes! (by cardinality)

1. {𝐿 ∣ 𝐿 ⊆ Σ∗} is not enumerable.2. {𝑃 ∣ 𝑃 is a program} is enumerable (lexicographical order).3. There are more languages than programs.4. Therefore, there must be undecidable problems.


The hello-world Problem: An Undecidable Problem

“The first program to write is thesame for all languages: Print thewords hello, world.” [1978, §1.1]


The hello-world Problem

The hello-world problemTo determine whether a given program, with a given input, prints hello,world as the first 12 characters that it prints.

TheoremThe hello-world problem is an undecidable problem.



The hello-world problemTo determine whether a given program, with a given input, prints hello,world as the first 12 characters that it prints.

TheoremThe hello-world problem is an undecidable problem.



Informal proof1. We assume that the program H exists:

hello-word tester

H

P Yes

I No



Informal proof (cont.)2. To change the behaviour of H to:

H₁

P Yes

I hello, world



Informal proof (cont.)3. Restrict H₁ to take only input P (i.e. P is the input of the program P):

H₂P

Yes

hello, world



Informal proof (cont.)4. What does H₂ do when given itself as input?

H₂H₂

Yes

hello, world

Therefore, the program H₂ cannot exist.5. Therefore, the program H cannot exists.




H₂H₂

Yes

hello, world

Therefore, the program H₂ cannot exist.

5. Therefore, the program H cannot exists.




H₂H₂

Yes

hello, world

Therefore, the program H₂ cannot exist.5. Therefore, the program H cannot exists.


Computability: Historical Remarks

1900 David Hilbert’s problemsWir müssen wissen — wir werden wissen!(We must know — we will know!)

1931 Kurt Gödel’s undecidability theorems

1936 The year of the confluence!𝜆-calculus (Alonzo Church, Stephen Kleene)Partial recursive functions (Kurt Gödel)Post machines (Emil Post)Turing machines (Alan Turing)

1936–40 The Church-Turing thesis: A (number-theoretic) function iseffectively calculable if and only if there is a Turing machinewhich computes it.




















Computability: Historical Remarks (cont.)

1985 Deutsch’s quantum Turing machine

1940–Today Many equivalents models of computation


Computability: Historical Remarks (cont.)

1985 Deutsch’s quantum Turing machine1940–Today Many equivalents models of computation


Problems Reduction

How to prove that a problem is undecidable?Proof of negation

Problem reductionReduction from a problem 𝑃1 to a problem 𝑃2:

yes

no

yes

no

𝑃1 𝑃2


Problems Reduction

How to prove that a problem is undecidable?Proof of negationProblem reductionReduction from a problem 𝑃1 to a problem 𝑃2:

yes

no

yes

no

𝑃1 𝑃2


Problems Reduction

TheoremIf 𝑃1 is undecidable and there is a reduction of 𝑃1 to 𝑃2, then 𝑃2 is unde-cidable too.

Proof by contradiction.

reduction decide𝑃1 instance 𝑃2 instance

no

yes

1. Let’s suppose 𝑃2 decidable2. There is a reduction from 𝑃1 to 𝑃2

⇒ 𝑃1 decidable⇒ ⊥ (contradiction)

3. Therefore, 𝑃2 is undecidable


Problems Reduction




no

yes





Problems Reduction




no

yes





Problems Reduction

Example (The calls-foo problem)Problem: Does a program Q with an input z calls the function foo?

Prove that the calls-foo problem is undecidable.


Problems Reduction

Example (The calls-foo problem)Problem: Does a program Q with an input z calls the function foo?Prove that the calls-foo problem is undecidable.


Problems Reduction

Example (cont.)

Proof.Idea: Reduction of the hello-world problem to the calls-foo problem.

That is, P with input y prints hello, world iff Q with input z calls thefunction foo.Reduction:

1. P₁: Rename foo in P.2. P₂: Add foo to P₁.3. P₃: Save the first 12 characters that prints P₂.4. P₄: When P₃ executes an output statement if output is hello, world

then calls foo.5. Q = P₄ and y = z.


Problems Reduction

Example (cont.)

Proof.Idea: Reduction of the hello-world problem to the calls-foo problem.That is, P with input y prints hello, world iff Q with input z calls thefunction foo.

Reduction:1. P₁: Rename foo in P.2. P₂: Add foo to P₁.3. P₃: Save the first 12 characters that prints P₂.4. P₄: When P₃ executes an output statement if output is hello, world



Problems Reduction

Example (cont.)

Proof.Idea: Reduction of the hello-world problem to the calls-foo problem.That is, P with input y prints hello, world iff Q with input z calls thefunction foo.Reduction:

1. P₁: Rename foo in P.2. P₂: Add foo to P₁.3. P₃: Save the first 12 characters that prints P₂.4. P₄: When P₃ executes an output statement if output is hello, world



Problems Reduction

Exercise (The halting problem, 8.1.1.a)Problem: Given a program and an input, does the program eventuallyhalt; i.e., does the program not loop forever on the input?

Prove that the halting problem is undecidable.


Problems Reduction

Exercise (The halting problem, 8.1.1.a)Problem: Given a program and an input, does the program eventuallyhalt; i.e., does the program not loop forever on the input?Prove that the halting problem is undecidable.


Problems Reduction

Exercise (cont.)

Proof (based on the solutions to Hopcroft, Motwani and Ullman (2007)).Idea: Reduction of the hello-world problem to the halting problem.

That is, P with input y prints hello, world iff Q with input z halt.Reduction:

1. P₁: Add an infinite loop (e.g. while(1){x=x;}) to the end of main.2. P₂: Save the first 12 characters that prints P₁.3. P₃: When P₂ executes an output statement if output is hello, world

then P₃ halts by going to the end of main.4. Q = P₃ and y = z.


Problems Reduction

Exercise (cont.)

Proof (based on the solutions to Hopcroft, Motwani and Ullman (2007)).Idea: Reduction of the hello-world problem to the halting problem.That is, P with input y prints hello, world iff Q with input z halt.

Reduction:1. P₁: Add an infinite loop (e.g. while(1){x=x;}) to the end of main.2. P₂: Save the first 12 characters that prints P₁.3. P₃: When P₂ executes an output statement if output is hello, world



Problems Reduction

Exercise (cont.)

Proof (based on the solutions to Hopcroft, Motwani and Ullman (2007)).Idea: Reduction of the hello-world problem to the halting problem.That is, P with input y prints hello, world iff Q with input z halt.Reduction:

1. P₁: Add an infinite loop (e.g. while(1){x=x;}) to the end of main.2. P₂: Save the first 12 characters that prints P₁.3. P₃: When P₂ executes an output statement if output is hello, world



Problems Reduction

TheoremIf 𝑃1 is undecidable and there is a reduction of 𝑃1 to 𝑃2, then 𝑃2 is unde-cidable too.Be careful! The reduction is from 𝑃1 to 𝑃2, it is not from 𝑃2 to 𝑃1:

From a reduction of 𝑃1 to 𝑃2:

𝑃2 decidable ⇒ 𝑃1 decidable𝑃1 undecidable ⇒ 𝑃2 undecidable


𝑃1 decidable ⇒ 𝑃2 decidable (hypothesis false)𝑃2 undecidable ⇒ 𝑃1 undecidable (hypothesis unknown)


Problems Reduction







Problems Reduction







References

Hopcroft, J. E., Motwani, R. and Ullman, J. D. (2007). Introduction toAutomata theory, Languages, and Computation. 3rd ed. Pearson Education(cit. on pp. 32–34).


Automata and Formal Languages - CM0081Turing Machines


Universidad EAFIT

Semester 2018-1

Turing Machines

Alan Mathison Turing(1912 – 1954)

Turing Machines 2/45

Turing Machines

Unbounded tape divided into discrete squares which contain symbolsfrom a finite alphabet

Read/Write headFinite set of instructions (transition function)Move of a Turing machine (tape symbol under the head, currentstate): change state, rewrite the symbol and move the head onesquare


Turing Machines

Unbounded tape divided into discrete squares which contain symbolsfrom a finite alphabetRead/Write head

Finite set of instructions (transition function)Move of a Turing machine (tape symbol under the head, currentstate): change state, rewrite the symbol and move the head onesquare


Turing Machines

Unbounded tape divided into discrete squares which contain symbolsfrom a finite alphabetRead/Write headFinite set of instructions (transition function)

Move of a Turing machine (tape symbol under the head, currentstate): change state, rewrite the symbol and move the head onesquare


Turing Machines

Unbounded tape divided into discrete squares which contain symbolsfrom a finite alphabetRead/Write headFinite set of instructions (transition function)Move of a Turing machine (tape symbol under the head, currentstate): change state, rewrite the symbol and move the head onesquare


Turing Machines

A Turing machine is a 7-tuple 𝑀 = (𝑄, Σ, Γ, 𝛿, 𝑞0, 𝐵, 𝐹) where

𝑄: A finite set of statesΣ: An alphabet of input symbolsΓ: An alphabet of tape symbols (Σ ⊆ Γ)

𝛿 ∶ 𝑄 × Γ → 𝑄 × Γ × 𝐷: A transition (partial) functionwhere 𝐷 = {𝐿, 𝑅} is the set of moves

𝑞0 ∈ 𝑄: A start state𝐵: The blank symbol (𝐵 ∈ Γ, 𝐵 ∉ Σ)

𝐹 ⊆ 𝑄: A set of final or accepting states


Transition Diagrams for Turing Machines

Example

𝑞0

start

𝑞1 𝑞2

𝑞3 𝑞4

0/𝑋 →

𝑌 /𝑌 →

𝑌 /𝑌 →0/0 →

1/𝑌 ←

𝑌 /𝑌 ←0/0 ←

𝑋/𝑋 →

𝑌 /𝑌 →𝐵/𝐵 →

where Σ = {0, 1} and Γ = {0, 1, 𝑋, 𝑌 , 𝐵}.


Transition Tables for Turing Machines

ExampleThe machine of the previous example is given by𝑀 = ({𝑞0, 𝑞1, 𝑞2.𝑞3, 𝑞4}, {0, 1}, {0, 1, 𝑋, 𝑌 , 𝐵}, 𝛿, 𝑞0, 𝐵, {𝑞4}) where 𝛿 isgiven by

state 0 1 𝑋 𝑌 𝐵𝑞0 (𝑞1, 𝑋, 𝑅) — — (𝑞3, 𝑌 , 𝑅) —𝑞1 (𝑞1, 0, 𝑅) (𝑞2, 𝑌 , 𝐿) — (𝑞1, 𝑌 , 𝑅) —𝑞2 (𝑞2, 0, 𝐿) — (𝑞0, 𝑋, 𝑅) (𝑞2, 𝑌 , 𝐿) —𝑞3 — — — (𝑞3, 𝑌 , 𝑅) (𝑞4, 𝐵, 𝑅)𝑞4 — — — — —


Quintuples for Turing Machines

ExampleThe machine of the previous example is given by𝑀 = ({𝑞0, 𝑞1, 𝑞2.𝑞3, 𝑞4}, {0, 1}, {0, 1, 𝑋, 𝑌 , 𝐵}, 𝛿, 𝑞0, 𝐵, {𝑞4}), where 𝛿 isgiven by

𝑞0, 0, 𝑋, 𝑅, 𝑞1𝑞0, 𝑌 , 𝑌 , 𝑅, 𝑞3𝑞1, 0, 0, 𝑅, 𝑞1𝑞1, 1, 𝑌 , 𝐿, 𝑞2𝑞1, 𝑌 , 𝑌 , 𝑅, 𝑞1𝑞2, 0, 0, 𝐿, 𝑞2

𝑞2, 𝑋, 𝑋, 𝑅, 𝑞0𝑞2, 𝑌 , 𝑌 , 𝐿, 𝑞2𝑞3, 𝑌 , 𝑌 , 𝑅, 𝑞3𝑞3, 𝐵, 𝐵, 𝑅, 𝑞4


Instantaneous Descriptions for Turing Machines

DefinitionAn instantaneous description of a Turing machine is a string

𝑋1𝑋2 ⋯ 𝑋𝑖−1𝑞𝑋𝑖𝑋𝑖+1 ⋯ 𝑋𝑛,

wherei) 𝑞 is the state of the Turing machine,ii) the head is scanning the 𝑖-th symbol from the left andiii) 𝑋1𝑋2 ⋯ 𝑋𝑛 is the portion of the tape between the leftmost and

rightmost non-blank.



Notation⊢𝑀

: Move of the Turing machine 𝑀 from an instantaneous descriptions toanother.∗⊢𝑀

: Zero o more moves of the Turing machine 𝑀 .



Example0 0 1

𝑞0

𝑋 0 1

𝑞1

𝑋 0 1

𝑞2

𝑞0001 ⊢𝑀

𝑋𝑞101 ⊢𝑀

𝑋0𝑞21

𝑞0001∗⊢𝑀

𝑋0𝑞21


Recursively Enumerable Languages

DefinitionLet 𝑀 = (𝑄, Σ, Γ, 𝛿, 𝑞0, 𝐵, 𝐹) be a Turing machine. The language acceptedby 𝑀 is

𝐿(𝑀) = {𝑤 ∈ Σ∗ ∣ 𝑞0𝑤∗⊢𝑀

𝛼𝑝𝛽},

where 𝑝 ∈ 𝐹 and 𝛼, 𝛽 ∈ Γ∗.

DefinitionA language 𝐿 is recursively enumerable (RE) if exists a Turing machine 𝑀such 𝐿 = 𝐿(𝑀).



DefinitionLet 𝑀 = (𝑄, Σ, Γ, 𝛿, 𝑞0, 𝐵, 𝐹) be a Turing machine. The language acceptedby 𝑀 is

𝐿(𝑀) = {𝑤 ∈ Σ∗ ∣ 𝑞0𝑤∗⊢𝑀

𝛼𝑝𝛽},

where 𝑝 ∈ 𝐹 and 𝛼, 𝛽 ∈ Γ∗.

DefinitionA language 𝐿 is recursively enumerable (RE) if exists a Turing machine 𝑀such 𝐿 = 𝐿(𝑀).



ExampleLet 𝑀 be the machine described by the previous diagram. Then𝐿(𝑀) = {0𝑛1𝑛 ∣ 𝑛 ≥ 1}.See the simulation in the course website.


Recursive Languages

ConventionWe assume that a Turing machine halts if it accepts.

What about if the Turing machine does not accept?

RecallRecall that a language 𝐿 is recursively enumerable if exists a Turing ma-chine 𝑀 such that 𝐿 = 𝐿(𝑀).

DefinitionA language 𝐿 is recursive if exists a Turing machine 𝑀 such that

i) 𝐿 = 𝐿(𝑀) andii) 𝑀 always halt (even if it does not accept)


Recursive Languages







Recursive Languages







Recursive Languages







Turing Machine Computable Functions

Number-theoretical functionsThe number-theoretical functions are the functions

{𝑓 ∣ 𝑓 ∶ ℕ𝑘 → ℕ}.

ExampleThe following functions are number-theoretical functions:

𝑧(𝑥) = 0 (zero function)𝑠(𝑥) = 𝑥 + 1 (successor function)

𝐼𝑛𝑘 (𝑥1, … , 𝑥𝑛) = 𝑥𝑘 (𝑛-ary projection functions)

𝑖(𝑥) = 𝑥 (identity function)𝑓𝑘(𝑥) = 𝑘 (𝑘-constant function)

𝑥 + 𝑦, 𝑥𝑦, 𝑥𝑦 (addition, multiplication and exponentiation)




{𝑓 ∣ 𝑓 ∶ ℕ𝑘 → ℕ}.

ExampleThe following functions are number-theoretical functions:







Codification of 𝑘-tuple of natural numbers

�� = 0𝑛 = 0 ⋯ 0⏟𝑛 times

for 𝑛 ∈ ℕ

(𝑛1, 𝑛2, … , 𝑛𝑘) = 𝑛11 𝑛21 ⋯ 1 𝑛𝑘 for (𝑛1, 𝑛2, … , 𝑛𝑘) ∈ ℕ𝑘



DefinitionA function 𝑓 ∶ ℕ → ℕ is Turing machine computable iff exists a machine𝑀 = (𝑄, {0, 1}, Γ, 𝛿, 𝑞0, 𝐵) (there are not accepting states) such that forall 𝑛 ∈ ℕ:From the initial instantaneous description 𝑞0 �� the machine halts with 𝑓(𝑛)on its tape, surrounded by blanks.

RemarkThe definition extends to functions 𝑓 ∶ ℕ𝑘 → ℕ.



DefinitionA function 𝑓 ∶ ℕ → ℕ is Turing machine computable iff exists a machine𝑀 = (𝑄, {0, 1}, Γ, 𝛿, 𝑞0, 𝐵) (there are not accepting states) such that forall 𝑛 ∈ ℕ:From the initial instantaneous description 𝑞0 �� the machine halts with 𝑓(𝑛)on its tape, surrounded by blanks.

RemarkThe definition extends to functions 𝑓 ∶ ℕ𝑘 → ℕ.



ExamplesThe functions𝑧(𝑥) (zero function),𝑠(𝑥) (successor function),𝐼𝑛

𝑘 (𝑥1, … , 𝑥𝑛) (𝑛-ary projection functions),𝑖(𝑥) (identity function),𝑓𝑘(𝑥) (𝑘-constant function),𝑥 + 𝑦, 𝑥𝑦 and 𝑥𝑦

are Turing machine computable functions.



ExampleThe proper subtraction function is Turing machine computable.

𝑚 ∸ 𝑛 = {𝑚 − 𝑛, if 𝑚 ≥ 𝑛;0, otherwise.

Initial instantaneous description: 𝑞00𝑚10𝑛

Final information on the tape: 0𝑚∸𝑛

See the simulation in the course homepage.


Equivalence between Function Computation andLanguage Recognition

Exercise (8.2.4)Define the graph of a function 𝑓 ∶ ℕ → ℕ to be the set of all stringsof the form [ 𝑥, 𝑓(𝑥)].

A Turing machine is said to compute the function 𝑓 ∶ ℕ → ℕ if,started with 𝑥 on its tape, it halts (in any state) with 𝑓(𝑥) on its tape.

Answer the following, with informal, but clear constructions.



Exercise (8.2.4)Define the graph of a function 𝑓 ∶ ℕ → ℕ to be the set of all stringsof the form [ 𝑥, 𝑓(𝑥)].A Turing machine is said to compute the function 𝑓 ∶ ℕ → ℕ if,started with 𝑥 on its tape, it halts (in any state) with 𝑓(𝑥) on its tape.




Exercise (8.2.4)Define the graph of a function 𝑓 ∶ ℕ → ℕ to be the set of all stringsof the form [ 𝑥, 𝑓(𝑥)].A Turing machine is said to compute the function 𝑓 ∶ ℕ → ℕ if,started with 𝑥 on its tape, it halts (in any state) with 𝑓(𝑥) on its tape.




Exercise (cont.)1. Show how, given a Turing machine that computes 𝑓 , you can construct

a Turing machine that accepts the graph of 𝑓 as a language.

2. Show how, given a Turing machine that accepts the graph of 𝑓 , youcan construct a Turing machine that computes 𝑓 .

3. A function is said to partial if it may be undefined for some arguments.If we extend the ideas of this exercise to partial functions, then we donot require that the Turing machine computing 𝑓 halts if its input 𝑥 isone of the natural numbers for which 𝑓(𝑥) is not defined.Do your constructions for parts (1) and (2) work if the function 𝑓 ispartial? If not, explain how you could modify the constructions to makeit work.




a Turing machine that accepts the graph of 𝑓 as a language.2. Show how, given a Turing machine that accepts the graph of 𝑓 , you

can construct a Turing machine that computes 𝑓 .

3. A function is said to partial if it may be undefined for some arguments.If we extend the ideas of this exercise to partial functions, then we donot require that the Turing machine computing 𝑓 halts if its input 𝑥 isone of the natural numbers for which 𝑓(𝑥) is not defined.Do your constructions for parts (1) and (2) work if the function 𝑓 ispartial? If not, explain how you could modify the constructions to makeit work.





can construct a Turing machine that computes 𝑓 .3. A function is said to partial if it may be undefined for some arguments.

If we extend the ideas of this exercise to partial functions, then we donot require that the Turing machine computing 𝑓 halts if its input 𝑥 isone of the natural numbers for which 𝑓(𝑥) is not defined.

Do your constructions for parts (1) and (2) work if the function 𝑓 ispartial? If not, explain how you could modify the constructions to makeit work.





can construct a Turing machine that computes 𝑓 .3. A function is said to partial if it may be undefined for some arguments.

If we extend the ideas of this exercise to partial functions, then we donot require that the Turing machine computing 𝑓 halts if its input 𝑥 isone of the natural numbers for which 𝑓(𝑥) is not defined.Do your constructions for parts (1) and (2) work if the function 𝑓 ispartial? If not, explain how you could modify the constructions to makeit work.


Restrictions to Turing Machines

RestrictionsTuring machines with semi-unbounded tapesMulti-stack machines

TheoremThe previous restrictions are equivalents to Turing machines.



RestrictionsTuring machines with semi-unbounded tapes

Multi-stack machines











Extensions to Turing Machines

ExtensionsMulti-tape Turing machinesMutil-dimensional tape Turing machinesMulti-head Turing machinesNon-deterministic Turing machinesSubroutines

TheoremThe previous extensions are equivalents to Turing machines.



ExtensionsMulti-tape Turing machines

Mutil-dimensional tape Turing machinesMulti-head Turing machinesNon-deterministic Turing machinesSubroutines




ExtensionsMulti-tape Turing machinesMutil-dimensional tape Turing machines

Multi-head Turing machinesNon-deterministic Turing machinesSubroutines




ExtensionsMulti-tape Turing machinesMutil-dimensional tape Turing machinesMulti-head Turing machines

Non-deterministic Turing machinesSubroutines




ExtensionsMulti-tape Turing machinesMutil-dimensional tape Turing machinesMulti-head Turing machinesNon-deterministic Turing machines

Subroutines











Automata and Formal Languages - CM0081A Language That Is Not Recursively Enumerable


Universidad EAFIT

Semester 2018-1

Undecidability

RecallA language 𝐿 is recursively enumerable if exists a Turing machine 𝑀such that 𝐿 = 𝐿(𝑀).

A language 𝐿 is recursive if exists a Turing machine 𝑀 such thati) 𝐿 = 𝐿(𝑀) andii) 𝑀 always halt (even if it does not accept).

DefinitionA language 𝐿 is undecidable if 𝐿 is not recursive.

A Language That Is Not Recursively Enumerable 2/19

Undecidability

RecallA language 𝐿 is recursively enumerable if exists a Turing machine 𝑀such that 𝐿 = 𝐿(𝑀).A language 𝐿 is recursive if exists a Turing machine 𝑀 such that

i) 𝐿 = 𝐿(𝑀) andii) 𝑀 always halt (even if it does not accept).



Undecidability

RecallA language 𝐿 is recursively enumerable if exists a Turing machine 𝑀such that 𝐿 = 𝐿(𝑀).A language 𝐿 is recursive if exists a Turing machine 𝑀 such that

i) 𝐿 = 𝐿(𝑀) andii) 𝑀 always halt (even if it does not accept).



Why “Recursive”?

The term “recursive” as synonym for “decidable” comes fromMathematics (prior to computers).

Equivalent formalization to Turing-machine computability based onrecursive functions.A function is recursive if only if it is Turing-machine computable (see,for example, (Boolos, Burges and Jeffrey 2007), (Hermes 1969) or(Kleene 1952)).Recursive problem: “it is sufficiently simple that I can write arecursive function to solve it, and the function alwaysfinishes.” (Hopcroft, Motwani and Ullman 2007, p. 385)



The term “recursive” as synonym for “decidable” comes fromMathematics (prior to computers).Equivalent formalization to Turing-machine computability based onrecursive functions.

A function is recursive if only if it is Turing-machine computable (see,for example, (Boolos, Burges and Jeffrey 2007), (Hermes 1969) or(Kleene 1952)).Recursive problem: “it is sufficiently simple that I can write arecursive function to solve it, and the function alwaysfinishes.” (Hopcroft, Motwani and Ullman 2007, p. 385)



The term “recursive” as synonym for “decidable” comes fromMathematics (prior to computers).Equivalent formalization to Turing-machine computability based onrecursive functions.A function is recursive if only if it is Turing-machine computable (see,for example, (Boolos, Burges and Jeffrey 2007), (Hermes 1969) or(Kleene 1952)).

Recursive problem: “it is sufficiently simple that I can write arecursive function to solve it, and the function alwaysfinishes.” (Hopcroft, Motwani and Ullman 2007, p. 385)



The term “recursive” as synonym for “decidable” comes fromMathematics (prior to computers).Equivalent formalization to Turing-machine computability based onrecursive functions.A function is recursive if only if it is Turing-machine computable (see,for example, (Boolos, Burges and Jeffrey 2007), (Hermes 1969) or(Kleene 1952)).Recursive problem: “it is sufficiently simple that I can write arecursive function to solve it, and the function alwaysfinishes.” (Hopcroft, Motwani and Ullman 2007, p. 385)


Codification of Turing Machines

ConventionThe Turing machine 𝑀 is of the form:

𝑀 = ({𝑞1, … , 𝑞𝑛}, {0, 1}, {𝑋1, 𝑋2, 𝑋3, … , 𝑋𝑚}, 𝛿, 𝑞1, 𝐵, {𝑞2}),

where 𝑋1 = 0, 𝑋2 = 1 and 𝑋3 = 𝐵. Moreover, 𝐷1 = 𝐿 and 𝐷2 = 𝑅.

Codification of an instructionThe instruction 𝛿(𝑞𝑖, 𝑋𝑗) = (𝑞𝑘, 𝑋𝑙, 𝐷𝑚) is codified by

0𝑖10𝑗10𝑘10𝑙10𝑚.



ConventionThe Turing machine 𝑀 is of the form:

𝑀 = ({𝑞1, … , 𝑞𝑛}, {0, 1}, {𝑋1, 𝑋2, 𝑋3, … , 𝑋𝑚}, 𝛿, 𝑞1, 𝐵, {𝑞2}),

where 𝑋1 = 0, 𝑋2 = 1 and 𝑋3 = 𝐵. Moreover, 𝐷1 = 𝐿 and 𝐷2 = 𝑅.

Codification of an instructionThe instruction 𝛿(𝑞𝑖, 𝑋𝑗) = (𝑞𝑘, 𝑋𝑙, 𝐷𝑚) is codified by

0𝑖10𝑗10𝑘10𝑙10𝑚.



Codification of a Turing machineLet 𝐶1, 𝐶2, … , 𝐶𝑝 be the codifications of the instructions of a Turing ma-chine 𝑀 . The codification of 𝑀 is

𝑀 = 𝐶111𝐶211 … 11𝐶𝑝.

RemarkNote that there are other possible codes for 𝑀 .



Codification of a Turing machineLet 𝐶1, 𝐶2, … , 𝐶𝑝 be the codifications of the instructions of a Turing ma-chine 𝑀 . The codification of 𝑀 is

𝑀 = 𝐶111𝐶211 … 11𝐶𝑝.

RemarkNote that there are other possible codes for 𝑀 .



A enumeration for the binary stringsWe ordered the binary strings by [length-]lexicographical order (strings areordered by length, and strings of equal length are ordered lexicographically).If 𝑤 is a binary string, we call 𝑤 the 𝑖-th string where 1𝑤 is the binaryinteger 𝑖. We refer to the 𝑖-th string as 𝑤𝑖.

𝑖-th Turing machineGiven a Turing machine 𝑀 with code 𝑤𝑖, we can now associate a naturalnumber to it: 𝑀 is the 𝑖-th Turing machine, referred to as 𝑀𝑖.

ConventionIf 𝑤𝑖 is not a valid Turing machine code, we shall take 𝑀𝑖 to be the Turingmachine with one state and no transitions. Hence 𝐿(𝑀𝑖) = ∅ if 𝑤𝑖 is not avalid Turing machine code.












The Diagonalization Language

The diagonalization languageA language that is not recursively enumerable.

Ld = {𝑤𝑖 ∣ 𝑤𝑖 ∉ 𝐿(𝑀𝑖)}.

𝑤𝑗

𝑀𝑖

1 2 3 4 …1 0 1 1 0 …2 1 1 0 1 …3 0 1 1 0 …4 1 1 0 0 …⋮ ⋮ ⋮ ⋮ ⋮ ⋱

𝑎𝑖𝑗 = {1, if 𝑤𝑗 ∈ 𝐿(𝑀𝑖);0, if 𝑤𝑗 ∉ 𝐿(𝑀𝑖).

Vector for the language 𝐿(𝑀𝑖): 𝑖-th rowLd: Complement of the diagonalIs it possible that Ld be in a row?



The diagonalization languageA language that is not recursively enumerable.

Ld = {𝑤𝑖 ∣ 𝑤𝑖 ∉ 𝐿(𝑀𝑖)}.

𝑤𝑗

𝑀𝑖

1 2 3 4 …1 0 1 1 0 …2 1 1 0 1 …3 0 1 1 0 …4 1 1 0 0 …⋮ ⋮ ⋮ ⋮ ⋮ ⋱

𝑎𝑖𝑗 = {1, if 𝑤𝑗 ∈ 𝐿(𝑀𝑖);0, if 𝑤𝑗 ∉ 𝐿(𝑀𝑖).

Vector for the language 𝐿(𝑀𝑖): 𝑖-th rowLd: Complement of the diagonalIs it possible that Ld be in a row?



Theorem (9.2)Ld is not recursively enumerable.

ProofWhiteboard.


References

Boolos, G. S., Burges, J. P. and Jeffrey, R. C. (2007). Computability andLogic. 5th ed. Cambridge University Press (cit. on pp. 5–8).Hermes, H. (1969). Enumerability ⋅ Decidability ⋅ Computability. Secondrevised edition. Springer-Verlag (cit. on pp. 5–8).Hopcroft, J. E., Motwani, R. and Ullman, J. D. (2007). Introduction toAutomata theory, Languages, and Computation. 3rd ed. Pearson Education(cit. on pp. 5–8).Kleene, S. C. (1952). Introduction to Metamathematics. North-Holland(cit. on pp. 5–8).


Automata and Formal Languages - CM0081An Undecidable Problem That Is Recursively

Enumerable


Universidad EAFIT

Semester 2018-1

Theorems About Recursive Languages

Theorem (9.3)If 𝐿 is a recursive language, so is 𝐿.

ProofWhiteboard.

Theorem (9.4)If both 𝐿 and its complement are recursively enumerable, then 𝐿 is recursive(and by Theorem 9.3, 𝐿 is recursive as well).

ProofWhiteboard.

An Undecidable Problem That Is Recursively Enumerable 2/32


Theorem (9.3)If 𝐿 is a recursive language, so is 𝐿.

ProofWhiteboard.

Theorem (9.4)If both 𝐿 and its complement are recursively enumerable, then 𝐿 is recursive(and by Theorem 9.3, 𝐿 is recursive as well).

ProofWhiteboard.



Possible relations between a language 𝐿 and its complement 𝐿:i) Both 𝐿 and 𝐿 are recursive.

ii) Neither 𝐿 nor 𝐿 are recursively enumerable.iii) 𝐿 is recursively enumerable but not recursive and 𝐿 is not recursively

enumerable.iv) 𝐿 is recursively enumerable but not recursive and 𝐿 is not recursively

enumerable.



Possible relations between a language 𝐿 and its complement 𝐿:i) Both 𝐿 and 𝐿 are recursive.ii) Neither 𝐿 nor 𝐿 are recursively enumerable.

iii) 𝐿 is recursively enumerable but not recursive and 𝐿 is not recursivelyenumerable.

iv) 𝐿 is recursively enumerable but not recursive and 𝐿 is not recursivelyenumerable.



Possible relations between a language 𝐿 and its complement 𝐿:i) Both 𝐿 and 𝐿 are recursive.ii) Neither 𝐿 nor 𝐿 are recursively enumerable.iii) 𝐿 is recursively enumerable but not recursive and 𝐿 is not recursively

enumerable.

iv) 𝐿 is recursively enumerable but not recursive and 𝐿 is not recursivelyenumerable.



Possible relations between a language 𝐿 and its complement 𝐿:i) Both 𝐿 and 𝐿 are recursive.ii) Neither 𝐿 nor 𝐿 are recursively enumerable.iii) 𝐿 is recursively enumerable but not recursive and 𝐿 is not recursively

enumerable.iv) 𝐿 is recursively enumerable but not recursive and 𝐿 is not recursively

enumerable.



Exercise (9.2.5)Let 𝐿 be recursively enumerable and let 𝐿 be non-RE. Consider thelanguage

𝐿′ = {0𝑤 ∣ 𝑤 is in 𝐿} ∪ {1𝑤 ∣ 𝑤 is not in 𝐿}.Can you say for certain whether 𝐿′ is recursive, RE, or non-RE? Justifyyour answer.

Solution (from Hopcroft, Motwani and Ullman (2007))Suppose 𝐿′ were RE. Then we could design a Turing machine 𝑀 for 𝐿as follows. Given input 𝑤, 𝑀 changes its input to 1𝑤 and simulates thehypothetical Turing machine for 𝐿′. If that Turing machine accepts, then 𝑤is in 𝐿, so 𝑀 should accept. If the Turing machine for 𝐿′ never accepts,then neither does 𝑀 . Thus, 𝑀 would accept exactly 𝐿, which contradictsthe fact that 𝐿 is not RE. We conclude that 𝐿′ is not RE.


The Universal Language

Conventions1. (𝑀, 𝑤): Represents the Turing machine 𝑀 with input 𝑤.2. 𝑤 is a string of 0’s and 1’s.

Codification of a Turing machine with an inputLet 𝑤𝑖 be the codification of a Turing machine 𝑀 . The codification of(𝑀, 𝑤) is

(𝑀, 𝑤) = 𝑤𝑖111𝑤.



The universal language Lu

An undecidable language that is recursively enumerable.

The set of binary strings that encode a pair (𝑀, 𝑤) such that 𝑤 ∈ 𝐿(𝑀),

Lu = { (𝑀, 𝑤) ∣ 𝑤 ∈ 𝐿(𝑀)} .TheoremLu is recursively enumerable.

Idea of the proofExists a Turing machine U such that Lu = 𝐿(U). The machine U is calleda universal Turing machine.




An undecidable language that is recursively enumerable.The set of binary strings that encode a pair (𝑀, 𝑤) such that 𝑤 ∈ 𝐿(𝑀),

Lu = { (𝑀, 𝑤) ∣ 𝑤 ∈ 𝐿(𝑀)} .

TheoremLu is recursively enumerable.





An undecidable language that is recursively enumerable.The set of binary strings that encode a pair (𝑀, 𝑤) such that 𝑤 ∈ 𝐿(𝑀),

Lu = { (𝑀, 𝑤) ∣ 𝑤 ∈ 𝐿(𝑀)} .TheoremLu is recursively enumerable.




TheoremLu is not recursive.

Proof.Suppose Lu is recursive ⇒Lu is recursive ⇒Ld is recursive ⇒⊥ (contradiction).


Code for a Universal Turing Machine

Since 𝑈 is a Turing machine exists 𝑖 (1654 digits) such that 𝑈 = 𝑀𝑖 givenby (using a different codification) (Penrose 1991, pp. 56-57):7244855335339317577198395039615711237952360672556559631108144796606505059404241090310483613632359365644443458382226883278767626556144692814117715017842551707554085657689753346356942478488597046934725739988582283827795294683460521061169835945938791885546326440925525505820555989451890716537414896033096753020431553625034984529832320651583047664142130708819329717234151056980262734686429921838172157333482823073453713421475059740345184372359593090640024321077342178851492760797597634415123079586396354492269159479654614711345700145048167337562172573464522731054482980784965126988788964569760906634204477989021914437932830019493570963921703904833270882596201301773727202718625919914428275437422351355675134084222299889374410534305471044368695876405178128019437530813870639942772823156425289237514565443899052780793241144826142357286193118332610656122755531810207511085337633806031082361675045635852164214869542347187426437544428790062485827091240422076538754264454133451748566291574299909502623009733738137724162172747723610206786854002893566085696822620141982486216989026091309402985706001743006700868967590344734174127874255812015493663938996905817738591654055356704092821332221631410978710814599786695997045096818419062994436560151454904880922084480034822492077304030431884298993931352668823496621019471619107014619685231928474820344958977095535611070275817487333272966789987984732840981907648512726310017401667873634776058572450369644348979920344899974556624029374876688397514044516657077500605138839916688140725455446652220507242623923792115253181625125363050931728631422004064571305275802307665183351995689139748137504926429605010013651980186945639498


The Original Universal Turing Machine

Based on 𝑀 -functions (subroutines with parameters) (Sicard 1997;Copeland 2004b).The machine is composed by 12 symbols and 4.000 instructions,approximately (Sicard Ramírez 1998).


Small Universal Turing Machines

UTM(𝑚, 𝑛): Class of universal Turing machines with 𝑚 states and 𝑛symbols.

There exists universal Turing machines in the followingclasses (Rogozhin 1996):

UTM(24, 2),UTM(10, 3),UTM(7, 4),UTM(5, 5),UTM(4, 6),UTM(3, 10),UTM(2, 18).

The classes UTM(3, 2), UTM(2, 3) and UTM(2, 2) areempty (Rogozhin 1996).


Small Universal Turing Machines

UTM(𝑚, 𝑛): Class of universal Turing machines with 𝑚 states and 𝑛symbols.

There exists universal Turing machines in the followingclasses (Rogozhin 1996):

UTM(24, 2),UTM(10, 3),UTM(7, 4),UTM(5, 5),UTM(4, 6),UTM(3, 10),UTM(2, 18).

The classes UTM(3, 2), UTM(2, 3) and UTM(2, 2) areempty (Rogozhin 1996).


Wolfram Turing MachineTHE BOOK STORE DOWNLOADS EDUCATION ARCHIVES FORUM

© 2012 Stephen Wolfram, LLC stephenwolfram.com | wolfram.com | send a message | contact info

Announced May 14th, 2007: 5th Anniversary of the Publication of A New Kindof Science

Is this Turing machine universal, ornot?

The machine has 2 states and 3 colors, and is 596440 in Wolfram's numbering scheme.

If it is universal then it is the smallest universal Turing machine that exists.

BACKGROUND » TECHNICAL DETAILS » GALLERY » NEWS »

PRIZE COMMITTEE » RULES & GUIDELINES » FAQs »

A universal Turing machine is powerful enough to emulate any standard computer.

The question is: how simple can the rules for a universal Turing machine be?

Since the 1960s it has been known that there is a universal 7,4 machine. In A New

Kind of Science, Stephen Wolfram found a universal 2,5 machine, and suggested that

the particular 2,3 machine that is the subject of this prize might be universal.

The prize is for determining whether or not the 2,3 machine is in fact universal.

What is a Turing Machine? » | Notable Universal Turing Machines »

INTERACTIVE DEMONSTRATIONS » PRINTABLE POSTER »

NKS|ONLINE »

SPONSORED BY WOLFRAM RESEARCH & STEPHEN WOLFRAM

Wolfram 2,3 Turing Machine Research Prize http://www.wolframscience.com/prizes/tm23/

1 of 1 10/11/2012 09:31 AM


Conway’s Game of Life

From: WikipediaNote: Adobe Reader is required to see the simulation (tested with AdobeReader 9.9.5 on Ubuntu).

ExamplesPulsar (Oscillator)Author: Jokey Smurf

Glider (Spaceship)Author: Rodrigo Silveira Camargo



Rulesi) Any live cell with fewer than two live neighbours dies, as if caused by

under-population.

ii) Any live cell with two or three live neighbours lives on to the nextgeneration.

iii) Any live cell with more than three live neighbours dies, as if byovercrowding.

iv) Any dead cell with exactly three live neighbours becomes a live cell,as if by reproduction.

TheoremIt is possible codified a universal Turing machine in Conway’s Game ofLife (Rendell 2011).




under-population.ii) Any live cell with two or three live neighbours lives on to the next

generation.

iii) Any live cell with more than three live neighbours dies, as if byovercrowding.







generation.iii) Any live cell with more than three live neighbours dies, as if by

overcrowding.








overcrowding.iv) Any dead cell with exactly three live neighbours becomes a live cell,

as if by reproduction.







overcrowding.iv) Any dead cell with exactly three live neighbours becomes a live cell,

as if by reproduction.



The Halting Problem

The original Turing machine accepted by halting, no by final state.

H(𝑀) = {𝑤 ∣ 𝑀 halts given the input 𝑤}.

The halting problem (language)

Lhp = { (𝑀, 𝑤) ∣ 𝑤 ∈ H(𝑀)} .

Example (Exercise 9.2.1)Show that Lhp is recursively enumerable but not recursive.


The Halting Problem




Lhp = { (𝑀, 𝑤) ∣ 𝑤 ∈ H(𝑀)} .



The Halting Problem




Lhp = { (𝑀, 𝑤) ∣ 𝑤 ∈ H(𝑀)} .



The Halting Problem230 M. Margenstern / Theoretical Computer Science 231 (2000) 217–251

Fig. 2. The state of the art. Black squares indicate undecidable points, while squares indicate decidablepoints, points marked with plus between the white squares lines and the line of black squares in the shapeof a hyperbola correspond to sets for which the status of the halting problem is not known.

author improved some of the results obtained in 1982 [94–96]. Yet his paper of 1982had remained unnoticed by the scienti�c community for ten years. The technique usedin order to obtain these results will be explained in our survey, in Section 3.4.2.Let us have a look now in the opposite direction: it is trivial that Turing machines on

a single letter alphabet have a decidable halting problem. It is not that trivial that thesame property holds for machines with a single state, whatever the number of symbolsis in the machine alphabet. This was de�nitely proved by Hermann in 1966 [32].For more than one state, one does not know much. As for proving the decidability

of the halting problem, M. Minsky writes in his famous book of 1967 that he andone of his students “did this for all 2× 2 machines [1961, unpublished] by a tediousreduction to thirty-odd cases (unpublishable)” [72, p. 281, last two lines]. Six yearslater, Pavlotskaya proved the same theorem [82] in a compact, very short proof. Thispaper has also remained unnoticed for many years. A few years later [84], she provedthat the point 3× 2 is also decidable.All these results about the decidability and undecidability of the halting problem for

Turing machines can be represented as in Fig. 2.

3.1. Decidability criteria

Let us now introduce a more precise notion of decidability criterion, respectivelystrong decidability criterion. Let c be an integer-valued function de�ned on a set Mof Turing machines in M with the following property: there is an integer f such that

State of theart†. Blacksquare(undecidableproblem),square(decidableproblem), plus(unknownstatus).

†Figure from (Margenstern 2000).An Undecidable Problem That Is Recursively Enumerable 28/32

The Halting Problem

RemarkThe halting problem was introduced and named by Davis (1958, p. 70.)not by Turing himself, contrary to popular belief (Copeland 2004a, p. 40).

Practical approach“In contrast to popular belief, proving termination is not always im-possible.” (Cook, Podelski and Rybalchenko 2011, p. 1)


The Halting Problem

RemarkThe halting problem was introduced and named by Davis (1958, p. 70.)not by Turing himself, contrary to popular belief (Copeland 2004a, p. 40).

Practical approach“In contrast to popular belief, proving termination is not always im-possible.” (Cook, Podelski and Rybalchenko 2011, p. 1)


References

Cook, B., Podelski, A. and Rybalchenko, A. (2011). Proving ProgramTermination. Communications of the ACM 54.5, pp. 88–98. doi:10.1145/1941487.1941509 (cit. on pp. 29, 30).Copeland, B. J. (2004a). Hypercomputation: Philosophical Issues.Theoretical Computer Science 317.1–3, pp. 251–267. doi:10.1016/j.tcs.2003.12.014 (cit. on pp. 29, 30).Davis, M. (1958). Computability and Unsolvability. McGraw-Hill (cit. onpp. 29, 30).Hopcroft, J. E., Motwani, R. and Ullman, J. D. (2007). Introduction toAutomata theory, Languages, and Computation. 3rd ed. Pearson Education(cit. on p. 8).Margenstern, M. (2000). Frontier Between Decidability and Undecidability:A Survey. Theoretical Computer Science 231.2, pp. 217–251. doi:10.1016/S0304-3975(99)00102-4 (cit. on p. 28).Penrose, R. (1991). The Emperor’s New Mind. Penguin Books (cit. onp. 14).


https://doi.org/10.1145/1941487.1941509


https://doi.org/10.1016/S0304-3975(99)00102-4

References

Rendell, P. (2011). A Universal Turing Machine in Conway’s Game of Life.In: Proceedings of the 2011 International Conference on High PerformanceComputing & Simulation (HPCS 2011). Ed. by Smari, W. W. IEEE,pp. 764–772. doi: 10.1109/HPCSim.2011.5999906 (cit. on pp. 20–24).Rogozhin, Y. (1996). Small Universal Turing Machines. TheoreticalComputer Science 168, pp. 215–240. doi: 10.1016/S0304-3975(96)00077-1(cit. on pp. 16, 17).Sicard Ramírez, A. (1998). Máquinas de Turing Dinámicas: Historia yDesarrollo de una Idea. MA thesis. Departamento de Informática ySistemas. Universidad EAFIT (cit. on p. 15).Sicard, A. (1997). Máquina Universal de Turing: Algunas Indicaciones parasu Construcción. Rev. U. EAFIT 33.108, pp. 61–106 (cit. on p. 15).Sicard, A. and Copeland, B. J. (2004b). Appendix: Subroutines andM-functions. In: The Essential Turing. Vol. 317. 1–3, pp. 54–57. doi:10.1016/j.tcs.2003.12.014 (cit. on p. 15).


https://doi.org/10.1109/HPCSim.2011.5999906

https://doi.org/10.1016/S0304-3975(96)00077-1


Automata and Formal Languages - CM0081Undecidable Problems About Turing Machines


Universidad EAFIT

Semester 2018-1

Reductions

DefinitionLet 𝑃1 and 𝑃2 be two problems. A reduction from 𝑃1 to 𝑃2 is a Turingmachine that takes an instance of 𝑃1 written on its tape and halts with aninstance of 𝑃2 that have the same answer (i.e. a reduction is an algorithm).

yes

no

yes

no

𝑃1 𝑃2

Undecidable Problems About Turing Machines 2/28

Reductions

Theorem (9.7.a)If there is a reduction from 𝑃1 to 𝑃2 then if 𝑃1 is undecidable then so 𝑃2.(Hint: Suppose the 𝑃2 is decidable and find a contradiction).

Theorem (9.7.b)If there is a reduction from 𝑃1 to 𝑃2 then if 𝑃1 is not recursively enumerablethen so 𝑃2. (Hint: Suppose the 𝑃2 is recursively enumerable and find acontradiction).


Reductions

Theorem (9.7.a)If there is a reduction from 𝑃1 to 𝑃2 then if 𝑃1 is undecidable then so 𝑃2.(Hint: Suppose the 𝑃2 is decidable and find a contradiction).

Theorem (9.7.b)If there is a reduction from 𝑃1 to 𝑃2 then if 𝑃1 is not recursively enumerablethen so 𝑃2. (Hint: Suppose the 𝑃2 is recursively enumerable and find acontradiction).


Turing Machines that Accept the Empty Language

NotationHenceforth, we’ll regard strings as the Turing machines they represent.

Two languagesLe = {𝑀 ∣ 𝐿(𝑀) = ∅},

Lne = {𝑀 ∣ 𝐿(𝑀) ≠ ∅}.



Theorem (9.8)Lne is recursively enumerable.

ProofConstruction of a non-determinist Turing machine to accept Lne:†

†Figure from Hopcroft, Motwani and Ullman (2007, Fig. 9.8).Undecidable Problems About Turing Machines 6/28


Theorem (9.8)Lne is recursively enumerable.

ProofConstruction of a non-determinist Turing machine to accept Lne:†



Theorem (9.9)Lne is not recursive.



Proof.1. Reduction from Lu to Lne where the pair (𝑀, 𝑤) is converted in 𝑀 ′,

such that 𝑤 ∈ 𝐿(𝑀), if only if, 𝐿(𝑀 ′) ≠ ∅.

2. The key is to have 𝑀 ′ ignore its input.†

3. Lne is not recursive by Theorem 9.7.a.




such that 𝑤 ∈ 𝐿(𝑀), if only if, 𝐿(𝑀 ′) ≠ ∅.2. The key is to have 𝑀 ′ ignore its input.†





such that 𝑤 ∈ 𝐿(𝑀), if only if, 𝐿(𝑀 ′) ≠ ∅.2. The key is to have 𝑀 ′ ignore its input.†




Theorem (9.10)Le is not recursively enumerable. (Hint: Le is the complement of Lne).


Rice’s Theorem

Set of the recursively enumerable languages

ℒ = {𝐿 ∣ 𝐿 is a recursively enumerable language }.

Properties (subsets) of the recursively enumerable languagesProperty 𝑃 of ℒ: 𝑃 ⊆ ℒ.

Example (Trivial properties)𝑃 (𝐿) = ∅ or 𝑃(𝐿) = ℒ.

Example𝑃 (𝐿): 𝐿 is a language regular.


Rice’s Theorem







Rice’s Theorem







Rice’s Theorem







Rice’s Theorem

Theorem (Rice’s theorem, 9.11)Every non-trivial property of ℒ is undecidable (Rice 1953).

How to prove Rice’s theorem?We identify a property 𝑃 by the Turing machines 𝑀 such that 𝐿(𝑀) ∈ 𝑃 .

Theorem (Rice’s theorem (second version))If 𝑃 ⊆ ℒ is a non-trivial property then

𝐿𝑃 = {𝑀 ∣ 𝐿(𝑀) ∈ 𝑃}

is undecidable.


Rice’s Theorem




𝐿𝑃 = {𝑀 ∣ 𝐿(𝑀) ∈ 𝑃}

is undecidable.


Rice’s Theorem




𝐿𝑃 = {𝑀 ∣ 𝐿(𝑀) ∈ 𝑃}

is undecidable.


Rice’s Theorem

Proof: Case ∅ ∉ 𝑃 .1. Let 𝐿 be a language and 𝑀𝐿 be a Turing machine such 𝐿 ≠ ∅,

𝐿 ∈ 𝑃 and 𝐿 = 𝐿(𝑀𝐿). Reduction from Lu to 𝐿𝑃 where the pair(𝑀, 𝑤) is converted in 𝑀 ′ such that:†

i) 𝐿(𝑀 ′) = ∅ (i.e. 𝑀 ′ ∉ 𝐿𝑃 ) if 𝑤 ∉ 𝐿(𝑀) andii) 𝐿(𝑀 ′) = 𝐿 (i.e. 𝑀 ′ ∈ 𝐿𝑃 ) if 𝑤 ∈ 𝐿(𝑀).

2. 𝐿𝑃 is not recursive by Theorem 9.7.a.


Rice’s Theorem




2. 𝐿𝑃 is not recursive by Theorem 9.7.a.


Rice’s Theorem




2. 𝐿𝑃 is not recursive by Theorem 9.7.a.†Figure from Hopcroft, Motwani and Ullman (2007, Fig. 9.10).


Rice’s Theorem

Proof: Case ∅ ∈ 𝑃 .1. By the previous case, 𝑃 is undecidable, i.e. 𝐿𝑃 is undecidable.2. 𝐿𝑃 = 𝐿𝑃 .3. Suppose 𝐿𝑃 is decidable then 𝐿𝑃 would be also decidable

(contradiction).4. Therefore, 𝐿𝑃 is undecidable.


Rice’s Theorem

All the problems about Turing machines that involve only the language thatthe Turing machine accepts are undecidable.

ExamplesIs the language accepted by the Turing machine empty? Is it finite? Isit regular? Is it context-free?Does the language accepted by the Turing machine contain the string“hello world”? Does it contain all the even numbers?


Rice’s Theorem

All the problems about Turing machines that involve only the language thatthe Turing machine accepts are undecidable.

ExamplesIs the language accepted by the Turing machine empty? Is it finite? Isit regular? Is it context-free?Does the language accepted by the Turing machine contain the string“hello world”? Does it contain all the even numbers?


Rice’s Theorem

Rice’s theorem does not imply that everything about Turing machines isundecidable.

ExampleIt is decidable if a Turing machine has five states.


Rice’s Theorem

Rice’s theorem does not imply that everything about Turing machines isundecidable.ExampleIt is decidable if a Turing machine has five states.


References

Hopcroft, J. E., Motwani, R. and Ullman, J. D. (2007). Introduction toAutomata theory, Languages, and Computation. 3rd ed. Pearson Education(cit. on pp. 6, 7, 9–11, 20–22).Rice, H. G. (1953). Classes of Recursively Enumerable Sets and TheirDecision Problems. Trans. Amer. Math. Soc. 74.2, pp. 358–366. doi:10.1090/S0002-9947-1953-0053041-6 (cit. on pp. 17–19).


https://doi.org/10.1090/S0002-9947-1953-0053041-6

Automata and Formal Languages - CM0081Undecidable Problems


Universidad EAFIT

Semester 2018-1


(From Wikipedia)Problems in logicProblems about abstract machinesProblems about matricesProblems in combinatorial group theoryProblems in topologyProblems in analysis…

Undecidable Problems 2/21

Normal Forms for the 𝜆-Calculus

RemarkThis problem was the first undecidable problem. See (Church 1935).

DefinitionThe set of 𝜆-terms, denoted by Λ, is inductively defined by:

𝑥 ∈ 𝑉 ⇒ 𝑥 ∈ Λ (variable)𝑀, 𝑁 ∈ Λ ⇒ (𝑀𝑁) ∈ Λ (application)

𝑀 ∈ Λ, 𝑥 ∈ 𝑉 ⇒ (𝜆𝑥𝑀) ∈ Λ (abstraction)

where 𝑉 is a set of variables.



RemarkThis problem was the first undecidable problem. See (Church 1935).

DefinitionThe set of 𝜆-terms, denoted by Λ, is inductively defined by:

𝑥 ∈ 𝑉 ⇒ 𝑥 ∈ Λ (variable)𝑀, 𝑁 ∈ Λ ⇒ (𝑀𝑁) ∈ Λ (application)

𝑀 ∈ Λ, 𝑥 ∈ 𝑉 ⇒ (𝜆𝑥𝑀) ∈ Λ (abstraction)

where 𝑉 is a set of variables.



DefinitionThe 𝛽-reduction is defined by

(𝜆𝑥𝑀)𝑁 →𝛽 𝑀[𝑥 ∶= 𝑁],

where 𝑀[𝑥 ∶= 𝑁] denotes the substitution of 𝑁 for the free occurrencesof 𝑥 in 𝑀 .

Examples(𝜆𝑦.𝑦𝑦)𝑥 →𝛽 𝑥𝑥

(𝜆𝑥.(𝜆𝑦.𝑦𝑥)𝑧)𝑣 →𝛽 (𝜆𝑦.𝑦𝑣)𝑧 →𝛽 𝑧𝑣

Let Ω be (𝜆𝑥.𝑥𝑥)(𝜆𝑥.𝑥𝑥), then Ω →𝛽 Ω →𝛽 ⋯



























TheoremThe set {𝑀 ∈ Λ ∣ 𝑀 has normal form} is not recursive (Church 1935,1936). See also (Barendregt 1990).


Post’s Correspondence Problem (PCP)

An instance of the PCPAn instance of PCP consist of two lists of equal length 𝐴 = 𝑤1, … , 𝑤𝑘 and𝐵 = 𝑥1, … , 𝑥𝑘 of strings over an alphabet Σ. We say this instance of PCPhas a solution, if there is a sequence of one or more integers 𝑖1, … , 𝑖𝑚, with𝑚 ≥ 1 that, when interpreted as indexes for strings in the 𝐴 and 𝐵 lists,yield the same string, i.e. 𝑤𝑖1

⋯ 𝑤𝑖𝑚= 𝑥𝑖1

⋯ 𝑥𝑖𝑚. The sequence 𝑖1, … , 𝑖𝑚

is called a solution of the instance of PCP.

The problemGiven an instance of PCP, tell whether this instance has a solution.



An instance of the PCPAn instance of PCP consist of two lists of equal length 𝐴 = 𝑤1, … , 𝑤𝑘 and𝐵 = 𝑥1, … , 𝑥𝑘 of strings over an alphabet Σ. We say this instance of PCPhas a solution, if there is a sequence of one or more integers 𝑖1, … , 𝑖𝑚, with𝑚 ≥ 1 that, when interpreted as indexes for strings in the 𝐴 and 𝐵 lists,yield the same string, i.e. 𝑤𝑖1

⋯ 𝑤𝑖𝑚= 𝑥𝑖1

⋯ 𝑥𝑖𝑚. The sequence 𝑖1, … , 𝑖𝑚

is called a solution of the instance of PCP.

The problemGiven an instance of PCP, tell whether this instance has a solution.



ExampleAn instance of the PCP:

List A List B𝑖 𝑤𝑖 𝑥𝑖1 1 1112 10111 103 10 0

Solution: 2, 1, 1, 3, 𝑚 = 4.

Undecidability proofThe PCP problem is undecidable (Post 1946). Hopcroft, Motwani and Ull-man (2007) shows the undecidability via a reduction of Lu to PCP.



ExampleAn instance of the PCP:

List A List B𝑖 𝑤𝑖 𝑥𝑖1 1 1112 10111 103 10 0

Solution: 2, 1, 1, 3, 𝑚 = 4.

Undecidability proofThe PCP problem is undecidable (Post 1946). Hopcroft, Motwani and Ull-man (2007) shows the undecidability via a reduction of Lu to PCP.


The Mortal Matrix Problem (MMP)

The problemLet 𝑆 be a finite set of 𝑛 × 𝑛 matrices with integer entries. To determinewhether the zero matrix belongs to the semigroup generated by 𝑆, i.e.to determine whether the matrices in 𝑆 can be multiplied in some order,possibly with repetitions, to yield the zero matrix.

Some undecidable instancesThe MMP is undecidable for a set of seven 3 × 3 matrices, or a set of two21 × 21 matrices (Halava, Harju and Hirvensalo 2007).

Undecidability proofReduction of PCP to MMP.












Hilbert’s Tenth Problem (HTP)

DefinitionA Diophantine equation is an equation of the form

𝐷(𝑥1, … , 𝑥𝑘) = 0,

where 𝐷 is a polynomial with integer coefficients.

The problem (in present terminology)“Given a Diophantine equation with any number of unknowns: To devise aprocess according to which it can be determined by a finite number of op-erations whether the equation has non-negative integer solutions.” (Sicard,Ospina and Vélez 2006, p. 12542)

Undecidability proofA set is recursively enumerable if and only if it is Diophantine (Matiyasevich1993).




𝐷(𝑥1, … , 𝑥𝑘) = 0,







𝐷(𝑥1, … , 𝑥𝑘) = 0,





References

Barendregt, H. (1990). Functional Programming and Lambda Calculus. In:Handbook of Theoretical Computer Science. Ed. by van Leeuwen, J. Vol. B.Formal Models and Semantics. MIT Press. Chap. 7. doi:10.1016/B978-0-444-88074-1.50012-3 (cit. on p. 9).Church, A. (1935). An Unsolvable Problem of Elementary Number Theory.Preliminar Report (Abstract). Bull. Amer. Math. Soc. 41.5, pp. 332–333.doi: 10.1090/S0002-9904-1935-06102-6 (cit. on pp. 3, 4, 9).— (1936). An Unsolvable Problem of Elementary Number Theory.American Journal of Mathematics 58.2, pp. 345–363. doi:10.2307/2371045 (cit. on p. 9).Halava, V., Harju, T. and Hirvensalo, M. (2007). Undecidability Bounds forInteger Matrices Using Claus Instances. International Journal ofFoundations of Computer Science 18.5, pp. 931–948. doi:10.1142/S0129054107005066 (cit. on pp. 14–16).


https://doi.org/10.1016/B978-0-444-88074-1.50012-3

https://doi.org/10.1090/S0002-9904-1935-06102-6

https://doi.org/10.2307/2371045

https://doi.org/10.1142/S0129054107005066

References

Hopcroft, J. E., Motwani, R. and Ullman, J. D. (2007). Introduction toAutomata theory, Languages, and Computation. 3rd ed. Pearson Education(cit. on pp. 12, 13).Matiyasevich, Y. V. (1993). Hilbert’s Tenth Problem. MIT Press (cit. onpp. 17–19).Post, E. (1946). A Variant of a Recursively Unsolvable Problem. Bull.Amer. Math. Soc. 52, pp. 264–268. doi:10.1090/S0002-9904-1946-08555-9 (cit. on pp. 12, 13).Sicard, A., Ospina, J. and Vélez, M. (2006). Quantum HypercomputationBased on the Dynamical Algebra 𝔰𝔲(1, 1). J. Phys. A: Math. Gen. 39.40,pp. 12539–12558. doi: 10.1088/0305-4470/39/40/018 (cit. on pp. 17–19).


https://doi.org/10.1090/S0002-9904-1946-08555-9

https://doi.org/10.1088/0305-4470/39/40/018

Automata and Formal Languages - CM0081The Church-Turing Thesis


Universidad EAFIT

Semester 2018-1



{𝑓 ∣ 𝑓 ∶ ℕ𝑘 → ℕ}.

ExampleThe following functions are Turing machine computable:





The Church-Turing Thesis 2/31



{𝑓 ∣ 𝑓 ∶ ℕ𝑘 → ℕ}.









𝑥 ∸ 𝑦 = {𝑥 − 𝑦, if 𝑥 ≥ 𝑦;0, otherwise;

|𝑥 − 𝑦| = {𝑥 ∸ 𝑦, if 𝑥 ≥ 𝑦;𝑦 ∸ 𝑥, otherwise;

𝑠𝑔(𝑥) = {1, if 𝑥 = 0;0, otherwise;

𝑒𝑣𝑒𝑛(𝑥) = {1, if 𝑥 is even;0, otherwise.


Common Versions of the Church-Turing Thesis

“A function is computable (effectively calculable) if and only if there is aTuring machine which computes it.” (Galton 2006, p. 94)

“The unprovable assumption that any general way to compute will allowus compute only the partial-recursive functions (or equivalently, what Tur-ing machines or modern-day computers can compute) is know as Church’shypothesis or the Church-Turing thesis.” (Hopcroft, Motwani and Ullman2007, p. 236)


Common Versions of the Church-Turing Thesis

“A function is computable (effectively calculable) if and only if there is aTuring machine which computes it.” (Galton 2006, p. 94)“The unprovable assumption that any general way to compute will allowus compute only the partial-recursive functions (or equivalently, what Tur-ing machines or modern-day computers can compute) is know as Church’shypothesis or the Church-Turing thesis.” (Hopcroft, Motwani and Ullman2007, p. 236)


Alonzo Church: A Definition

“We now define the notion, already discussed,of an effectively calculable function of posit-ive integers by identifying it with the notion ofa recursive function of positive integers (or ofa 𝜆-definable function of positive integers).This definition is thought to be justified byconsiderations which follow, so far as positivejustification can ever be obtained for the se-lection of a formal definition to correspond tointuitive notion.” (Church 1936, p. 356)See also (Church 1935).


Alan Turing: A Definition

“The ‘computable’ numbers† include all num-bers which would naturally be regarded ascomputable.” (Turing 1936, p. 249)

†The number whose decimal representation can be generating progressively by aTuring machine.


Stephen Kleene: Church’s thesis and Turing’s thesis

“The thesis of Church and Turing werenot even called ‘thesis’ at all until Kleene(1943, p. 60)† referred to Church’s ‘defin-ition’ as ‘Thesis I’ and the 1952 Kleene‡

referred to ‘Church’s Thesis’ and ‘Turing’sThesis’.” (Soare 1996, pp. 295–296)

†Kleene, S. C. (1943). Recursive Predicates and Quantifiers.‡Kleene, S. C. (1952). Introduction to Metamathematics.


Stephen Kleene: The Church-Turing Thesis

“The term ‘Church-Turing thesis’ seems to have been first introduce byKleene, with a small flourish of bias in favor of Church:” (Copeland 2002)

“So Turing’s and Church’s thesis are equival-ent. We shall usually refer to them both asChurch’s thesis, or in connection with that one ofits…version which deal with ‘Turing machines’ asthe Church-Turing thesis.” (Kleene 1967, p. 232)


Intensional-Extensional Meaning

“Here we also use the phrase ‘Church-Turing thesis’ to refer to the am-algamation of the two theses (these and others) where we identify all in-formal concepts of Definition 1.1† with one another we identify all the formalconcepts of Definition 1.2‡, and their mathematical equivalents, with oneanother and suppress their intensional meanings.” (Soare 1996, p. 296).

†Definition 1.1: A function is ‘computable’ (also called ‘effectively calculable’ orsimply ‘calculable’) if it can be calculated by a finite mechanical procedure.

‡Definition 1.2: (i) A function is ‘Turing computable’ if it is definable by a Turingmachine, as defined by Turing 1936.


Possible Refutation

Idea: Turing machine computability ⇏ effective calculability“A function is considered effectively computable if its value can be computedin an effective way in a finite number of steps, but there is no bound onthe number of steps required for any given computation. Thus, the factthat there are effectively computable functions which may not be humanlycomputable has nothing to do with Church’s thesis.” (Mendelson 1963,p. 202).


Possible Refutation

Idea: Effective calculability ⇏ Turing machine computability(the interesting one!)

From a Church’s letter to Pepis (June 8, 1937):“Therefore to discover a function which was effectively calculable but nogeneral recursive would imply discovery of an utterly new principle of logic,not only never before formulated, but never before actually used in a math-ematical proof…Moreover this new principle of logic must be of so strange,and presumably complicated,…I should be inclined to scrutinize the allegedeffective applicability of the principle with considerable care.” (Sieg 1997,pp. 175–176)


A Precise Version of the Church-Turing Thesis

Turing’ analysis: Features of computations performed by human computers

“States of mind” ⇒ Finite number of states

A human cannot reliably discriminate infinitely many symbols ⇒Finite alphabet

Unlimited sheets of paper ⇒ Unbounded tape

The human read/write symbols on the paper ⇒ Read/Write head

Human’s shift of attention form one part of the paper to another ⇒Displacement of the read/write head



































Precise Church-Turing thesis

“Any procedure than can be carried out by anidealised human clerk working mechanicallywith paper and pencil can also be carried outby a Turing machine.” (Copeland and Sylvan1999).


Other Theses

Gandy’s theses (Gandy n.d.)

“Thesis P. A discrete deterministicmechanical device satisfies prin-ciples I-IV below.” (p. 126)

“Theorem. What can be calculatedby a device satisfying principles I-IVis computable.” (p. 126)

“Thesis M. What can be calculatedby a machine is Turing machinecomputable.” (p. 124)


Other Theses

Gandy’s theses (Gandy n.d.)

“Thesis P. A discrete deterministicmechanical device satisfies prin-ciples I-IV below.” (p. 126)

“Theorem. What can be calculatedby a device satisfying principles I-IVis computable.” (p. 126)

“Thesis M. What can be calculatedby a machine is Turing machinecomputable.” (p. 124)


Other Theses

Physical Church-Turing thesis“A function is computable by means of a physically possible computingdevice if and only if there is a Turing machine which computes it.” (Galton2006, p. 95).


Current Situation

At the moment, it does not exists a refutation to the preciseChurch-Turing thesis.

The hypercomputation models refute the theoretical version of thethesis M.

Open problem: the refutation of the realizable version of the thesis M(i.e. the physical Church-Turing thesis).


Current Situation





Current Situation





Bonus Slides: Higher Type Computability

ExampleLet 𝑓 and ⊥ be a terminating and a non-terminating function from 𝑎 to 𝑎,respectively. The parallel or function

por ∷ (𝑎 → 𝑎) → (𝑎 → 𝑎) → 𝑎 → 𝑎por 𝑓 ⊥ = 𝑓por ⊥ 𝑓 = 𝑓por ⊥ ⊥ = ⊥

is an effectively calculable function which is not 𝜆-definable (Turner 2006).



Solution: To add extra primitives to 𝜆-calculus (interleaving, concurrency,etc.).

General problem: A definition of higher type computability (see, forexample, (Longley 2003)).



Solution: To add extra primitives to 𝜆-calculus (interleaving, concurrency,etc.).General problem: A definition of higher type computability (see, forexample, (Longley 2003)).


References

Church, A. (1935). An Unsolvable Problem of Elementary Number Theory.Preliminar Report (Abstract). Bull. Amer. Math. Soc. 41.5, pp. 332–333.doi: 10.1090/S0002-9904-1935-06102-6 (cit. on p. 7).— (1936). An Unsolvable Problem of Elementary Number Theory.American Journal of Mathematics 58.2, pp. 345–363. doi:10.2307/2371045 (cit. on p. 7).Copeland, B. J. (19th Aug. 2002). The Church-Turing Thesis. StanfordEncyclopedia of Philosophy (cit. on p. 10).Copeland, B. J. and Sylvan, R. (1999). Beyond the Universal TuringMachine. Australasian Journal of Philosophy 77.1, pp. 44–66. doi:10.1080/00048409912348801 (cit. on p. 19).Galton, A. (2006). The Church-Turing Thesis: Still Valid after All TheseYears? Applied Mathematics and Computation 178.1, pp. 93–102. doi:10.1016/j.amc.2005.09.086 (cit. on pp. 5, 6, 22).


https://doi.org/10.1090/S0002-9904-1935-06102-6

https://doi.org/10.2307/2371045

https://doi.org/10.1080/00048409912348801

https://doi.org/10.1016/j.amc.2005.09.086

References

Gandy, R. (n.d.). Church’s Thesis and Principles for Mechanisms. In: TheKleene Symposium, pp. 123–148. doi: 10.1016/S0049-237X(08)71257-6(cit. on pp. 20, 21).Hopcroft, J. E., Motwani, R. and Ullman, J. D. (2007). Introduction toAutomata theory, Languages, and Computation. 3rd ed. Pearson Education(cit. on pp. 5, 6).Kleene, S. C. (1943). Recursive Predicates and Quantifiers. Trans. Amer.Math. Soc. 53.1, pp. 41–73. doi: 10.1090/S0002-9947-1943-0007371-8(cit. on p. 9).Kleene, S. C. (1952). Introduction to Metamathematics. North-Holland(cit. on p. 9).— (1967). Mathematical Logic. John Wiley & Sons (cit. on p. 10).Longley, J. R. (2003). Notions of Computability at Higher Types I. In: LogicColloquium 2000. Ed. by Cori, R., Razborov, A., Todorcěvić, S. andWood, C. Vol. 19. Lecture Notes in Logic. A K Peters/CRC Press,pp. 32–142 (cit. on pp. 27, 28).


https://doi.org/10.1016/S0049-237X(08)71257-6

https://doi.org/10.1090/S0002-9947-1943-0007371-8

References

Mendelson, E. (1963). On Some Recent Criticism of Church’s Thesis. NotreDame Journal of Formal Logic 4.3, pp. 201–205. doi:10.1305/ndjfl/1093957577 (cit. on p. 12).Sieg, W. (1997). Step by Recursive Step: Church’s Analysis of EffectiveCalculability. The Bulletin of Symbolic Logic 3.2, pp. 154–180. doi:10.2307/421012 (cit. on p. 13).Soare, R. I. (1996). Computability and Recursion. The Bulletin of SymbolicLogic 2.3, pp. 284–321. doi: 10.2307/420992 (cit. on pp. 9, 11).Turing, A. M. (1936). On Computable Numbers, with an Application to theEntscheidungsproblem. Proc. London Math. Soc. 42, pp. 230–265. doi:10.1112/plms/s2-42.1.230 (cit. on p. 8).Turner, D. (2006). Church’s Thesis and Functional Programming. In:Church’s Thesis After 70 Years. Ed. by Olszewski, A., Woleński, J. andJanusz, R. Ontos Verlag, pp. 518–544 (cit. on p. 26).


https://doi.org/10.1305/ndjfl/1093957577

https://doi.org/10.2307/421012

https://doi.org/10.2307/420992

https://doi.org/10.1112/plms/s2-42.1.230

Automata and Formal Languages - CM0081Hypercomputation


Universidad EAFIT

Semester 2018-1

Motivation

Absolute computabilityGödel 1946 (Gödel 1990, p. 150):

“the great importance of the concept of general recursiveness (orTuring’s computability)…is largely due to the fact that with thisconcept one has for the first time succeeded in giving an absolutedefinition of an interesting epistemological notion, i.e., one notdepending on the formalism chosen.”

Hypercomputation 2/56

Motivation

Absolute computabilityDavis (1958, p. 11):

“For how can we ever exclude the possibility of our being presen-ted, some day (perhaps by some extraterrestrial visitor), with a(perhaps extremely complex) device or “oracle” that “computes”a noncomputable function? However, there are fairly convincingreasons for believing that this will never happen.”


Motivation

Relative computabilitySylvan and Copeland (2000, p. 190):

“Troubles with absolutism are deeper and more extensive thanthese cracks (analogue procedures and newer physics) reveal. Forone thing, computability is relative not simply to physics, but moregenerally to systems of frameworks, which include or contain un-derlying logics.”


Hypercomputers

DefinitionA hypercomputer is any machine (theoretical or real) that compute functionsor numbers, or more generally solve problems or carry out tasks, that cannotbe computed or solved by a Turing machine (Copeland 2002b).

Super Turing Machines and Non Turing Machines

TMSuper-TM

𝐿 ⊆ Σ∗

non-TM

TM𝐿 ⊆ Σ∗


Hypercomputers

DefinitionA hypercomputer is any machine (theoretical or real) that compute functionsor numbers, or more generally solve problems or carry out tasks, that cannotbe computed or solved by a Turing machine (Copeland 2002b).

Super Turing Machines and Non Turing Machines

TMSuper-TM

𝐿 ⊆ Σ∗

non-TM

TM𝐿 ⊆ Σ∗


Possible Sources of Hypercomputation

MathematicsComputability Logic

Hypercomputation Model (HM)

BiologyPhysics ?


First Hypercomputation Model: Oracle Turing Machines

DefinitionA oracle Turing machine (OTM) is a Turing machine equipped with anoracle that is capable of answering questions about the membership of aspecific set of natural numbers (Turing 1939).

Hypercomputability featuresIf the oracle is a recursive set then OTM ≡ TM.If the oracle is a non-recursive set then OTM ≡ HM.


First Hypercomputation Model: Oracle Turing Machines

DefinitionA oracle Turing machine (OTM) is a Turing machine equipped with anoracle that is capable of answering questions about the membership of aspecific set of natural numbers (Turing 1939).

Hypercomputability featuresIf the oracle is a recursive set then OTM ≡ TM.If the oracle is a non-recursive set then OTM ≡ HM.


About the ‘Hypercomputation’ Term

Copeland and Proudfoot (1999):

Right WrongHypercomputation Super-Turing computation

Computing beyond Turing’s limitBreaking the Turing barrierEtc.


Hypercomputation Model: Accelerated Turing Machines

DefinitionAn accelerated Turing machine (ATM) is a Turing machine that performsits first step in one unit of time and each subsequent step in half the timeof the step before (Copeland 1998, 2002a).

Hypercomputability featuresSince

1 + 1/2 + 1/4 + 1/8 + … =∞

∑𝑖=0

12𝑖 = 2,

an ATM could complete an infinity of steps in two time units.


Hypercomputation Model: Accelerated Turing Machines

DefinitionAn accelerated Turing machine (ATM) is a Turing machine that performsits first step in one unit of time and each subsequent step in half the timeof the step before (Copeland 1998, 2002a).

Hypercomputability featuresSince

1 + 1/2 + 1/4 + 1/8 + … =∞

∑𝑖=0

12𝑖 = 2,

an ATM could complete an infinity of steps in two time units.


Hypercomputation Model: Analog Recurrent NeuralNetwork (ARNN)

Description (Siegelmann 1999)

𝑢1

𝑢2

𝑥1

𝑥2

𝑥3

𝑥4

𝑏11

𝑏12

𝑏13𝑏22𝑏23

𝑐3

𝑎14𝑎22

𝑎24

𝑎23𝑎34

𝑋(𝑡 + 1) = 𝜎( 𝐴 ⋅ 𝑋(𝑡) + 𝐵 ⋅ 𝑈(𝑡) + 𝐶)


Hypercomputation Model: Analog Recurrent NeuralNetwork (ARNN)

Hypercomputability features

𝑎𝑖𝑗 ∈ {ℕ, ℚ, ℝ} ⇒ ARNN ≡ {DFA, TM, HM}.


Standard Quantum Computation (SQC)

ModelsQuantum Turing machines (QTM) (Deutsch 1985) and quantum cir-cuits (Deutsch 1989).

Relation between the models

TMs Probabilistic TMs

Reversible TMs QTMs

≡

≡ ≡

≡


Standard Quantum Computation (SQC)

ModelsQuantum Turing machines (QTM) (Deutsch 1985) and quantum cir-cuits (Deutsch 1989).

Relation between the models

TMs Probabilistic TMs

Reversible TMs QTMs

≡

≡ ≡

≡


“Weak” Hypercomputation Based on SQC

Generation of truly random numbers

𝑈𝐻 | 0⟩ = 1√2

(| 0⟩ + | 1⟩) → measure

1. We observe the superposition state: “The act of observation causesthe superposition to collapse into either | 0⟩ or the | 1⟩ state withequal probability. Hence you can exploit quantum mechanicalsuperposition and indeterminism to simulate a perfectly fair cointoss.” (Williams and Clearwater 1997, p. 160)

2. The problem: It is not clear how to use this property to solve anon-computable Turing machine problem (Ord and Kieu 2009).




𝑈𝐻 | 0⟩ = 1√2

(| 0⟩ + | 1⟩) → measure






𝑈𝐻 | 0⟩ = 1√2

(| 0⟩ + | 1⟩) → measure




Others Quantum Computation Models

Common misunderstanding

quantum computation ≡ SQC≡ adiabatic quantum computation (AQC)

The real situationKieu’s hypercomputational quantum algorithm (KHQA) (Kieu 2003):

finite AQC ≡ SQCinfinite AQC ≡ KHQA


Others Quantum Computation Models

Common misunderstanding

quantum computation ≡ SQC≡ adiabatic quantum computation (AQC)

The real situationKieu’s hypercomputational quantum algorithm (KHQA) (Kieu 2003):

finite AQC ≡ SQCinfinite AQC ≡ KHQA


Hypercomputational Quantum Algorithm à la Kieu

Sicard, Ospina and Vélez (2006):

Classically non-computable 𝑃 problem

(Hilbert’s 10th problem)

Hypercomputationalquantum algorithm

Physical referent(Infinite square well)

Simulation

Dynamical algebra(Lie alg. 𝔰𝔲(1, 1))

Partialsolution

to 𝑃


Hypercomputation Model: Infinite Time Turing Machines

DefinitionAn infinite time Turing machine is a Turing machine working on a timeclocked by transfinite ordinals (Hamkins and Lewis 2000; Hamkins 2002,2007).

DescriptionFor convenience, the machines have three tapes:†

input:

scratch:

output:

start

1

0

0

1

0

0

0

0

0

1

0

0

1

0

0

0

0

0

· · ·

· · ·

· · ·

†Figure from Hamkins (2002, Fig. 1).Hypercomputation 23/56


DefinitionAn infinite time Turing machine is a Turing machine working on a timeclocked by transfinite ordinals (Hamkins and Lewis 2000; Hamkins 2002,2007).

DescriptionFor convenience, the machines have three tapes:†

input:

scratch:

output:

start

1

0

0

1

0

0

0

0

0

1

0

0

1

0

0

0

0

0

· · ·

· · ·

· · ·

†Figure from Hamkins (2002, Fig. 1).Hypercomputation 24/56


Description (cont.)In stage 𝛼 + 1 the machine works as usual.

In limit ordinal stages the machine works as follows (Hamkins and Lewis2000, p. 569–570):“To set up such a limit ordinal configuration, the head is plucked fromwherever it might have been racing towards, and placed on top of the firstcell. And it is placed in a special distinguished limit state.”“Now we need to take a limit of the cell values on the tape. And we will dothis cell by cell according to the following rule: if the values appearing in acell have converged, that is, if they are either eventually 0 or eventually 1before the limit stage, then the cell retains the limiting value at the limitstage. Otherwise, in the case that the cell values have alternated from 0to 1 and back again unboundedly often, we make the limit cell value 1.”



Description (cont.)In stage 𝛼 + 1 the machine works as usual.In limit ordinal stages the machine works as follows (Hamkins and Lewis2000, p. 569–570):“To set up such a limit ordinal configuration, the head is plucked fromwherever it might have been racing towards, and placed on top of the firstcell. And it is placed in a special distinguished limit state.”“Now we need to take a limit of the cell values on the tape. And we will dothis cell by cell according to the following rule: if the values appearing in acell have converged, that is, if they are either eventually 0 or eventually 1before the limit stage, then the cell retains the limiting value at the limitstage. Otherwise, in the case that the cell values have alternated from 0to 1 and back again unboundedly often, we make the limit cell value 1.”



Description (cont.)“This completely describes the configuration of the machine at any limitordinal stage 𝛽, and the machine can go on computing, 𝛽 + 1, 𝛽 + 2, andso on, eventually taking another limit at 𝛽 + 𝜔 co and so on through theordinals.”

Hypercomputability featuresThe halting problem is decidable in 𝜔 many steps by infinite time Turingmachines (Hamkins and Lewis 2000).

RemarkAlthough not related with computability but algorithmic complexity,𝑃 ≠ 𝑁𝑃 for infinite time Turing machines (Schindler 2003).













Theorem (Hamkins and Lewis (2000, Theorem 1.1))Every halting infinite time computation is countable.

RemarkThe infinite time Turing machines have been generalised by theordinal computability models, which are models based on ordinal numbers.These models include infinite time Turing machines or Turing machinesworking on tapes of transfinite “length”. Seyfferth (2013) shows a briefoverview of these models.



Theorem (Hamkins and Lewis (2000, Theorem 1.1))Every halting infinite time computation is countable.

RemarkThe infinite time Turing machines have been generalised by theordinal computability models, which are models based on ordinal numbers.These models include infinite time Turing machines or Turing machinesworking on tapes of transfinite “length”. Seyfferth (2013) shows a briefoverview of these models.


Church-Turing Thesis and Thesis M

The Church-Turing thesis“Any procedure than can be carried out by an idealised human clerk workingmechanically with paper and pencil can also be carried out by a Turingmachine.” (Copeland and Sylvan 1999)

Thesis M“What can be calculated by a machine is Turing machine comput-able.” (Gandy 1980)


Church-Turing Thesis and Thesis M

The Church-Turing thesis“Any procedure than can be carried out by an idealised human clerk workingmechanically with paper and pencil can also be carried out by a Turingmachine.” (Copeland and Sylvan 1999)

Thesis M“What can be calculated by a machine is Turing machine comput-able.” (Gandy 1980)


Physical Hypercomputation?

Open problemThe refutation of a general/physical version of Gandy’s Thesis M.

Based on quantum physics (infinite adiabatic quantum computation)?Based on relativistic physics (cosmological phenomena)? (Pitowsky1990; Hogarth 1992, 1994; Etesi and Németi 2002; Németi and Dávid2006)




Based on quantum physics (infinite adiabatic quantum computation)?

Based on relativistic physics (cosmological phenomena)? (Pitowsky1990; Hogarth 1992, 1994; Etesi and Németi 2002; Németi and Dávid2006)




Based on quantum physics (infinite adiabatic quantum computation)?Based on relativistic physics (cosmological phenomena)? (Pitowsky1990; Hogarth 1992, 1994; Etesi and Németi 2002; Németi and Dávid2006)


An Interesting Project: Formal Verification ofHypercomputation in Relativistic Physics

Stannett and Németi (2014) and Stannett (2015):

GoalsImplement first-order axiomatisations of theories of the relativityusing the proof assistant Isabelle;

Add a model of computation carried out by machines travelling alongspecific spacetime trajectories;



Stannett and Németi (2014) and Stannett (2015):

GoalsImplement first-order axiomatisations of theories of the relativityusing the proof assistant Isabelle;Add a model of computation carried out by machines travelling alongspecific spacetime trajectories;



Goals (cont.)Consider how the power of these computational systems changesaccording to the underlying topology of spacetime;

Select a recursively uncomputable problem 𝑃 (for example, theHalting Problem) and machine-verify the following claims:

in simpler relativistic settings, 𝑃 remains uncomputable;in some spacetimes, 𝑃 can be solved.



Goals (cont.)Consider how the power of these computational systems changesaccording to the underlying topology of spacetime;Select a recursively uncomputable problem 𝑃 (for example, theHalting Problem) and machine-verify the following claims:

in simpler relativistic settings, 𝑃 remains uncomputable;in some spacetimes, 𝑃 can be solved.


Is Hypercomputation a Myth?

Davis’ refutationsDavis, M. (2006). Why There is no Such Discipline asHypercomputation. Applied Mathematics and Computation 178.1,pp. 4–7. doi: 10.1016/j.amc.2005.09.066.

Davis, M. (2004). The Myth of Hypercomputation. In: Alan Turing:Life and Legaly of a Great Thinker. Ed. by Teuscher, C. Springer,pp. 195–211. doi: 10.1007/978-3-662-05642-4_8.

Refutation to DavisSundar, N. and Bringsjord, S. (2011). The Myth of ‘The Myth of Hyper-computation’. In: Combined Pre-Proceedings of P&C 2011 and HyperNet2011. Ed. by Stannett, M., pp. 185–196.



https://doi.org/10.1007/978-3-662-05642-4_8


Davis’ refutationsDavis, M. (2006). Why There is no Such Discipline asHypercomputation. Applied Mathematics and Computation 178.1,pp. 4–7. doi: 10.1016/j.amc.2005.09.066.Davis, M. (2004). The Myth of Hypercomputation. In: Alan Turing:Life and Legaly of a Great Thinker. Ed. by Teuscher, C. Springer,pp. 195–211. doi: 10.1007/978-3-662-05642-4_8.




https://doi.org/10.1007/978-3-662-05642-4_8


Davis’ refutationsDavis, M. (2006). Why There is no Such Discipline asHypercomputation. Applied Mathematics and Computation 178.1,pp. 4–7. doi: 10.1016/j.amc.2005.09.066.Davis, M. (2004). The Myth of Hypercomputation. In: Alan Turing:Life and Legaly of a Great Thinker. Ed. by Teuscher, C. Springer,pp. 195–211. doi: 10.1007/978-3-662-05642-4_8.




https://doi.org/10.1007/978-3-662-05642-4_8

Academic Community

CommunitiesHypercomputation Research NetworkComputability in Europe (CiE)

Books and dedicated journal issuesSyropoulos, A. (2008). Hypercomputation. Computing Beyond theChurch-Turing Barrier. Springer.Applied Mathematics and Computation. Vol. 178(1), 2006.Burgin, M. (2005). Super-Recursive Algorithms. Springer.Theoretical Computer Science. Vol. 317(1-3), 2004.Mind and Machines. Vols. 12(4)/13(1), 2002/2003.


Academic Community

CommunitiesHypercomputation Research NetworkComputability in Europe (CiE)

Books and dedicated journal issuesSyropoulos, A. (2008). Hypercomputation. Computing Beyond theChurch-Turing Barrier. Springer.Applied Mathematics and Computation. Vol. 178(1), 2006.Burgin, M. (2005). Super-Recursive Algorithms. Springer.Theoretical Computer Science. Vol. 317(1-3), 2004.Mind and Machines. Vols. 12(4)/13(1), 2002/2003.


Final Remarks

“Once upon on time, back in the golden age of the recursive functiontheory, computability was an absolute.” (Sylvan and Copeland 2000,p. 189)

“Via the great pioneers of electronic computing…the 1930s analysis ofcomputation led to the modern computing era. Who knows where a21st-century overhaul of that classical analysis might lead.” (Copeland,Dresner, Proudfoot and Shagrir 2016, p. 38)“Is there any limit to discrete computation, and more generally, toscientific knowledge?” (Calude and Dinneen 1998, p. 1)““In breaking the Turing barrier, our knowledge of the world, and thereforeour control of it, would be altered forever,” Professor Cooper added.”†

†Computing a way through the Turing barrier. The Reporter. The University of LeedsNewsletter. No. 505. 21 February 2005.


Final Remarks

“Once upon on time, back in the golden age of the recursive functiontheory, computability was an absolute.” (Sylvan and Copeland 2000,p. 189)“Via the great pioneers of electronic computing…the 1930s analysis ofcomputation led to the modern computing era. Who knows where a21st-century overhaul of that classical analysis might lead.” (Copeland,Dresner, Proudfoot and Shagrir 2016, p. 38)

“Is there any limit to discrete computation, and more generally, toscientific knowledge?” (Calude and Dinneen 1998, p. 1)““In breaking the Turing barrier, our knowledge of the world, and thereforeour control of it, would be altered forever,” Professor Cooper added.”†



Final Remarks

“Once upon on time, back in the golden age of the recursive functiontheory, computability was an absolute.” (Sylvan and Copeland 2000,p. 189)“Via the great pioneers of electronic computing…the 1930s analysis ofcomputation led to the modern computing era. Who knows where a21st-century overhaul of that classical analysis might lead.” (Copeland,Dresner, Proudfoot and Shagrir 2016, p. 38)“Is there any limit to discrete computation, and more generally, toscientific knowledge?” (Calude and Dinneen 1998, p. 1)

““In breaking the Turing barrier, our knowledge of the world, and thereforeour control of it, would be altered forever,” Professor Cooper added.”†



Final Remarks

“Once upon on time, back in the golden age of the recursive functiontheory, computability was an absolute.” (Sylvan and Copeland 2000,p. 189)“Via the great pioneers of electronic computing…the 1930s analysis ofcomputation led to the modern computing era. Who knows where a21st-century overhaul of that classical analysis might lead.” (Copeland,Dresner, Proudfoot and Shagrir 2016, p. 38)“Is there any limit to discrete computation, and more generally, toscientific knowledge?” (Calude and Dinneen 1998, p. 1)““In breaking the Turing barrier, our knowledge of the world, and thereforeour control of it, would be altered forever,” Professor Cooper added.”†



References

Burgin, M. (2005). Super-Recursive Algorithms. Springer (cit. on pp. 44,45).Calude, C. S. and Dinneen, M. J. (1998). Breaking the Turing barrier.Tech. rep. CDMTCS (cit. on pp. 46–49).Copeland, B. J. (1998). Super Turing-Machines. Complexity 4.1, pp. 30–32.doi:10.1002/(SICI)1099-0526(199809/10)4:1<30::AID-CPLX9>3.0.CO;2-8(cit. on pp. 11, 12).— (2002a). Accelerating Turing Machines. Minds and Machines 12.2,pp. 281–301. doi: 10.1023/A:1015607401307 (cit. on pp. 11, 12).— (2002b). Hypercomputation. Minds and Machines 12.4, pp. 461–502.doi: 10.1023/A:1021105915386 (cit. on pp. 5, 6).Copeland, B. J. and Proudfoot, D. (1999). Alan Turing’s Forgotten Ideas inComputer Science. Scientific American 280.4, pp. 76–81. doi:10.1038/scientificamerican0499-98 (cit. on p. 10).


https://doi.org/10.1002/(SICI)1099-0526(199809/10)4:1<30::AID-CPLX9>3.0.CO;2-8

https://doi.org/10.1023/A:1015607401307

https://doi.org/10.1023/A:1021105915386

https://doi.org/10.1038/scientificamerican0499-98

References

Copeland, B. J. and Sylvan, R. (1999). Beyond the Universal TuringMachine. Australasian Journal of Philosophy 77.1, pp. 44–66. doi:10.1080/00048409912348801 (cit. on pp. 32, 33).Copeland, J., Dresner, E., Proudfoot, D. and Shagrir, O. (2016). Time toReinspect the Foundations? Communications of the ACM 59.11, pp. 34–38.doi: 10.1145/2908733 (cit. on pp. 46–49).Davis, M. (1958). Computability and Unsolvability. McGraw-Hill (cit. onp. 3).— (2004). The Myth of Hypercomputation. In: Alan Turing: Life andLegaly of a Great Thinker. Ed. by Teuscher, C. Springer, pp. 195–211. doi:10.1007/978-3-662-05642-4_8 (cit. on pp. 41–43).— (2006). Why There is no Such Discipline as Hypercomputation.Applied Mathematics and Computation 178.1, pp. 4–7. doi:10.1016/j.amc.2005.09.066 (cit. on pp. 41–43).Deutsch, D. (1985). Quantum Theory, the Church-Turing Principle and theUniversal Quantum Computer. Proc. R. Soc. Lond. A 400, pp. 97–117.doi: 10.1098/rspa.1985.0070 (cit. on pp. 15, 16).


https://doi.org/10.1080/00048409912348801

https://doi.org/10.1145/2908733

https://doi.org/10.1007/978-3-662-05642-4_8


https://doi.org/10.1098/rspa.1985.0070

References

Deutsch, D. (1989). Quantum Computational Networks. Proc. R. Soc.Lond. A 425, pp. 73–90. doi: 10.1098/rspa.1989.0099 (cit. on pp. 15, 16).Etesi, G. and Németi, I. (2002). Non-Turing Computations ViaMalament-Hogart Space-Times. Int. J. Theor. Phys. 41.2, pp. 341–370.doi: 10.1023/A:1014019225365 (cit. on pp. 34–36).Gandy, R. (1980). Church’s Thesis and Principles for Mechanisms. In: TheKleene Symposium. Ed. by Barwise, J., Keisler, H. J. and Kunen, K.Vol. 101. Studies in Logic and the Foundations of Mathematics.North-Holland Publishing Company, pp. 123–148. doi:10.1016/S0049-237X(08)71257-6 (cit. on pp. 32, 33).Gödel, K. (1990). Remarks Before the Princeton Bicentennial Conferenceon Problems in Mathematics (1946). In: Kurt Gödel. Collected works.Ed. by Feferman, S. et al. Vol. I. Publications 1938–1974. Oxford UniversityPress, pp. 150–153 (cit. on p. 2).Hamkins, J. D. (2002). Infinite Time Turing machines. Minds and Machines12.4, pp. 521–539. doi: 10.1023/A:1021180801870 (cit. on pp. 23, 24).


https://doi.org/10.1098/rspa.1989.0099

https://doi.org/10.1023/A:1014019225365

https://doi.org/10.1016/S0049-237X(08)71257-6

https://doi.org/10.1023/A:1021180801870

References

Hamkins, J. D. (2007). A Survey of Infinite time Turing machines. In:Machines, Computations, and Universality (MCU 2007). Ed. byDurand-Lose, J. and Margenstern, M. Vol. 4664. Lecture Notes inComputer Science, pp. 62–71. doi: 10.1007/978-3-540-74593-8_5 (cit. onpp. 23, 24).Hamkins, J. D. and Lewis, A. (2000). Infinite Time Turing machines. TheJournal of Symbolic Logic 65.2, pp. 567–604. doi: 10.2307/2586556(cit. on pp. 23–31).Hogarth, M. (1992). Does General Relativity Allow an Observer to View anEternity in a Finite Time? Foundations on Physics Letters 5.2, pp. 173–181.doi: 10.1007/BF00682813 (cit. on pp. 34–36).— (1994). Non-Turing Computers and Non-Turing Computability. PSA1994, pp. 126–138 (cit. on pp. 34–36).Kieu, T. D. (2003). Computing the Non-Computable. ContemporaryPhysics 44.1, pp. 51–71. doi: 10.1080/00107510302712 (cit. on pp. 20, 21).


https://doi.org/10.1007/978-3-540-74593-8_5

https://doi.org/10.2307/2586556

https://doi.org/10.1007/BF00682813

https://doi.org/10.1080/00107510302712

References

Németi, I. and Dávid, G. (2006). Relativistic Computers and the TuringBarrier. Applied Mathematics and Computation 178.1, pp. 118–142. doi:10.1016/j.amc.2005.09.075 (cit. on pp. 34–36).Ord, T. and Kieu, T. D. (2009). Using Biased Coins as Oracles.International Journal of Unconventional Computing 5, pp. 253–265 (cit. onpp. 17–19).Pitowsky, I. (1990). The Physical Church Thesis and PhysicalComputational Complexity. Iyyun 39, pp. 81–99. url:http://www.jstor.org/stable/23350656 (cit. on pp. 34–36).Schindler, R. (2003). 𝑃 ≠ 𝑁𝑃 for Infinite Time Turing Machines.Monatshefte für Mathematik 139.4, pp. 335–340. doi:10.1007/s00605-002-0545-5 (cit. on pp. 27–29).Seyfferth, B. (2013). Three Models of Ordinal Computability. PhD thesis.University of Bonn. url:http://hss.ulb.uni-bonn.de/2013/3196/3196.pdf (visited on14/05/2018) (cit. on pp. 30, 31).



http://www.jstor.org/stable/23350656

https://doi.org/10.1007/s00605-002-0545-5

http://hss.ulb.uni-bonn.de/2013/3196/3196.pdf

References

Sicard, A., Ospina, J. and Vélez, M. (2006). Quantum HypercomputationBased on the Dynamical Algebra 𝔰𝔲(1, 1). J. Phys. A: Math. Gen. 39.40,pp. 12539–12558. doi: 10.1088/0305-4470/39/40/018 (cit. on p. 22).Siegelmann, H. T. (1999). Neural Networks and Analog Computation.Beyond the Turing Limit. Progress in Theorical Computer Science.Birkhäuser (cit. on p. 13).Stannett, M. (2015). Towards Formal Verification of Computations andHypercomputations in Relativistic Physics. In: Machines, Computations,and Universality (MCU 2015). Ed. by Durand-Lose, J. and Nagy, B.Vol. 9288. Lecture Notes in Computer Science, pp. 17–27. doi:10.1007/978-3-319-23111-2_2 (cit. on pp. 37, 38).Stannett, M. and Németi, I. (2014). Using Isabelle/HOL to VerifyFirst-Order Relativity Theory. Journal of Automated Reasoning 52 (4),pp. 361–378. doi: 10.1007/s10817-013-9292-7 (cit. on pp. 37, 38).Sundar, N. and Bringsjord, S. (2011). The Myth of ‘The Myth ofHypercomputation’. In: Combined Pre-Proceedings of P&C 2011 andHyperNet 2011. Ed. by Stannett, M., pp. 185–196 (cit. on pp. 41–43).


https://doi.org/10.1088/0305-4470/39/40/018

https://doi.org/10.1007/978-3-319-23111-2_2

https://doi.org/10.1007/s10817-013-9292-7

References

Sylvan, R. and Copeland, J. (2000). Computability is Logic-Relative. In:Sociative Logics and Their Applications: Essays by the late Richard Sylvan.Ed. by Priest, G. and Hyde, D. Ashgate Publishing Company, pp. 189–199(cit. on pp. 4, 46–49).Syropoulos, A. (2008). Hypercomputation. Computing Beyond theChurch-Turing Barrier. Springer (cit. on pp. 44, 45).Turing, A. M. (1939). Systems of Logic Based on Ordinales. Proc. LondonMath. Soc. 45.2239, pp. 161–228. doi: 10.1112/plms/s2-45.1.161 (cit. onpp. 8, 9).Williams, C. P. and Clearwater, S. H. (1997). Explorations in QuantumComputing. Springer-Telos (cit. on pp. 17–19).


https://doi.org/10.1112/plms/s2-45.1.161

Documents

Automata and Formal Languages - CM0081 … J. E., Motwani, R. and Ullman, J. D. (2007). Introduction to Automata theory, Languages, and Computation. 3rd ed. Pearson Education. Hutton,