Upload
sundeep-chopra
View
215
Download
0
Embed Size (px)
Citation preview
7/28/2019 types of grammer
1/32
PersonalitiesLanguagesGrammars
Chomsky HierarchySummary
CHOMSKY HIERARCHY
Jie Jiang
October 4, 2009
Jie Jiang CHOMSKY HIERARCHY
7/28/2019 types of grammer
2/32
PersonalitiesLanguagesGrammars
Chomsky HierarchySummary
Noam ChomskyMarcel Schutzenberger
Avram Noam Chomsky
Born on December 7, 1928, currentlyprofessor emeritus of linguistics at MIT.
Americanlinguist, philosopher, cognitive scientist,political activist, author, and lecturer.
One of the fathers of modern linguistics.
Created the theory of generative
grammar, which has undergonenumerous revisions and has had a profound influence onlinguistics.
Sparkedthe cognitive revolution in psychology.
Jie Jiang CHOMSKY HIERARCHY
7/28/2019 types of grammer
3/32
PersonalitiesLanguagesGrammars
Chomsky HierarchySummary
Noam ChomskyMarcel Schutzenberger
Biography
From1945, studied philosophy and linguistics
at the University of Pennsylvania.PhD in linguistics fromUniversity of Pennsylvania in 1955.
1956, appointedfull Professor at MIT, Departmentof Linguistics and Philosophy.
1966,Ferrari P. Ward Chair; 1976, InstituteProfessor; currently Professor Emeritus.
Jie Jiang CHOMSKY HIERARCHY
7/28/2019 types of grammer
4/32
PersonalitiesLanguagesGrammars
Chomsky HierarchySummary
Noam ChomskyMarcel Schutzenberger
Contribution
Linguistics
Transformational grammars
Generative grammarLanguage aquisition
Computer Science
Chomsky hierarchyChomsky Normal Form
Context Free GrammarsPsychology
Cognitive Revolution (1959)Universal grammar
Jie Jiang CHOMSKY HIERARCHY
P li i
7/28/2019 types of grammer
5/32
PersonalitiesLanguagesGrammars
Chomsky HierarchySummary
Noam ChomskyMarcel Schutzenberger
Universal grammar
Noam Chomskys approach to the study of language is knownas universal grammar:
the human brain contains a limited set of rules for organizinglanguage. In turn, there is an assumption that all languageshave a common structural basis. This set of rules is known asuniversal grammaran innate set of linguistic principles shared by all humansattempts to explain language acquisition in general, not
describe specific languagesproposes a set of rules intended to explain language acquisitionin child development
Jie Jiang CHOMSKY HIERARCHY
P liti
7/28/2019 types of grammer
6/32
PersonalitiesLanguagesGrammars
Chomsky HierarchySummary
Noam ChomskyMarcel Schutzenberger
Marcel-Paul Schutzenberger
October
24, 1920 July 29, 1996.French mathematicianand Doctor of Medicine.
Professor of the Facultyof Sciences, University of Paris
Full Memberof French Academy of Sciences.
Jie Jiang CHOMSKY HIERARCHY
Personalities
7/28/2019 types of grammer
7/32
PersonalitiesLanguagesGrammars
Chomsky HierarchySummary
Noam ChomskyMarcel Schutzenberger
Biography
First trained as a physician,doctorate in medicine in 1948.
PhD in mathematics in 1953Professor at theUniversity of Poitiers, 1957-1963
Director of research
at the CNRS, 1963-1964Professorin the Faculty of Sciences at theUniversity of Paris, 1964-1996
Jie Jiang CHOMSKY HIERARCHY
Personalities
7/28/2019 types of grammer
8/32
PersonalitiesLanguagesGrammars
Chomsky HierarchySummary
Noam ChomskyMarcel Schutzenberger
Contribution
Formallanguages with Noam Chomsky
Chomsky-SchutzenbergerhierarchyChomsky-Schutzenbergertheorem
Automata
with Samuel EllenbergerBiology and Darwinism
Mathematical critique ofneodarwinism (1966)
Jie Jiang CHOMSKY HIERARCHY
Personalities
7/28/2019 types of grammer
9/32
PersonalitiesLanguagesGrammars
Chomsky HierarchySummary
AlphabetsLanguages
Alphabet
In computational practice, data are encoded in the computersmemory as strings of bits or other symbols appropriate formanipulation by a computer.How to understand the mathematics of strings of symbols?
Definition: An alphabet is a finite, nonempty set of symbols.We use to denote this alphabet.
the Roman alphabet {a, b, . . . , z}.binary alphabet {0, 1} :the alphabet particularly pertinent to the theory of
computation.Note: any object can be in an alphabet.In a formal point of view, an alphabet is simply a finite set ofany sort.For simplicity, however, we use as symbols only letters,
numerals, and other common characters such as $, or #.Jie Jiang CHOMSKY HIERARCHY
Personalities
7/28/2019 types of grammer
10/32
PersonalitiesLanguagesGrammars
Chomsky HierarchySummary
AlphabetsLanguages
String
A string is a finite sequence of symbols from .
We simply juxtapose the symbols to denote a string.
watermelon is a string over the alphabet {a, b, . . . , z}.0111011 is a string over {0, 1}.
A string has no symbols at all the empty string denoted by .
We generally use u, v, x, y, z, and Greek letters to denotestrings.
Example: we use w as a name for the string abc.
Jie Jiang CHOMSKY HIERARCHY
Personalities
7/28/2019 types of grammer
11/32
so t sLanguagesGrammars
Chomsky HierarchySummary
AlphabetsLanguages
String
The length of a string s, denoted |s|, is the number ofsymbols in it.
|101| = 3|| = 0
A string w , can be considered as a functionw : {1, . . . , |w|} ,the value of w(j), where 1 j |w|, is the symbol in the jthposition of w.
Example: if w = accordion, thenw(3) = w(2) = c, and w(1) = a.
Occurrences of a symbol in a string:the symbol occurs in the jth position of the stringw if w(j) = .
Jie Jiang CHOMSKY HIERARCHY
Personalities
7/28/2019 types of grammer
12/32
LanguagesGrammars
Chomsky HierarchySummary
AlphabetsLanguages
Operations on strings
Concatenation: x y ( or simply xy )Formally, w = x y iff |w| = |x| + |y|, w(j) = x(j) forj = 1, . . . , |x|, and w(|x| + j) = y(j) for j = 1, . . . , |y|.Example: 01 001 = 01001, beach boy = beathboy
Note: w = w for any string |w|Associative: (wx)y = w(xy) for any strings w, x and y.
A string v is a substring of a string w iff there are strings xand y such that w = xvy.
Note: every string is a substring of itself. (x = and y = )
is a substring of every string.If w = xv for some x, then v is a suffix of w.
If w = vy for some y, then v is a prefix of w.
Example: road is a prefix of roadrunner, a suffix of abroad,and a substring of both these and of broader.
Jie Jiang CHOMSKY HIERARCHY
Personalities
7/28/2019 types of grammer
13/32
LanguagesGrammars
Chomsky HierarchySummary
AlphabetsLanguages
Operations on strings
Definition wi:for each string w and each natural number i, the string wi isdefined as
w0 = , the empty stringwi+1 = wi w, for each i 0
Definition by induction : wi+1 is defined in terms of wi.
Example: (do)2 = ?
(do)2
= (do)1
do, (i = 1)(do)1 = (do)0 do, (i = 0)(do)0 = (do)2 = (e do) do = dodo
Jie Jiang CHOMSKY HIERARCHY
Personalities
7/28/2019 types of grammer
14/32
LanguagesGrammars
Chomsky HierarchySummary
AlphabetsLanguages
Operations on strings
Definition wR: reversal of a string w, denoted by wR, isdefined as
If |w| = 0, then wR = w = If |w| = n + 1 0, then w = ua for some a , andwR = auR
The string spelled backwards:
Example: reverseR = esrever.
Note: (wx)R = xRwR, for any strings w and x.
Jie Jiang CHOMSKY HIERARCHY
PersonalitiesL
7/28/2019 types of grammer
15/32
LanguagesGrammars
Chomsky HierarchySummary
AlphabetsLanguages
Languages definition
denotes the set of all sequences of strings that arecomposed of zero or more symbols of .+ denotes the set of all sequences of strings composed ofone or more symbols of .
+
=
{}A language is a subset of .A language is a collection of sentences of finite length allconstructed from a finite alphabet of symbols.
, and are languages.
{aba, czr, d, f} is a language over {a, b, . . . , z}.The infinite languages are specified by the scheme:
L = {w : w has property P}
{w {0, 1} : w has an equal number of 0s and ls}{w : w = wR}
Jie Jiang CHOMSKY HIERARCHY
PersonalitiesL
7/28/2019 types of grammer
16/32
LanguagesGrammars
Chomsky HierarchySummary
AlphabetsLanguages
Operations on languages
Concatenation: if L1 and L2 are languages over , theirconcatenation is L = L1 L2, or simply L = L1L2, where
L = {w : w = x y for some x L1 and y L2}
Example: = {0, 1},L1 = {w : w has an even number of 0s},L2 = {w
: w starts with a 0 and the rest of the symbols
are ls} L1 L2 = {w : w has an odd number of 0s}
Jie Jiang CHOMSKY HIERARCHY
PersonalitiesLanguages
7/28/2019 types of grammer
17/32
LanguagesGrammars
Chomsky HierarchySummary
AlphabetsLanguages
Operations on languages
Kleene star of a language L, denoted by L, is the set of allstrings obtained by concatenating zero or more strings from L:
L = {w : w = w1 wk for some k 0 and somew1, . . . ,wk L}
Example: = {0, 1},L = {01, 1, 100}, 110001110011 L
since 110001110011 = 1 100 01 1 100 1 1, and each ofthese strings is in Ls.
Jie Jiang CHOMSKY HIERARCHY
PersonalitiesLanguages Introduction
7/28/2019 types of grammer
18/32
LanguagesGrammars
Chomsky HierarchySummary
IntroductionFormal definitionExamples
Grammars
A grammar can be regarded as a device that enumerates thesentences of a language.
Types of grammarsPrescriptive: prescribes authoritative norms for a languageDescriptive: attempts to describe actual usage rather thanenforce arbitrary rulesFormal: a precisely defined grammar, such as context-free
Generative: a formal grammar that can generate naturallanguage expressions
Jie Jiang CHOMSKY HIERARCHY
PersonalitiesLanguages Introduction
7/28/2019 types of grammer
19/32
LanguagesGrammars
Chomsky HierarchySummary
IntroductionFormal definitionExamples
Formal grammars
Two broad categories of formal languages: generative andanalytic
A generative grammar formalizes an algorithm that generatesvalid strings in a language
An analytic grammar is a set of rules to reduce an inputstring to a boolean result that indicates the validity of thestring in the given language
A generative grammar describes how to write a language, andan analytic grammar describes how to read it (a parser).
Jie Jiang CHOMSKY HIERARCHY
PersonalitiesLanguages Introduction
7/28/2019 types of grammer
20/32
LanguagesGrammars
Chomsky HierarchySummary
IntroductionFormal definitionExamples
Formal Grammars
How to determine whether a string is valid within a givenlangauge?
A grammar is a set of formation rules that describe whichstrings formed from the alphabet of a language are
syntactically valid within the language.Grammars are composed of:a set of string rewriting rulesan assigned start symbol
The language described by a grammar is the set of stringsthat can be generated by applying these rules arbitrarily,starting with the start symbol.
Grammars could be thought of language generators or stringrecognizers for languages.
Jie Jiang CHOMSKY HIERARCHY
PersonalitiesLanguages Introduction
7/28/2019 types of grammer
21/32
g gGrammars
Chomsky HierarchySummary
Formal definitionExamples
Example
Assume the alphabet consists of a and b, the start symbol isS, rules are:1. S aSb2. S ba
Operations: start with S, and choose a rule to apply to itchoose rule 1, obtain the string aSbchoose rule 1 again, replace S with aSb aaSbbrepeated at will until all occurrences of S are removed, andonly symbols from the alphabet remain (i.e., a and b).
choose rule 2, replace S with ba aababbS aSb aaSbb aababb
The language of the grammar is the set of all the strings thatcan be generated using this process:
{ba, abab, aababb, aaababbb, . . .}
Jie Jiang CHOMSKY HIERARCHY
PersonalitiesLanguages Introduction
7/28/2019 types of grammer
22/32
g gGrammars
Chomsky HierarchySummary
Formal definitionExamples
Grammars
Defined as a quadruple Vt, Vn, P, S, where:
Vt = the terminal vocabularyi.e. the words of the language: finite (but productive ...)
Vn = the non-terminals
i.e. the set of categories, used for the benefit of the grammarto express generalizations over items in Vt
P = the set of productions,i.e. rules in the grammar: finite, and of the form
where , (Vn Vt)
S = the distinguished symbol, or start symbol (sentence)
S Vn,P must include at least one rule where = S.
Jie Jiang CHOMSKY HIERARCHY
PersonalitiesLanguages Introduction
7/28/2019 types of grammer
23/32
GrammarsChomsky Hierarchy
Summary
Formal definitionExamples
Generative
A grammar is generative if it predicts (explicitly defines, orcharacterizes) all and only all strings the language.
The language of a grammar G, denoted as L(G), is defined asall those strings over that can be generated by G.
L(G) = {w : S G
w}
Jie Jiang CHOMSKY HIERARCHY
PersonalitiesLanguages Introduction
7/28/2019 types of grammer
24/32
GrammarsChomsky Hierarchy
Summary
Formal definitionExamples
Example
Consider the grammar G where Vn = {S, B}, Vt = a, b, c, Sis the start symbol, and P consists of :1. S aBSc
2. S abc3. Ba aB4. Bb bb
Some examples of the derivation of strings in L(G) are:
S
2 abc
S 1 aBSc 2 aBabcc 3 aaBbcc 4 aabbccS 1 aBSc 1 aBaBScc 2 aBaBabccc 3 aaBBabccc 3aaBaBbccc 3 aaaBBbccc 4 aaaBbbccc 4 aaabbbccc
Jie Jiang CHOMSKY HIERARCHY
PersonalitiesLanguagesG
7/28/2019 types of grammer
25/32
GrammarsChomsky Hierarchy
Summary
Chomsky Hierarchy
Type Grammar Language Automata
3 Finite State Regular Finite
2 Context-Free C-F Pushdown
1 Context-Sensitive C-S Linear-Bounded
0 General Rewrite Unrestricted Turing Machines
occasionally referred to as ChomskySchutzenberger hierarchy
described by Noam Chomsky in 1956hierarchy of grammars
Type 3 Type 2 Type 1 Type 0
Jie Jiang CHOMSKY HIERARCHY
PersonalitiesLanguagesG
7/28/2019 types of grammer
26/32
GrammarsChomsky Hierarchy
Summary
Type-3 grammars
Type Grammar Language Automata
3 Finite State Regular Finite
2 Context-Free C-F Pushdown
1 Context-Sensitive C-S Linear-Bounded
0 General Rewrite Unrestricted Turing Machines
regular grammars: A a, and A aBa single nonterminal on the left-hand side.a single terminal, possibly followed (or preceded, but not both)by a single nonterminal on the right-hand side.S is allowed if S is NOT on the right side of any rule
generate the regular languages:can be decided by finite state automata.can be obtained by regular expressions (search patterns &lexical structure of programming languages)
Jie Jiang CHOMSKY HIERARCHY
PersonalitiesLanguagesGrammars
7/28/2019 types of grammer
27/32
GrammarsChomsky Hierarchy
Summary
Type-2 grammars
Type Grammar Language Automata
3 Finite State Regular Finite
2 Context-Free C-F Pushdown
1 Context-Sensitive C-S Linear-Bounded
0 General Rewrite Unrestricted Turing Machines
context-free grammars: A a single nonterminal on the left-hand side.a string of terminals and nonterminals on the right-hand side.
(less restrictive than Type-3)generate the context-free languages:
can be recognized by non-deterministic pushdown automata.theoretical basis for the syntax of most programminglanguages.
Jie Jiang CHOMSKY HIERARCHY
PersonalitiesLanguagesGrammars
7/28/2019 types of grammer
28/32
GrammarsChomsky Hierarchy
Summary
Type-1 grammars
Type Grammar Language Automata
3 Finite State Regular Finite
2 Context-Free C-F Pushdown
1 Context-Sensitive C-S Linear-Bounded
0 General Rewrite Unrestricted Turing Machines
context-sensitive grammars: A and may be empty.A is a single nonterminal. (less restrictive than Type-2) is a nonempty string of nonterminals and terminals. (lessrestrictive)S is allowed if S is NOT on the right side of any rule.
generate the context-sensitive languages:can be recognized by linear bounded automata (anondeterministic Turing machine whose tape is bounded by a
constant times the length of the input).Jie Jiang CHOMSKY HIERARCHY
PersonalitiesLanguagesGrammars
7/28/2019 types of grammer
29/32
GrammarsChomsky Hierarchy
Summary
Type-0 grammars
Type Grammar Language Automata
3 Finite State Regular Finite
2 Context-Free C-F Pushdown1 Context-Sensitive C-S Linear-Bounded
0 General Rewrite Unrestricted Turing Machines
unrestricted grammars:
generate the recursively enumerable languages:
can be recognized by Turing machines.
Jie Jiang CHOMSKY HIERARCHY
PersonalitiesLanguagesGrammars
7/28/2019 types of grammer
30/32
GrammarsChomsky Hierarchy
Summary
Exercise 1
Given = {0, 1}, please provide the specifications for thefollowing languages:The language consists of all strings begin with 0.
{0}{0, 1}
The language consists of all strings begin with 0, and endwith 1.
{0}{0, 1}{1}The language consists of all strings with odd lengths.
{0, 1}2n1, n = 1, 2, . . .The language consists of all strings with substring of threeconsecutive 0.
{0, 1}000{0, 1}
The language consists of all strings without substring of threeconsecutive 0.
{001, 01, 1}
Jie Jiang CHOMSKY HIERARCHY
PersonalitiesLanguagesGrammars
7/28/2019 types of grammer
31/32
Chomsky HierarchySummary
Exercise 2
Consider the grammar G where Vn = {S, B, C}, Vt = a, b, c,S is the start symbol, and P consists of :1. S aBC2. S aSBC
3. aB ab4. bB bb5. CB BC6. bC bc7. cC cc
try to provide one derivation for sentence aaabbbccc.S 2 aSBC 2 aaSBCBC 1 aaaBCBCBC 3aaabCBCBC 5 aaabBCCBC 4 aaabbCCBC 5aaabbCBCC 5 aaabbBCCC 4 aaabbbCCC 6aaabbbcCC 7 aaabbbccC 7 aaabbbccc
Jie Jiang CHOMSKY HIERARCHY
PersonalitiesLanguagesGrammars
7/28/2019 types of grammer
32/32
Chomsky HierarchySummary
1
PersonalitiesNoam ChomskyMarcel Schutzenberger
2 Languages
AlphabetsLanguages
3 GrammarsIntroduction
Formal definitionExamples
4 Chomsky Hierarchy
Jie Jiang CHOMSKY HIERARCHY