Upload
katrina-henry
View
219
Download
1
Embed Size (px)
Citation preview
Computational Grammars
Azadeh Maghsoodi
History
Before 18001800-1900First 20s20sWorld War IILast 1950sNowadays
Before 1800
Traditional Grammar Correct Speech of a specific language Not scientific Rejected Useful issues: POS
1800-1900
Indian-European languages Language vs. Other languages Language vs. its history
Early 20s
Enough Philology! Language in a specific time
20s
America & Western Europe Intellectual Pattern Understanding Processes in human being
World War II
Math. Logic as a study tool Computer invention caused new App Abstract Mind model ends Behaviorism
Late 1950
Chomsky is coming! Formal Language Theory “Syntactic Structures” Language Categories
– Type 0: Natural (Irregular)– Type 1: Context sensitive– Type 2: Context free– Type 3: Regular
Late 1950 (continue)
Chomsky followers professes:– Generative grammar: Accurate and definite
enough for testing Generative Grammars
– Goal: Unaware knowledge of users– Biologic and inborn basis for linguistic abilities
Universal Grammar Shared structures
Nowadays
Motives – Discover human mind structure – Language process technology
Applications – Word processors– MT– Word predictors– Text predictors– UFIs / DB Queries– Information retrieval
Syntactic Model
Grammars
Parse Algorithms
Computational Grammars
Generative Grammars– Caused by Natural Language Theory– Introduced by Chomsky– Accurate and definite structures– Transformational grammar (TG)– Constraint-Based Lexicalist grammar (CBLG)
TG
Less computational efficiency Theoretical basis Complex rules Simple lexicons
TG (continue)
Chomsky hierarchy & First TG Standard Theory (1965) Extended Standard Theory Government & Binding Theory (1981-1988)
Standard Theory
Sentence – Deep structure– Surface structure
Generative TG– Basic part
Produce deep structure CFG
– Transformational part Transformational Rules
Transformational Rules
Convert deep structure to surface structure Transformational Rule ~ Transformation Example: (same deep structures)
– (i) The boys place the book on the table.– (ii) The boy has placed the book on the table.– (iii) Did the boy place the book on the table?
Transformational Rules (example)
A deep structure:
S
NP VP
N D
the boy
Aux V NP
will place The book
Transformational Rules (example)
To produce yes/no question:– Using a Move Transformation– S[NP VP [AUX V NP]] S[AUX NP VP[V NP]]
S
NP VP
Aux V NP
S
Aux NP VP
V NP
Government and Binding Theory (GB)
Universal grammar theory Learning a language = confirming a small set
of parameters + learning lexicons Move α: deep structure to surface structure ‘Move α’ moves anything to anywhere Some constraints correct ‘Move α’
GB (continue)
Lexicons
Deep Struct
Surface Struct
Logical FormPhonological Form
Move-α
LF Move-αStylistic &
Phonological Rules
GB (continue)
Minimalist Program (MP)– Choose the best candidate instead of direct
production– Under study
CBLG
Based on TGs Increase computational efficiency of
grammars Simple rules Complex lexicons Psychological Computational
CBLG (continue)
Constraint-Based architecture– Constraint satisfaction more important than
transformational derivation
Strict lexicalism– Lexicons: syntactic atoms of a language– Independent Internal structure from syntactic
constraints
CBLG (continue)
Surface structures are produced directly Most computational grammars are CBLG
Computational Grammars
Unification grammar (UG) Categorical grammar (CG) Dependency grammar (DG) Link grammar Lexical/Functional grammar (LFG) Tree Adjoining grammar (TAG) Generalized Phrase Structure grammar (GPSG) Head Driven Phrase Structure grammar (HPSG)
Unification Grammar (UG)
Lots of CBLs are UG Augmented CFG
– CFG can’t recognize long distance dependencies– A generalized form of CFG + A set of features– Augmented Transition Network (ATN)– Definite Clause Grammar (DCG)
Unification Grammars
UG (continue)
Unification Grammars– Feature structures are extended– No need to CFGs– Grammar ~ A set of constraints between feature
structures– Key concept: Subsumption relation
UG (continue)
CAT verb
ROOT cry CAT verb
ROOT cry
CAT verb VFORM present
VFORM present
(Unificator)
UG (example)
S NP VP Unification grammar:
X0 X1 X2 CAT 0 = 5
CAT 1 = NP
CAT 2 VP
AGR 0 = AGR 1 = AGR 2
VFORM 0 = VORM 2
UG (continue)
More grammar information are stored in lexicons
Less grammar rules Using DAGs
ATN Grammar
Transitive network ~ Expanded Finite-State machine
ATN Grammar ~ A set of transitive networks Features Constraints
Categorical Grammar (CG)
Lots of bases are omitted No difference between lexicons and none-
lexicons Part Of Speech is replaced by some complex
category NP/S : NP is on the right NP\S : NP is on the left
CG (example)
Peter : NP
Likes : (NP\S)/NP
Peanuts : NP
Passionately : (NP\S)\(NP\S)
Peter likes peanuts passionately.
CG (example)
S
NP NP\S
Peter NP\S (NP\S)\(NP\S)
(NP\S)/NP NP
Likes peanuts
passionately
Dependency Grammar (DG)
American linguists Based on TGs Dependencies between words Dependency tree
V
N play Adv
boys well
Link Grammar
Planarity phenomenon Legal sequence of words:
– Satisfy local necessities (satisfaction)– No crossed conjunctions (planarity)– One connected graph (connectivity)
CFG Lexical grammars
– Grammar is distributed between words Probability models Voice recognition Hand-written recognition
Link Grammar (example)
linking requirements:
Link Grammar (example)
linking requirements are satisfied
Link grammar (example)
Not part of a language
Lexical-Functional Grammar (LFG)
Unification grammar Not TG ATN research and its deficiencies introduced
LFG Group structures 4 structures
Tree Adjoining Grammar (TAG)
Between CFG and CSG Grammar rules are a set of initial trees Initial trees are anchored trees Two main operations:
– Substitution– Adjoin
High accuracy
TAG (example)
S VP S
NP VP + VP ADV NP VP
V NP VP ADV
V NP
TAG (continue)
High accuracy Apps in NLP
– MT– Information retrieval– …
Generalized Phrase Structure grammar (GPSG)
Only CFLs CFG Rules
– Immediate Dominance (ID)– Linear Precedence (LP)
Head Driven Phrase Structure grammar (HPSG)
Lexical grammar Based on unification Increase computational potency of GPSG Simple CFG Complex lexicons
Applications
Parse Algorithms
Top-Down parsing Bottom-Up parsing (*)
Parse Algorithms
Top-Down parsing Chart parser
– Dynamic Programming Recursive Transition Network (RTN)
– ATN grammar LR parser
– Shift-Reduce algorithms Cocke-Younger-Kasami parser (CYK)
– Dynamic Programming– CNF grammar
Efficient Algorithms
Chart parser CYK parser
Questions???