Embed Size (px)
Citation preview
Why Syntax is ImpossibleWhy Syntax is Impossible
Mike DowmanMike Dowman
SyntaxSyntax
Languages have tens of thousands of words
Some combinations of words make valid sentences
Others don’tNo one understands the grammar
of any language
Languages have tens of thousands of words
Some combinations of words make valid sentences
Others don’tNo one understands the grammar
of any language
Syntax is Complicated!Syntax is Complicated!
I saw Bill with Mary yesterday.You saw WHO with Mary yesterday?!Who did you see with Mary yesterday?
I saw Bill with Mary yesterday.You saw WHO with Mary yesterday?!Who did you see with Mary yesterday?
Syntax is Complicated!Syntax is Complicated!
I saw Bill with Mary yesterday.You saw WHO with Mary yesterday?!Who did you see with Mary yesterday?
I saw Bill and Mary yesterday.You saw WHO and Mary yesterday?!
I saw Bill with Mary yesterday.You saw WHO with Mary yesterday?!Who did you see with Mary yesterday?
I saw Bill and Mary yesterday.You saw WHO and Mary yesterday?!
Syntax is Complicated!Syntax is Complicated!
I saw Bill with Mary yesterday.You saw WHO with Mary yesterday?!Who did you see with Mary yesterday?
I saw Bill and Mary yesterday.You saw WHO and Mary yesterday?!Who did you see and Mary yesterday?
I saw Bill with Mary yesterday.You saw WHO with Mary yesterday?!Who did you see with Mary yesterday?
I saw Bill and Mary yesterday.You saw WHO and Mary yesterday?!Who did you see and Mary yesterday?
Generative GrammarGenerative Grammar
An explicit formal system that defines the set of valid sentences in a language
And maybe also explains what each one means
Generative grammar is the core research topic in linguistics
Includes strongly nativist theories and theories proposing that languages are primarily learned
An explicit formal system that defines the set of valid sentences in a language
And maybe also explains what each one means
Generative grammar is the core research topic in linguistics
Includes strongly nativist theories and theories proposing that languages are primarily learned
Grammar WritingGrammar Writing
Linguists take a selection of possible sentences
And obtain grammaticality judgments for those sentences
Then they produce a grammar that accounts for all the data
Linguists take a selection of possible sentences
And obtain grammaticality judgments for those sentences
Then they produce a grammar that accounts for all the data
Grammar CoverageGrammar Coverage
Linguists’ grammars only work for selected sentences
They can’t explain most naturally occurring sentences
The more data we consider the more surprising quirks of syntax that emerge
Linguists’ grammars only work for selected sentences
They can’t explain most naturally occurring sentences
The more data we consider the more surprising quirks of syntax that emerge
Children’s Language Acquisition
Children’s Language Acquisition
Kid’s observe a limited number of example sentences
But quickly internalize a system that correctly characterizes the whole language
Kid’s observe a limited number of example sentences
But quickly internalize a system that correctly characterizes the whole language
I-languageE-language LAD
How can kids do syntax when linguists can’t?
How can kids do syntax when linguists can’t?
Innate component of language (provided by genes)
Learned component of language (provided by language data)
Innate component of language (provided by genes)
Learned component of language (provided by language data)
How can kids do syntax when linguists can’t?
How can kids do syntax when linguists can’t?
Innate component of language (provided by genes)
Learned component of language (provided by language data)
Linguists have to infer bothChildren only the learned compo
nent
Innate component of language (provided by genes)
Learned component of language (provided by language data)
Linguists have to infer bothChildren only the learned compo
nent
Information TheoryInformation Theory
Both components of language must contain some amount of information
Data available to children must provide at least enough information as is in the learned component
This puts a limit on the complexity of the learned component of language
Both components of language must contain some amount of information
Data available to children must provide at least enough information as is in the learned component
This puts a limit on the complexity of the learned component of language
Linguists’ TaskLinguists’ Task
Linguists need to have at least as much information as is in the learned and innate components together
Can use data from multiple languages to try to characterize innate components
And can use positive and negative data
Linguists need to have at least as much information as is in the learned and innate components together
Can use data from multiple languages to try to characterize innate components
And can use positive and negative data
Correspondence to Linguistic Theories
Correspondence to Linguistic Theories
Small learned component = parameter setting
Large learned component = learned languages
Small innate component = general learning mechanism
Large innate component = universal grammar
Small learned component = parameter setting
Large learned component = learned languages
Small innate component = general learning mechanism
Large innate component = universal grammar
Size of Each ComponentSize of Each Component
Innate Component
small large huge
small learn = easy
ling = easy
learn = easy
ling = hard
learn = easy
ling = impossible
large learn = hard
ling = hard
learn = hard
ling = hard
learn = hard
ling = impossible
Learned
Component
huge learn = impossible
ling = impossible
learn = impossible
ling = impossible
learn = impossible
ling = impossible
Which component is large?
Which component is large?
As we haven’t yet managed to produce a generative grammar, at least one of innate or learned components must be large
Children learn relatively easily, so the learned component can’t be too big
As we haven’t yet managed to produce a generative grammar, at least one of innate or learned components must be large
Children learn relatively easily, so the learned component can’t be too big
Size of Each ComponentSize of Each Component
Innate Component
small large huge
small learn = easy
ling = easy
learn = easy
ling = hard
learn = easy
ling = impossible
large learn = hard
ling = hard
learn = hard
ling = hard
learn = hard
ling = impossible
Learned
Component
huge learn = impossible
ling = impossible
learn = impossible
ling = impossible
learn = impossible
ling = impossible
How big could the innate component be?
How big could the innate component be?
Genome contains 3 billion base pairs = 6 billion bits
Cell metabolism adds more information
Each base pair can be modified
Huge amount of information!
Genome contains 3 billion base pairs = 6 billion bits
Cell metabolism adds more information
Each base pair can be modified
Huge amount of information!
What could be in a huge innate component?
What could be in a huge innate component?
Not words forms - vary from language to language
Grammaticality patternsRules of syntax would be hugely
complex
Not words forms - vary from language to language
Grammaticality patternsRules of syntax would be hugely
complex
Impossibility of SyntaxImpossibility of Syntax
Grammaticality judgments on average can provide no more than one bit of information each
If syntax is hugely complex, there will be many grammars that are compatible with any given body of data
But all but one of these grammars would fail when tested on enough new data
Grammaticality judgments on average can provide no more than one bit of information each
If syntax is hugely complex, there will be many grammars that are compatible with any given body of data
But all but one of these grammars would fail when tested on enough new data
A Concrete ExampleA Concrete Example
A multi-agent modelEach agent has:innate componentlearned componentBoth are bit strings of fixed
lengthSentences are 100 bit strings
A multi-agent modelEach agent has:innate componentlearned componentBoth are bit strings of fixed
lengthSentences are 100 bit strings
Deciding on the Grammaticality of a Sentence 1
Deciding on the Grammaticality of a Sentence 1
Treat the sentence as a binary number Find:bi = s mod ni
bl = s mod nl
b is an index to a bit in the innate (bi) or learned (bl) component
n is the number of bits in the innate (ni) or learned (nl) component
s is the length of the sentences
Treat the sentence as a binary number Find:bi = s mod ni
bl = s mod nl
b is an index to a bit in the innate (bi) or learned (bl) component
n is the number of bits in the innate (ni) or learned (nl) component
s is the length of the sentences
Deciding on the Grammaticality of a Sentence 2
Deciding on the Grammaticality of a Sentence 2
A pseudo-random function maps from the two selected bits plus the sentence to a Boolean grammaticality judgment
It’s therefore typically necessary to know every bit of the sentence and both the innate and learned bits to predict the grammaticality of the sentence
Every bit counts
Usually about half of sentences are grammatical, half ungrammatical
A pseudo-random function maps from the two selected bits plus the sentence to a Boolean grammaticality judgment
It’s therefore typically necessary to know every bit of the sentence and both the innate and learned bits to predict the grammaticality of the sentence
Every bit counts
Usually about half of sentences are grammatical, half ungrammatical
4 Kinds of Agent4 Kinds of Agent
TeacherInnate: 10101000
Learned: 10010101
RelatedInnate: 10101000
Learned: 11110001
UnrelatedInnate: 10110101
Learned: 00111000
LinguistInnate: 00110100
Learned: 10001100
Learning by Related, Unrelated
Learning by Related, Unrelated
Observe a sentence from the teacher
Work out if it is grammatical according to current I-language
If not, invert the relevant bit of the learned component
Observe a sentence from the teacher
Work out if it is grammatical according to current I-language
If not, invert the relevant bit of the learned component
Grammar Inference by Linguists
Grammar Inference by Linguists
Choose random sentencesAsk the teacher if they are grammatic
alStore all sentences and grammaticali
ty judgmentsSearch for a setting of innate and lea
rned components that assigns the correct grammaticality rating to every sentence
Choose random sentencesAsk the teacher if they are grammatic
alStore all sentences and grammaticali
ty judgmentsSearch for a setting of innate and lea
rned components that assigns the correct grammaticality rating to every sentence
1,000 Bit Innate and Learned Components1,000 Bit Innate and
Learned Components
0.6
0.7
0.8
0.9
1
0 5000 10000 15000 20000
Number of Example Sentences
relatedunrelatedlinguist
1,000 Bit Innate Component 1,000,000 Bit Learned Component
1,000 Bit Innate Component 1,000,000 Bit Learned Component
0.6
0.7
0.8
0.9
1
0 5000 10000 15000 20000
Number of Example Sentences
relatedunrelatedlinguist
1,000,000 Bit Innate Component 1,000 Bit Learned Component
1,000,000 Bit Innate Component 1,000 Bit Learned Component
0.6
0.7
0.8
0.9
1
0 5000 10000 15000 20000
Number of Example Sentences
relatedunrelatedlinguist
Implications of Impossible Syntax
Implications of Impossible Syntax
A linguist can write a grammar that will adequately characterize any body of data
But it will fail when tested on new data
Partial grammars are not a stepping stone to complete generative grammars
A linguist can write a grammar that will adequately characterize any body of data
But it will fail when tested on new data
Partial grammars are not a stepping stone to complete generative grammars
A Universal Law of Generative Grammar
A Universal Law of Generative Grammar
Generative grammar is impossible if:
H(learned component) + H(innate component) > H(language data)
Unless we can use information from another source (genetic, neuroscientific, psycholinguistic)
Generative grammar is impossible if:
H(learned component) + H(innate component) > H(language data)
Unless we can use information from another source (genetic, neuroscientific, psycholinguistic)
Why do Syntax?Why do Syntax?
Studying generative grammar may tell us something about the human mind
It won’t help us build natural language processing systems
Is studying rare and obscure constructions the best way to do syntax?
Studying generative grammar may tell us something about the human mind
It won’t help us build natural language processing systems
Is studying rare and obscure constructions the best way to do syntax?
ConclusionConclusion
The idea that we can characterize a language by considering enough linguistic data is a hypothesis
It’s very unlikely that it’s possible to write a complete generative grammar
The idea that we can characterize a language by considering enough linguistic data is a hypothesis
It’s very unlikely that it’s possible to write a complete generative grammar
