37
Computation Representation Processes Learning Cognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept. of Linguistics and Institute for Advanced Computational Science Stony Brook University Tri-Co Dept of Linguistics, Haverford College May 7, 2019 1

Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

Learning with Locality Across Speech and Sign

{Jon Rawski

Jane Chandlee

}Dept. of Linguistics and Institute for Advanced Computational Science

Stony Brook UniversityTri-Co Dept of Linguistics, Haverford College

May 7, 2019

1

Page 2: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

Workshop Goal

I Understand phonological rules across speech and sign

I How do they constrain grammar hypotheses learners inferfrom data?

I How do learners balance these limits with modality properties?

Computational Perspective

I Mathematically precise representational claims

I Mathematically restricted grammars

I Efficient, provably correct learning algorithms

1

Page 3: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

Learning Depends on Hypothesis Spaces

Computably Enumerable

Context-Sensitive

MildlyContext-Sensitive

Context-FreeRegularFinite

Yoruba copying

Kobele 2006

Swiss German

Shieber 1985English nested embedding

Chomsky 1957

English consonant clusters

Clements and Keyser 1983 Kwakiutl stress

Bach 1975

Chumash sibilant harmony

Applegate 1972

Heinz & Idsardi 2013, Rawski & Heinz 2019

2

Page 4: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

Learning Depends on Hypothesis Spaces

Computably Enumerable

Context-Sensitive

MildlyContext-Sensitive

Context-FreeRegularFinite

Yoruba copying

Kobele 2006

Swiss German

Shieber 1985English nested embedding

Chomsky 1957

English consonant clusters

Clements and Keyser 1983 Kwakiutl stress

Bach 1975

Chumash sibilant harmony

Applegate 1972

Heinz & Idsardi 2013, Rawski & Heinz 2019

2

Page 5: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

Learning Depends on Hypothesis Spaces

Computably Enumerable

Context-Sensitive

MildlyContext-Sensitive

Context-FreeRegularFinite

Yoruba copying

Kobele 2006

Swiss German

Shieber 1985English nested embedding

Chomsky 1957

English consonant clusters

Clements and Keyser 1983 Kwakiutl stress

Bach 1975

Chumash sibilant harmony

Applegate 1972

Heinz & Idsardi 2013, Rawski & Heinz 2019

2

Page 6: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

Phonology is Subregular

I Regular/finite-state: Memory does not grow with input sizeI Sufficient for phonology (Johnson 1972, Kaplan & Kay 1994)

I Underlying/Surface pairs: (ba:d, ba:t),I Rewrite Rules: -son → -voice / #,I Constraint Interaction: *[-son,+voice]# >> IDENT(voi)

I New hypothesis: phonology only needs subregular power

3

Page 7: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

Input Strictly Local Functions

ISL Functions (Chandlee 2014, Chandlee & Heinz 2018)

I output u tracks contiguous input substrings x of length kI (CAD, CBD), A → B / C D, *CAD >> FAITH(A→B)I Intervocalic Voicing is 3-ISL: VTV → VDV

I About 95% of processes in P-Base (Mielke 2008)I Substitution, deletion, epenthesis, general affixation,

metathesis (local & bounded nonlocal), partial reduplication, ...

4

Page 8: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

What information is present in a string?

b

1

a

2

b

3

a

4

I Domain of sequence elements {1,2,3,4}I Labeling relations {a,b}(IPA, features, orthography, etc)

I Ordering functions (successor and predecessor)

5

Page 9: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

What information is present in a string?

b

1

a

2

b

3

a

4

s s s

s

pppp

I Domain of sequence elements {1,2,3,4}I Labeling relations {a,b}(IPA, features, orthography, etc)

I Ordering functions (successor and predecessor)

5

Page 10: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

What information is present in a string?

b

1

a

2

b

3

a

4

H

5

L

6

A A

I Domain of sequence elements {1,2,3,4,5,6}I Labeling relations {a,b,H,L} (now including tone)

I Association relations

I Ordering functions (successor and predecessor)

6

Page 11: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

Signs as Strings

I Models signs as sequential hold & movement segments withfeatures (Liddell 1984)

I Explicit claim: Only difference between sign and speech is size& content of feature system

I Rawski 2017: metathesis, compound reduction, partialreduplication are ISL in speech/sign

7

Page 12: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

ASL Final Syllable Reduplication

pics from Sandler & Lillo-Martin 2006

8

Page 13: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

ASL Final Syllable Reduplication

4-ISL function: /0 → LML / LML nInput String: LMLML

o L M L M L n

9

Page 14: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

ASL Final Syllable Reduplication

4-ISL function: /0 → LML / LML nInput String: LMLML

o L M L M L n

9

Page 15: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

ASL Final Syllable Reduplication

4-ISL function: /0 → LML / LML nInput String: LMLML

o L M L M L nL

9

Page 16: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

ASL Final Syllable Reduplication

4-ISL function: /0 → LML / LML nInput String: LMLML

o L M L M L nL

9

Page 17: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

ASL Final Syllable Reduplication

4-ISL function: /0 → LML / LML nInput String: LMLML

o L M L M L nL M

9

Page 18: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

ASL Final Syllable Reduplication

4-ISL function: /0 → LML / LML nInput String: LMLML

o L M L M L nL M

9

Page 19: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

ASL Final Syllable Reduplication

4-ISL function: /0 → LML / LML nInput String: LMLML

o L M L M L nL M L M

9

Page 20: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

ASL Final Syllable Reduplication

4-ISL function: /0 → LML / LML nInput String: LMLML

o L M L M L nL M L M

9

Page 21: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

ASL Final Syllable Reduplication

4-ISL function: /0 → LML / LML nInput String: LMLML

o L M L M L nL M L M

9

Page 22: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

ASL Final Syllable Reduplication

4-ISL function: /0 → LML / LML nInput String: LMLML

o L M L M L nL M L M L LML

9

Page 23: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

Signs as Graphs

I encodes autosegmental relations (Sandler 1989, van der Hulst1993, Brentari 1998)

picture from (Sandler & Lillo-Martin 2006)

10

Page 24: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

Compound Reduction

pictures from (Sandler & Lillo-Martin 2006) 11

Page 25: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

Compound Reduction

pictures from (Sandler & Lillo-Martin 2006)

11

Page 26: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

ISL Across Speech and Sign

Process Strings GraphsMetathesis ISL A-ISL

Partial Reduplication ISL A-ISL

Compound Reduction ISL A-ISL

Interpretation

I Strict Locality is salient across spoken and signed phonology

I Locality ranges over the representationI “Adapted systems” view

I signers exploit nonlinear structureI restrict sequential structure to preserve ISL requirements

I Locality as unified inductive learning bias

12

Page 27: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

Learning Input Strictly Local Functions

ISL Function Learning Algorithm (Chandlee et al 2014)

I Input strict locality as an inductive principle

I Generalize ISL functions from positive data.

I Successful Identification in the Limit (Gold 1967)

13

Page 28: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

State merging approachFinal devoicing:{(DND, DNT), (DNN, DNN), (DNT, DNT)...}1. Prefix Tree Representation of data2. Merge states that have the same suffix of length k−1.

! DD:DN DN

DND

D:T

DNNN:N

DNT

T:T

N:!

#:!

#:!

#:!

14

Page 29: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

State merging approachFinal devoicing:{(DND, DNT), (DNN, DNN), (DNT, DNT)...}1. Prefix Tree Representation of data2. Merge states that have the same suffix of length k−1.

! DD:DN DN

DND

D:T

DNNN:N

DNT

T:T

N:!

#:!

#:!

#:!

14

Page 30: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

State merging approachFinal devoicing:{(DND, DNT), (DNN, DNN), (DNT, DNT)...}1. Prefix Tree Representation of data2. Merge states that have the same suffix of length k−1.

λ DD:DN N

N:NDND

D:T

#:λ

DNT

T:T

N:λ

#:λ

#:λ

14

Page 31: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

State merging approach

Final devoicing:{(DND, DNT), (DNN, DNN), (DNT, DNT)...}1. Prefix Tree Representation of data2. Merge states that have the same suffix of length k−1.

λ DD:DNN

N:N

D:T

#:λ

DNT

T:TN:λ

#:λ#:λ

14

Page 32: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

State merging approachFinal devoicing:{(DND, DNT), (DNN, DNN), (DNT, DNT)...}1. Prefix Tree Representation of data2. Merge states that have the same suffix of length k−1.

λ DD:λ

N

N:N

T

T:T

D:D

N:DN

T:DT

#:T

D:λN:N

T:T

#:λ

D:λ

N:N

T:T

#:λ

14

Page 33: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

Cognitive Implications

Our proposal: Learners are amodally sensitive to

I particular locality representations (e.g. substrings/subgraphs)I ISL memory restrictions (e.g. bounded substrings).

I reflected experimentally (Finley 2011; Lai 2015, Avcu 2017)

Any cognitive mechanism with ISL complexity is sensitive to lengthk blocks of consecutive events occuring in the underlying structure.(Rogers et al 2013)

If structures occur in time, this means sensitivity, at each point, toimmediately prior sequence of k−1 events.

All learning systems necessarily structured by representational andcomputational nature of their domains (Rawski & Heinz 2019).

15

Page 34: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

Answering Poeppel’s “Mapping Problem”

Maps problem: Find brain areas correlating with cognitive tasksMapping problem: Decompose cognition into neuronal operations

Poeppel 2012

“focus on the operations and algorithms that underpin languageprocessing”

“commitment to an algorithm or computation in this domaincommits one to representations of one form or another withincreasing specificity and also provides clear constraints for whatthe neural circuitry must accomplish.”

16

Page 35: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

Where to go from here?

I Further linguistic work on computational comparisons ofphonology across speech and sign

I Learning algorithms/theorems, integrating learnability withlinguistic theory

I Representational tradeoffs (strings, trees, graphs, etc)I Phonology, morphology, syntax, may be local over the right

representations (Graf et al 2018).

I Computationally motivated experimental work on modality

17

Page 36: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

Metathesis

pictures from (Sandler & Lillo-Martin 2006) and Wilbur (2012)18

Page 37: Learning with Locality Across Speech and Sign · ComputationRepresentationProcessesLearningCognition Learning with Locality Across Speech and Sign n Jon Rawski Jane Chandlee o Dept

Computation Representation Processes Learning Cognition

Metathesis

pictures from (Sandler & Lillo-Martin 2006) and Wilbur (2012)

18