View
1
Download
0
Category
Preview:
Citation preview
2/1/12
1
Learning I
Overview
• Biological background • Types of learning • Modern versions of learning
What is learning?
• What goes with what • Profiting from experience* • Storing information in memory
– Without [necessarily] acting on it immediately
Non-learning influences on behaviors
Fixed-action patterns
• Non-learned behaviors • Organism does reflexively • Initiated by releasers • Once initiated, run from start to finish
Fixed action patterns
• Babies: – Grasping, head-turning, sucking – All aid in nursing
• Adults: yawning? • Birds:
– Greylag goose and egg-rolling – Oystercatchers and larger eggs
• “supernormal” stimulus
2/1/12
2
Critical periods
• Limited time for learning
• Extra plasticity • Found in humans,
animals • Language • Also, vision
From Johnson & Newport
From Banks & Aslin
Critical periods
• Imprinting – Halfway between built-in behaviors and
flexible learning – Baby duck: Follows moving object – Mama duck, person, wooden duck
• Very quickly learns to prefer it
Critical periods
• Birdsong learning – Some birds are ‘born with’ song – Others learn their species’ song
• E.g. white-crowned sparrow
– Often compared to (human) language acquisition
Critical periods
• Birdsong learning – White-crowned sparrow (Marler, 1970) – Must hear species’ song between 10 & 50d – Doesn’t actually sing till 150-200d – Raised in lab:
• Can learn from recording of w.-c. s. song • Can’t learn from song sparrow song (diff.
species)
Critical periods
• Birdsong learning – Marler, 1970:
• Crude ‘template’ of species-typical song • Hooked into intrinsic feedback/reinforcement
system • Like human speech, which also isn’t linked to
an extrinsic reward – I.e., it must be internally rewarding
Critical periods
• Language acquisition – Critical period for learning language[s]?
• The island experiment • Umm, illegal.
– Test case: late learners of ASL
2/1/12
3
Critical periods
• Language acquisition – Late ASL learners (Newport & Suppalla, 1989)
• ASL first language • Acquired at different ages • 10+ years experience using ASL
– Better performance the earlier it was acquired – Johnson & Newport (1990)
• Similar results with L2 learners of English
Critical periods
• Language acquisition – Newport (1990): “Less-is-more” hypothesis
• Language [sign] elements a, b, c • Meaning elements m, n, o • Many ways to combine (7 * 7 = 49) • Kids have worse working memory so they can
only consider small # of combinations – Much more likely to be right
• Cochran, McDonald, & Parault (1999) – Concurrent task = better generalization
Other constrained learning
• Some things very difficult to condition to some stimuli (Bolles 1970) – Run away from shock, press bar for food – Sweetness & nausea, click & pain – Vice versa
• Species-specific constraints – Pigeons learn visual cues to illness
Behaviorism & learning
• Very quantitative • In the end, couldn’t fully account for all
real-world learning • Still useful in some contexts today
– Drug treatment – Chemotherapy & food aversions – Learning rules in many computational
models
Behaviorism & learning
• Habituation • Classical conditioning • Operant conditioning
– Also “trial and error learning” or “instrumental learning”
Habituation
• Decline in organism’s reaction as stimulus loses novelty
• Don’t react to stable, consequence-free event
• May also contribute to ease of learning the right associations – Unusualness heuristic
2/1/12
4
Classical conditioning
• Pavlov • Terms:
– Unconditioned stimulus – Unconditioned response – Conditioned stimulus – Conditioned response
Classical conditioning
Food
Drool
Bell
Drool Drool Drool Drool Drool
US
UR
CS
CR
What’s ‘set in stone’ What’s learned
Classical conditioning
Toxin
Nausea
Scapegoat food
Nausea Nausea Nausea Nausea Nausea
US
UR
CS
CR
What’s ‘set in stone’ What’s learned
Classical conditioning
Reading important
Heightened pulse
Bold font in email inbox
Heightened pulse
Heightened pulse
Heightened pulse
Heightened pulse
Heightened pulse
US
UR
CS
CR
What’s ‘set in stone’ What’s learned
Classical conditioning
• Extinction – CR can go away if US/CS relationship broken
• Blocking – Kamin, 1969
• Two CS’s • Simultaneous: both elicited CR • One conditioned first: only first one elicited CR
Classical conditioning
• Predictiveness of the CS is crucial for C.C. to occur (not simple cooccurrence) – Rescorla (1967)
Group Prob. US follows CS
Prob. US by itself
Learning?
1 .80 .80 N 2 .80 .40 Y 3 .40 .40 N 4 .40 0 Y
2/1/12
5
Classical conditioning
• Stimulus substitution? – CS takes place of US – Too simple
• Adaptation – Sometimes the CR is opposite of UR – Compensatory response model
• Dinitrophenol: + O2, + temp • Conditioned response: -O2, -temp
Classical conditioning
• Compensatory response model – Explanation for drug tolerance? – Need increasing doses to get effect
• CR in opposite direction counteracts drug
– Heroin: Gutierrez-Cebollada et al. (1994) • Addicts admitted to hospital for OD
– 50% had injected normal dose in unusual environment • Addicts admitted for unrelated stuff (controls)
– All in familiar environment • No CS -> lower tolerance
Classical conditioning
Altoid
Good taste
Computer noise
Bad taste
US
UR
CS
CR
What’s ‘set in stone’ What’s learned
Overview
• Biological background • Types of learning • Modern versions of learning
Operant conditioning
• Other names – Instrumental learning – Trial-and-error learning
• Behavior that acts (“operates”) on the environment
• Voluntary response is encouraged by rewarding the response
Operant conditioning
Behavior Consequence (Reinforce/Punish)
Outcome Type of reinforcement
X Appetitive (“good”) stimulus occurs
+ behavior Positive reinforcement
X Appetitive stimulus ends
- behavior Negative punishment
X Aversive (“bad”) stimulus occurs
- behavior Positive punishment
X Aversive stimulus ends
+ behavior Negative reinforcement
2/1/12
6
Operant conditioning
• Reinforcers – Primary (food, water, mating) – Secondary
• Strong association with primary • E.g. money
Operant conditioning
• In the lab – “Skinner box”
• Behavior: keypeck, bar press – Pigeon/rat will do sometimes randomly
• Reinforcement: food • (Or punishment, e.g. footshock or nausea) • Directly manipulate frequency of behavior by
changing contingencies of behavior
Operant conditioning
Beg
ging
beh
avio
r Likelihood of receiving ice cream for begging
high
medium
low
0 # of trials
A lot
Operant conditioning Ju
mp
on fu
rnitu
re
Likelihood of being squirted with water
low
medium
high 0
# of trials
A lot
Operant conditioning
Ala
rm c
lock
sla
ps
Likelihood of alarm ceasing
high
medium
low 0 # of trials
A lot
Operant conditioning
Rud
e co
mm
ents
Likelihood of party invitations
high
medium
low 0
# of trials
A lot
2/1/12
7
Paired associates
• Analog to instrumental learning • Pair words, flashcard-style
(“car”-”coche”) • Anticipate word 2 till it’s readily recalled
– Bidirectional (word 2 --> word 1)
Paired associates
• Each time a pair is learned, strengthened associations
• Learn nonwords--”new” associations – But depends greatly on existing knowledge
Conditioning wrap-up
• Weaknesses of ‘conditioning’ approach – Ignore biological influences on behavior – Ignores role of top-down info (knowledge)
• Nonethless, still useful in many practical contexts, including…
Infant methods • Definitely cognitive stuff going
on. • But how to investigate in non-
verbal organisms? • By looking at behavior.
Also used with
xkcd.com
Habituation
• Bore kids to death with something (e.g. sound, picture).
• Then present something new. • Do they dishabituate?
• If so, suggests they can tell the difference. • Do brains dishabituate?
– fMRI adaptation – Petkov et al., 2008:
• Present vocalizations to macaque in fMRI • Change to vocalizations from another animal • BOLD signal increases!
Learning
• Figuring out what things go together
• (And also what things don’t go together)
2/1/12
8
• Lookattheprettybabyeveryone • Whereisthebabynowprettygirl • Thisroastedbabyisprettygood
• Lookattheprettybabyeveryone • Whereisthebabynowprettygirl • Thisroastedbabyisprettygood
• ty->ba: unlikely (.33 transition probability)
• ba->by: likely (1.0 transition probability) • by->ev: unlikely (.33)
• Lookattheprettybabyeveryone • Whereisthebabynowprettygirl • Thisroastedbabyisprettygood
• ty->ba: unlikely (.33 transition probability)
• ba->by: likely (1.0 transition probability) • by->ev: unlikely (.33)
Other cues to word boundaries
• Stress – English: most words are stress-initial – ThePREttyBAbyWANTSaBOTtle
Other cues to word boundaries
• Jusczyk, Houston & Newsome – 7.5. Months:
• Familiarize infants with stress-initial words – KINGdom
• Play passages with KINGdom vs. HAMlet • Infants listen longer to kingdom >hamlet
passages • Recognize words
Other cues to word boundaries
• Jusczyk, Houston & Newsome – 7.5. Months:
• Stress-initial OK • Familiarize with guiTAR • Test with guiTAR, surPRISE • No difference in looking times
2/1/12
9
Other cues to word boundaries
• Jusczyk, Houston & Newsome – 7.5. Months:
• Stress-initial OK • Stress not initial XX • Familiarize with tar, prize • Play guiTAR/surPRISE--get it
Other cues to word boundaries
• Jusczyk, Houston & Newsome – 7.5. months:
• Stress-initial (kingdom) OK • Stress not initial (guitar) XX • Stressed syllable (tar) OK
– 10.5 months: • Stress not initial (guitar) OK
But…
• Not all languages have this kind of stress pattern.
• (Similar to the segmentation problem in object recognition--how do you know what the properties of objects/words are if you don’t know what the objects/words are themselves???)
Statistical learning
• Saffran, Aslin & Newport (1996) – 8-month-old infants
golabubidakutupiropadotitupirobidakupadotigolabupadotibidakugolabutupiro golabupadotitupirobidakupadotibidakutupirogolabupadotibidakugolabutupiro …
Statistical learning
• Saffran, Aslin & Newport (1996) – 8-month-old infants
golabubidakutupiropadotitupirobidakupadotigolabupadotibidakugolabutupiro golabupadotitupirobidakupadotibidakutupirogolabupadotibidakugolabutupiro …
Statistical learning
• Saffran, Aslin & Newport (1996) – 8-month-old infants
golabubidakutupiropadotitupirobidakupadotigolabupadotibidakugolabutupiro golabupadotitupirobidakupadotibidakutupirogolabupadotibidakugolabutupiro …
2/1/12
10
Statistical learning
• Saffran, Aslin & Newport (1996) – 8-month-old infants
golabubidakutupiropadotitupirobidakupadotigolabupadotibidakugolabutupiro golabupadotitupirobidakupadotibidakutupirogolabupadotibidakugolabutupiro …
Statistical learning
bi da ku
pi ro bi
tu pa go
… ti ro bu
tu da
.33 .33
.33
1.0 1.0
1.0 1.0 1.0
Statistical learning
• Play for 2 minutes • Present kids with bidaku or piro.bi type
words • What do they listen to longer?
bidaku < piro.bi – (novelty preference)
• At 8 months infants can segment words based on statistical cues
But what about stress?
• BRILligSLIthyTOVES.
But what about stress?
• BRILligSLIthyTOVES. • Johnson & Jusczyk (2001)
– 8-month-olds – Stress vs. statistics: Stress wins
But what about stress?
• BRILligSLIthyTOVES. • Johnson & Jusczyk (2001)
– 8-month-olds – Stress vs. statistics: Stress wins
• Thiessen & Saffran (2003) – Statistics wins @ 7 mos – Stress wins @ 9 mos
2/1/12
11
But what about stress?
• BRILligSLIthyTOVES. • Johnson & Jusczyk (2001)
– 8-month-olds – Stress vs. statistics: Stress wins
• Thiessen & Saffran (2003) – Statistics wins @ 7 mos – Stress wins @ 9 mos
• Segmentation precedes stress.
Statistical learning
• What’s getting learned? – Frequency
• Thedog > hassock • But “thedog” isn’t a word, and hassock is (srsly)
– Conditional probability • Given X, Y is __ likely to happen • Predictiveness
• Aslin, Saffran & Newport (1996)
Statistical learning
• What’s getting learned? – Aslin, Saffran & Newport (1996)
• golabu/padoti 2x as often as tupiro/bidaku • So ti.gola is as frequent as bidaku (45x each) • But
bi->da->ku
ti->go->la
1.0 1.0
.33 1.0
Statistical learning
• What’s getting learned? – Aslin, Saffran & Newport (1996)
• golabu/padoti 2x as often as tupiro/bidaku • So tiba.go is as frequent as bidaku (45x each) • 3 min of listening; then test
bi->da->ku
ti->go->la
1.0 1.0
.33 1.0
Frequency:
=
Cond. prob.:
<
Statistical learning
• Infants are indeed learning conditional probabilities, not just frequency.
Language specificity?
• Works for language • Works for tone sequences (C-C#-G…) • Works for visual stuff • Tamarin monkeys do it (Hauser et al.)
– (or not?)
• Parallel results: language and music • Slightly differing: sound and sight
2/1/12
12
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
1
Always vs impossible
Always vs possible
Possible vs impossible
Prop
ortio
n co
rrec
t
***
*** ***
Statistical learning: your data pre.tty
vs pre.ba
pre.tty vs
ty.ba
ty.ba vs
pre.ba
(some word analogs of the visual stimuli [prettybaby])
Nonadjacent dependencies
• Phenomenon in language • Not only…but also
– Either…or – Who…with? – Got _____ed
• Dependent on linguistic structure, or learnable from input?
Nonadjacent dependencies
• Nonadjacent dependency learning:
Nonadjacent dependencies
Nonadjacent dependencies Nonadjacent dependencies
Syllables: No (Newport & Aslin, 2004)
2/1/12
13
Nonadjacent dependencies
Consonant->consonant Vowel->vowel
(Newport & Aslin, 2004)
0
500
1000
1500
2000
2500
3000
0 500 1000 1500 2000 2500
Time (ms)
Fre
quency
(Hz)
A B C
⇓ ⇓ ⇓
0
500
1000
1500
2000
2500
3000
0 500 1000 1500 2000 2500
Time (ms)
Fre
quency
(Hz)
A B C
2/1/12
14
0.5
0.5
1.0
1.0
1.0
1.0
0.5 0.5 0.5 0.5 0.5 0.5
1.0 1.0 0.5...
1.0 1.0 0.5...
0.5 0.5 0.5 0.5 0.5 0.5 0.5...
chance
2/1/12
15
Is high v. low the only grouping cue?
Redo different-pitch condition, but with complex tones of two different timbres
– Perceived fundamentals of each tone set in same frequency range – Actual harmonics don’t overlap
0
500
1000
1500
2000
2500
3000
0 500 1000 1500 2000 2500
Time (ms)
Fre
quency
(Hz)
A B C
All-or-none auditory effect? – Could be a low-level perceptual phenomenon – Not related to language-learning
– Speech doesn’t “fly apart” like that
New timbre difference: flute vs. violin – Discernibly different but harmonics overlap
0 500 1000 1500 2000
10000
5000
0
Freq
uenc
y (H
z)
Time (ms)
Learned both structures!
2/1/12
16
Vision vs. audition
• Temporal sequences: – Audition better
• Simultaneous cooccurrence: – Vision better – But some auditory cooccurrences learned
Recommended