Upload
jj
View
18
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Lecutre
Citation preview
Week lecture 4
Rescorla-Wagner Model; Neurobiology of Prediction Error
Surprisingness of the US
Robert Rescorla & Allan Wagner The model is a mathematical expression of surprise:
Learning will occur only when the subject is surprised - that is, when what happens is different from what the subject expected to happen
Blocking (Leon Kamin)
Group Phase 1 Phase 2 Test Result Blocking L-US L & T-US T no CR
Control L & T-US T CR
This experiment is important because it shows that:
1. Conditioning is not an automatic result of CS-US pairings
2. For conditioning to occur, the CS must be informative and the US surprising
V = associative strength between CS and US
Vmax = maximum associative strength
V = change in associative strength
on each conditioning trial
Measure of size of CR during CS-US conditioning trials
Reality
Theory
Quantification of surprisingness of the US
More surprise
Less surprise
V = Vmax - Vn Vn = strength of the association at the beginning of trial n
Vn = change is the strength of the association produced by trial n
Learning curves can differ in terms of: 1. Vmax
2. Rate of acquisition
Vn = (Vmax - Vn)
Vmax is determined by the magnitude of the US
and relate to the salience of the CS and the US, respectively. Their values are between 0 and 1.
Vn = (Vmax -Vn)
Rescorla-Wagner model: valuable predictions
This model precludes quantitative predictions but allows interesting qualitative predictions (increases, decreases, and more).
Vn = (Vmax -Vn)
ACQUISITION Assume = 0.3 and Vmax = 1
Trial Vn Vn = (Vmax - Vn) 1 0.00 V1 = 0.3 (1 - 0.00) = 0.30
2 0.00 + 0.30 V2 = 0.3 (1 - 0.30) = 0.21
3 0.00 + 0.30 + 0.21 V3 = 0.3 (1 - 0.51) = 0.15
4 0.00 + 0.30 + 0.21 + 0.15 V4 = 0.3 (1 - 0.66) = 0.10
Trial Vn Vn = (Vmax - Vn) 1 0.00 V1 = 0.3 (1 - 0.00) = 0.30
2 0.00 + 0.30 V2 = 0.3 (1 - 0.30) = 0.21
3 0.00 + 0.30 + 0.21 V3 = 0.3 (1 - 0.51) = 0.15
4 0.00 + 0.30 + 0.21 + 0.15 V4 = 0.3 (1 - 0.66) = 0.10
1 Ext 0.00 + 0.30 + 0.21 + 0.15 + 0.10 V5 = 0.3 (0 - 0.76) = - 0.22
Vmax = 0
CONDITIONED INHIBITORS have negative associative strength
EXTINCTION The weakening of a conditioned response when a CS is presented by itself
BLOCKING
When two CSs are used (a & b), the association or expectation at the beginning of a trial would be the sum of the strengths of each of the stimuli present
Vab = Va + Vb
Therefore, the amount of conditioning on a compound trial in which a and b occur together would be
Va = Vb = (Vmax - Vab)
In the blocking group, if the Vmax for the light (L) = 1.0, then:
VL = 1.0 at the end of Phase 1 (because of extensive L conditioning)
when the light and the tone (T) are presented in combination on trial 1 of Phase 2
VLT = VL + VT = 1.0 + 0 = 1.0
Therefore, the amount of conditioning to the T in the blocking group after 1 trial of conditioning with the LT compound is:
VT = (Vmax - VLT) = 0.3 (1.0 - 1.0) = 0
Tone ------>shock Light ------>shock
Tone + Light-------> shock
Tone ?
Assume that only few trials were given before the compound trial, and that Vmax = 1 and = 0.3
VT = 0.2 and VL = 0.2, thus VTL = 0.4
VT = VL = 0.3 (1.0 - 0.4) = 0.18
The model predicts an increase in associative strength for both T and L when presented during the compound trial
But, if there was extensive conditioning before the compound trial such that:
VT = 0.9 and VL = 0.9, thus VTL = 1.8
VT = VL = 0.3 (1.0 - 1.8) = - 0.24
Therefore, the model predicts a decrease associative strength for both T and L when presented during the compound trial
OVEREXPECTATION EFFECT
Rescorla (1970) - Extensively trained rats
Tone ---------->shock
Light ---------->shock
Experimental group (E) Tone + Light----------> shock
Control group (C) Nothing
No conditioning to the CS does not mean no conditioning at all
Contextual stimuli
Context
Trial 1 CS + Context ------> US = + associative strength to compound Trial 2 Context alone ------>US = + associative strength to context
Trial 20 CS + Context ------> US = - associative strength to compound
Trial 21 Context alone ------> US = + associative strength to context
When the US is not contingent to the CS, conditioning will be strong to contextual (background) stimuli but not to the CS
= context
Some problems with the Rescorla-Wagner model
1. Exclusive focus on the surprisingness of the US Nicholas Mackintosh John Pearce & Geoffrey Hall
The Mackintosh Model The Pearce-Hall Model
It is important to consider how the salience of the CS () changes
during conditioning
2. The conclusion that extinction destroys the original learning
2. Spontaneous Recovery: the reappearance of a CR to a CS after a period of time following the last extinction trial
Renewal: the reappearance of a CR to a CS due to return to the training environment, instead of the environment used during extinction Reinstatement: the reappearance of a CR to a CS due to a brief presentation of the US
Rapid Reacquisition: rapid return of a CR to a previously extinguished CS
Surprise and Prediction Error
US not predicted and therefore very surprising
US somehow predicted and therefore less surprising
US more predicted and therefore much less surprising
V = Prediction Error
Neurobiology of Prediction Error
DOPAMINE A neurotransmitter involved in learning, motivation and a variety of psychobiological functions
Agonist: Cocaine, Amphetamine, Methylphenidate
Antagonists: Chlorpromazine, Haloperidol (Anti-Psychotic drugs)
US = Drops of juice UR = Lick
CSs
DA neurons VTA
Wolfram Schultz
Prior to CS-US conditioning
After CS-US conditioning
During extinction
Dopaminergic neurons in the VTA encode a Prediction Error
CS
Auto-shaping
US
NAC = nucleus accumbens
Homeostatic hypothesis of learning
Optimal level
Disturbance Actual level
Receptors
Activation of learning systems in the brain
Past knowledge
New event
Current knowledge
DA in VTA Error Signal
Response system
DA neurons VTA Prediction Error Signal
Learning
Learning
Learning