Animal models of frustration
Experimental model: mother-infant separation in monkeys
• Advantage: matches clinical results.
• Disadvantages: expensive, slow, restricted to few animals.
Ideal animal model
• Matches clinical results.
• Affordable, rapid results, access to many animals.
Why do we need animal models?
• To move from descriptive to experimental research.
• To identify causal factors.
• To understand the neural basis of the behaviors involved.
• As a source of ideas for treatments and interventions.
Tinklepaugh, 1928, J Comp Psychol, 8, 197-236.
Drawing by Katsuo & Chiharu Tomita
Frustration: early studies An early experiment with moneys
illustrates the general strategy.
1 - Monkey sees a piece of banana
(favorite food) placed underneath a cup.
2 - Screen is lowered, monkey keeps an
expectation of the reward active.
3-4 - Money chooses and consumes
reward.
1 - In other trials, monkey sees a piece of
lettuce (less desirable food).
2 - Retains a lettuce representation
active.
3-4 - Makes choice and consumes
reward, although it is not its favorite food.
1 - Occasional tests: monkey sees
banana under the cup.
2 - Lettuce placed under the cup.
3 - Monkey expects banana (favorite),
but gets lettuce (less desirable).
4 - Lettuce rejected if expecting banana.
Monkey displays aggressive behavior
toward the experimenter.
Frustration
• Frustration is induced in situations in which a larger or more preferred reward is expected, but a smaller or less preferred reward is obtained.
• Surprising nonreward: expected better than obtained.
•Surprising nonreward promotes two kinds of learning:
✓ Allocentric: learning about a change in the environment (cognitive).
✓ Egocentric: learning about my own reaction to that change (emotional).
Papini, 2003, Brain Behav Evol, 62, 83-95.
Aftereffects: immediate (seconds to minutes) consequences of
surprising nonreward that result from the induction of frustration.
Anticipatory effects: an expectation of the aversive internal state
previously triggered by surprising nonreward in a similar situation.
Behavioral persistence: pairings of the expectancy of frustration
with reward reduces the tendency of frustration to induce
avoidance responses.
Frustration: effects
Frustration
Stimulus Reward: 4%
Expectancy: 32%
Secondary frustration Avoidance
Approach
Negative discrepancy:
Expected more value
than received
Aftereffects
of frustration
Primary frustration
Anticipatory
effects
Frustration
Stimulus Reward: 4%
Expectancy: 32%
Secondary frustration
Approach
Negative discrepancy:
Expected more value
than received
Primary frustration
Behavioral
persistence(counterconditioning)
Frustration
Stimulus Reward: 4%
Expectancy: 32%
Approach
Negative discrepancy:
Expected more value
than received
Aftereffects
of frustration
Primary frustration
• Immediate (seconds to minutes) consequences of surprising nonreward that result
from the induction of frustration.
• Aftereffects depend on the acquisition of a reward expectancy.
• Examples of behaviors that change following surprising nonreward in rats:
• The frustration effect (FE): invigoration of food-reinforced behavior.
• Changes in agonistic behavior: aggressive behavior can increase or decrease,
depending on the conditions.
• Increased general activity: sniffing, rearing, and exploratory activity.
• Increased distress vocalizations: low-frequency, aversive vocalizations.
• Odor emission: rats respond to odors left over by other rats that experienced
surprising nonreward in a given place.
Aftereffects of frustration
What do these effects have in common?
Immediate consequences of surprising nonreward.
Aftereffects of frustration: FE in runway (instrumental)
Double-runway procedure (instrumental)
•Rats learned to find food in Goal 1 and then run for food in Goal 2.
•Then, food in Goal 1 was available on some trials but not on other trials.
•Rats had no way of predicting whether food would or not be present on Goal 1 (surprising nonreward).
•Rats ran faster immediately after no food than after food on Goal 1 (red arrow longer than green arrow).
•Frustration effect (FE): faster responding after nonreward than after reward.
•This invigoration of responding is reminiscent of the person pushing the key in the vending machine
faster when the machine fails to deliver a Coke (see picture top right).
Rewarded Rewarded
Nonrewarded Rewarded
Runway 1 Runway 2Start Goal 1 Goal 2
food
Amsel & Roussel, 1952, J Exp Psychol, 43, 363-368.
0
10
20
30
40
50
0 1 2 3 4 5 6
Sessions
Res
po
nse
s p
er M
inu
te
S/Sham
S/ADX
M/Sham
M/ADX
Pre 1 5
Thomas & Papini, 2001, Physiol Behav, 72, 543-547.
Aftereffects of frustration: adrenalectomy (ADX) eliminates the extinction spike
•Glucocorticoids (GCs), including cortisol (humans)
and corticosterone (rats), are secreted by the
adrenal cortex.
•GCs provide feedback to the hypothalamus and
other brain areas that participate in emotional
stress.
•Adrenalectomy (ADX) eliminates de adrenal cortex
and thus reduces the amount of circulating GCs.
Autoshaping procedure (Pavlovian)
•Adrenalectomy (ADX) does not affect the acquisition or
extinction of lever pressing in autoshaping.
•However, ADX eliminates the invigoration of responding
that typically occurs in early extinction trials in the
autoshaping procedure (called “extinction spike”).
•Arrows show the elimination of the extinction spike in
animals trained under spaced (S) or massed (M)
conditions (related to the distribution of practice).
(a) Mustaca et al. (2000)
0
10
20
30
40
50
60
32-W 32-W W-W W-W
Perc
enta
ge
Dominant Submissive Dominant Submissive
Increased aggression (instrumental)
•Rats learned to find food in the goal box of a runway.
•Then, they received some trials without food.
•Rats were released in an arena immediately after a
reinforced vs. after a nonreinforced trial.
•Aggressive responses increase after no food relative
to the frequency of aggressive behaviors after a
food-reinforced trial.
Reduced aggression (consummatory)
•Pretest: dominant and submissive males identified.
•Training: dominants had access to 32% sucrose or
water (W) for 10 sessions, followed by a session with
access to water for all animals.
•Posttest: downshifted dominants were not aggressive,
whereas unshifted dominants remained aggressive.
Aftereffects of frustration: Changes in agonistic behavior
Start Goal
Aftereffects of frustration: Increased activity
Pellegrini & Mustaca, 2000, Learn Motiv, 31, 200-209.
Increased activity (consummatory)
•Preshift: two groups received access to either vanilla
cockies (preferred) or rabbit pellets (less desirable) in
5-min sessions. Several behaviors were recorded.
•Postshift: one group was shifted to the rabbit pellets
(downshifted), while the other continued to have
access to the rabbit pellets (unshifted). The same
behaviors were recorded.
•Results: downshift led to an increase in ambulation
and rearing, while simultaneously consumption and
approach to the feeder were reduced.
OR
Vanilla cockies Rabbit pellets
Consumption
Ambulation
Approach to
the feeder
Rearing
Preshift Postshift
Aftereffects of frustration: distress vocalizations
Increased ultrasound emissions(instrumental)
•Acquisition: 11-day old infant rats, still nursing,
learned to walk from the start to the goal of a
runway, where they found an anesthetized dam.
Infants were rewarded with the opportunity to
suck for 30 s. Simultaneously, ultrasound
emissions were recorded. Latencies and
ultrasounds decreased as infants learned to
approach the goal.
•Extinction: infants walked toward the goal, but
found that the dam was absent.
•Results: the time to reach the goal increased
during extinction, which is expected. In addition,
infants increased the frequency of distress
vocalizations.
•Similar vocalizations were observed in adults
exposed to surprising nonreward.
Anesthesized
dam
Infant
StartRun
GoalT
ime
(s
)
Ult
raso
un
ds
Start Run Goal
Session Blocks Amsel et al., 1977, Science, 197, 786-788.
Aftereffects of frustration: odor emissions
Start Goal
Food cupPaper floor Donor
“Observer”
Odor emissions (instrumental)
•Donors: these rats experience surprising
nonreward in the goal box of a runway
(partial reinforcement).
•Observers: these rats are placed in the
runway without reward, but with a piece of
paper that was used by donors.
•Donors leave an odor after experiencing surprising nonreward in the goal box.
•Most observers respond to this frustration odor by avoiding the location.
•Rats switch to a nonpreferred location if they smell the odor in a preferred site.
•However, with continued exposure, most rats develop an attraction to the odor.
•After donors were exposed to partial reinforcement for more than 500 trials, the
odor continued to effectively induce avoidance responses in naïve observers.
•To the extent these odors reflect a negative emotion induced by surprising
nonreward, this evidence suggests that tolerance to frustration does not develop
with extensive experience.
Ludvigson et al., 1979, Anim Learn Behav, 7, 251-258.
Frustration
Stimulus Reward: 4%
Expectancy: 32%
Secondary frustration Avoidance
Approach
Negative discrepancy:
Expected more value
than received
Aftereffects
of frustration
Primary frustration
Anticipatory
effects
• Successive negative contrast (SNC) • Instrumental (iSNC)
• Consummatory (cSNC)
• Magnitude of reinforcement extinction effect (MREE)
Anticipatory effects of frustration
What do these effects have in common?
Anticipation of frustration after experience with reward failure.
Elliott, 1928, Univ Cal Pub Psychol
StartGoal
Frustration: instrumental successive negative contrast (iSNC)
iSNC•Rats learned to find food
in the goal of this complex
maze.
•One group found wet
cereal (favorite).
•The other found sunflower
seeds (less desirable).
Preshift Postshift
Wet cereal Sunflower seeds
Sunflower seeds
•Rats learned the task faster when
rewarded with wet cereal than
sunflower seeds.
•But a downshift to sunflower
seeds disrupted behavior.
•Unshifted rats always rewarded
with sunflower seeds continued to
reduce errors.
•Animals refused to go directly to
the goal and searched for the
missing food.
Group Preshift Postshift
Downshifted Wet cereal Sunflower seeds
Unshifted Sunflower seeds Sunflower seeds
Err
ors
Daily Trials
Maze learning procedure
Group Preshift Postshift
32-4 32% 4%
4-4 4% 4%
Frustration: consummatory successive negative contrast (cSNC)
Bottle with
sucrose
solution
Papini, 2006, Jap J Anim Psychol, 56, 35-54.
•Rats had access to sucrose
solutions of different
concentrations.
•One group found 32% sucrose
(favorite).
•The other found 4% sucrose
(less desirable). Consummatory procedure•Preshift: Rats consumed more 32% sucrose than 4% sucrose.
•Postshift: A downshift from 32% to 4% sucrose disrupted consumption.
•Unshifted rats always exposed to 4% sucrose continued consumption.
•Rejection of the downshifted sucrose was transient.
0
2
4
6
8
10
12
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Trials
Flu
id In
take
(m
l)
32-4
4-4
Preshift Postshift
cSNC
32% sucrose 4% sucrose
Always 4% sucrose
Frustration: magnitude of reinforcement extinction effect (MREE)
Papini et al., 2001, Learn Motiv, 32, 434-456.
Group Acquisition Extinction
12 Lever → 12 pellets Lever → 0
1 Lever → 1 pellet Lever → 0
Acquisition Extinction
MREE
Autoshaping procedure (Pavlovian)
•Acquisition: two groups received training in autoshaping.
•Lever presentations were paired with either 12 or 1 pellet.
•Rats press the lever, although they would receive the
pellets whether or not they press the lever (Pavlovian).
•Acquisition lasted 40 sessions (figure shows the last three
sessions).
•Extinction: same lever presentations, but not followed by
food.
•MREE: faster extinction after acquisition with a large
reward (12 pellets) than after a small reward (1 pellet).
•Similarity between MREE and SNC: in both cases, the
size of the discrepancy between expected (large) and
received reward (small or zero) leads to response
suppression.
Frustration
Stimulus Reward: 4%
Expectancy: 32%
Secondary frustration
Approach
Negative discrepancy:
Expected more value
than received
Primary frustration
Behavioral
persistence(counterconditioning)
Frustration
Reward: 4%
Approach
Behavioral persistence(counterconditioning)
Remember counterconditioning? Used by Pavlov and Watson:
Pavlov Watson
Shock Food
Secondary frustration
Leg flexion Salivation
White rat Food
Fear Positive
Avoidance
Frustration
Reward: 4%
Approach
Behavioral persistence(counterconditioning)
Remember counterconditioning? Used by Pavlov and Watson:
Pavlov Watson
Food
Secondary frustration
Salivation
Food
Positive
Shock White rat
•Partial reinforcement extinction effect
(PREE)
•Partial delay of reinforcement extinction effect
(PDREE)
•Variable magnitude of reinforcement extinction effect
(VMREE)
Frustration: behavioral persistence
What do these effects have in common?
Coping with reward uncertainty via counterconditioning.
Partial reinforcement extinction effect (PREE)
Gonzalez & Bitterman, 1969, J Comp Physiol Psychol, 67, 94-103.
0.0
0.4
0.8
1.2
1.6
2.0
0 5 10 15
Lo
g L
ate
ncy
+ 1
Days
P, L
C, L
C, S
Extinction
Partial/Large
Continuous/Large
Continuous/SmallMREE
PREE
Group Acquisition Extinction
Continuous/Small Run → 100% small Run → 0
Continuous/Large Run → 100% large Run → 0
Partial/Large Run → 50% large Run → 0
Runway procedure (instrumental)
•Acquisition (not shown in figure): rats learned
to find food in goal box.
•Food: always (100%) small amount.
•Food: always (100%) large amount.
•Food: 50% large, 50% nothing (randomly).
•Extinction (shown in figure): food withheld.
•MREE (see earlier slides): Continuous/Large
extinguished faster than Continuous/Small.
•PREE: Continuous/Large extinguished faster
than Partial/Large.
•Partial reinforcement (reward uncertainty)
increased behavioral persistence in
extinction.
Start Goal
Hippocampal lesions eliminate the PREE
Rawlins et al., 1980, Exp Brain Res, 38, 273-283.
Group Acquisition Extinction
Partial/Sham Run → 50% Reward Run → 0
Continuous/Sham Run → 100% Reward Run → 0
Partial/Hipp Run → 50% Reward Run → 0
Continuous/Hipp Run → 100% Reward Run →0
0 . 0
0 . 2
0 . 4
0 . 6
0 . 8
1. 0
1. 2
0 1 2 3 4 5 6 7
S e s s io n s
Ru
n S
peed
(1
/s) PR/SO
CR/SO
PR/Hipp
CR/Hipp
Extinction
Continuous/Sham
Partial/Sham
Partial/Hipp
Continuous/Hipp
Hipp: lesion of the hippocampus.
Sham: simulated lesion (control).
Lesions were administered before acquisition.
Runway procedure (instrumental)
•Acquisition (not shown in figure): rats learned to
find food in goal box.
•Hippocampal lesions did not affect acquisition.
•Extinction (shown in figure): food withheld.
•PREE: Continuous/Sham extinguished faster than
Partial/Sham.
•Hippocampal lesions eliminated the PREE (no
difference between Partial vs. Continuous with
hippocampal lesion.
•Hippocampus: necessary for coping with reward
uncertainty and develop behavioral persistence.
Start Goal
Partial delay of reinforcement extinction effect (PDREE)
Rashotte & Surridge, 1969, Quart J Exp Psychol, 21, 156-161.
Start Goal
Group Acquisition Extinction
Partial Run → 50% Reward Run → 0
Continuous Run → 100% Reward Run → 0
Partial delay Run → 50% Immediate Run → 0
Runway procedure (instrumental)
•Acquisition: rats learned to find food in goal box.
•Partial: food and no food on a random 50% of trials.
•Continuous: food in 100% of trials.
•Partial delay: food was immediately available on a random 50% of
trials, but it was delayed by 30 s on the other 50% of trials.
•Extinction: reward (delay) uncertainty increased persistence.Ru
nn
ing
Sp
eed
(fe
et/
seco
nd
)
4-Trial Blocks
Partial
Partial delay
Continuous
Rela
tive R
un
nin
g S
peed
5-Trial Blocks
Continuous: 9 pellets
Varied magnitude:
9 or 1 pelletsPartial:
9 or 0 pellets
Extinction
Logan et al., 1956, J Exp Psychol, 52, 65-70.
Variable magnitude of reinforcement extinction effect (VMREE)
Group Acquisition Extinction
Partial Run→50% Large, 50% nothing Run → 0
Continuous Run→100% Large Run → 0
Varied Run→50% Large, 50% small Run → 0
Runway procedure (instrumental)
•Acquisition (not shown in figure): rats learned to find food in
goal box.
•Partial: food and no food on a random 50% of trials.
•Continuous: food in 100% of trials.
•Varied magnitude: large reward on a random 50% of trials (9
pellets), but small reward on the other 50% of trials (1 pellet).
•Extinction: reward (magnitude) uncertainty increased
persistence.
Start Goal
Frustration: transfer
Stimulus
Secondary frustration
Approach
Other stimuli
Behavioral
persistence(counterconditioning)
Una experiencia de pérdida “inmuniza” contra experiencias posteriores
Glueck et al, 2018, Learn Motiv, 63, 105-125.
0
50
100
150
200
250
300
0 5 10 15
Go
al-tr
ackin
g T
ime
(s)
Trials
32 (n=8)
4 (n=10)
0
10
20
30
40
50
0 5 10 15 20
Res
po
nse
s p
er M
inu
te
Sessions
32 (n=8)
4 (n=10)
cSNC
Autoshaping
extinction
Frustration: transfer
•Animals experience two tasks that differ in terms of the reward, the response, and the environment.
•Phase 1: cSNC, reward: sucrose solution (fluid), response: consummatory, box: only a sipper tube.
Group 32: exposed to a 32-to-4% sucrose downshift | Group 4: always received 4% sucrose.
•Phase 2: Autoshaping, reward: food pellets (solid), response: lever pressing, box: Skinner box with lever.
•All animals had the same autoshaping experience, but differed in their experience during Phase 1.
•Results of Phase 1: 32-to-4% sucrose downshift led to consummatory suppression relative to the unshifted
group.
•Problem: if recovery from the downshift involves counterconditioning, then the reactivation of secondary
frustration in a different task should lead to response persistence.
•Results of Phase 2: previously downshifted and unshifted groups did not differ during acquisition. However,
previously downshifted animals exhibited more persistence in extinction that previously unshifted animals.
Phase 1 Phase 2
Phase 1:
Acquisition 1
Phase 2:
Acquisition 2
Phase 3:
Extinction
Apparatus→Short, black
wide box
Long, white
narrow runway
Long, white
narrow runway
Motivation → Hunger Thirst Thirst
Reward → Food pellets Water None
Response and
Schedule →
Jumping CR
Jumping PRRunning CR Running EXT
Response and
Schedule →
Climbing CR
Climbing PRRunning CR Running EXT
Response and
Schedule →
Running CR
Running PRRunning CR Running EXT
•Six groups participated in this experiment (three responses
and two schedules of reinforcement in Phase 1).
•CR: continuous reinforcement. PR: partial reinforcement.
•EXT: extinction training.
•Jumping and running are compatible.
•Climbing and running are incompatible.
Frustration: transfer
Tim
e(s
)
4-Trial Blocks
Phase 3: Extinction
PR compatible:
jumping, running
PR incompatible: climbing
CR
•Phase 3 results: Despite differences in the
apparatus, motivation, and reward, training under
PR in Phase 1 affected extinction in Phase 2.
•Compatible responses: increased persistence.
• Incompatible responses: decreased persistence.
•Frustration counterconditioned in Phase 1
reactivated persistence in Phase 3.
•The internal state overrode internal and external
condition and control behavior in extinction.
Ross, 1964, J Exp Psychol, 68, 492-502.