Frustration:

Experimental models

Animal models of frustration

Experimental model: mother-infant separation in monkeys

• Advantage: matches clinical results.

• Disadvantages: expensive, slow, restricted to few animals.

Ideal animal model

• Matches clinical results.

• Affordable, rapid results, access to many animals.

Why do we need animal models?

• To move from descriptive to experimental research.

• To identify causal factors.

• To understand the neural basis of the behaviors involved.

• As a source of ideas for treatments and interventions.

Tinklepaugh, 1928, J Comp Psychol, 8, 197-236.

Drawing by Katsuo & Chiharu Tomita

Frustration: early studies An early experiment with moneys

illustrates the general strategy.

1 - Monkey sees a piece of banana

(favorite food) placed underneath a cup.

2 - Screen is lowered, monkey keeps an

expectation of the reward active.

3-4 - Money chooses and consumes

reward.

1 - In other trials, monkey sees a piece of

lettuce (less desirable food).

2 - Retains a lettuce representation

active.

3-4 - Makes choice and consumes

reward, although it is not its favorite food.

1 - Occasional tests: monkey sees

banana under the cup.

2 - Lettuce placed under the cup.

3 - Monkey expects banana (favorite),

but gets lettuce (less desirable).

4 - Lettuce rejected if expecting banana.

Monkey displays aggressive behavior

toward the experimenter.

Frustration

• Frustration is induced in situations in which a larger or more preferred reward is expected, but a smaller or less preferred reward is obtained.

• Surprising nonreward: expected better than obtained.

•Surprising nonreward promotes two kinds of learning:

✓ Allocentric: learning about a change in the environment (cognitive).

✓ Egocentric: learning about my own reaction to that change (emotional).

Papini, 2003, Brain Behav Evol, 62, 83-95.

Aftereffects: immediate (seconds to minutes) consequences of

surprising nonreward that result from the induction of frustration.

Anticipatory effects: an expectation of the aversive internal state

previously triggered by surprising nonreward in a similar situation.

Behavioral persistence: pairings of the expectancy of frustration

with reward reduces the tendency of frustration to induce

avoidance responses.

Frustration: effects

Frustration

Stimulus Reward: 4%

Expectancy: 32%

Secondary frustration Avoidance

Approach

Negative discrepancy:

Expected more value

than received

Aftereffects

of frustration

Primary frustration

Anticipatory

effects

Frustration

Stimulus Reward: 4%

Expectancy: 32%

Secondary frustration

Approach

Negative discrepancy:

Expected more value

than received

Primary frustration

Behavioral

persistence(counterconditioning)

Aftereffects of frustration

Frustration

Stimulus Reward: 4%

Expectancy: 32%

Approach

Negative discrepancy:

Expected more value

than received

Aftereffects

of frustration

Primary frustration

• Immediate (seconds to minutes) consequences of surprising nonreward that result

from the induction of frustration.

• Aftereffects depend on the acquisition of a reward expectancy.

• Examples of behaviors that change following surprising nonreward in rats:

• The frustration effect (FE): invigoration of food-reinforced behavior.

• Changes in agonistic behavior: aggressive behavior can increase or decrease,

depending on the conditions.

• Increased general activity: sniffing, rearing, and exploratory activity.

• Increased distress vocalizations: low-frequency, aversive vocalizations.

• Odor emission: rats respond to odors left over by other rats that experienced

surprising nonreward in a given place.

Aftereffects of frustration

What do these effects have in common?

Immediate consequences of surprising nonreward.

Aftereffects of frustration: FE in runway (instrumental)

Double-runway procedure (instrumental)

•Rats learned to find food in Goal 1 and then run for food in Goal 2.

•Then, food in Goal 1 was available on some trials but not on other trials.

•Rats had no way of predicting whether food would or not be present on Goal 1 (surprising nonreward).

•Rats ran faster immediately after no food than after food on Goal 1 (red arrow longer than green arrow).

•Frustration effect (FE): faster responding after nonreward than after reward.

•This invigoration of responding is reminiscent of the person pushing the key in the vending machine

faster when the machine fails to deliver a Coke (see picture top right).

Rewarded Rewarded

Nonrewarded Rewarded

Runway 1 Runway 2Start Goal 1 Goal 2

food

Amsel & Roussel, 1952, J Exp Psychol, 43, 363-368.

0

10

20

30

40

50

0 1 2 3 4 5 6

Sessions

Res

po

nse

s p

er M

inu

te

S/Sham

S/ADX

M/Sham

M/ADX

Pre 1 5

Thomas & Papini, 2001, Physiol Behav, 72, 543-547.

Aftereffects of frustration: adrenalectomy (ADX) eliminates the extinction spike

•Glucocorticoids (GCs), including cortisol (humans)

and corticosterone (rats), are secreted by the

adrenal cortex.

•GCs provide feedback to the hypothalamus and

other brain areas that participate in emotional

stress.

•Adrenalectomy (ADX) eliminates de adrenal cortex

and thus reduces the amount of circulating GCs.

Autoshaping procedure (Pavlovian)

•Adrenalectomy (ADX) does not affect the acquisition or

extinction of lever pressing in autoshaping.

•However, ADX eliminates the invigoration of responding

that typically occurs in early extinction trials in the

autoshaping procedure (called “extinction spike”).

•Arrows show the elimination of the extinction spike in

animals trained under spaced (S) or massed (M)

conditions (related to the distribution of practice).

(a) Mustaca et al. (2000)

0

10

20

30

40

50

60

32-W 32-W W-W W-W

Perc

enta

ge

Dominant Submissive Dominant Submissive

Increased aggression (instrumental)

•Rats learned to find food in the goal box of a runway.

•Then, they received some trials without food.

•Rats were released in an arena immediately after a

reinforced vs. after a nonreinforced trial.

•Aggressive responses increase after no food relative

to the frequency of aggressive behaviors after a

food-reinforced trial.

Reduced aggression (consummatory)

•Pretest: dominant and submissive males identified.

•Training: dominants had access to 32% sucrose or

water (W) for 10 sessions, followed by a session with

access to water for all animals.

•Posttest: downshifted dominants were not aggressive,

whereas unshifted dominants remained aggressive.

Aftereffects of frustration: Changes in agonistic behavior

Start Goal

Aftereffects of frustration: Increased activity

Pellegrini & Mustaca, 2000, Learn Motiv, 31, 200-209.

Increased activity (consummatory)

•Preshift: two groups received access to either vanilla

cockies (preferred) or rabbit pellets (less desirable) in

5-min sessions. Several behaviors were recorded.

•Postshift: one group was shifted to the rabbit pellets

(downshifted), while the other continued to have

access to the rabbit pellets (unshifted). The same

behaviors were recorded.

•Results: downshift led to an increase in ambulation

and rearing, while simultaneously consumption and

approach to the feeder were reduced.

OR

Vanilla cockies Rabbit pellets

Consumption

Ambulation

Approach to

the feeder

Rearing

Preshift Postshift

Aftereffects of frustration: distress vocalizations

Increased ultrasound emissions(instrumental)

•Acquisition: 11-day old infant rats, still nursing,

learned to walk from the start to the goal of a

runway, where they found an anesthetized dam.

Infants were rewarded with the opportunity to

suck for 30 s. Simultaneously, ultrasound

emissions were recorded. Latencies and

ultrasounds decreased as infants learned to

approach the goal.

•Extinction: infants walked toward the goal, but

found that the dam was absent.

•Results: the time to reach the goal increased

during extinction, which is expected. In addition,

infants increased the frequency of distress

vocalizations.

•Similar vocalizations were observed in adults

exposed to surprising nonreward.

Anesthesized

dam

Infant

StartRun

GoalT

ime

(s

)

Ult

raso

un

ds

Start Run Goal

Session Blocks Amsel et al., 1977, Science, 197, 786-788.

Aftereffects of frustration: odor emissions

Start Goal

Food cupPaper floor Donor

“Observer”

Odor emissions (instrumental)

•Donors: these rats experience surprising

nonreward in the goal box of a runway

(partial reinforcement).

•Observers: these rats are placed in the

runway without reward, but with a piece of

paper that was used by donors.

•Donors leave an odor after experiencing surprising nonreward in the goal box.

•Most observers respond to this frustration odor by avoiding the location.

•Rats switch to a nonpreferred location if they smell the odor in a preferred site.

•However, with continued exposure, most rats develop an attraction to the odor.

•After donors were exposed to partial reinforcement for more than 500 trials, the

odor continued to effectively induce avoidance responses in naïve observers.

•To the extent these odors reflect a negative emotion induced by surprising

nonreward, this evidence suggests that tolerance to frustration does not develop

with extensive experience.

Ludvigson et al., 1979, Anim Learn Behav, 7, 251-258.

Anticipatory effects of frustration

Frustration

Stimulus Reward: 4%

Expectancy: 32%

Secondary frustration Avoidance

Approach

Negative discrepancy:

Expected more value

than received

Aftereffects

of frustration

Primary frustration

Anticipatory

effects

• Successive negative contrast (SNC) • Instrumental (iSNC)

• Consummatory (cSNC)

• Magnitude of reinforcement extinction effect (MREE)

Anticipatory effects of frustration

What do these effects have in common?

Anticipation of frustration after experience with reward failure.

Elliott, 1928, Univ Cal Pub Psychol

StartGoal

Frustration: instrumental successive negative contrast (iSNC)

iSNC•Rats learned to find food

in the goal of this complex

maze.

•One group found wet

cereal (favorite).

•The other found sunflower

seeds (less desirable).

Preshift Postshift

Wet cereal Sunflower seeds

Sunflower seeds

•Rats learned the task faster when

rewarded with wet cereal than

sunflower seeds.

•But a downshift to sunflower

seeds disrupted behavior.

•Unshifted rats always rewarded

with sunflower seeds continued to

reduce errors.

•Animals refused to go directly to

the goal and searched for the

missing food.

Group Preshift Postshift

Downshifted Wet cereal Sunflower seeds

Unshifted Sunflower seeds Sunflower seeds

Err

ors

Daily Trials

Maze learning procedure

Group Preshift Postshift

32-4 32% 4%

4-4 4% 4%

Frustration: consummatory successive negative contrast (cSNC)

Bottle with

sucrose

solution

Papini, 2006, Jap J Anim Psychol, 56, 35-54.

•Rats had access to sucrose

solutions of different

concentrations.

•One group found 32% sucrose

(favorite).

•The other found 4% sucrose

(less desirable). Consummatory procedure•Preshift: Rats consumed more 32% sucrose than 4% sucrose.

•Postshift: A downshift from 32% to 4% sucrose disrupted consumption.

•Unshifted rats always exposed to 4% sucrose continued consumption.

•Rejection of the downshifted sucrose was transient.

0

2

4

6

8

10

12

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Trials

Flu

id In

take

(m

l)

32-4

4-4

Preshift Postshift

cSNC

32% sucrose 4% sucrose

Always 4% sucrose

Sucrose!

Trial 1

Sucrose!

Trial 10

Sucrose!What is

this?!

Trial 11

Sucrose!Frustration!

Trial 12

?Sucrose!

Frustration!

CONFLICT!

Frustration: magnitude of reinforcement extinction effect (MREE)

Papini et al., 2001, Learn Motiv, 32, 434-456.

Group Acquisition Extinction

12 Lever → 12 pellets Lever → 0

1 Lever → 1 pellet Lever → 0

Acquisition Extinction

MREE

Autoshaping procedure (Pavlovian)

•Acquisition: two groups received training in autoshaping.

•Lever presentations were paired with either 12 or 1 pellet.

•Rats press the lever, although they would receive the

pellets whether or not they press the lever (Pavlovian).

•Acquisition lasted 40 sessions (figure shows the last three

sessions).

•Extinction: same lever presentations, but not followed by

food.

•MREE: faster extinction after acquisition with a large

reward (12 pellets) than after a small reward (1 pellet).

•Similarity between MREE and SNC: in both cases, the

size of the discrepancy between expected (large) and

received reward (small or zero) leads to response

suppression.

Behavioral persistence

Frustration

Stimulus Reward: 4%

Expectancy: 32%

Secondary frustration

Approach

Negative discrepancy:

Expected more value

than received

Primary frustration

Behavioral

persistence(counterconditioning)

Frustration

Reward: 4%

Approach

Behavioral persistence(counterconditioning)

Remember counterconditioning? Used by Pavlov and Watson:

Pavlov Watson

Shock Food

Secondary frustration

Leg flexion Salivation

White rat Food

Fear Positive

Avoidance

Frustration

Reward: 4%

Approach

Behavioral persistence(counterconditioning)

Remember counterconditioning? Used by Pavlov and Watson:

Pavlov Watson

Food

Secondary frustration

Salivation

Food

Positive

Shock White rat

•Partial reinforcement extinction effect

(PREE)

•Partial delay of reinforcement extinction effect

(PDREE)

•Variable magnitude of reinforcement extinction effect

(VMREE)

Frustration: behavioral persistence

What do these effects have in common?

Coping with reward uncertainty via counterconditioning.

Partial reinforcement extinction effect (PREE)

Gonzalez & Bitterman, 1969, J Comp Physiol Psychol, 67, 94-103.

0.0

0.4

0.8

1.2

1.6

2.0

0 5 10 15

Lo

g L

ate

ncy

+ 1

Days

P, L

C, L

C, S

Extinction

Partial/Large

Continuous/Large

Continuous/SmallMREE

PREE

Group Acquisition Extinction

Continuous/Small Run → 100% small Run → 0

Continuous/Large Run → 100% large Run → 0

Partial/Large Run → 50% large Run → 0

Runway procedure (instrumental)

•Acquisition (not shown in figure): rats learned

to find food in goal box.

•Food: always (100%) small amount.

•Food: always (100%) large amount.

•Food: 50% large, 50% nothing (randomly).

•Extinction (shown in figure): food withheld.

•MREE (see earlier slides): Continuous/Large

extinguished faster than Continuous/Small.

•PREE: Continuous/Large extinguished faster

than Partial/Large.

•Partial reinforcement (reward uncertainty)

increased behavioral persistence in

extinction.

Start Goal

Hippocampal lesions eliminate the PREE

Rawlins et al., 1980, Exp Brain Res, 38, 273-283.

Group Acquisition Extinction

Partial/Sham Run → 50% Reward Run → 0

Continuous/Sham Run → 100% Reward Run → 0

Partial/Hipp Run → 50% Reward Run → 0

Continuous/Hipp Run → 100% Reward Run →0

0 . 0

0 . 2

0 . 4

0 . 6

0 . 8

1. 0

1. 2

0 1 2 3 4 5 6 7

S e s s io n s

Ru

n S

peed

(1

/s) PR/SO

CR/SO

PR/Hipp

CR/Hipp

Extinction

Continuous/Sham

Partial/Sham

Partial/Hipp

Continuous/Hipp

Hipp: lesion of the hippocampus.

Sham: simulated lesion (control).

Lesions were administered before acquisition.

Runway procedure (instrumental)

•Acquisition (not shown in figure): rats learned to

find food in goal box.

•Hippocampal lesions did not affect acquisition.

•Extinction (shown in figure): food withheld.

•PREE: Continuous/Sham extinguished faster than

Partial/Sham.

•Hippocampal lesions eliminated the PREE (no

difference between Partial vs. Continuous with

hippocampal lesion.

•Hippocampus: necessary for coping with reward

uncertainty and develop behavioral persistence.

Start Goal

Partial delay of reinforcement extinction effect (PDREE)

Rashotte & Surridge, 1969, Quart J Exp Psychol, 21, 156-161.

Start Goal

Group Acquisition Extinction

Partial Run → 50% Reward Run → 0

Continuous Run → 100% Reward Run → 0

Partial delay Run → 50% Immediate Run → 0

Runway procedure (instrumental)

•Acquisition: rats learned to find food in goal box.

•Partial: food and no food on a random 50% of trials.

•Continuous: food in 100% of trials.

•Partial delay: food was immediately available on a random 50% of

trials, but it was delayed by 30 s on the other 50% of trials.

•Extinction: reward (delay) uncertainty increased persistence.Ru

nn

ing

Sp

eed

(fe

et/

seco

nd

)

4-Trial Blocks

Partial

Partial delay

Continuous

Rela

tive R

un

nin

g S

peed

5-Trial Blocks

Continuous: 9 pellets

Varied magnitude:

9 or 1 pelletsPartial:

9 or 0 pellets

Extinction

Logan et al., 1956, J Exp Psychol, 52, 65-70.

Variable magnitude of reinforcement extinction effect (VMREE)

Group Acquisition Extinction

Partial Run→50% Large, 50% nothing Run → 0

Continuous Run→100% Large Run → 0

Varied Run→50% Large, 50% small Run → 0

Runway procedure (instrumental)

•Acquisition (not shown in figure): rats learned to find food in

goal box.

•Partial: food and no food on a random 50% of trials.

•Continuous: food in 100% of trials.

•Varied magnitude: large reward on a random 50% of trials (9

pellets), but small reward on the other 50% of trials (1 pellet).

•Extinction: reward (magnitude) uncertainty increased

persistence.

Start Goal

Transfer

Frustration: transfer

Stimulus

Secondary frustration

Approach

Other stimuli

Behavioral

persistence(counterconditioning)

Una experiencia de pérdida “inmuniza” contra experiencias posteriores

Glueck et al, 2018, Learn Motiv, 63, 105-125.

0

50

100

150

200

250

300

0 5 10 15

Go

al-tr

ackin

g T

ime

(s)

Trials

32 (n=8)

4 (n=10)

0

10

20

30

40

50

0 5 10 15 20

Res

po

nse

s p

er M

inu

te

Sessions

32 (n=8)

4 (n=10)

cSNC

Autoshaping

extinction

Frustration: transfer

•Animals experience two tasks that differ in terms of the reward, the response, and the environment.

•Phase 1: cSNC, reward: sucrose solution (fluid), response: consummatory, box: only a sipper tube.

Group 32: exposed to a 32-to-4% sucrose downshift | Group 4: always received 4% sucrose.

•Phase 2: Autoshaping, reward: food pellets (solid), response: lever pressing, box: Skinner box with lever.

•All animals had the same autoshaping experience, but differed in their experience during Phase 1.

•Results of Phase 1: 32-to-4% sucrose downshift led to consummatory suppression relative to the unshifted

group.

•Problem: if recovery from the downshift involves counterconditioning, then the reactivation of secondary

frustration in a different task should lead to response persistence.

•Results of Phase 2: previously downshifted and unshifted groups did not differ during acquisition. However,

previously downshifted animals exhibited more persistence in extinction that previously unshifted animals.

Phase 1 Phase 2

Phase 1:

Acquisition 1

Phase 2:

Acquisition 2

Phase 3:

Extinction

Apparatus→Short, black

wide box

Long, white

narrow runway

Long, white

narrow runway

Motivation → Hunger Thirst Thirst

Reward → Food pellets Water None

Response and

Schedule →

Jumping CR

Jumping PRRunning CR Running EXT

Response and

Schedule →

Climbing CR

Climbing PRRunning CR Running EXT

Response and

Schedule →

Running CR

Running PRRunning CR Running EXT

•Six groups participated in this experiment (three responses

and two schedules of reinforcement in Phase 1).

•CR: continuous reinforcement. PR: partial reinforcement.

•EXT: extinction training.

•Jumping and running are compatible.

•Climbing and running are incompatible.

Frustration: transfer

Tim

e(s

)

4-Trial Blocks

Phase 3: Extinction

PR compatible:

jumping, running

PR incompatible: climbing

CR

•Phase 3 results: Despite differences in the

apparatus, motivation, and reward, training under

PR in Phase 1 affected extinction in Phase 2.

•Compatible responses: increased persistence.

• Incompatible responses: decreased persistence.

•Frustration counterconditioned in Phase 1

reactivated persistence in Phase 3.

•The internal state overrode internal and external

condition and control behavior in extinction.

Ross, 1964, J Exp Psychol, 68, 492-502.