Schedules of Reinforcement

CH 17,18,19

Divers of Nassau

• Diving for coins

• Success does not follow every attempt

• Success means reinforcement

Intermittent Reinforcement

• A reinforcer follows the response only once in a while.

Continuous Reinforcement (CRF)

• A reinforcer follows every response.

• Best used for shaping or maintaining difficult behavior.

Shaping with Reinforcement

Before:

Andrew has no gum

Behavior Initial: NA

Intermed: Moves lips only

Terminal: Says words unclearly

Behavior Initial: Andrew moves lips

Intermed: Andrew makes croaking sound

Terminal: Says words clearly

After:

Andrew receives no gum

After:

Andrew receives gum

Reinforcement

Extinction

Schedule of Reinforcement

• The way reinforcement occurs because of the number of responses, time between responses, and stimulus conditions.

Schedules of Reinforcement

• Ratio– Fixed Ratio (FR)– Variable Ratio (VR)

• Interval– Fixed Interval (FI)– Variable Interval (VI)

Fixed Ratio (FR) Schedule of Reinforcement

• A reinforcer follows a fixed number of responses.

FR Responding

• After a response is reinforced, no responding occurs for a period of time, then responding occurs at a high, steady rate until the next reinforcer is delivered.

Pauses

Reinforcers Reinforcement always immediately follows the 5th response!!!!

This is an FR5 schedule of reinforcement.

Postreinforcement Pause (PRP)

• The name for the pause after the consumption of the reinforcer and before the next ratio of responses begins.

FR & PRP

• Length of the pause is proportional to the size of the ratio.

• Small ratio – pausing is short

• Large ratio – pausing is long

General Rule for establishing intermittently reinforced

behavior:First use CRF and gradually increase the intermittency of reinforcement as responding

stabilizes at a high rate.

FR120

• What are the response requirements?

• When will reinforcement be delivered?

• What will the pattern of responding look like?

Cumulative Graph for FR

0

20

40

60

80

100

120

140

Time

Cum

ulat

ive

Num

ber

of R

espo

nses

Reinforcers

Post reinforcement pauses

Variable Ratio (VR) Schedule

• A reinforcer follows after a variable number of responses

VR Responding

• VR schedules produce a high, constant rate of responding, with almost no postreinforcement pausing.

0

20

40

60

80

100

120

140

160

180

Time

Cum

ulat

ive N

umbe

r of R

espo

nses

VR 50

VR 50

• Schedule notation

• Number designates the average number of responses required for reinforcement

How do typical schedules of reinforcement differ from

gambling?

• See pages 290-291

Skinner Box vs. 1 Armed Bandit

Typical VR Gambling Schedule

Many interspersed learned reinforcers

No Yes

Amount of reinforcer varies from ratio to ratio

No Yes

Small ratio No Yes

Emotional reinforcers No Yes

Time Dependent Schedules

• Fixed Interval (FI)

• Variable Interval (VI)

FI Schedule of Reinforcement

• A reinforcer is contingent on the first response after a fixed interval of time since the last opportunity for reinforcement.

Fixed Interval 10s Schedule of Reinforcement

0

50

100

150

200

0 10 20

Seconds

Cum

ulat

ive

Num

ber

of

Res

pons

es

Reinforcer

Fixed Interval 200-s Schedule of Reinforcement

0

20

40

60

80

100

120

140

160

180

200

Seconds

Cu

mu

lati

ve N

um

ber

of

Resp

on

ses

Reinforcer

Example: Fixed Interval (FI) Schedule of Reinforcement

FI 200-s

200 seconds

400 seconds

600 seconds

Fixed-Interval Scallop

• A FI schedule often produces a scallop – a gradual increase in the rate of responding with responding occurring at a high rate just before reinforcement is available. No responding occurs for some time after reinforcement (Post Reinforcement Pause.

Fixed Interval 200-s Schedule of Reinforcement

0

20

40

60

80

100

120

140

160

180

200

Seconds

Cu

mu

lati

ve N

um

ber

of

Resp

on

ses

Reinforcer

Example: Fixed Interval (FI) Schedule of Reinforcement

FI 200-s

200 seconds

400 seconds

600 seconds

Scallop

PRP

Fixed Interval 200 s

0

0.1

0.2

0.3

20 40 60 80 100 120 140 160 180 200

Time (sec)

Rel

ativ

e F

requ

ency

+/+

+/rl

rl/rl

A

Lever Pressing of Mice after Training

Term paper vs. FI

• Term paper does not have a deadline.

FI vs. Term Paper

FI Term PaperDoes early responding effect anything?

No Yes

Do you get more if you work harder?

No Yes

Is the relevant response class clear?

Yes No

Are there calendars and clocks?

No Yes

Is there a deadline? No YesIs the reinforcer too delayed?

No Yes

Fixed Time & Superstitious Behavior

• Fixed time schedules of reinforcement– A reinforcer is delivered after the passage of a

fixed period of time, independently of the response.

• Superstitious behavior– Behaving as if the response causes some

specific outcome when it really does not.

Superstitious behavior of the pigeon:

Experiment by SkinnerFixed Time Schedule

Variable Interval (VI) Schedule of Reinforcement

• A reinforcer is contingent on the first response after a variable interval of time since the last opportunity for reinforcement.

VI Responding

• VI schedules produce a moderate rate of responding with almost no postreinforcement pausing.

Comparing/Contrasting Ratio and Interval Schedules

• Pg. 305: table

• Pg. 305: cumulative records

Continuous vs. Intermittent

• CRF – every response is reinforced

• Intermittent reinforcement – only some responses produce a reinforcer.

Intermittent Reinforcement

• Makes the response more resistant to extinction than does continuous reinforcement.

Resistance to Extinction

• The number of responses or the amount of time before a response extinguishes

Resistant to Extinction and Intermittent Reinforcement

• Intermittent reinforcement makes the response more resistant to extinction than does continuous reinforcement.

• Why? Hard for the rat to tell the difference between only an occasional reinforcement and no reinforcement

• CRF followed by EXT: rats quickly stop responding: easy discrimination between reinforcement & extinction contingencies

4 Types of Concurrent Contingencies

1. Two compatible responses

2. Compatible contingencies for a single response

3. Incompatible contingencies for a single response

4. Two incompatible responses

Concurrent Contingencies

• Contingencies that are available at the same time.

• More than one contingency of reinforcement or punishment is available at the same time.

Concurrent Contingencies

• Two levers in chamber– VI 300 s on Left lever– VI 30 s on Right lever

• Will animal allocate more responses to the left or the right lever?

• Matching Law (pg. 330)% of Left lever presses = % Left-lever reinforcers

Matching Law

• Relative Frequency of responding on two concurrent schedules of reinforcement equals the relative value of reinforcement on those two schedules