Outline• B.F. Skinner

– Biography– Theoretical notions

> Respondent and Operant Behaviour

> Operant Conditioning Principles

> The Skinner Box

> Shaping and Extinction

> Superstitious behavior

> Discriminative responding

> Secondary reinforcers

“Stimulus, response! Stimulus, response! Don’t you ever think?”

B. F. Skinner (1904-1990)

• Biography– Wanted to be writer

>B.A. in English Literature (1926)– Entered graduate school at Havard (1928)

>Mentored by the Chair of Physiology (W. Crozier) Who studied the “animal as a whole” without

appealing to internal processes.

>Obtained his Ph.D. in 1931

– Taught at University of Minnesota (1936 - 1945)>Published “The Behaviour of Organisms” (1938)

– Affiliated with Havard until he died (1990)

B. F. Skinner

B. F. Skinner (1904-1990)

• Biography– Inventions

> The air-crib Easily-cleaned, temperature and humidity-controlled Somewhat controversial (but effective) Commercially manufactured

>Project pigeon Received a 25K grant to develop a cruise missile

• Guided by trained pigeons• U.S. Navy passed on it

(but retested the idea

in the 1980s)

B. F. Skinner

• Biography– Inventions

> The Skinner Box (AKA, operant chamber)

Animal can respond multiple times Operant response: Bar pressing Operant conditioning: Increased bar pressing when food is

delivered following the response.

B. F. Skinner

B. F. Skinner (1904-1990)

• Biography– Inventions

> Cumulative recorder Keeps track of the animal’s responding

• Time is recorded on the ‘X’ axis• Total number of responses is recorded on the ‘Y’ axis• Faster rates of responding lead to steeper slopes

B. F. Skinner

B. F. Skinner (1904-1990)

• Major Theoretical Notions– Respondent and operant behaviour

>Respondent behaviour - Behaviour elicited by a known stimulus

E.g., Unconditioned responses• Elicited by unconditioned stimuli• Reflexive

>Operant behaviour - Behaviour not elicited by a known stimulus

E.g., Most of our everyday behaviour• Occurs spontaneously

B. F. Skinner

B. F. Skinner (1904-1990)

• Major Theoretical Notions– Type S and Type R conditioning

> Two kinds of conditioning Type S (respondent conditioning) - classical

(Pavlovian) conditioning• ‘S’ to emphasize the role of the eliciting Stimulus• Strength is measured by the magnitude of the CR

Type R (operant conditioning) - learning that involves operant behaviour

• ‘R’ to emphasize Response• Strength is measured by the response rate

B. F. Skinner

B. F. Skinner (1904-1990)

• Major Theoretical Notions– Type S and Type R conditioning

> Comparison with Thorndike’s approach Thorndike’s puzzle box

• Dependent variable was time-to-solution

-> I.e., how long it took to learn a (single) response Skinner

• Dependent variable was rate of responding

-> I.e., What variables affect the rate

of responding

B. F. Skinner

B. F. Skinner (1904-1990)

Edward Lee Thorndike (1874 - 1949)

• Major Theoretical Notions– Operant conditioning principles

>Two general principles Any response that is followed by a reinforcing

stimulus tends to be repeated A reinforcing stimulus is anything that

increases the rate with which an operant response occurs

• I.e., anything that increases the probability of a response’s re-occurring

B. F. Skinner

B. F. Skinner (1904-1990)

• Major Theoretical Notions– Operant conditioning principles

>Contingent reinforcement Emphasis on behaviour and its consequences

• Gaining reinforcement depends (i.e., is contigent) on making the appropriate response

Culture as a set of reinforcement contigencies• Different cultures reinforce different behaviour

patterns Controlling reinforcement -> controls behaviour

• E.g, child rearing

B. F. Skinner

B. F. Skinner (1904-1990)

• Major Theoretical Notions– Conditioning the lever-pressing response

>Three steps

1. Deprivation• Food/water deprived for 23 hours per day• Animal is held at 80% of its free-feeding body

weight

2. Magazine training• Food pellets are delivered by the experimenter• Animal learns to associate the sound of the

delivery mechanism with food

B. F. Skinner

B. F. Skinner (1904-1990)

• Major Theoretical Notions– Conditioning the lever-pressing response

>Three steps

3. Lever pressing• Animal is placed in the box• Eventually hits the lever (operant response)• Delivery of food pellet reinforces the response.

B. F. Skinner

B. F. Skinner (1904-1990)

• Major Theoretical Notions– Conditioning the lever-pressing response

>Shaping A faster method of teaching the rat to lever press Two components

1. Differential reinforcement

-> Some responses are reinforced, others are not

2. Successive approximations

-> Only reinforce responses that become progressively closer to the desired response

B. F. Skinner

B. F. Skinner (1904-1990)

• Major Theoretical Notions– Extinction and spontaneous recovery

>Extinction - removing the reinforcer removes the operant response

>Spontaneous recovery - The reoccurrence of a response that had been extinguished, with no additional training.

B. F. Skinner

B. F. Skinner (1904-1990)

Cum

ulat

ive

resp

onse

s

Time

Res

t

Extinction

Spontaneous recovery

• Major Theoretical Notions– Superstitious behavior

>What if we delivered pellets noncontingently? Random behaviour would get reinforced

• E.g., Dog running in circles while waiting to get fed Humans are susceptible to similar conditioning

• E.g., Athletes/ coaches’ game rituals

B. F. Skinner

B. F. Skinner (1904-1990)

• Major Theoretical Notions– Discriminative operant

>Skinner box is set so that reinforcement is only available when the light is on.

The light is the discriminative stimulus• I.e., indicates that reinforcement is available

SD = light on, S = light off, SR = reinforcing stimulus• A discriminative operant is symbolized as:

SD -> R -> SR • Skinner was interested in the SD -> R association

-> Cf. respondent conditioning Stimulus control of behaviour

B. F. Skinner

B. F. Skinner (1904-1990)

• Major Theoretical Notions– Secondary reinforcement

>A neutral stimulus paired with a reinforcer can take on reinforcing qualities of its own

>To test this notion Lever press -> light -> food Extinguish the response

• Neither light nor food is delivered Allow lever - press to deliver light (not food)

• Response rate increases

B. F. Skinner

B. F. Skinner (1904-1990)

• Major Theoretical Notions– Secondary reinforcement

>A secondary reinforcer can be used to reinforce other responses

Clicker training for dogs• Warning: secondary reinforcers can be

extinguished! Money for humans

• Generalized reinforcer

B. F. Skinner

B. F. Skinner (1904-1990)