C82LEA Biology of learning and memory Learning about time Charlotte Bonardi

C82LEA Biology of learning and memory

Learning about time

Charlotte Bonardi

We are pretty good at estimating time periods, and makingjudgements about whether intervals are shorter or longer than each other.

We are also sensitive to the day/night time cycle:

jetlag (timing obviously not just to do with the sun)

waking up just before your alarm goes off

We are pretty good at estimating time periods, and makingjudgements about whether intervals are shorter or longer than each other.

We are also sensitive to the day/night time cycle:

jetlag (timing obviously not just to do with the sun)

waking up just before your alarm goes off

How do we do it? Can animals do these things?

How do they do it?

Distinguish periodic (learning to respond at a particular timeof day) and interval timing (learning to respond after a particularinterval of time).

Distinguish periodic (learning to respond at a particular timeof day) and interval timing (learning to respond after a particularinterval of time).

PERIODIC TIMING

e.g. Circadian rhythms. Question: is the cyclical behaviour really controlled by time perse? Or is it controlled by stimuli that are always present at thatparticular time?

Wheel running in the rat (described in Carlson):

4am 8am Midday 4pm 8pm Midnight

ACTIVITY

Light off Light on


ACTIVITY

ACTIVITY

Light off

Light off Light on

Light on


Constant dim light

When no light cues are available they maintain behaviouron an approximately 25-hour cycle

Cockroaches (Roberts, 1965). Increased activity at dusk. Whenremoved visual cues cycle drifted until increased activity started15 hours before dusk (cycle slightly less than 24 hours).

Restoring visual cues produced a gradual shift back to correcttime. Entrainment : light acts as a zeitgeber synchronising theinternal clock.


Restoring visual cues produced a gradual shift back to correcttime. Entrainment : light acts as a zeitgeber synchronising theinternal clock

Question: Is the apparent internal 24-hour clock the result of environmental experience?


Restoring visual cues produced a gradual shift back to correcttime. Entrainment : light acts as a zeitgeber synchronising theinternal clock

Question: Is the apparent internal 24-hour clock the result of environmental experience?

Bolles & Stokes (1965)

Subjects born and reared under either 19, 24 or 29 hour light/dark cycles. Then fed at a regular point in their own particular cycle....

animals on the 24-hour cycle learned to anticipate food....

29 22 15 8 1

Light change

Light change

29 22 15 8 1

but the others didn’t....

29 22 15 8 1

Light change

Is there any evidence for a physiological system that could provide this 24-hour clock?

The suprachiasmatic nucleus (SCN) of the hypothalamusmay be a candidate.

The metabolic rate in the SCN appears to vary as a functionof the day-night cycle.

Lesions of the SCN will abolish the circadian regularity offoraging and sleeping in the rat.

It also receives direct and indirect inputs from the visual system,which could keep circadian rhythms entrained with the realday-night cycle.

INTERVAL TIMING

Consider a normal classical conditioning procedure:

Tone (20 sec) --> food

? ? ?

INTERVAL TIMING

Consider a normal classical conditioning procedure:

Tone (20 sec) --> food

? ? ?

.....so what happens if thestimulus keeps on going(and you omit the food)?

The peak procedure

.....so what happens if thestimulus keeps on going(and you omit the food)?

The peak procedure

Church & Gibbon, 1982

Rats in lit chamber. Occasionally houselight went off for a 0.8, 4.0 or 7.2 sec (the CS). When the lights went on again a lever was presented for five seconds. If the rat pressed the lever after a 4-sec CS it got food, otherwise it did not. Then tested with a range of stimulus durations (0.8 - 7.2 secs).

Response probability

0.8

0.4

0 2 4 6 8

Church & Gibbon, 1982

Rats in lit chamber. Occasionally houselight went off for a 0.8, 4.0 or 7.2 sec (the CS). When the lights went on again a lever was presented for five seconds. If the rat pressed the lever after a 4-sec CS it got food, otherwise it did not. Then tested with a range of stimulus durations (0.8 - 7.2 secs).

0 2 4 6 8

0.8

0.4

Food after2 seconds

0 2 4 6 8

0.8

0.4

Food after2 seconds

0.8

0.4

0 2 4 6 8

Food after4 seconds

0 2 4 6 8

0.8

0.4

Food after2 seconds

0.8

0.4

0 2 4 6 8

Food after4 seconds

0 2 4 6 8

0.8

0.4Food after8 seconds

Weber’s Law

The generalisation that the just noticeable difference isproportional to the magnitude of the stimulus.

Hence small amounts judged more accurately than large amounts

Weber’s Law

The generalisation that the just noticeable difference isproportional to the magnitude of the stimulus.

Hence small amounts judged more accurately than large amounts

One versus two

Weber’s Law

one versus two – absolute difference of one

- difference as % of whole 0.5

One versus two

Nineteen versus twenty

Weber’s Law

19 versus 20 – absolute difference of one

- difference as % of whole 0.05

Weber’s Law

for comparable difficulty to the one versus two comparison youneed to compare 20 versus 10...

ten versus twenty

Weber’s Law

This may be called the scalar property of timing (it appliesto other judgements too).

I / I = k

I = Just discriminable change in intensityI = original intensity k = constant

One versus twoDifference = 2-1 = 1Ratio = (2-1)/2 = 0.5

One versus twoDifference = 2-1 = 1Ratio = (2-1)/2 = 0.5

Nineteen versus twentyDifference = 20-19 = 1Ratio = (20-19)/20 = 0.05

pacemakert pulses per second

response?

Scalar timing theory

e.g. Gibbon, Church & Meck, (1984)

Working memory N * t

Reference memory K * N * t

comparator

response?

Pacemaker emits pulses at aconstant rate t (although theremay be some random variation).



comparator


response?

Pacemaker emits pulses at aconstant rate t (although theremay be some random variation).

When a stimulus is presented, aswitch is operated, and the pulsesare allowed to accumulate in working memory. This will equalt multiplied by the number of seconds that have passed (N).



comparator


Process 1:

Storing duration of a stimulus inShort term memory



response?

5-second stimulus:successive pulses stored in working memory

t = 1 per second(ish)

1

comparator


comparator

response?



2




response?



3



comparator


response?



4



comparator


response?



5



comparator


Process 2:

Storing duration of a stimulus inReference memory


response?

When the reinforcement occurs,pulses stop accumulating; the numberof pulses in working memory (N * t)is now stored in reference memory

5


comparator

5.1


response?

When the reinforcement occurs,pulses stop accumulating; the numberof pulses in working memory (N * t)is now stored in reference memory;

this storage is not always completelyaccurate -- there is some memory distortion. This is represented by K, a number that is close to 1.If K=1 then the memory is accurate;if K<1 then a smaller number of pulses isstored; if K>1 then a greater numberis stored.



comparator

5.1


response?

After several trials there will be severalnumbers stored in reference memoryNm1, Nm2, Nm3, etc -- each equal to theK * N * t for that particular trial.



comparator

5.1


response?




comparator

5.1 4.7


response?




comparator

5.1 4.7 4.9


response?


Remember the error on each trial will not be the same



comparator

5.1 4.7 4.9 5.0


Process 3:

Using stored value in reference memory to decide whether or not to respond on the

next trial

response?

On each trial the animal compares thenumber of pulses in working memory(N * t) with a random value drawn fromthose stored in reference memory Nmx.



comparator

pacemakert pulses per second 5.1 4.7 4.9 5.0

response?

On each trial the animal compares thenumber of pulses in working memory(N * t) with a random value drawn fromthose stored in reference memory Nmx.

This is done by the comparator. If the valuesare close, then the animal responds.



comparator

pacemakert pulses per second 5.1 4.7 4.9 5.0

response?

1.0

Another stimulus occurs, and the successive number of pulses is stored in working memory


comparator



5.1 4.7 4.9 5.0

response?

2.0

Another stimulus occurs, and the successive number of pulses is stored in working memory


comparator



5.1 4.7 4.9 5.0

response?

2.0

The animal uses ONE of the values in reference memory to decide when to respond

Suppose 2 seconds have passed; N * t =2


comparator



5.1 4.7 4.9 5.0

response?

The comparator works out how closethe values are using a ratio rule -- NOTa difference rule

i.e. NOT N * t - NMx

but N * t - NMx / NMx

This is one of the reasons that accuracyis better with short intervals.

comparator

pacemakert pulses per second 2.0



5.1 4.7 4.9 5.0

NO RESPONSE

comparator

pacemakert pulses per second 2.0



5.1 4.7 4.9 5.0

2 - 5.1 / 5.1 = .61

Large therefore don’t respond


response?

3.0


comparator



5.1 4.7 4.9 5.0

response?

4.0


comparator



5.1 4.7 4.9 5.0

response?

5.0

The animal uses ONE of the values in reference memory to decide when to respond



comparator



5.1 4.7 4.9 5.0

5.0



comparator



5.1 4.7 4.9 5.0

5 - 4.7 / 4.7 = .06

Small therefore respond

RESPONSE

Potential problems with scalar timing theory

1) There is as yet no physiological evidence for a pacemaker

Alternatives have been proposed:

(i) Instead of a pacemaker, it has been proposed that timingcould be achieved by a series of oscillators, each of which hastwo states, on or off. If each oscillator switches after a differentperiod of time, then the entire pattern of activation could be used to determine the exact time (e.g., Gallistel, 1990; Church& Broadbent, 1991):





R G





RR RG GR GG





RRR RRG RGR RGG GRR GRG GGR GGG


(ii) Another solution that has been proposed is the Behavioural theory of timing (e.g., Killeen & Fetterman, 1988).

When the animal gets a reward, this stimulates behaviour.The animal moves across an invariant series of behaviouralclasses in between reinforcements. A pulse from an internalpacemaker will change the behaviour from one class toanother. The behaviour that is occurring when the nextreinforcer occurs becomes a signal for that reinforcer.


(ii) Another solution that has been proposed is the Behavioural theory of timing (e.g., Killeen & Fetterman, 1988).

When the animal gets a reward, this stimulates behaviour.The animal moves across an invariant series of behaviouralclasses in between reinforcements. A pulse from an internalpacemaker will change the behaviour from one class toanother. The behaviour that is occurring when the nextreinforcer occurs becomes a signal for that reinforcer.

Wipe whiskersWash faceEat the food Explore corner


2) Conditioning and timing supposedly occur at the same time,and yet are controlled by completely different learning mechanisms.

Some theories of timing try and explain conditioning; e.g.,Gibbon & Balsam (1977).




Calculate rate of reinforcement during stimulus, and rate ofreinforcement during background. If first is higher than second,get conditioning.




Calculate rate of reinforcement during stimulus, and rate ofreinforcement during background. If first is higher than second,get conditioning.

6 reinforcers in 60 minutes of background = 1/10 = 0.14 reinforcers in 15 minutes of stimulus = 4/15 = 0.27 0.27 > 0.1

This theory cannot explain basic phenomena, like blocking.

Some conditioning models try to explain timing -- e.g.Real time models (e.g., Sutton & Barto, 1981).

They work with the Rescorla- Wagner model, just likeregular conditioning theories. However, the stimulus isassumed to change over the course of its presentation, andthis allows the animal to learn about when a reinforcer occurs.

This theory cannot explain basic phenomena, like blocking.

Some conditioning models try to explain timing -- e.g.Real time models (e.g., Sutton & Barto, 1981).

They work with the Rescorla- Wagner model, just likeregular conditioning theories. However, the stimulus isassumed to change over the course of its presentation, andthis allows the animal to learn about when a reinforcer occurs.

Stimulus trace:

Food

General references

Bouton, M.E. (2007). Learning and Behavior. Sinauer Associates.

Carlson, N.R. (2001) Physiology of Behaviour. Allyn & Bacon. Chapter 9.

Domjan, M. (1988). The principles of learning and behavior. Brooks/ColePublishing Company. Chapter 12.

Pearce, J.M. (1997). Animal Learning and Cognition. Lawrence Erlbaum Associates. Chapter 7.

Shettleworth, S.J. (1998). Cognition, Evolution and Behaviour. Oxford University Press. Chapter 8.

Wynne, C.D.L. (2000). Animal Cognition. Macmillan. Chapter 5 pp.96-101

Specific references

Bolles, R.C., & Stokes, L.W. (1965). Rat’s anticipation of diurnal and a-diurnal feeding. Journal of Comparative and Physiological Psychology, 60, 290-294.Church, R.M., & Broadbent, H.A. (1991). A connectionist model of timing. In M.L.Commons, S Grossberg, & J.E.R. Staddon (Eds.) Neural network models of conditioning and timing (pp.225-240). Hillsdale, N.J.: Lawrence Erlbaum AssociatesChurch, R.M., & Gibbon, J. (1982). Temporal generalization. Journal ofExperimental Psychology: Animal Behaviour Processes, 8, 165-186.Gallistel, C.R. (1990). The organisation of learning. Cambridge, MA: MIT Press.Gibbon, J., Church, R.M., & Meck, W.H. (1984). Scalar timing in memory.In J. Gibbon & L. Allen (Eds.) Time and time perception, Annals of the New York Academy of Sciences (Vol. 423, pp.52-77). New York: New YorkAcademy of Sciences.Gibbon, J., & Balsam, P. (1981). Spreading association in time. In C.M. Locurto,H.S. Terrace & J. Gibbon (Eds.) Autoshaping and conditioning theory (pp.219-253).New York: Academic Press.Killeen, P.R., & Fetterman, J.G. (1988). A behavioral theory of timing. PsychologicalReview, 95, 274-295.Roberts, S.K. (1965). Photoreception and entrainment of cockroach activityrhythms. Science, 148, 958-960.Sutton, R.S., & Barto, A.G. (1981). Toward a modern theory of adaptive networks:Expectation and prediction. Psychological Review, 88, 135-170.

Documents

C82LEA Biology of learning and memory Learning about time Charlotte Bonardi