Forward models in the cerebellum: Predicting, Planning, Learning
Adrian Haith, PhDAssistant Professor, Department of Neurology
Johns Hopkins University
Forward ModelsKnowing the state of the body is critical for accurate movementBut: long (~100ms) delays in receiving sensory feedback
Solution: Forward Model
• Simulates consequences of motor commands
• Predicts future state of the body
• Believed to be a primary function of the cerebellum
Forward Model
Uses of Forward Models
• Potential uses of forward models:– Forward models provide an alternative estimateof the state of the body during active movement
– This should enable rapid compensation for erratic motor commands
– Forward models allow one to plan appropriate movements to achieve desired goals
Moving without a forward model
• Cerebellar ataxia
Forward models improve proprioception
Passive Movement by robot
Active Movement(stopped by a virtual wall)
Report PercievedLimb Position
(Cerebellar Ataxia)
(Estimation Varia
bility)
Bhanpuri et al., 2013
Integrating knowledge of motor commands improves perception in controls but not cerebellar patients
Control
Role of forward models in feedback control• Our motor commands are always variable– Knowledge of bad motor commands can be used to correct an errant movement online
Example: Saccades
Xu-‐Wilson et al., 2009
Cerebellar
Compensation for slow start absent in patients
If cerebellum really implements forward models, cerebellar output should resemble movement outcomes, not motor commands
Does appear to be the case for saccades:
Simple spike population response encodes impending velocity profile of a saccadeAlso some evidence for arm movements.
Encoding of eye kinematics in the cerebellum
Herzfeld et al., 2015
Co-‐ordination versus prediction
Manipulating an object with both hands requires precise coordination
Common argument: Such coordination is enabled by forward models
Can easily co-‐ordinate effectors without having to do prediction
e.g. co-‐ordinatingeyelids while blinking!
Wolpert & Flanagan, 2001
Coordination is not cerebellum-‐dependentUnloading
timeExternal Unloading
SelfUnloading
Anticipatory reduction of force
Diedrichsen et al., 2004
Healthy Controls
Intact in cerebellar patients
NB – colors here reflect unloading speed
Cerebellar Patients
Co-‐ordination Prediction
Cerebellum is required to learn novel coordination
Participants taught a novel coordination pattern:Button-‐trigeredunloading
Diedrichsen et al., 2004
No learning in cerebellar patients
Downward acceleration in catch trials (surprise lack of unloading) -‐ LEARNING
Cerebellum does play a critical role in learning new patterns of coordination
Visuomotor rotation
Adaptation
Cursor path rotated relative to hand
Force Field
Force field applied to the hand• Strength/direction of force
depends on hand position/speed
Time course of adaptation
• Typically occurs in minutes• Stereotypical exponential learning curve:
• Characterizable through an error-‐driven learning rule:
𝑥"#$ = 𝐴𝑥" + 𝐵𝑒"
Adaptation depends on the cerebellum
Maschke et al., 2004
• Healthy controls regain baseline levels of performance• Cerebellar patients never do
Adaptation depends on the cerebellum
Another example:– Patients with severe cerebellar ataxia show limited adaptation
– Gradual introduction of perturbation enables learning
Criscimagna-‐Hemminger et al., 2010
Adaptation is associated with altered prediction
Synofzik et al., 2008
Is adaptation mediated through a change in the cerebellar forward model?
Exposure to perturbation affects both movement andperception
Both effects blunted in cerebellar patients
Adaptation is driven by sensory prediction errors
45 degree visuomotor rotation
Subset of subjects instructed how to “cheat”
Mazzoni & Krakauer, 2006
Adaptation is driven by sensory prediction errors
Similar behavior also seen when throwing balls while wearing prism goggles
Martin et al., Brain, 1996
No implicit recalibration in cerebellar patients
Taylor et al., 2010
Multiple components of adaptation
• Not ALL adaptation is due to forward model recalibration
• At least two dissociable components of learning:
• Cerebellum believed to be responsible for slow, implicit component. What about other component?
Cerebellum
Reaction Time and Motor Learning
• RT increases during motor adaptation:
Fernandez-‐Ruiz et al., 2011
Reaction Time and Motor Learning
Haith et al., 2015
Lowering RT limits expression of learning
Haith et al., 2015
Reaction Time and Savings
Haith et al., 2015
Reaction Time and SavingsDay-‐by-‐day comparison:
Haith et al., 2015
Implicit vs Explicit learning
+=
Taylor et al., 2014
Adaptation composed of multiple components• Implicit recalibration (based
on forward model)• Explicit re-‐aiming
Cerebellar damage impairs both implicit andimplicit learning
Same paradigm, in controls versus cerebellar patients
Butcher et al., In Prep
Controls
Patients
Net Learning
Re-‐Aiming Recalibration
+
=
Beyond Adaptation• Cerebellum also critical for less structured forms of learning– Learning arbitrary associations between stimuli and actions
• Neuroimaging studies consistently implicate cerebellum
• Learning of arbitrary visuomotorassociations is impaired in individuals with cerebellar ataxia
Balsers & Ramnani, 2011
De Novo Learning
• Is the cerebellum involved in ALL kinds of learning?– Probably not
“De novo” learning task-‐ Cannot be learned through adaptation of an existing controller
De Novo Learning
• Is the cerebellum involved in ALL kinds of learning?– Probably not
“De novo” learning task-‐ Cannot be learned through adaptation of an existing controller
De novo learning and the cerebellum
• Preliminary data suggest that learning of this kind is unaffected in patients with cerebellar ataxia
Cerebellum and cognition
• Clear that cerebellum has role in cognitive functions
• Could some regions of the cerebellum implement models of the cortex, rather than the body?
CortexInput Output
PredictedOutput
• Could account for cerebellar role in “automation” of responses…
Cerebellum
Summary• The cerebellum implements a forward model, predicting
the future state of the body based on current state and current motor commands
• Strong evidence that the output of a forward model influences the estimated state of the body
• Fair evidence that forward models also support movement planning
• The cerebellum likely does much more besides implement forward models– Many open questions!