Forward  models  in  the  cerebellum:  Predicting,  Planning,    Learning

Adrian  Haith,  PhDAssistant  Professor,  Department  of  Neurology

Johns  Hopkins  University

Forward  ModelsKnowing  the  state  of  the  body  is  critical  for  accurate  movementBut:  long  (~100ms)  delays  in  receiving  sensory  feedback

Solution:  Forward  Model

• Simulates  consequences  of  motor  commands

• Predicts  future  state  of  the  body

• Believed  to  be  a  primary  function  of  the  cerebellum

Forward  Model

Uses  of  Forward  Models

• Potential  uses  of  forward  models:– Forward  models  provide  an  alternative  estimateof  the  state  of  the  body  during  active  movement

– This  should  enable  rapid  compensation for  erratic    motor  commands

– Forward  models  allow  one  to  plan appropriate  movements  to  achieve  desired  goals  

Moving  without  a  forward  model

• Cerebellar  ataxia

Forward  models  improve  proprioception

Passive  Movement  by  robot

Active  Movement(stopped  by  a  virtual  wall)

Report  PercievedLimb  Position

(Cerebellar  Ataxia)

(Estimation  Varia

bility)

Bhanpuri et  al.,  2013

Integrating  knowledge  of  motor  commands  improves  perception  in  controls  but  not  cerebellar  patients

Control

Role  of  forward  models  in  feedback  control• Our  motor  commands  are  always  variable– Knowledge  of  bad  motor  commands  can  be  used  to  correct  an  errant  movement  online

Example:  Saccades

Xu-­‐Wilson  et  al.,  2009

Cerebellar

Compensation  for  slow  start  absent  in  patients

If  cerebellum  really  implements  forward  models,  cerebellar  output  should  resemble  movement  outcomes,  not  motor  commands

Does  appear  to  be  the  case  for  saccades:

Simple  spike  population  response  encodes  impending  velocity  profile  of  a  saccadeAlso  some  evidence  for  arm  movements.

Encoding  of  eye  kinematics  in  the  cerebellum

Herzfeld  et  al.,  2015

Co-­‐ordination  versus  prediction

Manipulating  an  object  with  both  hands  requires  precise  coordination

Common  argument:  Such  coordination  is  enabled  by  forward  models

Can  easily  co-­‐ordinate  effectors  without  having  to  do  prediction

e.g.  co-­‐ordinatingeyelids  while  blinking!

Wolpert &  Flanagan,  2001

Coordination  is  not  cerebellum-­‐dependentUnloading

timeExternal  Unloading

SelfUnloading

Anticipatory  reduction  of  force

Diedrichsen et  al.,  2004

Healthy  Controls

Intact  in  cerebellar  patients

NB  – colors  here  reflect  unloading  speed

Cerebellar  Patients

Co-­‐ordination                Prediction

Cerebellum  is  required  to  learn novel  coordination

Participants  taught  a  novel  coordination  pattern:Button-­‐trigeredunloading

Diedrichsen et  al.,  2004

No  learning   in  cerebellar  patients

Downward  acceleration  in  catch  trials  (surprise   lack  of  unloading) -­‐ LEARNING

Cerebellum  does  play  a  critical  role  in  learning  new patterns  of  coordination

Visuomotor rotation

Adaptation

Cursor  path  rotated  relative  to  hand

Force  Field

Force  field  applied  to  the  hand• Strength/direction  of  force  

depends  on  hand  position/speed

Time  course  of  adaptation

• Typically  occurs  in  minutes• Stereotypical  exponential  learning  curve:

• Characterizable through  an  error-­‐driven  learning  rule:

𝑥"#$ = 𝐴𝑥" + 𝐵𝑒"

Adaptation  depends  on  the  cerebellum

Maschke et  al.,  2004

• Healthy  controls  regain  baseline  levels  of  performance• Cerebellar  patients  never  do

Adaptation  depends  on  the  cerebellum

Another  example:– Patients  with  severe  cerebellar  ataxia  show  limited  adaptation

– Gradual  introduction  of  perturbation  enables  learning

Criscimagna-­‐Hemminger et  al.,  2010

Adaptation  is  associated  with  altered  prediction

Synofzik et  al.,  2008

Is  adaptation  mediated  through  a  change  in  the  cerebellar  forward  model?

Exposure  to  perturbation  affects  both  movement  andperception  

Both  effects  blunted  in  cerebellar  patients

Adaptation  is  driven  by  sensory  prediction  errors

45  degree  visuomotor rotation

Subset  of  subjects  instructed  how  to  “cheat”

Mazzoni &  Krakauer,  2006

Adaptation  is  driven  by  sensory  prediction  errors

Similar  behavior  also  seen  when  throwing  balls  while  wearing  prism  goggles

Martin  et  al.,  Brain,  1996

No  implicit  recalibration  in  cerebellar  patients

Taylor  et  al.,  2010

Multiple  components  of  adaptation

• Not  ALL adaptation  is  due  to  forward  model  recalibration

• At  least  two  dissociable  components  of  learning:

• Cerebellum  believed  to  be  responsible  for  slow,  implicit  component.  What  about  other  component?

Cerebellum

Reaction  Time  and  Motor  Learning

• RT  increases  during  motor  adaptation:

Fernandez-­‐Ruiz  et  al.,  2011

Reaction  Time  and  Motor  Learning

Haith  et  al.,  2015

Lowering  RT  limits  expression  of  learning

Haith  et  al.,  2015

Reaction  Time  and  Savings

Haith  et  al.,  2015

Reaction  Time  and  SavingsDay-­‐by-­‐day  comparison:

Haith  et  al.,  2015

Implicit  vs Explicit  learning

+=

Taylor  et  al.,  2014

Adaptation  composed  of  multiple  components• Implicit  recalibration  (based  

on  forward  model)• Explicit  re-­‐aiming

Cerebellar  damage  impairs  both  implicit  andimplicit  learning

Same  paradigm,  in  controls  versus  cerebellar  patients

Butcher  et  al.,  In  Prep

Controls

Patients

Net  Learning

Re-­‐Aiming Recalibration

+

=

Beyond  Adaptation• Cerebellum  also  critical  for  less  structured  forms  of  learning– Learning  arbitrary  associations  between  stimuli  and  actions

• Neuroimaging  studies  consistently  implicate  cerebellum

• Learning  of  arbitrary  visuomotorassociations  is  impaired  in  individuals  with  cerebellar  ataxia

Balsers &  Ramnani,  2011

De  Novo  Learning

• Is  the  cerebellum  involved  in  ALL  kinds  of  learning?– Probably  not

“De  novo”  learning  task-­‐ Cannot  be  learned  through   adaptation  of  an  existing  controller

De  Novo  Learning

• Is  the  cerebellum  involved  in  ALL  kinds  of  learning?– Probably  not

“De  novo”  learning  task-­‐ Cannot  be  learned  through   adaptation  of  an  existing  controller

De  novo  learning  and  the  cerebellum

• Preliminary  data  suggest  that  learning  of  this  kind  is  unaffected  in  patients  with  cerebellar  ataxia

Cerebellum  and  cognition

• Clear  that  cerebellum  has  role  in  cognitive  functions

• Could  some  regions  of  the  cerebellum  implement  models  of  the  cortex,  rather  than  the  body?

CortexInput Output

PredictedOutput

• Could  account  for  cerebellar  role  in  “automation”  of  responses…

Cerebellum

Summary• The  cerebellum  implements  a  forward  model,  predicting  

the  future  state  of  the  body  based  on  current  state  and  current  motor  commands

• Strong  evidence  that  the  output  of  a  forward  model  influences  the  estimated  state  of  the  body

• Fair  evidence  that  forward  models  also  support  movement  planning

• The  cerebellum  likely  does  much  more  besides  implement  forward  models– Many  open  questions!