Embed Size (px)
Citation preview
Jerusalem in Motion
Dec, 2003
Amir Karniel
Feedback, Adaptation, Learning or Evolution: How Does the Brain
Coordinate and Time Movements?
Amir Karniel
Department of Biomedical Engineering
Ben Gurion University of the Negev
The studies presented were done in collaboration with:
Gideon Inbar, Ronny Meir, and Eldad Klaiman - Technion
Sandro Mussa-Ivaldi - Northwestern University
The first workshop of THE CENTER FOR MOTOR RESEARCH December 18-21, 2003
Jerusalem in Motion
Dec, 2003
Amir Karniel
The Hierarchy of Wide Sense AdaptationEquilibrium trajectories and internal modelsReaching movements muscle models and adaptation
Adaptation to Force PerturbationsTime representationSequence learning and switching
Bimanual CoordinationSymmetry at the perceptual level as an invariant feature Tapping experiments and first indications for internal
models
Summary and Future Research
Outline
Jerusalem in Motion
Dec, 2003
Amir Karniel
Two Important Concepts in the Theory of Motor Control
Equilibrium Inverse Model
x yyd F-1(yd) F(x)
Feldman
Bizzi et al.
+Minimum Jerk, Flash and Hogan
+Force fields, primitives, Mussa-Ivaldi
Albus (cerebellum)
Inbar and Yafe (signal adaptation)
+feedback error, Kawato
+distal teacher, Jordan
Adaptation
Change of Impedance Change of the inverse
Jerusalem in Motion
Dec, 2003
Amir Karniel
Reaching movements
• Feed-Forward Control
• Invariant Features: Roughly straight line, bell shaped speed profile (Flash & Hogan 1985)
Key QuestionsWhat is the origin of the invariance ?
How do we handle external perturbations ?
MJT
Jerusalem in Motion
Dec, 2003
Amir Karniel
X
BK s
T0
X0
B p
11 + s
n
n i
Fm
F0
0
0
0
x-= v1=b 3=a 2a
0< v Ta
0v vb/Ta=B
A Hill-type mechanical muscle model The viscose element B is not a constant !
Jerusalem in Motion
Dec, 2003
Amir Karniel
Linear Vs. Nonlinear Muscle Model
Linear model
0 0.5 10
0.2
0.4
0.6
0.8End point speed
0 0.5 10
0.2
0.4
0.6
0.8End point speed
The nonlinear Hill-type model
The physiologically plausible nonlinear model can produce the typical speed profile with a simple control signals
Karniel and Inbar (1997) Biol. Cybern. 77:173-183
Jerusalem in Motion
Dec, 2003
Amir Karniel
0 0.05 0.1 0.15 0.2 0.2
0.22
0.24
0.26 Duration
Amplitude
0 0.05 0.1 0.15 0.2 0
0.5
1
1.5 Maximum Speed
Other typical features of rapid movements are also facilitated by the nonlinear muscle properties
In this set of simulations the one-fifth power law model was used.
05
1
eqxxkxbxm
Karniel and Inbar (1999) J. Motor Behav. 31:203-206
0 10 20 30 Amplitude (deg)
1.0 0.5 0.0
80 40 0
Vm
ax (
deg/
s)
Dur
atio
n (
s)
Jerusalem in Motion
Dec, 2003
Amir Karniel
Adaptation to force perturbations
Modified with permission from Patton and Mussa-Ivaldi
No ForceAfter-Effects
Force FieldAfter Learning
Force FieldInitial Exposure
• Force field exposure recovery of unperturbed pattern
• Removal of field “after-effects”
(Shadmehr & Mussa-Ivaldi 1994)
Jerusalem in Motion
Dec, 2003
Amir Karniel
Hierarchical system with feedback adaptation and learning
Musculoskeletal system
Dynamics determine the control signal
(e.g., EPH, CPG, …)
Internal models for control
Desired Target
Actual Performance
Feedback
Learning Adaptation
Jerusalem in Motion
Dec, 2003
Amir Karniel
Time Scale
Change Scale
No Change Feedback
Parameters Change
Structural Change
Functional Change
Adaptation
Learning
Evolution
mSec Minutes Years Myears
The hierarchy of wide sense adaptationKarniel and Inbar (2001), Karniel (In preparation)
Jerusalem in Motion
Dec, 2003
Amir Karniel
The Hierarchy of Wide Sense AdaptationEquilibrium trajectories and internal modelsReaching movements muscle models and adaptation
Adaptation to Force PerturbationsTime representationSequence learning and switching
Bimanual Coordination Symmetry at the perceptual level as an invariant feature Tapping experiments and first indications for internal
models
Summary and Future Research
Outline
Jerusalem in Motion
Dec, 2003
Amir Karniel
What are the limitations of adaptation?
Key Questions:
tqCqD d ,Plant & Environment Controller
F +Gxxx,+CxxH Example:
?ˆ,, EtqCtqEqD d
Force Field
Internal Representation of the field
What is the structure of the modifier ? ?E
Could it be a function of position, velocity, time, … ?
Jerusalem in Motion
Dec, 2003
Amir Karniel
Time Representation
These systems are indistinguishable therefore
1. The existence of time variable isn’t sufficient to define time
representation.
2. It is sufficient to consider the following form:
txtugy
txtufx
,,
,,
e
Tee
xtugy
xtufx
,
1,,
00
,
tx
xxx
t
Tte
xtugy
xtufx
,
,
Jerusalem in Motion
Dec, 2003
Amir Karniel
Time Representation - Definition
The system is said to be capable of time representation if there
exists a deterministic function h(x) such that for any u(t).
The system is said to be capable of time representation of up to T
seconds with ε accuracy if there exists a deterministic function h(x)
such that for t<T and for any u(t).
txht
xtugy
xtxxtufx
,
0 , 0
εxht
Jerusalem in Motion
Dec, 2003
Amir Karniel
The experiment
Null Learning Generalization
No external field External Force field time/state/sequence dependent
Number of movements ~100 ~500 ~100
Jerusalem in Motion
Dec, 2003
Amir Karniel
Time Varying Force Field
0
6cos13
y
x
f
tf The force field is not correlated with the movement initiation, therefore there is no way to use state information.
Only time representation would allow adaptation and after-effects for this field.
Jerusalem in Motion
Dec, 2003
Amir Karniel
Result: No adaptation to this TV force field
A control experiment with the viscous curl field
The maximum distance from a straight line during “learning”
Karniel and Mussa-Ivaldi (2003) Biol. Cybern.
Jerusalem in Motion
Dec, 2003
Amir Karniel
Viscous Curl Force Field
-1 0 1-1
0
1B-
Vx
Vy
-1 0 1-1
0
1B+
Vx
Vy
xy
yx
vBf
vBf
15
15
1
1
B
B ,,,,,, BBBBBBB
Jerusalem in Motion
Dec, 2003
Amir Karniel
Result: There is Significant Adaptation with This Sequence of Force Fields
The maximum distance from a straight line during “learning”
A control experiment with the viscous curl field
Jerusalem in Motion
Dec, 2003
Amir Karniel
Direction Error Calculation
“B+”
DE is Positive
Therefore:
Positive DE: Yielding to the field
Negative DE: Over resisting the field
2. If the deviation is to the right multiply by –1
1. Find the Euclidean distance from a straight line at the point of maximum velocity(The feed-forward part of the movement)
3. If the curl field in the sequence is B- multiply by –1
Jerusalem in Motion
Dec, 2003
Amir Karniel
Catch trials – After Effects
1 2-0.01
0
0.01 A few trials without force field were introduced unexpectedly.
The left bar is the mean of the error (DE) during these trials in the first part of the learning.
The right bar is in the last part.
Significant expectation to the correct field after learning
i.e., learning of an internal model of the force field
Jerusalem in Motion
Dec, 2003
Amir Karniel
Mid – Summary
• No adaptation in the case of the time dependent force field
• Adaptation in the case of the simplest sequence of curl viscous fields with four targets.
What is learned in the second case?
Jerusalem in Motion
Dec, 2003
Amir Karniel
Odd and Even Movement
• During the learning it is possible to assign a unique force field to each movement instead of learning the sequence of force fields.
• The generalization phase would violate this representation.
Force Field: B+ B-
Jerusalem in Motion
Dec, 2003
Amir Karniel
Refuting the Sequence Learning Assumption
1. Analysis of errors in the last part where diagonal movements are introduced
Force Field: B+ B-
The same sequence is applied in this part; sequence learning predicts similar errors
Jerusalem in Motion
Dec, 2003
Amir Karniel
Distance Error Analysis of movements in part 1 and part 5
1 2 3-0.01
0
0.01
0.02
0.03
0.04ba
1 2 3-0.02
0
0.02
0.04
0.06bb
1 2 3-0.05
0
0.05bc
1 2 3-0.02
0
0.02
0.04
0.06bd
1 2 3-0.01
0
0.01
0.02
0.03
0.04zj
1 2 3-0.02
0
0.02
0.04
0.06be
The sequence learning assumption predicts similar errors in the right two bars that is smaller than the first, left bar
Left bar: Catch trials in part 1.
Middle bar: Movements in part 5 that are inconsistent with the learning phase.
Right bar: Movements in part 5 that are consistent with the learning phase.
All movements are consistent with the sequence of force field.
However, ANOVA of the data shows similar error in the first two bars and significantly smaller error in the right bar!
Jerusalem in Motion
Dec, 2003
Amir Karniel
Refuting the Sequence Learning Assumption
We found that when the perturbation can be modeled both as a function of sequence and as a function of the state, the brain generates a state dependent model.
We tried to train subject with the same sequence but with three targets.In this case one needs to follow the temporal sequence in order to adapt
Can we design an experiment where only sequence representation would allow adaptation?
Would the brain adapt to this perturbation?
Jerusalem in Motion
Dec, 2003
Amir Karniel
Result: No Adaptation to the Sequence of Force Fields!
A control experiment with the viscous curl field
The maximum distance from a straight line during “learning”
Karniel and Mussa-Ivaldi (2003) Biol. Cybern. 89:10-21
Jerusalem in Motion
Dec, 2003
Amir Karniel
Catch trials – No After Effects
1 2-0.01
0
0.01A few trials without force field were introduced unexpectedly.
The left bar is the mean of the error (DE) during these trials in the first part of the learning.
The right bar is in the last part.
No significant expectation to the correct field after learning
i.e., no learning of an internal model to the sequence!
Jerusalem in Motion
Dec, 2003
Amir Karniel
Mid – Summary (2)• No adaptation in the case of time dependent force field• Adaptation when the temporal sequence coincide with
single state mapping• No adaptation in the case of sequence of force fields
Maybe it is too difficult to construct two internal models simultaneously
Multiple Models Conjecture (“soft” version): If each force field is experienced separately and enough time is given for consolidation of each model, then the multiple model would be constructed
Karniel and Mussa-Ivaldi (2003) Biol. Cybern. 89:10-21
Jerusalem in Motion
Dec, 2003
Amir Karniel
Day 1 Day 2 Day 3 Day 4
Early Training
Late Training
Late TrainingCatch-Trials
Karniel and Mussa-Ivaldi EBR 2002
Jerusalem in Motion
Dec, 2003
Amir Karniel
Result: Clear learning of each perturbation, but No evidence for ability to utilize multiple models and context switching
0
5
10
15
20
1E 1L 2E 2L 3E 3L 4E 4L-20
-15
-10
-5
0
5
(Subject E)
Error [DE, mm] during early and late training
Error [DE, mm] during catch trials
Day 1 Day 2 Day 3 Day 4
Jerusalem in Motion
Dec, 2003
Amir Karniel
Does the brain employs clocks counters or switches ?
In contrast to artificial devices that are based on clock counters and switches the brain
seems to prefer state dependent maps
Jerusalem in Motion
Dec, 2003
Amir Karniel
The Hierarchy of Wide Sense AdaptationEquilibrium trajectories and internal modelsReaching movements muscle models and adaptation
Adaptation to Force PerturbationsTime representationSequence learning and switching
Bimanual CoordinationSymmetry at the perceptual level as an invariant feature Tapping experiments and first indications for internal
models
Summary and Future Research
Outline
Jerusalem in Motion
Dec, 2003
Amir Karniel
Bimanual Coordination (1)
• Preference for in-phase symmetry
• Stable vs. Unstable
• Homologous muscles
Figure from Kelso and Schöner (1988)
Jerusalem in Motion
Dec, 2003
Amir Karniel
Bimanual Coordination (2)
• It was recently shown that the preference for symmetry in bimanual coordination is perceptual
Figure from Mechsner et al. (2001)
Jerusalem in Motion
Dec, 2003
Amir Karniel
Bimanual Coordination (3)• Untrained individuals are unable
to produce non-harmonic polyrhythms
• However, with altered feedback (gear) they are able to generate symmetrical movement of the flags and non-symmetrical movements of the hands.
• Again: The preference for symmetry is perceptual
• Figure from Mechsner et al. (2001)
Jerusalem in Motion
Dec, 2003
Amir Karniel
Bimanual Coordination (4)• The preference for symmetry was explained in terms of
stable solution of dynamic system without employing internal models.
• Following the vast literature about reaching movements we propose an alternative Hypothesis:
The brain contains internal representation of the transformation between the perceptual level and the
execution level in order to maintain the symmetry invariance in face of altered feedback or other
external perturbations.• Predictions: 1. Learning curves, 2. After effects
Jerusalem in Motion
Dec, 2003
Amir Karniel
Bimanual Index Tapping Bimanual Index Tapping ExperimentExperiment
• The right hand received slower feedback such that when the display shows rotation at equal speeds the subject eventually produces a non-harmonic polyrhythm, with a left/right tapping frequency ratio of 2/3
Jerusalem in Motion
Dec, 2003
Amir Karniel
Learning Curve Regression (Standardized Data)
-0.2 -0.1 0 0.1 0.2-0.2
-0.1
0
0.1
0.2
Time [min]
LOG
(L/R
Tap
ping
Rat
io E
rror
)Learning-Phase Ratio Errors
Confidence: 0.01
From: Karniel A, Klaiman E, and Yosef V, Society for Neuroscience 2003
Jerusalem in Motion
Dec, 2003
Amir Karniel
After-Effect IndicationsAfter-Effect IndicationsThe last 60 seconds of each half in the experiment
0 10 20 30 40 50 600
2
4
6
8
Firs
t ha
lf (A
sym
)Tapping Ratios in last 60 secs of Experiment
0 10 20 30 40 50 600
2
4
6
8
Time [sec]
Sec
ond
half
(Sym
)
Jerusalem in Motion
Dec, 2003
Amir Karniel
Bimanual Adaptation Hypothesis
• Symmetry Invariance
• Adaptable transformation from the perception level to the execution level
• After effects
• The structure, learning rates and generalization capabilities are subjects for future research
Jerusalem in Motion
Dec, 2003
Amir Karniel
Future Research
• Relative role of each level, muscles, spinal cord, central nervous system
• The structure of internal models (learning capabilities and generalization capabilities)
• Virtual Haptic Reality
• The Robo-Sapiens age
Mathematical Analysis, Simulation, Experiments
Jerusalem in Motion
Dec, 2003
Amir Karniel
Turing-like test for motor intelligence: The Robo-Sapiens age
Building a robot that would be indistinguishable from human being
