The Visual Computer manuscript No.(will be inserted by the editor)

Simulating the Task-level Control of Human Motion: A Methodology

and Framework for Implementation

Vincent De Sapio1, James Warren1, Oussama Khatib1, Scott Delp2

1 Artificial Intelligence LaboratoryComputer Science DepartmentStanford UniversityStanford, CA 94305, USAe-mail: {vdesap, warren, khatib}@robotics.stanford.edu

2 Neuromuscular Biomechanics LaboratoryMechanical Engineering and Bioengineering DepartmentsStanford UniversityStanford, CA 94305, USAe-mail: delp.stanford.edu

Received: date / Revised version: date

Abstract A task-level control framework is proposedfor providing feedback control in the simulation of goal-directed human motion. An operational space approach,adapted from the field of robotics, is used for this pur-pose. This approach is augmented by a significant newextension directed at addressing the control of muscle-driven systems. Task/posture decomposition is intrinsi-cally exploited, allowing human musculoskeletal proper-ties to direct postural behavior during the performanceof a task. This paper also describes a simulation architec-ture for generating musculoskeletal simulations of humancharacters. The evolving capabilities of the collective en-vironment are directed toward autonomously generatingrealistic motion control for virtual actors in interactivecomputer graphics applications, as well as synthesizingthe control of human-like motion in robotic systems.

Key words Human animation - Musculoskeletal dy-namics - Robotics - Task-level control

1 Introduction

In recent years there has been an emerging desire tosynthetically generate human-like motion in both sim-ulated and physical settings. In the computer graphicscommunity this desire is directed toward autonomouslygenerating realistic motion for virtual actors [5] [6] [29].

Send offprint requests to: V. De Sapio, Clark Center RoomE100, 318 Campus Dr., Stanford, CA 94305-5450

The intent is to direct these virtual actors using high-level goal directed commands for which low-level motioncontrol is automatically generated.

Motivated by similar desires, the robotics communityseeks a high-level control framework for robotic systems.With the recent advent of complex humanoid robots [19][25] this challenge grows more demanding. Consistentwith their anthropomorphic design, humanoid robots areintended to operate in a human-like manner within man-made environments and to promote interaction with theirbiological counterparts. Common control strategies in-volve generating joint space trajectories [19] or learningspecific motions [10], but these approaches require exten-sive motion planning computations and do not generalizewell to related tasks.

The challenge of synthesizing low-level human mo-tion control from high-level commands can be addressedby integrating approaches from the biomechanics androbotics communities. The biomechanics community hasinvestigated the phenomenon of neuromuscular dynam-ics and control through the use of computational musclemodels [3] [4] [31]. This characterization allows us todescribe muscle strength limitations, activation delays,and overall muscle contraction dynamics. Properly ac-counting for these characteristics is critical to authenti-cally simulating human motion since all human motion isrooted in, and bounded by, physiological capabilities. Ina complementary manner, the robotics community hasinvestigated the task-level feedback control of robots us-ing the operational space approach [12] [13] [15]. This ap-proach recasts the dynamics of the robotic system into arelevant task space description. This provides a naturalmechanism for specifying high-level motion commands

2 Vincent De Sapio et al.

that can be executed in real-time using feedback con-trol.

While the use of computational musculoskeletal mod-els and the task-level control of robots have been re-searched in the past, the integration of these approachesinto a unified framework offers a promising methodologyfor synthesizing goal-directed human-like motion con-trol. Recent work [15] addresses multi-objective task-level control of humanoid systems. However, the authorsdo no not address the biomechanical factors that influ-ence human motion control. Such factors, like muscu-loskeletal dynamics, form an important part of emulat-ing human motion in a synthetic environment.

This paper focusses on the biomechanical aspects ofhuman motion in order to provide a physiological frame-work for simulating human motion. We present a con-trol architecture that integrates elements of biomechani-cal and robotic approaches. This control architecture ex-tends the operational space approach by addressing thecontrol of muscle-driven systems. This extension com-plements other work done in operational space controlof humanoid systems [15] that addresses multiple taskobjectives but not the physiological basis for motion.

In addition to the new control framework this pa-per presents a simulation architecture for the task-levelcontrol of human-like motion. This consists of a simula-tion implementation that generates feed-forward muscu-loskeletal simulations of human-like characters. This isbuilt upon a core environment that has already been de-veloped for the physics-based simulation of robotic sys-tems. The evolving capabilities of the overall environ-ment are facilitated by an object-oriented design thatprovides a natural mechanism for connecting the mus-culoskeletal plant to task-level controllers. The environ-ment offers real-time simulation functionality allowingthe user to control and interact with many degree-of-freedom muscle-driven models in a task-level setting.

2 Previous Work

The operational space framework [12] [13] provides thepoint of departure for the task-level control approachthat will be addressed in this paper. This frameworkprovides a natural vocabulary for studying human mo-tion control. It does so by providing dynamic models atthe task level and structures for decoupled task and pos-ture control. This allows for posture objectives to be con-trolled without dynamically interfering with the primarytask(s). More recently a multi-objective formulation hasbeen proposed for the control of humanoid systems [15].The efficacy of the operational space framework has beendemonstrated for robotic systems; however, to date ithas only had limited application for biomechanical sys-tems [27]. Specifically, past work in formulating opera-tional space control, including the multi-objective formu-lation [15], makes no accommodation for muscle-driven

control. This paper extends the application of the oper-ational space approach to biomechanical simulations.

Several tools have emerged in the biomechanics com-munity for studying the dynamics and neuromuscularcontrol of movement. Many biomechanics labs have madeuse of SIMM (Software for Integrated MusculoskeletalModeling) [4], a modeling environment for defining skele-tal topology, muscle geometry, and joint kinematics. Whenused in conjunction with SD-FAST (Symbolic Dynam-ics) [9] to generate the multi-body equations of motionfor the skeletal system, the user can simulate the feed-forward dynamic response of the musculoskeletal systemto neural inputs. There are no native control capabilitiesin SIMM, so users must specify the control in open-loopusing their own feed-forward optimization routine [22] orin closed-loop using their own feedback control routine[28]. In addition to SIMM there have been several othermusculoskeletal simulation systems described in the lit-erature [8] [17] [18] [23].

3 Musculoskeletal Model

The musculoskeletal model presented here involves a de-scription of the skeletal system as a rigid multi-bodysystem spanned by a set of musculotendon actuators.The musculotendon actuator model consists of a stan-dard two state Hill-type model [30] [26]. A stiff tendonsimplification of the two state model will also be pre-sented.

While this musculoskeletal model has been broadlyused, Section 5 describes a unique integration of thismodel into a task-level control framework. This involvesencoding musculoskeletal parameters into a postural fieldto facilitate control.

3.1 Skeletal Dynamics

The skeletal system is modeled as a system of constrainedrigid bodies whose configuration is described by a setof generalized coordinates, q. These generalized coordi-nates are usually taken to be the joint angles betweenlimb segments but may also involve other displacementparameters. The n equations of motion can thus be rep-resented in standard form as,

Γ = A(q) q + b(q, q) + g(q) (1)

where Γ is the set of joint torques, A(q) is the systemmass matrix, b(q, q) is the vector of centrifugal and Cori-olis terms, and g(q) is the vector of gravity terms. Forconciseness we will often refrain from explicitly denot-ing the functional dependence of these quantities on q

and q. This practice will also be employed with otherquantities throughout the paper.

Simulating the Task-level Control of Human Motion: A Methodology and Framework for Implementation 3

3.2 Muscle Kinematics

Having defined a dynamical model of the skeletal systemwe may now assign a system of musculotendon actuatorsto it. For the purposes of this model we will assume thatall musculotendon lengths, l, can be uniquely determinedfrom the system configuration, q. That is, l = l(q). Asa consequence of this assumption differential variationsin l are given by,

δl = L(q) δq (2)

where L(q) is the muscle Jacobian. From the principleof virtual work we conclude,

Γ = −LT fT (3)

where fT is the vector of net muscle forces (active andpassive components). The negative sign is due to theconvention of taking contractile muscle forces as positive.

3.3 Neuromuscular Dynamics

Neuromuscular dynamics refers to the time evolution ofmuscle states (length and force) in response to neuralexcitation (control input) [11] [21]. This net dynamic ef-fect can be divided into activation dynamics and muscu-lotendon contraction dynamics. A Hill-type active statemodel [26] [30] has been used to model these effects.

3.3.1 Activation Dynamics Activation dynamics refersto the process of muscle activation in response to neuralexcitation. This can be modeled by the following equa-tion of state written in terms of the activation, a,

a =u − a

τ(u, a)(4)

where u ∈ [0, 1] is the neural input. The term τ(u, a) isa time constant given by,

τ(u, a) =

{(τa − τd)u + τd for u > a

τd for u < a(5)

where τa and τd are the activation and deactivation timeconstants respectively.

3.3.2 Contraction Dynamics Musculotendon contractiondynamics refers to the process of force generation in themuscle based on muscle contraction, rate of contraction,and activation. A number of different active state modelsexist. One such model is formulated with muscle length,lM , as the state [26] [30]. Figure 1 depicts the configura-tion of forces in this model. In addition to an active ele-ment there is a passive viscoelastic element (in parallel)and an elastic tendon element (in series). The relativeangle associated with the muscle fibers, α, is referred toas the pennation angle.

Fig. 1 Active state musculotendon model. The active con-tractile element and passive viscoelastic element are in par-allel. The passive elastic tendon element is in series.

Fig. 2 Force-length-velocity relationship showing muscleforce, fM , as a function of muscle length, lM , and rate of

contraction,lM (all quantities are normalized).

Based on the geometry of Figure 1 we have the fol-lowing relationships,

l(q) = lM cos α + lT

l(q, q) = lM cos α + lT(6)

and the following force equilibrium equation,

fT = (fA + fP ) cos α (7)

Using this force equilibrium equation in conjunction withconstitutive relationships (Figure 2) we can express anequation of state in the following functional form,

lM = lM (l(q), l(q, q), lM , a) (8)

4 Vincent De Sapio et al.

For a system of r musculotendon actuators we canexpress the following system of 2r first order state equa-tions,

a =

1/τ1 0

. . .

0 1/τr

u1 − a1

...ur − ar

(9)

and,lM = lM (l(q), l(q, q), lM ,a) (10)

3.4 Musculoskeletal Dynamics

Combining neuromuscular dynamics with multi-body skele-tal dynamics allows us to model the overall musculoskele-tal system. Given a set of neural inputs, u, a set of mus-cle activations, a, arise as dictated by the activation dy-namics. Based on these activations as well as the skeletalconfiguration, q and q, a set of muscle forces, fT , arise.These muscle forces are related to the joint torques, Γ ,through the muscle Jacobian, L. The skeletal dynamicsare driven by these joint torques, resulting in motion ofthe system. This overall process is described by the feed-forward path of Figure 3. This path, however, does notaddress the feedback mechanisms used by the centralnervous system to arrive at appropriate compensationfor controlling the musculoskeletal plant.

3.4.1 Stiff Tendon Model If we make the assumptionthat the tendon is infinitely stiff, lM is no longer an in-dependent state but rather is algebraically related to theoverall musculotendon length, l(q). This provides a use-ful simplification to the full musculoskeletal dynamicsdescribed in the previous sections. If we define the mus-cle saturation force, fS(q, q) , fA(q, q, 1), we can thendefine the muscle force-activation gain, Kf , as the mag-nitude of force generation in the muscle per change inunit activation. Thus,

Kf ,∂fT

∂a= fS cos α (11)

For a system of muscles the muscle force-activation gainmatrix is then,

Kf =

fS1

cos α1 0

. . .

0 fSrcos αr

(12)

and the muscle forces can now be expressed in terms ofthis gain,

fT (q, q,a) = fP (q, q) + Kf (q, q)a (13)

where,

fP =(fP1

cos α1 · · · fPrcos αr

)T(14)

As with the musculotendon forces it is useful to ex-press our generalized forces using a gain relationship.

To this end will define the muscle torque-activation gainmatrix,

KΓ , −LT Kf (15)

The generalized forces can now be expressed in terms ofthis gain,

Γ (q, q,a) = ΓP (q, q) + KΓ (q, q)a (16)

where,ΓP = −LT fP (17)

The overall musculoskeletal dynamics can then be ex-pressed as,

Γ (q, q,a) = A(q) q + b(q, q) + g(q) (18)

4 Task-Level Control Framework

Within the context of our control framework a task isany formal description of desired activity that can beexplicitly represented as a function of the generalizedcoordinates. This could be as simple as specifying theposition of a limb to be in a certain location. Multipletasks can be combined into a single task definition, aslong as they are kinematically consistent with each other.The task coordinates will be denoted by x = x(q). TheJacobian, J(q), of the task is defined as,

Jij ,∂xi

∂qj

(19)

Defining the dynamically consistent inverse of the Ja-cobian as [12] [13],

J , A−1JT (JA−1JT )−1 (20)

we can map the configuration space description of theskeletal dynamics (1) into an operational space descrip-tion,

JT

(Aq + b + g = Γ )

⇓

Λ(q) x + µ(q, q) + p(q) = f

(21)

This provides us a description of the dynamics in taskcoordinates rather than generalized coordinates (whilegeneralized coordinates are still present in (21), the in-ertial term involves task space accelerations rather thangeneralized accelerations). This is useful for control ap-plications since we can frame our control problem interms of the relevant task coordinates, x, that we wishto control using a relevant operational space force, f .The terms in (21) are defined as the operational spacemass matrix,

Λ(q) , (JA−1JT )−1 (22)

the operational space centrifugal-Coriolis vector,

µ(q, q) , JTb − ΛJq (23)

Simulating the Task-level Control of Human Motion: A Methodology and Framework for Implementation 5

Fig. 3 Musculoskeletal system (feed-forward path). Neural excitations provide input to the activation dynamics. Output ofthe activation dynamics provides input to the contraction dynamics. Output of the contraction dynamics provides input tothe skeletal dynamics through the joint torques.

and the operational space gravity vector,

p(q) , JTg (24)

We will always take the dimension of the task to beless than or equal to the dimension of the configurationspace. In the specific case that the dimension of the taskis strictly less than the dimension of the configurationspace we refer to the system as redundant with respectto the task (see Figure 4). In this case the operationalspace dynamics do not fully describe the system behav-ior and f does not fully describe the system control.However, the null space of the task is associated with acomplementary posture space of motion that does notinterfere with the task [13]. This posture space may beutilized to satisfy other criteria through the followingdecomposition of the control torques,

Γ = Γtask + Γposture = JTf + (I − JT J

T)Γ o (25)

A more concise expression of this is,

Γtask + Γposture = JTf + NT Γ o (26)

where,

NT , I − JT JT

(27)

Khatib et al. [15] consider a multi-objective formula-tion of operational space control, allowing for a hierarchyof separate tasks that may conflict with each other. Thismulti-objective approach will not be used for the expo-sition presented throughout the rest of this paper. Thebasic formulation summarized above will suffice, givenan assumption of non-conflicting tasks.

5 Task-Level Musculoskeletal Control

We now wish to apply a task/posture control decomposi-tion to our musculoskeletal system. This entails choosing

Fig. 4 A task description illustrating two correspondingtask consistent postures. Redundancy with respect to taskintroduces task dynamics as well as posture dynamics (sim-ulation sequence created by L. Sentis).

physiologically characteristic task and posture controlstructures that model certain aspects of the motion con-trol behavior of the central nervous system. Because weare choosing feedback control structures the system canbe designed to behave in a robust manner in the presenceof disturbances. This is important for real-time interac-tion with a physics-based simulation or a physical sys-tem. In the absence of such feedback mechanisms, feed-forward optimization approaches like dynamic optimiza-tion would be required. Such approaches have the draw-back of being computationally costly, thereby precludingreal-time execution. Additionally, due to the open-loopnature of the controls they generate, the behavior of thesystem is not robust in the presence of disturbances andexternal user interactions.

6 Vincent De Sapio et al.

5.1 Task Control

We will begin our task/posture control synthesis by choos-ing a task control structure. This entails designing a con-trol response, f . An artificial potential field, Ut(x), de-scribed in task coordinates serves as a straightforwardcontrol primitive. For example,

Ut(x) ,1

2(x − xo)

T Kx(x − xo) (28)

can be used as a simple quadratic potential field, relativeto a goal location, xo. Seeking to minimize this potentialwe generate the following control force,

f⋆ = −∇Ut = −Kx(x − xo) (29)

Adding a damping term we have,

f⋆ = −Kx(x − xo) − Kv x (30)

as the control input of the decoupled system. The gainmatrices, Kx and Kv, can be chosen to reflect physiologi-cal properties, such as natural limb endpoint impedances.Using estimates (denoted by · ) of the operational spacedynamic properties we have the following dynamic com-pensation expression,

f = Λf⋆ + µ + p (31)

This expression will be complemented by a posture ex-pression, to be developed in the following section.

5.2 Posture Control

We will complete our task/posture control synthesis bychoosing a posture control structure. This entails de-signing a control input, Γp, to serve as the null spaceterm, Γo, in (26). A potential field, Up(q), can be chosento represent any of a number of different physiologicalmeasures. Specifically, where these measures relate tomusculoskeletal properties we will refer them as muscu-lar effort measures. Our control then seeks to minimizeUp while satisfying the task.

The expression for Up(q) proposed in [16] involvesmuscle strength capacities projected at the joint leveland was given by,

Up(q) =r∑

i=1

wi

(gi(q)

ΓBi(q)

)2

(32)

where, depending on the sign of the gravity terms, gi,the term ΓBi

is either the upper or lower muscle in-duced torque boundary about a joint i. These terms arecomputed using the stiff tendon musculoskeletal modeldescribed in Section 3.4.1. The wi terms are weightingterms.

The physiological rationale for this measure was dis-cussed in detail in [16] and will only be briefly described

here. In this study a number of motion capture exper-iments were performed on human subjects. The exper-imental postures associated with a given reaching taskwere compared with other postures in the kinematic nullspace of the task. The set of these postures comprise theself-motion manifold associated with a fixed task point,x(q) = xo. This set is given by M(xo) , {q|x(q) = xo}.In [16] good correlation was observed between the exper-imental postures and the postures that minimized theeffort measure of (32).

It is noted that by projecting muscle strength ca-pacities to the joint level in a decoupled manner, as isdone in (32), the cross-joint coupling associated withmulti-articular muscles (muscles that span more thanone joint) is ignored. Consequently, in this paper we pro-pose an enhanced and more general muscle effort mea-sure to that which was proposed in [16]. This is givenby,

Up(q) = g(q)T (KΓ KTΓ )−1g(q) (33)

where KΓ is the muscle torque-activation gain matrixdefined in (15).

Using the measure of (33), experimental studies wereperformed and analyzed. Motion capture experimentsperformed on five subjects who were instructed to movedifferent weights to various target locations. Figure 5 de-picts the results of a particular subject trial. The effortassociated with the subject’s nominal configuration, qo,at the target can be compared with the effort associatedwith other configurations in a subset of the self-motionmanifold, q = [q

1, q

2], of the target.

Not all trials show as precise a correlation betweenthe subject’s nominal configuration and the minimumof the muscle effort curve as can be seen in the trialof Figure 5. However, for most trials the minimum ofthe effort curve is typically within several degrees of thesubject’s nominal configuration. More experiments areplanned; nevertheless, the results are promising in thatthey suggest good correlation between muscle effort min-imization and subject motion. The reader may refer to[16] for a more extensive discussion of experimental val-idation with human subjects.

In [16] the approach employed with the muscle effortmeasure (32) was effectively a kinematic one. While aprioritized task-level control framework is summarized,the authors do not explicitly integrate the musculoskele-tal model into the dynamics and control setting. Further,the analysis done in [16] is purely kinematic. By con-trast, we wish to apply an effort minimization approachto the control, where we seek to minimize the muscleeffort potential in a task consistent manner. To do thiswe generate the following posture control torque,

Γp = −∇Up (34)

Adding a damping term and gains we have,

Γp = −Kp ∇Up − Kd q (35)

Simulating the Task-level Control of Human Motion: A Methodology and Framework for Implementation 7

Fig. 5 Variation of muscle effort (33) over the self-motionmanifold for a given hand target location. The horizontalaxis represents variation from the subject’s nominal config-uration, qo, in degrees along the self-motion manifold. Formost trials the minimum of the effort curve is typically withinseveral degrees of the subject’s nominal configuration.

5.3 Unified Musculoskeletal Task/Posture Control

With the individual task and posture control structuresdefined we can implement the overall feedback controlstructure. In summary, our control torque is given by,

Γ = JTf + NT Γ p (36)

where the operational space compensation is,

f = Λf⋆ + µ + p (37)

and the control input of the decoupled system is,

f⋆ = −Kx(x − xo) − Kv x (38)

Finally, the posture term is,

Γp = −Kp ∇Up − Kd q (39)

where Up is the muscle effort measure given by (32), (33),or some other muscle based potential. Figure 6 depicts a

block diagram of the task-level neuromuscular controllerdescribed by equations (36) through (39).

Figure 7 depicts responses associated with the task-level neuromuscular controller of Figure 6. In this illus-trative example a muscle-driven redundant chain is con-trolled to move to a target. In one case the distal musclesare stronger than the proximal muscles. In the other casethe proximal muscles are stronger than the distal mus-cles. Different behaviors autonomously emerge from thecontroller based on these different physiological condi-tions.

The tight integration of our musculoskeletal modeland task-level control framework is unique in that itposes the control objective for the musculoskeletal plantin terms of a concise set of task coordinates rather thanthe much larger set of generalized coordinates. Addition-ally, while the musculoskeletal plant can be driven by in-dividual neural excitations, as is conventionally done, itsintegration with the task-level control framework allowsit to be controlled through a unique task/posture de-composition. In this way the posture controller encodesthe properties of the musculoskeletal plant in an efficientmanner, using a potential, without the need for specify-ing a large set of control inputs (neural excitations).

This approach also offers advantages over other ap-proaches. For example, a standard approach like dy-namic optimization exhibits a significant computationalburden since a large input space of neural excitationsmust be searched to drive the musculoskeletal plant overa fixed time interval. Additionally, the control is inher-ently open-loop as the feed-forward path of the muscu-loskeletal plant is repeatedly cycled in order to generatethe control inputs. This prohibits the simulation frombeing interactive and responding in a robust manner todisturbances.

6 Simulation Architecture

A simulation environment has been developed for simu-lating musculoskeletal dynamics and task-level control.This has been built upon a simulation and active in-terface framework (SAI), that is comprised of a set oflibraries developed to perform interactive simulation ofcomplex robotic systems [14].

6.1 Core Simulation Environment

At the core of SAI is a multi-body dynamics engine forsimulating kinematic chains. Additionally, collisions andcontact between objects are simulated. A haptics inter-face library allows for interaction with the system usinga force feedback device. Outside of the core dynamicsand interaction capabilities most of SAI’s infrastructureis devoted to algorithms for task-level planning and con-trol.

8 Vincent De Sapio et al.

Fig. 6 Task-level feedback control employing a neuromuscular criteria for posture control. The task servo block represents acontrol law in task coordinates. This can be defined as the gradient of a task potential field plus a damping term (38). Estimatesof the operational space dynamic parameters, Λ, µ, and p are used to provide the appropriate dynamic compensation, f (37).The posture control emerges in a task consistent manner where the posture servo block represents a neuromuscular basedcontrol law. This can be defined as the gradient of a muscle effort potential plus a damping term (39). The sum of the taskand posture terms is applied to the skeletal plant as a control input, Γ (36).

Fig. 7 A muscle-driven redundant chain controlled by the system of Figure 6. The case where the distal muscles are strongerthan the proximal ones is illustrated in (a). The controller drives the musculoskeletal plant to the target (task) from to to tf ina manner that minimizes the effort (Kx = 100, Kv = 20, Kp = .3, Kd = 3). The case where the proximal muscles are strongerthan the distal ones is illustrated in (b). Again, the controller drives the musculoskeletal plant to the target in a manner thatminimizes the effort (Kx = 100, Kv = 20, Kp = .4, Kd = 4).

Simulating the Task-level Control of Human Motion: A Methodology and Framework for Implementation 9

The multi-body dynamics component of SAI focuseson recursive algorithms [1] [7] [20] for computing thedynamics and control of n-joint branching, redundantrobotic mechanisms with m operational degrees of free-dom. The complexity of these algorithms scales accord-ing to O(nm+m3) as compared to O(n3 +m3) for manyexisting symbolic algorithms. Typically m is small com-pared to n which implies near linear scaling, O(n). Amuch more detailed description of the multi-body dy-namics algorithms used in SAI is given in [1].

The collision handling capabilities of SAI implementan impulse based approach for modeling rigid contactand collision. Objects in the simulated world are mod-eled as the union of convex polyhedra. A contact spaceis defined to describe the contact state between any twopolyhedra with a finite number of contact points. Whena contact or collision occurs a Jacobian and operationalspace mass matrix, defined with respect to the contactspace, are computed. In this approach only the true de-grees of freedom of the multi-body system are modeled,providing added efficiency by eliminating the internalconstraints of the system. The collision model employsa standard coefficient of restitution relation as the con-straint on velocities before and after collision. A muchmore detailed description of the collision approach usedin SAI, as well as its benefits over standard approaches,is given in [24].

While collision capability is useful in the general ap-plication of physics-based simulation we have not specif-ically exploited it in our task-level musculoskeletal con-trol approach. As a result, we have only provided a briefsummary of this core simulation capability and havenot discussed it within the specific context of task-levelmusculoskeletal control. Future applications of our mus-culoskeletal control approach may exploit the collisionand contact capabilities of SAI. These capabilities wouldbe particularly relevant for gait and contact force con-trollers, both of which would require force sensing infor-mation in the feedback loop. Figure 8 depicts simulationsin SAI illustrating its handling of multi-body dynamicsas well as collision.

6.2 Musculoskeletal Simulation Environment

The core SAI environment has been extended to incorpo-rate the simulation of muscle-driven systems. Characterscan be specified with a set of musculotendon actuators,modeled as described in Section 3. The environment al-lows feed-forward operation in which control inputs aresent to each of the muscles, as well as an implementa-tion of the task-level control methodology of Section 5.The object-oriented design employed provides a natu-ral mechanism for connecting the musculoskeletal plantto task-level controllers. Appendix A provides a detaileddescription of the class design.

The addition of muscle simulation capabilities to SAIis illustrated in Figure 9 which depicts a sequence from

a task-level control simulation. The model shown in thesequence consists of 30 joints and 144 muscles. Muscu-loskeletal data used in this model has been derived fromSIMM models [4]. The hands are defined as task controlpoints that can be moved interactively by the user. Abalancing task constraint is imposed as well.

Because the control of the human characters in thesesimulations is robust and executed in real-time (or nearreal-time) the user can interact with the simulation andapply external disturbances to the characters. Figure 10shows an example of disturbance forces interactively ap-plied by the user. The character’s controller applies com-pensation in response to the disturbance, resulting instable behavior of the character’s dynamics.

7 Conclusion and Future Work

We have presented a framework for the task-level controlof muscle-driven human motion. This framework inte-grates a musculoskeletal model and a task-level controlapproach; allowing the control objective for the muscu-loskeletal plant to be stated in terms of a natural set oftask coordinates. The unique task/posture decomposi-tion implements a posture controller that efficiently en-codes musculoskeletal properties into a posture poten-tial. The feedback nature of this framework inherentlyprovides stable response to disturbances and externalinteractions; as opposed to feed-forward approaches likedynamic optimization.

A simulation environment that implements our task-level musculoskeletal control framework has also beenpresented. This is built upon an efficient multi-body dy-namics engine. The overall environment provides a user-friendly interface for producing physically realistic task-driven human-like motions. We believe that such a sys-tem is useful in the study and synthesis of human-likemotion for computer graphics applications as well as thesynthesis of human-like motion control for anthropomor-phic robotic systems.

The current focus of our work has been on whole-body postural control rather than walking. There aregait controllers currently being tested for future use. Re-garding the muscle models; while the current system sim-ulates full muscle dynamics in the feed-forward path, astiff tendon muscle model is employed for control pur-poses. This simplifies the control problem but neglectssome important dynamical properties of muscle. Plansfor improvement of the system include the implementa-tion of other muscle control routines. This will include anoperational space version of the computed muscle con-trol method [28]. Such an approach involves perform-ing a task constrained minimization of muscular effortdefined in terms of muscle activations. The conceptualframework for this approach has been developed and animplementation is planned.

The implementation of constraint enforcement capa-bilities is planned as well. This will be useful for modeling

10 Vincent De Sapio et al.

Fig. 8 Sequences illustrating core simulation capabilities of SAI. Virtual acrobats interacting in a dynamic environmentsubject to realistic physical constraints (a). This SAI simulation sequence illustrates real-time multi-body dynamics andcollision response (simulation sequence created by F. Conti). Interaction with a virtual skeleton in SAI (b). The skeleton iscommanded to sit on the block. The user then pulls on the block with a virtual force causing the skeleton to summersault(simulation sequence created by V. De Sapio).

closed kinematic chains in addition to chains that involveconstraints between the generalized coordinates. Suchconstraints are particularly relevant in the case of thehuman shoulder complex and the knee joint. A controlmethodology has been developed to address such cases[2]. This makes use of a unique operational space de-composition that generates separate structures for taskmotion control and constraint force control.

The muscle path generation facilities of the SAI mus-culoskeletal environment do not include wrapping sur-faces, which are useful for modeling complex muscle rout-ing. SIMM includes these facilities and has been usedto augment the capabilities of SAI in this area. Conse-quently, an implementation of wrapping surfaces wouldbe a particularly useful addition to a future version ofthe SAI environment.

8 Acknowledgements

The contributions of Luis Sentis, Irena Paschenko, FrancoisConti, Diego Ruspini, and Kyong-Sok Chang to the SAIsimulation environment are gratefully acknowledged. Theauthors would also like to thank Katherine Holzbaurfor providing an enhanced model of the human shouldercomplex which was implemented in some of our work.

Simulating the Task-level Control of Human Motion: A Methodology and Framework for Implementation 11

Fig. 9 Task-level control of a simulated human in a dynamic setting. While the environment allows feed-forward operation inwhich control inputs are sent to each of the 144 muscles, a more efficient task-level control is used in this interactive sequence.Users can interactively move the control points, defined at the wrists, while the simulated human responds.

Fig. 10 Response of a human character to disturbance forces (vectors) interactively applied by the user. The feedbackcontroller applies compensation resulting in stable behavior of the system and a return to equilibrium. This is not possiblewith feed-forward simulation approaches that generate open-loop controls.

12 Vincent De Sapio et al.

Fig. 11 Muscle paths are generated as Catmull-Rom splinesthrough muscle origin, insertion, and via points.

A Musculoskeletal Class Design

Skeletal dynamics has been modeled in the SAI environ-ment using a set of link, joint, and robot classes. At thetop of the class hierarchy is a world class that containsa list of pointers to any number of robot objects. Eachrobot can be loaded individually into the world by theuser. An XML file structure is used for storing geometric,kinematic, and inertial properties for each robot.

Muscle mechanics has been modeled using a classhierarchy consisting of an activator class, a contractorclass, a muscle class, and a muscle system class. Collec-tively, the classes are used to define a system of control-lable musculotendon actuators. Each robot object pointsto a muscle system object. An XML file structure is usedfor storing properties for the muscles that are containedin each muscle system. A set of structures are used forstoring muscle and tendon lengths, velocities, and forces,as well as activation properties, contraction properties,and path data. The path structure contains a list of at-tachment points and a list of pointers to correspond-ing attachment links. There are also linear and cubicspline interpolation functions. The linear interpolationfunction is used for interpolating over the muscle force-length data. The spline interpolation function employsa Catmull-Rom spline algorithm for generating musclepaths (Figure 11).

The activator class encapsulates muscle activationdynamics. Each object contains activation properties dataspecific to an individual muscle. Based on a neural in-put, u, muscle activation, a, is computed at each timestep. Figure 12 depicts the block diagram for this class.

The contractor class encapsulates musculotendon con-traction dynamics. Each object contains musculotendonpath data and contraction properties data specific to an

Fig. 12 The activator class state model. Activation, a, isthe system state (and output) responding to neural input.

Fig. 13 The contractor class state model. Muscle length,lM , is the system state responding to activation input. Theoutput is the tendon force, fT .

individual muscle-tendon pair. The overall musculoten-don path length, l, is computed given the path data andthe current configuration of the skeleton, q. Based onan activation input, a, the force generated in the ten-don, fT , as well as active and passive forces generated inthe muscle, fA and fP , are computed at each time step.Figure 13 depicts the block diagram for this class.

The muscle class integrates the activator and con-tractor classes. It contains a pointer to an activator ob-ject and contractor object and thus represents a singlemusculotendon actuator (motor nucleus and associatedmuscle-tendon body). Methods are provided to interfaceto the activator and contractor objects. Figure 14 de-picts the block diagram for this class.

The muscle system class is composed of a vector ofpointers to individual muscle objects. There are meth-

Simulating the Task-level Control of Human Motion: A Methodology and Framework for Implementation 13

Fig. 14 The muscle class model consisting of pointers to a contractor and an activator object. A muscle object coordinatesactivation and muscle dynamics for a given muscle-tendon unit.

ods to coordinate the initialization and update of in-dividual muscle objects. Additionally, at each updatesystem-wide quantities are computed. These include themuscle Jacobian, L, and the vector of joint torques, Γ .The muscle Jacobian is computed given the path dataand the current configuration of the skeleton.

The set of classes previously described simulate feed-forward musculoskeletal dynamics. A set of controls (neu-ral excitations) can be specified as input to drive thefeed-forward dynamics. In practice these controls wouldbe produced by a dynamic optimization routine [22] or inclosed-loop by a feedback control routine [28]. To imple-ment the feedback control structure described in Section5, a set of classes have been implemented that employ thestiff tendon assumption described earlier. These are sim-plifications of the muscle classes already described, andare used specifically for task-level musculoskeletal con-trol. At each update additional system wide quantitiesare computed. These include the muscle force-activationgain matrix, Kf , and the muscle torque-activation gainmatrix, KΓ , as well as the muscle induced torque bound-aries ΓBi

. Various muscle effort measures, like the onesdescribed in Section 5.2, can be computed from thesesystem wide quantities.

References

1. Chang K-S, Khatib O (2000) Operational space dy-namics: efficient algorithms for modeling and control ofbranching mechanisms. In: Proceedings of the 2000 Inter-national Conference on Robotics and Automation. IEEE,1:850–856

2. De Sapio V, Khatib O (2005) Operational space controlof multibody systems with explicit holonomic constraints(accepted for publication). In: Proceedings of the 2005International Conference on Robotics and Automation.IEEE

3. Delp SL, Loan JP (1995) A software system to developand analyze models of musculoskeletal structures. Com-puters in Biology and Medicine 25:21–34

4. Delp SL, Loan JP (2000) A computational framework forsimulating and analyzing human and animal movement.IEEE Computing in Science and Engineering 2(5):46–55

5. Faloutsos P, van de Panne M, Terzopoulos D (2003)Autonomous reactive control for simulated humanoids.In: Proceedings of the 2003 International Conference onRobotics and Automation. IEEE, 1:917–924

6. Faloutsos P, van de Panne M, Terzopoulos D (2001)Composable controllers for physics-based character an-imation. In: Proceedings of the 2001 SIGGRAPH Con-ference. ACM, pp 251–260

7. Featherstone R (1987) Robot dynamics algorithms.Kluwer, Boston

8. Hase K, Miyashita K, Ok S, Arakawa Y (2003) Humangait simulation with a neuromusculoskeletal model andevolutionary computation. Journal of Visualization andComputer Animation 14(2):73–92

9. Hollars M, Rosenthal D, Sherman M (1991) SD/Fastusers manual. Technical report, Symbolic Dynamics In-corporated

10. Ijspeert A, Nakanishi J, Schaal S (2002) Movement im-itation with nonlinear dynamical systems in humanoidrobots. In: Proceedings of the 2002 International Confer-ence on Robotics and Automation. IEEE, 2:1398–1403

11. Kandel ER, Schwartz JH, Jessell TM (2000) Principlesof neural science. 4th edition, McGraw-Hill, New York

12. Khatib O (1987) A unified approach to motion and forcecontrol of robot manipulators: the operational space for-mulation. International Journal of Robotics Research3(1):43–53

13. Khatib (1995) Inertial properties in robotic manipula-tion: An object level framework. International Journalof Robotics Research 14(1):19–36

14. Khatib O, Brock O, Chang K-S, Conti F, Ruspini D, Sen-tis L (2002) Robotics and interactive simulation. Com-municatons of the ACM 45(3):46–51

15. Khatib O, Sentis L, Park J, Warren J (2004) Whole bodydynamic behavior and control of human-like robots. In-ternational Journal of Humanoid Robotics 1(1):29–43

16. Khatib O, Warren J, De Sapio V, Sentis L (2004)Human-like motion from physiologically-based potentialenergies. In: Lenarcic J, Galletti C (eds) On advances inrobot kinematics. Kluwer, Boston, pp 149–163

17. Komura T, Shinagawa Y, Kunii TL (1999) Calculationand visualization of the dynamic ability of the human

14 Vincent De Sapio et al.

body. Journal of Visualization and Computer Animation10(2):57–78

18. Komura T, Shinagawa Y, Kunii TL (2000) Creating andretargetting motion by the musculoskeletal human bodymodel. The Visual Computer 16(5):254–270

19. Kuroki Y, Blank B, Mikami T, Mayeux P, MiyamotoA, Playter R, Nagasaka K, Raibert M, Nagano M, Yam-aguchi J (2003) Motion creating system for a small bipedentertainment robot. In: Proceedings of the 2003 Inter-national Conference on Intelligent Robots and Systems.IEEE/RSJ, 2:1394-1399

20. Lilly KW (1992) Efficient dynamics simulation of roboticmechanisms. Kluwer, Boston

21. McMahon TA (1984) Muscles, reflexes, and locomotion.Princeton

22. Pandy MG, Anderson FC, Hull DG (1992) A parameteroptimization approach for the optimal control of large-scale musculoskeletal systems. Journal of BiomechanicalEngineering 114:450–460

23. Rasmussen J, Damsgaard M, Surma E, Christensen S,de Zee M (2003) Designing a general software system formusculoskeletal analysis. In: Proceedings of the Interna-tional Symposium on Computer Simulation in Biome-chanics. International Society of Biomechanics

24. Ruspini D, Khatib O (2000) A framework for multi-contact multi-body dynamic simulation and haptic dis-play. In: Proceedings of the 2000 International Con-ference on Intelligent Robots and Systems. IEEE/RSJ,2:1322-1327

25. Sakagami Y, Watanabe R, Aoyama C, Matsunaga S, Hi-gaki N, Fujimura K (2002) The intelligent asimo: systemoverview and integration. In: Proceedings of the 2002International Conference on Intelligent Robots and Sys-tems. IEEE/RSJ, 3:2478-2483

26. Schutte LM (1992) Using musculsokeletal models to ex-plore strategies for improving performance in electri-cal stimulation-induced leg cycle ergometry. PhD thesis,Stanford University

27. Thelen DG, Anderson FC (2001) An operational spacetracking algorithm for generating dynamic simulationsof movement. In: Proceedings of the 5th InternationalSymposium on Computer Methods in Biomechanics andBiomedical Engineering. International Society of Biome-chanics

28. Thelen DG, Anderson FC, Delp SL (2003) Generatingdynamic simulations of movement using computed mus-cle control. Journal of Biomechanics 36:321–328

29. Yamane K (2004) Simulating and generating motions ofhuman figures. Springer, New York

30. Zajac FE (1989) Critical reviews in biomedical engineer-ing. In: Bourne JR (ed), Muscle and tendon: properties,models, scaling, and application to biomechanics and mo-tor control. CRC Press, 17(4)359–411

31. Zajac FE (1993) Muscle coordination of movement: aperspective. Journal of Biomechanics 26:109–124

VINCENT DE SAPIO is aPh.D. student in the Artifi-cial Intelligence Laboratoryat Stanford University and aSenior Member of the Tech-nical Staff at Sandia NationalLaboratories. He received aB.S. degree from RensselaerPolytechnic Institute in 1990and an M.S. degree from Stan-ford University in 1994, bothin Mechanical Engineering.He is currently investigating

task-level control within the context of biomechanicalsystems. This involves the implementation of neuromus-cular models in conjunction with a task/posture controldecomposition. The goal is to emulate aspects of humanmotor control in robotic systems.

JAMES WARREN is a Ph.D.student in the Artificial In-telligence Laboratory at Stan-ford University. He receivedhis B.S. degree in Mathe-matics from Texas A&M Uni-versity in 1996 and his M.S.degree in Scientific Comput-ing and Computational Math-ematics from Stanford Uni-versity in 2000. His researchis in identifying human mo-tion behaviors from motion

capture data and applying these strategies for humanoidrobotic control.

OUSSAMA KHATIB is Pro-fessor of Computer Scienceat Stanford University. Hereceived his Ph.D. in 1980from Sup’Aero, Toulouse, France.His current research is in human-centered robotics, human mo-tion synthesis, humanoid robotics,dynamic simulation, hapticinterfaces, and human-friendlyrobot design. This builds upona large body of work pursuedover the past 25 years and

published in over 200 contributions in the field. ProfessorKhatib was Program Chair of ICRA2000 and editor ofThe Robotics Review. He served as director of the Stan-ford Computer Forum and is currently president of theInternational Foundation of Robotics Research and ed-itor of Springer Tracts in Advanced Robotics. ProfessorKhatib is an IEEE Fellow, a Distinguished Lecturer ofIEEE, and a recipient of the JARA Award.

Simulating the Task-level Control of Human Motion: A Methodology and Framework for Implementation 15

SCOTT DELP is Professorand Chair of the Bioengi-neering Department at Stan-ford University. After receiv-ing the Ph.D. degree fromStanford University in 1990,he joined the faculty of North-western University, where hewas jointly appointed in theSchools of Engineering andMedicine. He returned to Stan-ford in 1999 in the depart-ments of Mechanical Engi-

neering and Bioengineering. His work draws on com-putational mechanics, biomedical imaging, and neuro-muscular biology to study human movement. He has re-ceived numerous awards including a National Young In-vestigator Award from the National Science Foundationand Faculty Fellowships from the Baxter Foundation andPowell Foundation. He is a Fellow of the American In-stitute of Medical and Biological Engineering.