On the Use of Homogeneous Transformations to MapHuman Hand Movements onto Robotic Hands

G. Salvietti1, M. Malvezzi2, G. Gioioso1,2, and D. Prattichizzo1,2

Abstract— Replicating the human hand capabilities is agreat challenge in telemanipulation as well as in autonomousgrasping and manipulation. One of the main issue is relatedto the difference between human and robotic hand in terms ofkinematic structure, which does not allow a direct correlationof the joints. We recently proposed an object-based mappingalgorithm able to replicate on several robotic hand models thehuman hand synergies. In such approach the virtual objectshapes were a-priori defined (e.g. a sphere or an ellipsoid),and the transformation was represented as the composition ofa rigid body motion and a scale variation. In this work weintroduce a generalization of the object-based mapping, thatovercomes the definition of a shape for the virtual object. Weconsider only a set of reference points on the hands. We estimatea homogeneous transformation matrix that represents how thehuman hand motion changes its reference point positions. Thesame transformation is then imposed to the reference pointson the robotic hand and the joints values obtained througha kinematic inversion technique. The mapping approach issuitable also for telemanipulation scenarios where the handjoint motions are combined with a wrist displacement.

I. INTRODUCTION

Robotic hands present a high variability of kinematicstructures, actuation and control systems. They differ in thenumber of fingers, in the number of Degrees of Freedom(DoFs) per finger, in the type of joints and actuators, etc.[1]. In most of the telemanipulation scenarios, however, themotion of such a heterogeneous set of devices is related to themotion of a unique complex kinematic structure: the humanhand. This has led to the development of several mappingstrategies that strongly depend on the robotic hand structures.Examples of these approaches are the fingertip mapping[2], the pose mapping [3] and the joint-to-joint mapping[4]. The main drawbacks of these methods are mainly thelack of generality and the need of empirical or heuristicconsiderations to define the correspondence between humanand robotic hands.

To overcome such limits, in [5] we presented a mappingdefined in the task space and mediated by a virtual object (asphere). The method was detailed in [6] and generalized in[7], considering an ellipsoid as virtual object and extendingthe possible transformations that can be imposed to it. Theobject-based mapping is obtained considering two virtualobjects, one on the human and one on robotic hand. They arecomputed considering the minimum volume object contain-ing reference points suitably defined, placed on the respective

1Department of Advanced Robotics, Istituto Italiano di Tecnologia, ViaMorego 30, 16163 Genoa, Italy. gionata.salvietti@iit.it

2Universita degli Studi di Siena, Dipartimento di Ingegneriadell’Informazione, Via Roma 56, 53100 Siena, Italy. {malvezzi,gioioso, prattichizzo}@dii.unisi.it

Fig. 1: Mapping in the task space, main idea. The algorithmtries to reproduce on the robotic hand the same homogeneoustransformation induced by a motion on some referencepoints placed on the human hand model, defines here as theparadigmatic hand.

hands. A configuration variation of the human hand induces amotion and a deformation of the virtual object. Then we im-posed that the object defined on the robotic hand moves anddeforms according to that defined on the human hand. Themapped motion of the robotic hand is then obtained throughpseudo-inversion techniques. This mapping procedure, wasadopted to map human hand synergies [8] onto hands withvery dissimilar kinematics. However, the definition of a shapefor the virtual object implies that the method obtains betterresults if the manipulated object has a shape similar tothe virtual one. Moreover, sharing deformation cannot bereproduced and only in-hand manipulation has been testedso far, since the few available parameters are not sufficientto describe both joints and wrist motions.

In this paper we introduce a new solution which overcomesthe definition of a virtual object shape and allows replicatingthe motion of the human hand in a telemanipulation scenariowhere also wrist motion is considered. The mapping is basedon the definition of a series of reference points, both onthe human and on the robotic hand: the reference points onthe human hand are necessary to evaluate the transformationproduced by the hand motion, the points on the robotichands are necessary to map such transformation on therobotic hand. A configuration change on the human handcauses a transformation of the reference point positions,which can be generally represented by a six-dimensionaldisplacement and/or a non rigid deformation. In this paper,we assume that this transformation can be represented asa linear transformation, estimated from the displacement ofthe reference points. The same linear transformation is thenimposed to the robotic hand reference points and the handjoint displacement is consequently defined by solving its

inverse kinematics.A linear transformation matrix can be, in general, de-

composed as the combination of different elementary trans-formations, e.g., translation and rotations, scale variationsand shear deformations. Such decomposition is adopted inthe proposed mapping procedure to separately reproducethe contribution in terms of internal forces [9], which areparamount for grasp control, and in terms of the rigidbody displacement imposed by the hand on the manipulatedobject [10].

We simulated two possible scenarios to test the appli-cability and effectiveness of the proposed approach. In thefirst one, human hand synergies are reproduced in a modelof a three fingered hand to demonstrate the capability ofthe method to capture in-hand motion. In the second ateleoperation task is simulated involving both hand and wristmotion.

The paper is organized as follows: Section II summa-rizes the properties of homogeneous transformations andrecalls the definition of primitive transformations, Section IIIdescribes the procedure to map human hand movementsto robotic hands, based on homogeneous transformations,Section IV shows how the proposed procedure works withsome numerical tests, in which a robotic hand with a nonanthropomorphic structure is simulated, finally Section Vprovides some concluding considerations and describes fur-ther developments of the work.

II. THE HOMOGENEOUS TRANSFORMATION PROPERTIES

The mapping procedure is based on the analysis of thetransformation of a set of reference points on the humanhand during hand motion. In the paper we assume that thetransformation is linear, so, indicating with pi and pf thecoordinates of a generic reference point in its initial andfinal configuration, respectively, we have that

pf = Api + b,

where A is a 3 × 3 matrix representing the linear map andb is a three-dimensional vector representing the translationin the transformation. Introducing the augmented matrix andvector notation, it is possible to represent both the translationand the linear map using a single matrix multiplication, i.e.

pf = T pi,

where pi,f =[pTi,f 1

]Tand

T =

[A p

0 0 0 1

]. (1)

.Homogeneous 4 × 4 matrices are widely used in 3D

computer graphics system to represent solid bodies trans-formations [11]. Homogeneous transformations are able torepresent all the transformations required to move an objectand visualize it: translation, rotation, scale, shear, and per-spective. Any number of transformations can be multiplied toform a composite matrix. Transformation matrices are widelyadopted also in continuum mechanics to describe material

displacements and strains, and methods to decompose fromthe deformation gradient the contribution of rigid body mo-tion and non rigid deformation are available in the literature,see for instance [12], [13].

Rigid body motions are particular types of transformationthat preserve the distance between points and the anglesbetween vectors. They can be represented as the combinationof a rotation, defined by the rotation matrix R ∈ SO(3), anda translation motion, defined by the vector p ∈ <3. SO(3)(special orthogonal) is the set of all the 3 × 3 orthogonalmatrices with determinant equal to 1 [14]. The correspondinghomogeneous matrix can be expressed as shown in Fig. 2-a).T is in this case a representation of a generic element of theSE(3) group (special Euclidean).

Homogeneous matrices can be adopted also to describenon rigid transformations: isotropic transformations, whichmodifies the object size by a scaling factor α, withoutmoving it; non isotropic transformations, which modify theobject size by scaling factors [α, β, γ], in the x, y, z direc-tions respectively, and shear transformations, that displaceseach point in fixed direction, by an amount proportionalto its signed distance from a line that is parallel to thatdirection. A generic non rigid transformation is qualitativelyrepresented in Fig. 2-b). In this study we do not considerperspective transformations for the sake of simplicity. Thesebasic homogeneous transformations are usually referred to asprimitive transformations. Each of them can be representedwith a more meaningful and concise representation: a scalarfor the isotropic transformation, a vector for translation, 3Dscaling and shear, a quaternion for rotations. The recover ofthe concise form from the primitive transformation matrix isstraightforward, but, on the other hand, once primitives havebeen multiplied into a composite matrix, the recovery of theprimitive is not direct in general [15].

Different procedures to decompose a generic 4× 4 matrixinto a series of primitive transformations are available inthe literature. In this paper we consider the extractionof the rigid and non rigid motions contributions from ageneric linear transformation matrix. Consider, for instance,the human hand/arm system moving along a trajectory whilethe hand is changing the grasp forces exerted onto an object.In this case a large rigid arm displacement is coupled witha smaller non rigid deformation. The displacements of thereference points on the human hand however contains boththe contributions. So that, in the mapping procedure, whenthe arm and wrist is involved in the motion, we proposeto extract the rigid part of the motion from the completetransformation matrix and to reproduce it with the roboticarm, while the non rigid contribution to the reference pointconfiguration variation is reproduced acting on the robotichand fingers. This decomposition will be be better explainedin the second numerical experiment proposed in Sec. IV.

In this type of application we need to express the trans-formation matrix as follows

T = TdefTrb (2)

where Trb = TtrTrot represents the rigid part of the displace-

Trb =

[R p0 1

]Tdef =

1 hxy hxz 0hyx 1 hyz 0hzx hzy 1 00 0 0 1

a) b)

Fig. 2: Examples of linear transformations applied to a cubeand corresponding homogeneous matrices: a) rigid bodymotion, b) shear and scale transformation.

ment, composed of a rotation and a translation, and Tdeftakes into account the non rigid deformation, as shown inFig. 2-a) and b) respectively. The extraction of the translationpart of the rigid body motion from the starting matrix T isstraightforward considering eq. (1). The matrix A in eq. (1)can be written, with the polar decomposition, as the product

A = UR (3)

in which R is orthogonal and U is an Hermitian semi-positivematrix, R ∈ SO(3) represents a rigid rotation, while U takesinto account the non rigid deformation [16].

III. MAPPING PROCEDURE BASED ON HOMOGENEOUSTRANSFORMATION

The considered reference hand is a model of the humanhand. The definition of the following reference frames isnecessary to identify hand motion. Let {Nw} be an inertialreference frame, adopted to describe hand/wrist motion. Let{Nh} indicate a frame on the human hand palm. Theconfiguration of {Nh} reference frame with respect to theinertial one depends on the arm motion, and is described bythe homogeneous transformation matrix Th, which dependson six parameters, namely the coordinates of {Nh} originand the relative orientation between the frames, described forinstance by Euler angles. Let pw ∈ <6 be a vector containing{Nh} position and orientation with respect to {Nw}.

Various kinematic models of the human hand are availablein the literature, we chose a 20 DoFs model, in which eachfinger has four DoFs [17]. Let us indicate with qh ∈ <nqh ,with nqh = 20, the generic configuration of hand joints.

The number and position of the reference points on thehuman hand, nh, can be arbitrarily set. Reference points canbe placed on the fingertips, in the intermediate phalanges, incorrespondence of the joint axis, etc. The fingertips representa natural choice for the reference points, since they are atthe end of the kinematic chains that define the fingers, sotheir configuration depends on all the joints [6].

Let us indicate with pk, with k = 1, · · · , nh the referencepoints on the human hand. The vector phk,c ∈ <3 represents

the coordinates of the generic reference point pk with respectto {Nh} when the hand assumes a configuration C and thejoint values are qhc . Let furthermore us indicate with phk,c ∈<4 the corresponding augmented vector, adopted to representaffine transformations, i.e. phk,c = [phk,c

T1]T. Finally, let us

indicate with phc ∈ <3nh a vector containing the coordinatesof all the reference points in the generic configuration C.

Let phi denote the initial position of the reference points onthe human hand. Their position is a function of hand initialconfiguration vector qhi and the wrist initial configurationphi,w, and can be evaluated by the direct kinematic analysisof the hand, i.e.

phi = fk(qhi , phi,w). (4)

Let us then assume that, starting from this initial configura-tion, the hand and the wrist are moved, let qhf and phf,w bethe final hand joint and wrist configurations. The referencepoint positions on the human hand vary according to handand wrist kinematics, i.e. phf = fk(qhf , p

hf,w), [18].

We assume that the configuration variation of the refer-ence points from phi to phf can be represented as a lineartransformation, i. e., for each point pk, with k = 1, · · · , nh,the following linear relationship can be written as

phk,f = T phk,i. (5)

Given the initial and final reference point configurations phk,iand phk,f , we can evaluate the linear transformation T ineq. (5) by solving the following linear system

phk,f = Mt (6)

where t ∈ <12, the unknown vector, contains the componentsof the linear transformation T , i.e.:

T =

t1 t2 t3 t4t5 t6 t7 t8t9 t10 t11 t120 0 0 1

,and the system matrix M ∈ <3nh×12 is defined as

M =

M1

· · ·Mnh

,in which the generic matrix Mk ∈ <3×12 is given by

Mk =

ph Tk,i 01,4 01,4

01,4 ph Tk,i 01,4

01,4 01,4 ph Tk,i

.As already mentioned in Sec. II, T matrix can then be de-

composed as the product between a rigid body transformationmatrix Trb and a non rigid transformation Tdef .

The idea behind the proposed mapping procedure is thento reproduce, on the reference points defined on the robotichand, the same linear transformation computed on the humanhand. Note that the homogeneous transformation matrix Tobtained by solving the linear system in eq. (6) depends onthe human hand and wrist configuration variation, and alsoon the initial configuration of reference points phi .

Let us indicate with {Nw,r} an inertial reference frame,adopted to describe robotic hand and arm motion, and {Nr}a reference frame on the robotic hand palm, let pw,r ∈ <6

be a vector describing the position and orientation of frame{Nr} with respect to {Nw,r}, and let qrc ∈ <nqr indicatethe robotic hand joint vector. In general, since the robotichand has a kinematic structure different from the human one,nqr 6= nqh .

A set of reference points are defined also on the robotichand, indicated with P r

k , with k = 1, · · · , nr. In general, nhand nr are not related and nh 6= nr.

In the initial reference configuration the coordinates ofthe reference points on the robotic hand are defined by thevectors prk,i, that can be collected in the vector pri ∈ <3nr .The final configuration of these points, according to theabove defined linear transformation, can be evaluated as thecomposition of two motions

prk,f = TdefTrbprk,i = Tdef p

rk,rb. (7)

The reference point configurations after the rigid transfor-mation can be collected in the vector prrb ∈ <3nr , whileprf ∈ <3nr contains the final reference point configurations.The displacement vector ∆pr due to the non rigid part ofthe transformation is thus defined as

∆pr = prf − prrb.

This displacement has to be reproduced by modifying therobotic hand joint values, according to robotic hand inversekinematics. If the displacement ∆pr is sufficiently small, thelinear approximation of the kinematics of the robot can beconsidered. Consequently, the displacement that has to beimposed to the robotic hand joints, can be evaluated as

∆qr = J#r ∆pr +NJrα (8)

where Jr is the robotic hand Jacobian matrix, the index # de-notes a generic pseudoinverse, NJr

is a basis of Jr nullspaceand α is a vector parametrizing the homogeneous part ofthe inverse differential kinematics problem and managing thepresence of eventual redundant hand DoF [14]. Robotic handjoint variation ∆qr is the displacement that has to be imposedto the robotic hand joints in order to obtain, on the robotichand reference points, the same linear transformation of thereference points on the human hand.

IV. NUMERICAL EXPERIMENTS

A. Mapping human hand synergies on a three fingeredrobotic hand

In the first part of numerical experiments we consideredonly in–hand motions and we did not take into account wristmotions. We simulated the mapping of human hand synergiesonto a robotic hand with a non anthropomorphic structure.Postural synergies represent a way to simplify human handstructure, reducing the number of DoFs necessary to defineits posture [19]. The synergy idea has been recently broughtto robotics, to reduce the number of inputs necessary toactuate a robotic hand, thus simplifying their mechanical and

a) b)

Fig. 3: Models of the human and robotic hands for theanalysis of mapping procedure, a) 20 Dof human handmodel, b) modular hand model.

control structure [20], [8]. Mapping human hand synergieson robotic hands with different kinematics in the task spaceusing the definition of a virtual object was analyzes anddiscussed in [6].

The considered robotic hand is a the three fingered mod-ular hand composed of three identical planar fingers, eachof them has three joints [7]. Globally the hand has 9 DoFs.Each finger can be represented as a three DoFs planar robot,so the motion of all the points on each finger is on a plane,normal to the revolute joint axes.

The numerical experiments were performed using andadapting the functions available in SynGrasp, a MatlabToolbox for the simulation and the analysis of grasping withseveral hand models [21]. This tool includes functions forthe definition of hand kinematic structure and of the contactpoints with a grasped object, the coupling between jointsinduced by a synergistic control, compliance at the contact,joint and actuator levels. Its analysis functions can be used toinvestigate the main grasp properties: controllable forces andobject displacements, manipulability analysis, grasp qualitymeasures. Furthermore, functions for the graphical represen-tation of the hand, the object and the main analysis resultsare provided. The SynGrasp numerical models of the humanand robotic hand adopted in the simulations are shown inFig. 3.

We considered the first two synergies. We activated onesynergy per time and we mapped the corresponding motionto the robotic hand. In synergy actuated hands, joint variablevariations are constrained by a linear relationship

∆qh = S∆z, (9)

in which S ∈ <nq×nz is the synergy matrix and ∆z ∈ <nz

represents the synergy input values. Synergy matrix for theused human hand model is available in the SynGrasp toolboxand has been computed from the data collected in [19].A mathematical model of synergy actuated hands, and ananalysis on the role of synergies in the controllability of forceand object motions in grasp tasks were presented in [22].

We chose the fingertips of the first four fingers as referencepoints on the human hand. For the robotic hand we assumedthe three fingertips. The dimensions of contact point vectors

synergy 1 synergy 2

−20 −10 0 10 20 30

−30

−25

−20

−15

−10

−5

0

5

10

15

x [mm]

y [m

m]

−30 −20 −10 0 10 20 30

−30

−20

−10

0

10

20

x [mm]

y [m

m]

Fig. 4: Mapping the first and the second synergy: the humanhand. First row: human hand displacement. Second row:displacement directions of the reference points on the humanhand, projection on the xy plane, the blue vectors representthe reference point displacements, the black arrow the objectcenter displacement.

are phi ∈ <12 and pri ∈ <9 for the human and robotic hand,respectively. From eq. (6), it results that M ∈ <12×12. Asstarting configuration of the human hand, qhi we assumed themean configuration obtained from the provided experimentalmeasures in [19].

By applying the mapping procedure detailed in Sec. III,human hand synergies were mapped on the robotic hand.Fig. 4 shows the human hand motion and the displacementsof the reference points when the first two synergies areactuated. Fig. 5 reports the corresponding robotic handmovements and reference points displacements. In particular,the red arrows represent the displacement evaluated bythe homogeneous transformation, while the blue ones thedisplacements that can be effectively reproduced by the hand,due to its kinematic structure, that constraints the motion ofeach reference point on a plane.

The second simulation involves grasping. The human androbotic hands grasp an object and the reference points are thecontact points with the object. The objects in this simulationis defined through its contact points and no specification onshape and size are reported. The human hand is actuated inorder to produce a rigid body motion of the grasped object,and the corresponding motion of the object on the robotichand obtained with the proposed mapping is analyzed. Rigidbody motions are displacements of the grasped object thatdo not involve any variations of the contact forces. A morecomplete description of this type of motions is detailed in[20], [22], where it was also demonstrated that the subspacedimension of the controllable rigid body motions depends

synergy 1 synergy 2

−50 −40 −30 −20 −10 0 10 20 30 40 50−50

−40

−30

−20

−10

0

10

20

30

x [mm]

y [m

m]

−50 −40 −30 −20 −10 0 10 20 30 40 50

−40

−30

−20

−10

0

10

20

30

x [mm]

y [m

m]

Fig. 5: Mapping the first and the second synergy: the robotichand. First column: displacement of the robotic hand. Secondrow: robotic hand reference points: the red vectors representthe displacement directions predicted by the homogeneousmatrix, the blue ones the effective displacements that therobotic hand can reproduce with its kinematics.

−30 −20 −10 0 10 20 30−35

−30

−25

−20

−15

−10

−5

0

5

10

15

x [mm]

y [m

m]

−50 −40 −30 −20 −10 0 10 20 30 40 50−50

−40

−30

−20

−10

0

10

20

30

x [mm]

y [m

m]

a) b)

Fig. 6: Mapping the rigid body motion on the robotic hand,a) human hand reference point (blue) and object center(black) displacement directions, b) robotic hand referencepoint displacement directions, red arrows represents thedisplacements evaluated by the homogeneous transformation,the blue ones the displacement effectively reachable by therobotic hand.

on the actuation and on the number and type of contacts.Let us assume that the human hand is grasping an objectthrough the four contact points phi and that it moves it bymaintaining invariant the contact forces. According to [20],it results that the internal force subspace dimensions is ne =6. Consequently, the minimum number of inputs necessaryto generate at least one rigid body motion is nz = 7. Thesynergy matrix considered in this simulation is then S ∈<20×7 and is given by the first seven columns of the completesynergy matrix.

In this configuration, the vector of synergies that allowsto produce on the human hand model a rigid body motion,evaluated according to the procedure detailed in [20] is

∆zrb = [0.1, 0.47,−0.13, 0.29, 0.23, 0.34, 0.71]T

0 1 2 3 4 5 6 70

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

rotation [rad]

erro

r

norm errordirection error [rad]

Fig. 7: Difference between the displacement of the robotichand object center evaluated by the homogeneous transfor-mation and reached by the robotic hand.

For the sake of simplicity, the object centers in thehuman and robotic hands are defined as the hypotheticalcenters of mass of the reference points, evaluated assuminga unitary mass for each point. Fig. 6 shows human androbotic reference points and object centers displacementswhen an object rigid body motion is imposed to the humanhand. In the robotic hand diagram, the red vectors repre-sents the displacements of the reference points predicted bythe homogeneous transformation evaluated on the basis ofhuman hand displacement, while the blue ones represent thedisplacements that can be effectively reached by the robotichand, due to its kinematic structure. It is evident that robotichand reference points are constrained to move on a plane,and then the required displacements, necessary to realize thetransformation, cannot be realized. For this reason the actualobject displacement, on the robotic hand, is different from thenominal one. More precisely, the displacement evaluated bythe homogeneous transformation is (mm): ∆orh = [−1.62 −4.10 −1.84]T while the displacement obtained by the robotichand is (mm): ∆or = [0.31 −1.56 −1.68]T. The differencebetween displacement norms, normalized with respect tothe first one, is 0.51, i.e. the actual displacement norm isabout one half of the value predicted by the homogeneoustransformation matrix, while the angle between displacementvectors is 0.62 rad.

The difference between the transformation predicted bythe homogeneous transformation and the reached one de-pends on the kinematic limits of the robotic hand is sen-sitive to the relative configuration between the hands. Asan example, Fig. 7 shows the object center displacementerror obtained by rotating the robotic hand with respect to zaxis, while keeping the human hand in the same orientation.The blue curve in the diagram represents the differencebetween the norms of the vectors, normalized with respectto the nominal displacement evaluated by the homogeneoustransformation, while the green curve represents the angle(expressed in radians) between displacement vectors.

a) b)

Fig. 8: a) Human hand and three–fingered robotic handgrasping a cubic object: initial configuration b) object center(black dot), contact points on the human hand (blue dots)and on the robotic hand (red dots).

B. Teleoperating a three fingered robotic hand

The second set of numerical simulations are aimed atmapping on a robotic hand, a task in which the human handis grasping a cubic object, its wrist moves along a giventrajectory and, at the same time, the first hand synergy isactivated. The synergy activation in this case produces botha variation of the contact forces and a displacement of theobject center, which can be evaluated using the procedurediscussed in [22]. This wrist and in–hand motions weremapped on a robotic system represented by a three fingeredhand resembling the Barrett Hand but with eight actuatedjoints (two fingers with three joints and one with two), anda six DoF arm. Both the hands were grasping the sameobject, a cube with side 50mm. The two hands started froma given grasp obtained through the grasp planner availablein Syngrasp, which consider a procedure similar to thatdescribed in [23].

As reference points for the mapping we assumed thecontact points of the hands with the object. In Fig. 8-a)human and robotic hand configurations are sketched, andin Fig. 8-b) contact points are shown. The grasp plannerprovided seven contact points on the human hand and threecontact points on the robotic hand.

We considered a generic trajectory for the human handwrist represented by a cosine arc whose length was 470 mmin the x direction and whose height in the z direction was150 mm with respect to the {Nw} reference frame. Whilethe human hand is following this trajectory, hand finger jointreference values are varied along the first hand synergy. Thetrajectory was sampled in a series of steps. For each steps,human hand and arm motion was mapped on the roboticsystem with the proposed mapping approach.

Fig. 9-a) shows the human hand trajectory. The blue linerepresents object center displacement. Fig. 9-b) shows the re-sulting motion on the robotic hand. The red curve representsrobotic hand object center trajectory during the simulation.As it can be seen, the robotic hand is able to follow the

a)

0 50 100 150 200 250 300 350 400 450−100

0

100

200

−100

−50

0

50

100

150

200

250

y

x

z

b)

Fig. 9: Mapping human hand motion on a robotic handduring the execution of a trajectory combined with theactivation of hand synergies: a) human hand and objecttrajectory; b) robotic hand trajectory, the blue line representshuman hand object center trajectory, the red one shows theobtained robotic hand object center trajectory.

human hand trajectory, even if it is the combination of ageneric six-dimensional wrist displacement with an in–handmotion.

Fig. 10 shows, for the first sampling step, the direction ofthe object center displacement produced by the activation ofthe hand synergies, i.e. without considering wrist motion, andthe corresponding variations of the contact forces, evaluatedaccording to [22]. While the object motion directions arequite comparable, a simple comparison between contactforce variations is not possible, due to the different positionsand number of contact points between the human and robotichand.

Finally, Fig.11 shows the sensitivity of the proposed map-ping procedure with respect to some operative parameters.The upper diagram shows the effect of the applied synergies:the final reference values of the human hand joints were

Fig. 10: a) In–hand object center motion direction: humanhand (blue arrow) and robotic hand (red arrow). b) contactforce variations in the human hand contact points (bluearrow) and in the robotic hand contact points (red arrows)

evaluated as qhf = qhi +αS∆z, with α varying from 0.1 and1 and ∆z = [1 0 0 · · · ]T for the first synergy, ∆z =[0 1 0 · · · ]T for the second one, etc.. As it can be seen,the sensitivity of α parameter on the trajectory error, definedas the distance between object centers at the end of trajectoryexecution, is quite evident. The sensitivity is furthermoredifferent for the different synergies. This effect is due to thedifferent kinematics between human and robotic hand, thatwas pointed also in the preceding set of simulations. Thethree fingered robotic hand, due to its kinematic constraints,is not able to fully reproduce the object in–hand motionproduced by the human hand. The lower diagram shows thesensitivity of the trajectory error on the size of trajectorydiscretization step. In this case the sensitivity is quite lower,i.e. the proposed mapping procedure is quite robust withrespect to the length of trajectory discretization step.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

5

10

15

synergy activation factor

traje

ctor

y er

ror [

mm

]

0 50 100 1500

2

4

6

8

trajectory sampling step [mm]

traje

ctor

y er

ror [

mm

]

syn. 1syn. 2syn. 3

Fig. 11: Trajectory error sensitivities. Upper diagram: sensi-tivity of the trajectory error on the coefficient of synergyactivation, for the first three synergies. Lower diagram:sensitivity on the trajectory discretization step.

V. CONCLUSIONS

The complex and different structures that characterizerobotic hands requires methods to unify their control. Thereare applications, e.g. telemanipulation or learning by demon-stration, in which a mapping between the human hand androbotic hands is necessary. The development of a mappingfunction between human and robotic hands, even with dis-similar kinematics, is necessary to solve these issues. Inthis paper we describe a mapping procedure based on thetask space, whose main requirement is the definition of aseries of reference points both on the human and on therobotic hand. When the human hand is activated, through itssynergies, the displacement of its reference points allows todefine a homogeneous matrix that capture the transformationthat the human hand produces on a set of reference points.The same homogeneous transformation matrix is adoptedto evaluate the displacements of the robotic hand referencepoints, and consequently, through the inverse kinematics, therobotic hand joint variations.

The advantages of this type of mapping is that it doesnot requires empirical and heuristic considerations, it isgeneral and can be applied to robotic hands with a kinematicstructure very different from the anthropomorphic one. Themapping function is nonlinear, and depends on the initialreference configurations of human and robotic hand.

This is a preliminary presentation of a homogeneous trans-formation based mapping procedure. The numerical testspresented in the paper show that its performance dependson relative configuration of the hands and this aspect needsto be furthermore analyzed and compared with other map-ping methods. However, since the method is based on thetransformation of a virtual set of points in the task space,we expect that the dependency of mapping performance onthe relative configurations between the hands is substantiallydue to the kinematic limits of the robotic hand and not onthe mapping features, like, for instance, in the joint–to–jointmapping method. Other parameters have to be considered inthe analysis, for instance the role of the number and locationof the reference points on both the hands.

REFERENCES

[1] A. Bicchi, “Hands for dextrous manipulation and robust grasping:a difficult road towards simplicity,” IEEE Trans. on Robotics andAutomation, vol. 16, pp. 652–662, December 2000.

[2] A. Peer, S. Einenkel, and M. Buss, “Multi-fingered telemanipulation- mapping of a human hand to a three finger gripper,” in Robot andHuman Interactive Communication, 2008. RO-MAN 2008. The 17thIEEE International Symposium on, pp. 465–470, August 2008.

[3] N. Y. Lii, Z. Chen, M. A. Roa, A. Maier, B. Pleintinger, andC. Borst, “Toward a task space framework for gesture commandedtelemanipulation,” in RO-MAN, 2012 IEEE, pp. 925–932, IEEE, 2012.

[4] M. T. Ciocarlie and P. K. Allen, “Hand posture subspaces for dexterousrobotic grasping,” The International Journal of Robotics Research,vol. 28, pp. 851–867, July 2009.

[5] G. Gioioso, G. Salvietti, M. Malvezzi, and D. Prattichizzo, “Mappingsynergies from human to robotic hands with dissimilar kinematics:an object based approach,” in Workshop on manipulation underuncertainty, IEEE Int. Conf. on Robotics and Automation, (Shanghai,China), May 2011.

[6] G. Gioioso, G. Salvietti, M. Malvezzi, and D. Prattichizzo, “Mappingsynergies from human to robotic hands with dissimilar kinematics: anapproach in the object domain,” IEEE Trans. on Robotics, 2013.

[7] G. Gioioso, G. Salvietti, M. Malvezzi, and D. Prattichizzo, “An object-based approach to map human hand synergies onto robotic hands withdissimilar kinematics,” in Robotics: Science and Systems VIII, Sidney,Australia: The MIT Press, July 2012.

[8] M. Gabiccini, A. Bicchi, D. Prattichizzo, and M. Malvezzi, “On therole of hand synergies in the optimal choice of grasping forces,”Autonomous Robots, pp. 1–18.

[9] A. Bicchi, “On the closure properties of robotic grasping,” The Int. J.of Robotics Research, vol. 14, no. 4, pp. 319–334, 1995.

[10] G. Salvietti, L. Meli, G. Gioioso, M. Malvezzi, and D. Prattichizzo,“Object-based bilateral telemanipulation between dissimilar kinematicstructures,” in Proc. IEEE/RSJ Int. Symp. Intelligent Robots andSystems, (Tokyo, Japan), 2013.

[11] K. Shoemake and T. Duff, “Matrix animation and polar decomposi-tion,” in Proceedings of the conference on Graphics interface, vol. 92,pp. 258–264, Citeseer, 1992.

[12] D. Guan-Suo, “Determination of the rotation tensor in the polardecomposition,” Journal of elasticity, vol. 50, no. 3, pp. 197–207,1998.

[13] A. Hoger and D. E. Carlson, “Determination of the stretch and rotationin the polar decomposition of the deformation gradient,” Quarterly ofapplied mathematics, vol. 42, no. 1, pp. 113–117, 1984.

[14] R. Murray, Z. Li, and S. Sastry, A mathematical introduction to RoboticManipulation. 1994.

[15] R. Goldman, Recovering the Data from the Transformation Matrix,vol. VII-2 of GEMS. Academic Press, 1991.

[16] N. J. Higham and R. S. Schreiber, “Fast polar decomposition of an ar-bitrary matrix,” SIAM Journal on Scientific and Statistical Computing,vol. 11, no. 4, pp. 648–655, 1990.

[17] S. Mulatto, A. Formaglio, M. Malvezzi, and D. Prattichizzo, “An-imating a deformable hand avatar with postural synergies for hapticgrasping,” in Haptics: Generating and Perceiving Tangible Sensations.Eurohaptics 2010, Lecture Notes in Computer Science, pp. 203–210,Amsterdam, The Netherlands: Springer Verlag, 2010.

[18] D. Prattichizzo and J. Trinkle, “Grasping,” in Handbook on Robotics(B. Siciliano and O. Kathib, eds.), pp. 671–700, Springer, 2008.

[19] M. Santello, M. Flanders, and J. F. Soechting, “Postural hand synergiesfor tool use,” The Journal of Neuroscience, vol. 18, pp. 10105–10115,December 1998.

[20] D. Prattichizzo, M. Malvezzi, and A. Bicchi, “On motion and forcecontrollability of grasping hands with postural synergies,” in Proceed-ings of Robotics: Science and Systems, (Zaragoza, Spain), June 2010.

[21] M. Malvezzi, G. Gioioso, G. Salvietti, D. Prattichizzo, and A. Bicchi,“Syngrasp: a matlab toolbox for grasp analysis of human and robotichands,” in Proc. IEEE Int. Conf. on Robotics and Automation, 2013.

[22] D. Prattichizzo, M. Malvezzi, M. Gabiccini, and A. Bicchi, “On mo-tion and force controllability of precision grasps with hands actuatedby soft synergies,” IEEE Transactions on Robotics, vol. in press, pp. 1–17, 2013.

[23] A. T. Miller, S. Knoop, H. I. Christensen, and P. K. Allen, “Automaticgrasp planning using shape primitives,” in Robotics and Automa-tion, 2003. Proceedings. ICRA’03. IEEE International Conference on,vol. 2, pp. 1824–1829, IEEE, 2003.