1 MARS PI Meeting JSC 8/2004 Interaction Lab USC – Interaction Lab Skill Learning by Primitives-Based Demonstration & Imitation Maja Matarić - PI Marcelo

1 MARS PI Meeting JSC 8/2004

Interaction Lab

USC – Interaction Lab

Skill Learning by Primitives-Based Demonstration & Imitation

Maja Matarić - PI

Marcelo Kallmann, Chad Jenkins - postdocs

Amit Ramesh - PhD student

Nathan Miller - subcontracted engineer<If you want a photo>


Interaction Lab

Overall Goals

• Goal:– xx


Interaction Lab

Major Contributions

• Contributions:– Development of the USC motion suit for low cost, lightweight,

wireless, real-time motion collection – Cross-kinematics metrics for imitation learning, allowing for

comparison and mapping of motion data across various kinematic structures

– Prediction of expected sensory information for a successful grasping, from the analysis and interpolation of collected Robonaut sensory data

– The use of randomized motion planning for reaching and for motion sequencing


Interaction Lab

Major Contributions

• Demonstrations/Applications:– Teleoperation of Robonaut

– The implementation of human-robot cooperation applications for a variety of tasks. An application example was demonstrated using Robosim (NASA’s Robonaut simulator)


Interaction Lab

Architecture OverviewUSC Motion Suit

Cross-kinematics metrics for imitation learning are used for collecting motion from various types of kinematics

Other sources of motion collection

Robot-Ready Motion Database

Derivation of motion controllers based on collected motion and

sensory data

Robonaut Control

Learned skills are represented as parameterized motion controllers

Derivation of motion controllers based on collected motion and

randomized roadmaps


Interaction Lab

USC Motion Suit

• Applications:


Interaction Lab

Cross-Kinematics Metric

• Applications:


Interaction Lab

Learning Sensory Structures

• Applications:


Interaction Lab

Collision-Free Reaching• Goal: Efficient collision-free motion planning for

humanoid arms

• Applications:– manipulation tasks in environments with obstacles – relocating grasped objects in the workspace– maintenance tasks when the tools, controls, handles, etc, to be

reached are situated in difficult locations


Interaction Lab

Potential Application Example

• Truss Assembling– Collision-free reaching for tools and for grasping bars


Interaction Lab

Approach• Our tests indicate that sampling-based motion planners are more

efficient than other methods, e.g. such as IK with collision avoidance (Drumwright, Kallmann, Mataric, “Towards Single-Arm Reaching for Humanoids in Dynamic Environments”, submitted to Humanoids’04 )

• On-Line Arms Motion Planning– We have applied the on-line Rapidly-exploring Random Trees (RRT) on

the composite configuration space of the two arms

– Motions can be achieved in 1 or 2 secs in simple scenarios

• Dynamic Roadmaps– When the workspace has few changes, dynamic roadmaps can be very

efficient. Joins the advantages of multi-query methods (PRMs, PRTs, VGs) and single-query methods (RRTs, Exp. Spaces, SBLs).


Interaction Lab

On-line Arms Motion Planning• Example collision-free motions to pre-grasping targets

– Visualization geometry: 23930 triangles

– Collision geometry: 1016 triangles

– Computation time: 1 to 2s, including optimization (smoothing)

– Optimization takes about 0.3s

(Pentium III 2.8 GHz)


Interaction Lab

Path Optimization (1/2)• Incremental path linearization

– Simple and efficient in most cases– May be time-consuming as collision detection

must be invoked before each local linearization.


Interaction Lab

Path Optimization (2/2)• Decoupled, sub-configuration linearization

can be applied:– In the example, the top arm in the left video

makes an unnecessary motion– This can be only corrected when smoothing

arms independently (right video)


Interaction Lab

Dynamic Roadmaps: Overview• Evaluation of dynamic roadmaps for finding collision-free

arm motions in changing environments• Comparison with on-line bi-directional RRT

4 DOFs 7 DOFs 17 DOFs


Interaction Lab

Roadmap Computation• A grid is defined over the workspace• Desired number of configurations are sampled


Interaction Lab

Roadmap Computation• Sampled nodes are connected to the k neighbors if the

connection is valid, until linking all valid connections


Interaction Lab

Roadmap Computation• Cell localization in respect to the roadmap nodes:

– For each cell, test all edges and nodes which are invalid in respect to that cell, ie, which “collide” with that cell

– Each cell keeps references to the invalidated edges/nodes– More efficient methods:

• Hierarchical tests of cells

• Exploiting cell adjacency coherence(Leven and Hutchinson 2000) [2]

• Using graphics hardware


Interaction Lab

Roadmap Computation• Cells are localized and roadmap nodes and edges

intersecting with objects are invalidated


Interaction Lab

Roadmap Maintenance• Whenever obstacles in the workspace are detected to

change position, the affected cells are either liberated or occupied

• Reference counting management:– All roadmap nodes/edges maintain reference counters– When cells are occupied/liberated, the associated nodes/edges

have their counters incremented/decremented


Interaction Lab

Roadmap Maintenance• Examples:

24x32 2218 nodes 64x70 2406 nodes 48x64 6661 nodes


Interaction Lab


202 3146 nodes 402 2898 nodes


Interaction Lab


243 3575 nodes 243 5144 nodes


Interaction Lab


243 3575 nodes 243 5144 nodes


Interaction Lab

Roadmap Query• Find motion from initial configuration qi

to goal configuration qg

• Retrieve nearest nodes N(qi), N(qg) in the roadmap

• Retrieve a path from N(qi) to N(qg): A* search, however:– Path may not exist– qi to N(qi) may not be valid– qg to N(qg) may not be valid

• In case of failure, a Bi-RRT is used as on-line planner– to fix the path if one of the tests fails– Compute the full path


Interaction Lab

Experiments• 100 random problems for each scenario

– Random obstacles, initial and goal configurations– Failure considered after 10 seconds

• 7 Dynamic scenarios with:

One arm Two arms Robonaut


Interaction Lab

• Two arm planar manipulator, planar workspace, 7 DOFs• Robonaut model, 17 DOFs: 7 in each arm + 3 at the base

(Pentium III 2.8 GHz)

Experiments: Example Motions


Interaction Lab

• Up to 8 times faster in the planar workspaces• Roadmap/Grid size vs. maintenance cost tradeoff• Modest speed gains in 3d workspaces

ExperimentsScenario Grid Nodes Links DRM Time(s) RRT Time(s) Comparison

One arm 24x32 2218 7467 12.0 59.3 4.9

One arm 64x70 2406 8069 17.1 146.8 8.6

One arm 48x64 6661 22431 45.9 72.4 1.6

Two arm 202 3146 10629 18.2 151.3 8.3

Two arm 402 2898 9838 17.1 146.8 8.6

Robonaut 243 3575 12312 47.9 63.1 1.3

Robonaut 243 5144 17707 45.8 51.8 1.1


Interaction Lab

Integration with Robonaut• Motions generated on our Robonaut model can

be easily transferred to Robonaut• Same mapping mechanism as the one used for

the teleoperation with the motion suit


Interaction Lab

Sequencing Parameterized Motion• Motion control for complex operations require

mixed type of controllers• Sequencing of primitive movement controllers is

required• Ex: coordination of bimanual manipulations, of

multiple robots, or of biped locomotion


Interaction Lab

Example with Biped Locomotion• We propose a search method for sequencing movement

primitives• The method is based on randomized roadmaps and

discrete search methods• The method was applied to obtain statically stable biped

locomotion among obstacles


Interaction Lab

Movement Primitive: Definition• Any controller with a simpler parameterization over the

configuration space:

Pis: Si

s C

where:

s is the start configuration of primitive Pis

Sis is the parameter space of primitive Pi

s

C is the configuration space

• Primitives generate postures, not motions


Interaction Lab

• Total of 9 DOFs• Movement control only allowed trough motion

primitives parameterization, ensuring:correct support, balance, limits, collision-free

Primitives: Description

PR

PLPB


Interaction Lab

Primitives: DescriptionMovement Primitive

Instantiation Condition

Primitive Motion Parametric Space Dim.

PL support in left

foot

moves right leg articulations and body

rotation 4

PB support in both feet

moves body, feet fixed with IK

3

PR support in right foot

moves left leg articulations and body

rotation 4

(Robot has 9 DOFs)


Interaction Lab




PL support in left

foot


rotation 4



3



rotation 4

(Robot has 9 DOFs)


Interaction Lab




PL support in left

foot


rotation 4



3



rotation 4

(Robot has 9 DOFs)


Interaction Lab




PL support in left

foot


rotation 4



3



rotation 4

(Robot has 9 DOFs)


Interaction Lab

Sequencing: Problem Definition• Given:

– Primitives Pi

– Task completion test function t(q) : C {0,1} • Determine a sequence of concatenated valid

paths in configuration space, such that:– Each path is generated by a single primitive– The first point is the current position– The last point satisfies completion test


Interaction Lab

Sequencing: Search Tree• Expand a roadmap in the parametric space of

the motion primitive associated with c

search tree


Interaction Lab

Sequencing: Search Tree• Determine paths leading to configurations in a

different support mode

search tree


Interaction Lab

Sequencing: Search Tree• Select lowest cost leaf c

cost(c) = length(root,c) + dist(c,goal)

search tree

…


Interaction Lab

Sequencing: Search Tree• Expand a roadmap in the parametric space of

the new motion primitive

• Continue the process until close to goal point

search tree

…

…


Interaction Lab

Obtained Examples


Interaction Lab

Integrated demo with Robosim• Cooperative human-robot example application • Demonstration of example motions using the

USC motion suit• Segmentation into meaningful example• Derivation of a primitive controller, which is able

to interpolate the examples in order to obtain a scooping motion in any position inside the tray

• New tools can be reached using our motion planner


Interaction Lab

Integrated demo with Robosim

• Example of scooping motions to different instructed locations


Interaction Lab

Integrated demo with Robosim

• Example of sending scooping motions to Robosim


Interaction Lab

Summary• a

• b


Interaction Lab

Thanks• Doug

• JSC Robonaut Team

Documents

1 MARS PI Meeting JSC 8/2004 Interaction Lab USC – Interaction Lab Skill Learning by Primitives-Based Demonstration & Imitation Maja Matarić - PI Marcelo