Synchronization of movements

of a real humanoid robot with music

Amstutz Patrick

BIRG, EPFL

Goals of the project

• Study trajectories generation system

• Find an approach for adding sound capabilities for Webots videos

• Implement collision detection for visual feedback of drumming task

• Synchronize the robot with music: investigate possible solutions for extracting tempo information from music

System overview (1)

System overview (2)

Mathematical Aspect of CPGi

Discrete Part:

Rhythmic Part:

Matlab Simulation

Processschema

Tempo detection

• Matthew, Davies and Plumbley

• Aubio Library

Webots

• http://www.cyberbotics.com– Professional mobile robot simulation

Collision detection

• ODE Physic Plugin– http://www.ode.org

• Message to supervisor– Texture change– Sound info file

Matlab Music Generation

• Generation function:– generation(left, right, sound1, sound2, sound3)

• Parameters:– Two matrices of simulation information– Three sounds corresponding to each

instrument

• Output:– Wav file

Merging Video and Sound

• VirtualDub– Xvid– Mp3

• Script Generation

• Output avi files

References

Thank you

• Sarah Degallier, for your help

• Yvan Bourquin, for Webots help

• Alessandro Crespi, for VirtualDub help

• Pr. Auke Ijspeert for allowing me to perfom this project at BIRG