Motion CaptureEmilio Cantu, Jin He, Gerard Louis, Brian Orchosky, Chelsey Salberg

Motion capture is the process of recording the movement of objects or people.

What is Motion Capture?

Early Animation

Rotoscoping - animation technique where animators trace over footage, frame by frame for use in live action and animated films

In Early Film

First movie to use Motion Capture: Snow White and the Seven Dwarfs (1937). Live models were used to guide the drawings.

In Early Games

Highlander: The Last of the MacLeods (1995). Employed Motion Analysis

Traditional Animation

Keyframing - animation made up of key frames (drawings) that define the starting and ending points of any smooth transition.

Motion Capture vs. Keyframing

A Brief Look In Movies

• Motion capture has it’s roots from movies (Hollywood).

• Game developers as a consequence begin to implement some of these new ideas or technologies to enhance their games.

• There’s really no significant difference in translating motion capture technology from movies to games.To give the audience a better sense on how motion capture was incorporated into games, movie references using motion capture are given.

Motion Capture In Movies • Snow White and the Seven Dwarfs (1937)• The Lord of the Rings: The Fellowship of the Ring (2001)• Polar Express (2004) • King Kong (2005)• Happy Feet (2006)• Avatar (2009)• District 9 (2009)• Mars Needs Moms (2011)• The Adventures of Tintin (2011)• Rango (2011)• Rise of the Planet of the Apes (2011)• John Carter (2012)• The Avengers (2012)• Ted (2012)

Snow White and the Seven Dwarfs (1937):

• Actors and actresses would act out scenes and would be filmed. The animators would then use the individual frames as a guide to their drawings (early form of mocap technology).

Polar Express (2004) & Avatar (2009)

• Actors and actresses would use a mocap suit which translated their movement into a digital effect which was then animated into a 3D computer animation.

The Adventures of Tintin (2011)

• Uses a mix of real actors and props, mocap allows a more accurate blend of images and film that can all be pieced together in an amazing sight.

Ted (2012)

• Ted (the teddy bear) was actually animated by using motion capture on an actor, which is why Ted’s movements seemed so realistic.

More Movie Examples

Rise of the Planet of the Apes (2011)

District 9 (2009)

John Carter (2012)

Games

The term ‘performance capture’ is used to describe advanced motion capture

In addition to body movements, more minute details such as emotion, finger movement, and facial detection can be captured and used.

Examples

Since the movie Avatar, one of the earliest, most famous, examples of performance capture, it has also been used in games such as L.A. Noire and Heavy Rain

Facial Motion Capture Beginnings

100+ markers on an actor’s face to capture facial expressions.

Very cumbersome for the actor - takes a long time to put all dots on, and the

actor cannot touch their faceuntil they are removed

Facial Motion Capture

Markerless motion capture: http://www.youtube.com/watch?v=7bX0qpsLfpE

Mixamo recently released Face Plus for Unity - capture face data with only a webcam.

Early motion detection in games

Sega’s Heavyweight Champ (1976) for arcades

Two boxing glove controllers you punch,

one for each player. A 1987 remakegave one player control of two punchcontrollers. The cabinet can also turn from side to side, and the boxer moves

too.

Sega Hang On (1985)

The ‘world’s first full-bodyexperience video game’.Three arcade cabinet designs. The one pictured enables turning by leaning on the motorcycle.

NES Power Glove

1989 for NESMove the glove controller around. The time taken for ultrasonic sound to travel between speakers on the glove and microphones on a receiver unit is used to triangulate position.

Optical fibers can detect bending of the glove (light is obstructed when bent).

Considered difficult to use

Sega Activator (1993)

Octogonal device on the floor that had eight infrared light sensors beamed upwards for detecting movement. Like the Power Glove, also not well-received.

Dance Dance Revolution (1998)http://www.youtube.com/watch?v=ExjhGrVJ8Do

Most game modes/variants feature pressure sensors in four directions.

These sensors must be stepped on to match the arrows appearing on screen in time with music.

First major ‘exergaming’ (exercise gaming) series.

Konami’s Police 911 (2000)

Shooting game where playersduck for cover and dodge… by actually ducking for cover anddodging.Uses infrared sensors.

Notable motion capture games

Prince of Persia (1989)Rotoscoping - videos (like the one below) were used as reference.

http://www.youtube.com/watch?v=WAjRNU3DbSY

https://en.wikipedia.org/wiki/Prince_of_Persia_(1989_video_game) (for animated GIF)

Metal Gear Solid

Used from the second entry in the series (2001) onwards.

The upcoming Metal Gear Solid 5 uses facial capture. Actors had their face and expressions captured from many angles.

http://www.youtube.com/watch?feature=player_embedded&v=DbPxvBbZAhs

Enter The Matrix (2003)

The actors (and stunt doubles) from the original movies were the motion capture actors for the game. Motion capture took 3 months.

L.A. Noirehttps://www.youtube.com/watch?v=3FIviB90Ih8

Used MotionScan for performance capture, with 32 cameras for facial animation.

The blooper reel above shows the tech better than the game does because it features actors breaking character and being more natural.

Other well-known games using motion captureMadden (from 1994 on)The Legend Of Zelda: Ocarina of Time (1998)

Devil May Cry 3 and 4 (2005, 2008)Uncharted series (2007)Grand Theft Auto IV (2008)The Last of Us (2013)

Motion Capture Technologies

Many types:• Optical (most prevalent)

o a series of cameras with strobing lights are set up around an area to be “captured.”

• Inertialo Body suit or sensors attached to

a person that track gyroscopic movement and transmit to a receiver

o inertial sensors have 3d gyroscopes, accelerometers, and magnetometers

Motion Capture Technologies

Last type:• Mechanical

o Suit that subject wears that transmits motion data to a receiver

o Uses joint sensors to determine movement

Similarities between technologies

• All are fairly expensive, need between $5000-$10000 initial investment

• All require interfacing with a computer running specialized software

• Each brand has its own proprietary software or plugins for industry standard software

Optical Motion Capture

• OptiTrack series from NaturalPoint

• Prime 41 ($4,999)o Resolution: 2048 ×

2048o Frame Rate: 30–

180 FPS (adjustable)

o Infrared Strobe (won’t interfere with video cameras)

Inertial Motion Capture

• XSens MVNo Full body suit with 17 inertial sensorso Flexible, since there are no special

harnesses to hold the sensors, rather they are built in to the suit Used in “Paul”, by Seth Rogen to play the alien

Motion Capture Software

• Autodesk MotionBuildero Industry standard

3d character animation software

o Can import motion capture data and map motions to imported characters from Maya, 3ds, etc

Questions?