Face the Future

Steve Maddock

University of Sheffield

Beginnings…

Pong, 72

Space Invaders, 78Star Wars:

A New Hope, 78

Tron, 82

The Art of Graphics for the IBM PC (& Sinclair QL), summer 84

BBC Micro: Elite, 84

Siggraph, 87

The Wall, 80“Help me Obi-Wan…”

Alan Watt’s books

McGregor and Watt, 84

McGregor and Watt, 86

Watt, 2000Watt and Policarpo, 98

Watt and Watt, 92

Watt, 89 Watt, 93

Watt and Policarpo, 01

Watt and Policarpo, 03

Watt and Policarpo, 05

?

Graphics teaching at Sheffield

Pas

quar

iell

o, C

. Pel

acha

ud, G

reta

: A S

impl

e F

acia

l Ani

mat

ion

Eng

ine,

6th

Onl

ine

Wor

ld C

onfe

renc

e on

Sof

t Com

puti

ng in

In

dust

rial

App

icat

ions

, Sep

tem

bre

2001

Java and OpenGL

Modelling and Animation

Lee Cooper, 95-98, “Physically-based modelling of human limbs”

Agata Opalach, 92-95, “Implicit Surfaces for Modelling and Character Animation”

Visualisation, 90-93. Three PhDs: Fuller, De Geus, Hall

Games

1998, Jan-Jul: six month RAE-sponsored Industrial Sabbatical at Gremlin Interactive, Sheffield

1998-2002: 50 undergrad/masters dissertation projects with Infogrames

Modelling and animation

Maddock, S.C. (1999), "An Investigation of the Modelling-Animation Relationship in Computer Graphics“.

“Form follows function” – relating functional aspects of a model to structure gives better control of complexity.

More use of physics and behavioural models needed.

Computer facial animation

Early days: Parke, 72 and 74; Two categories

Geometric approaches Physics-based approaches

Much still eludes us.

Right Frontalis Outer

Right Frontalis Inner

Right Lateral Corrugator

Right Angular Depressor

Right Inner Labi Nasi

Right Labi Nasi

Right Frontalis Major

Right Secondary Frontalis

Right Zygomatic Major

Some landmarks in CFA

(Max Headroom, 1980s) Roger de Peltrie, 85 Waters, 87 Williams, 90 Toy Story, Pixar, 95 Virtual idol - Kyoto Date, 1996 Video rewrite, 97 (Forrest

Gump) Virtual Newsreader –

Ananova, 2000 Final Fantasy: the spirits

within, 01 Gollum, 2004

Face work in the UK

Computer Science/Vision Bath, Cambridge, Cardiff, UEA, Edinburgh,

Essex, Glasgow, Kent, Leeds, Leeds Met, QMW, Sheffield, Surrey, Sussex, UCL

Psychology Aberdeen, Cambridge, Central Lancashire,

Glasgow, Hertfordshire, Liverpool, Newcastle, Nottingham, Open, Sheffield, Southampton, St.Andrews, Stirling, UCL, York

Forensics/Medical Cambridge, Glasgow, Manchester, Oxford,

Sheffield, UCL Companies/Offices

3Q, Artem Digital, avatar-me, Biovirtual, BT, Digimask, Genemation, Image Metrics, Jim Henson’s creature shop, Lexicle, Life studio, Richard Neave partnership, Televirtual, Vicon, Virtual Clones Ltd

www.dcs.shef.ac.uk/

~steve/face/

Sheffield graphics research work

Visual Speech Realism in facial animation

Discrete differential geometry

Kinematics and biomechanics in adapting motion capture data for use in computer games

Using VR for police training

Facial modellingProcedural modelling

Information hiding

Audio-visual speech enhancement

Dialogue, personality and virtual characters

CFA at Sheffield

Edge and Maddock, 04, Using motion capture data to animate visual speech

Edge et al, 04, Constraint-based Synthesis of Visual Speech

CFA at Sheffield

Sanchez et al (submitted 04), Realistic Performance-driven Facial Animation using Hardware Acceleration

Near future?

More photorealistic rendering of skin in games characters. Background actors in games and films. Game characters that can ‘hold’ you in a simple conversation. Recorded messages on gravestones (bbc.co.uk, 04).

Distant future?

Beautifiers – always look your best on a videophone.

Synthespians – capture an actor: How to represent personality and behaviour? (Need a visual Turing test?)

Bring back to life famous people using gramophone recordings of their voice (e.g. Churchill, Lincoln) and models of their facial behaviour.

Acting/singing/competing with famous people games and entertainment.

Virtual clones, who can take your place while you are alive and live on as you after death.

Holographic beings…

References

Edge, J.D. and S. Maddock (2001), "Expressive visual speech using geometric muscle functions", Proc. 19th Eurographics UK Chapter Annual Conference, (UCL, 3-5 April, 2001), pp.11-18.

Edge, J. and S. Maddock, (2003) “Image-based Talking Heads using Radial Basis Functions”, Proc. EGUK2003, University of Birmingham, UK, 3-5 June, 2003.

Edge, J., M. Sanchez, S. Maddock (2004), “Using motion capture data to animate visual speech”, Symposium on Language, Speech and Gesture for Expressive Characters, March 30-31, 2004, part of the AISB 2004 Convention: Motion, Emotion and Cognition, University of Leeds, UK, March 29 - April 1, 2004

Edge, J. and S. Maddock (2004), “Constraint-based Synthesis of Visual Speech”, ACM SIGGRAPH'04 Technical Sketch.

Sanchez, M. and S. Maddock, (2003) “Planar bones for MPEG-4 facial animation”, Proc. EGUK2003, University of Birmingham, UK, 3-5 June, 2003

Sanchez Lorenzo, M., J.D. Edge, S. King, and S. Maddock (2003), "Use and Re-use of Facial Motion Capture Data", Proc. Vision, Video and Graphics 2003, University of Bath, July 10-11, 2003, pp. 135-142

Sanchez, M., J. Edge, S. Maddock (2004), “Realistic Performance-driven Facial Animation using Hardware Acceleration” (submitted).