Upload
others
View
10
Download
0
Embed Size (px)
Citation preview
Agents and Avatars 2
Ruth Aylett
Overview-
Crowds Speech and Expressive behaviour
– Embodied conversational characters
Creating autonomy– Scripting– Architectures
Crowds and flocking
Interactions amongmembers of a group– Local neighborhood
Reynolds Boids– Originally birds or
fish– Complex behaviour
determined bysimple rules
Separation: BoidAvoidance
Alignment
Cohesion
Motor Control
Steering Force Integrate to determine acceleration
– Thrust – determines speed– Lateral Steering Force – determines direction
Boid Object Representation
Point Mass Vehicle– Mass– Position– Velocity– Orientation
• Constrained to align with velocity– Force and Speed Limits
(No moment of inertia)
Same maths as point objects
acceleration = steering_force / mass– Given f = mavelocity = velocity + acceleration– Over a unit time period– Generically v = v x atposition = position + velocity– Over a unit time period– Generically s = s + vt
Seeking and Fleeing Aim towards target
Desired_velocity = Kp (position – target)Steering = desired_velocity – velocity
Seeking andFleeing Applet(Reynolds)
Pursuing and Avoiding
Target is another moving object Predict target’s future position Scale prediction time, T, based on distance to object, Dc
T=Dc
Pursuing andavoiding applet(Reynolds)
More Behaviors
Evasion– Like flee, but predict pursuer’s movement
Arrival– Like seek, but stop at target– Applet (Reynolds)
Obstacle Avoidance– Repulsive force– Aim to boundary– Adjust velocity to be perpendicular to surface normal
Do People Flock?
Social psychologist’sreport that peopletend to travel assingles or in groupsof size 2 to 5.
“Controlling Steering Behavior for Small Groups ofPedestrians in Virtual Urban Environments”
Terry Hostetler, Phd dissertation, 2002
Characteristics of Small Groups
Proximity Coupled Behavior Common Purpose Relationship Between
Members
Moving Formations
Pairs: Side by side Triples: Triangular shape
Stationary Formations
Moving pair approachesstationary triple
Stationary quintupleformed
Two Parameters - because in 2D– Acceleration
• Increase/reduce walking speed• Combination of step length and step rate
– Turn• Adjust orientation• Heading direction for forward walking
Locomotion Model for Walking
Avoiding an Obstacle --Trajectory
Small look-ahead distance Large look-ahead distance
ped 1
ped 2
walkway axis walkway axis
ped 1
ped 2
Interaction Between Pairs -- 1
Interaction Between Pairs -- 2
Interaction Between Pairs -- 3
Adding goals
People are usually going somewhere– Easy case: same goal
• Walking to a sports stadium• Political demonstrations
– Harder case: multiple goals• Campus traffic• Most public spaces: streets, shops, stations
Talking Heads
Human faces– Thus high expectations:
• Blinking, random head movement• Facial muscles• Lip sync
Rarely intelligent– Entirely scripted in most cases– No integration of speech production with
intelligent architecture and NL generation
Moving the face - 1
Simple approach– Define overlay frames with different facial
expressions• For example, six mouth shapes
– Use signal from TTS engine to pick amouth shape
– This works OK with cartoon-like characters
Moving the face - 2
Visemes– Mouth and lip position
for each phoneme– Correspond to a
submesh in thegeometry
Using visemes
Find viseme for each phoneme– Synchronise graphical change with sound
Catalan weather forecaster
Complete talking character– Weather forecasts have predictable content– Still have to match content and behaviour
• <demo via browser>
Embodied conversational characters
Beyond talking heads– Key difference is interactivity– Use of dialogue theory– Focus on body language - expressive
behaviour
Relating graphical to languagebehaviour
Expressive facial behaviour
Drawing on psychology:– FACS (Facial Action Coding System)– Developed by Ekman & Friesen (1978)– Distinguishes among 44 Action Units– Each action unit (AU) describes a separate,
visible, and muscle-based facial change
Duchenne de Bologne
French anatomist of19thC– In 1860s used
electric currents onfacially paralysedsubject
– Showed whichmuscles go withwhich expressions
The Duchenne smile
Associated withexpression ofhappiness
AU12 (Lip Corner Puller)
Plus
AU6 (Cheek Raiser)
Mapping onto mpeg-4
Defines FAPs– Facial action
parameters– Set of feature points– Model calibration– Morphology
parameters– Expression
parameters
Coordinating with language Use of Affective Performance Markup Language (APML)<performative type="inform"><theme> As far as <emphasis x-
pitchaccent="LplusHstar"> vitamins </emphasis> are concerned<boundary type="LH"/></theme></performative><rheme><emphasis x-pitchaccent="Hstar"> research </emphasis>
has shown <boundary type="LH"/></rheme> <performativetype="inform"> <rheme>that eating<emphasis x-pitchaccent="Hstar">the recommended </emphasis>levels of vitamin
<rheme affect="sorry-for">can have <emphasis x pitchaccent="Hstar">beneficial </emphasis><emphasis x-pitchaccent="Hstar">effects</emphasis>for your <emphasis x-pitchaccent="Hstar">appearance</emphasis> <boundary type="LH"/> and <emphasis x-pitchaccent="Hstar"> health</emphasis><boundary type="LL"/>
</rheme></performative>
Incorporating gesture
Additional markup to control gestures insynch with other expressive behaviour
Generating expressivebehaviour
Setting the internal state to be expressed Low-level accounts
– Focus on behaviour (non-symbolic)– Primitive emotions– Brain limbic system: physiological integration– Canamero, Velasquez, Damasio
High-level accounts– Focus on cognitive appraisal– Complex taxonomy of emotions– Cortex: cognitive integration– Ortony, Clore and Collins; many applications– Frijda; Lazarus; Sherer
Modelling emotion Appraisals
– Assessments of events, actions, objects
Valence– Whether emotion is positive or negative
Arousal– Degree of physiological response
Implementing appraisals– Domain-specific rules– Probability of impact on agent’s goals
Emotion theoryOrtony, Clore and Collins (OCC)
Extremely widely used for agents– Ortony, A; Clore, G. & Collins, A. 1988 The cognitive
structure of emotions. Cambridge University Press–
Group TypesWell-being joy, distressFortunes-of-others happy-for, gloating, resentment, sorry-forProspect-based hope, satisfaction, relief, fear, fears-confirmed, disappointmentAttribution pride, admiration, shame, reproachAttraction love, hateWell-being ! Attribution admiration + joy " gratitude(compound emotions) Reproach + distress " anger
Pride +joy " gratificationShame +distress " remorse
OCC Model - more detail
Group Specification Name & typeWellBeing
Appraisal ofsituation as event
Joy - pleased about eventDistress - displeased aboutevent
FortunesofOthers
Appraisal ofsituation as eventaffecting another
Happy-for: pleased about anevent desirable for anotherResentment: displeased aboutan event desirable for another
Prospect-based
Appraisal ofsituation as aprospective event
Hope: pleased about aprospective desirable eventFear: displeased about aprospective undesirable event
Emotions defined in terms ofsituations, goals, and others
Autonomy
An independent sense-reflect-act cycle: implies– An agent-based action-selection mechanism– Self-animation in real-time: cannot be pre-rendered– Local sensing– Virtual robotics
Not a natural graphics view– Global manipulation of polygons– Hierarchical scene-graph
Scripting
Typical of NPCs in computer games What representational content is
needed?– Can just invoke named animations– What parameters?
• Depends on animation approach
Parametrised ActionRepresentation
Badler: symbolic term -> movement Supports natural language interaction
– Named actions: walk, jump, run– Adverb modification: quickly, tiredly,
enthusiastically Laban Movement Analysis
– From choreography– Effort and shape components to
characterise movements
Badler’s PARs
Scripting with Finite State Automata
TennisgameFSMsequence
Limitations of scripting
Becomes predictable– Either independent of local sensing– Or single FSM transition stimulus
Limited ability to use internal state– Interaction memory– Affective state
Lacks sequencing flexibility– Sequence is hardwired
Low-level architecture for a fish…
Functional Architecture
High-level architecture
Using game engines
Offer some support for NPCs– More facilities than a scenegraph– Attaching animations to graphical bodies– Imports from popular 3D packages like character studio
Disadvantages– Proprietary; oriented to scripting– Hard (in some cases impossible) to implement sensing
• Fixed paths quite often– Often focused on shooting
Gamebots
Loosely coupled architecture based onUnreal Tournament– CMU– Agent ‘mind’ outside of UT– Defined interface: message types and
content– Javabots gives java interface
The mechanism Gamebots and TCP sockets
– Every robot possesses one TCP socket– Exchange data through the TCP socket– Unreal Client(s) can connect to the server at anytime
UnrealServer
(response to the command;
send out sensor data)
Gamebots
Robot Controller
Unreal Client(s)
Command
Sensor data
Command
Sensor data
Robot ControllerCommand
Sensor data
Command
Sensor data
………… TCP Socket
Virtual robot
Communication protocol Gamebots protocol
– Format: data_type {segment1} {segment2} …• data_type: the type of the data in upper case
characters– Examples: INIT, STA, SEN, DRIVE etc.
• segment: a list of name-value pairs separated by space– Examples: {Location 100,200,300}, {Name Left Range
800.0} etc.
– Example• INIT {ClassName USARBot.ATRVJr} {Location 200,600,-450}
– Two types of data• Message (from the server)• Command (from the client)
Communication protocol - 2 Messages
– State message: the robot’s current state• STA {Time t} {Camera pitch,yaw,roll} {Zoom fov} {Attitude
pitch,yaw,roll} {Location x,y,z} {Velocity x,y,z} {LightToggle bool}{LightIntensity int} {Battery float}
– Sensor message: the sensor data• Sonar Sensor SEN {Type Range} {Name string Range number}
{Name string Range number} …• Laser Sensor SEN {Type RangeScanner} {Name string} {Location
x,y,z Rotation pitch,yaw,roll} {Range r1,r2,r3…}• Human Motion Detection SEN {Type HumanMotion} {Name string}
{Prob float}• Sound Sensor SEN {Type Sound} {Name string} {Loudness float}
{Duration float}– Geometry message: sensor’s geometry information
• GEO {Type string} {Name string Position x,y,z Direction pitch,yaw,roll} {Namestring Position x,y,z Direction pitch,yaw,roll} …
– Configuration message: sensor’s configuration information• CONF {Type string} {Name Value} {Name Value} …
Communication protocol - 3
Command– Spawn robot
• INIT {ClassName robot_class} {Name robot_name} {Locationx,y,z}
– robot_class: the class name of the robot. Example:USARBot.P2AT, USARBot.P2DX etc.
– robot_name: the robot’s name. Can be any string.– x,y,z: the start position of the robot. For different arenas, we
need different positions.• Example
– INIT {ClassName USARBot.P2AT} {Location 312,-600,-305}