Scalable, Controllable, Efficient and Convincing Crowd Simulation

Mankyu SungUniversity of Wisconsin-Madison

What is Crowd?

Crowd A group of individuals Share a same environment Have a same goal but may

act in different way

UniqueNatural Phenomenon

What is Crowd Simulation? Crowd Simulation

Artificial creation of virtual crowds Collection of behaviors of each individual

Behaviors Selection of motions over time

Real Crowd Virtual Crowd

Challenges Highly complex crowd behaviors

• Ex) Street environment : crossing, sitting, window-shopping, walking with friends…

Not easy to maintain real-time performance• Real time animation of more than 1,000 characters is

hard

Not easy to achieve convincingness• Realistic motions, Realistic behaviors

Need to minimize the control parameters

Research Goal : Four demands for crowd simulation

Crowd simulation method

that meets demands

Scalability Controllability

Efficiency Convincingness

Our goal

(1) Scalability Definition: Ability of crowd simulation to

continue to function well when the environment become more complicated.

Scalable Character Memory• Agent’s memory is not proportional to the complexity of environment.

Scalable Authoring• The whole simulation should be broken into a set of small simulations.• Author new simulation through simple copy&paste

(2) Controllability Definition: Ability to specify particular

behaviors to characters by setting constraints

Constraints • High-level scenario for controlling crowd flow

• No specific individual

• Low-level individual constraints• Specific individual

(3)Efficiency

Definition: Fast processing of algorithm

Fast motion synthesis Fast collision detection Linear simulation performance

(4)Convincingness Definition: Ability to cause viewers to

believe specific aspects of crowds.

Visually convincing behavior• No discontinuity on motion• No collisions between characters

Semantically convincing behavior• No unreasonable behaviors (depending on

animator’s intention)

Outline1. Features of crowds2. Two-level crowd simulations method3. Probability scheme4. Collision detection through MOBB(Motion

Oriented Bounding Box) – (Briefly)5. Constrained motion synthesis6. Summary and future work

Solution : Observe real crowdsWhat features crowds have?

Anonymity Hard to distinguish one from others It does not matter who is doing, but what is

happening there matters Locality

Behaviors depend on location and social relationship

Variety People behave differently

Solution :Two-Level crowd simulation framework

Situations(What should each individual do ?)

Motion Synthesis(How to show motions?)

Motion Synthesis(How to show motions?)

High-level

Low-level

Situations(What should each individual do ?)

An EnvironmentExample:Model of Street

Simulation:Real time, Reactive

Rendering:Unreal Game Engine for playback

Observation:Behavior Depends on Situation

Store Window

In front of Store WindowPossibly: stop to window shop

Friends Together:Possibly: Stop to talkProbably: Have same goal

Doorway

In front of DoorwayPossibly: open door, enterUnlikely: stand blocking door

In a hurry:Check for trafficRun across street

In Crosswalk:Walk across street onceyou’ve started

Use crosswalk:Wait for green lightStart crossing

Sidewalk:Walk here

Street:Generally, don’t walk here

Managing Environmental Complexity:

Situation-Based Approach

Many different situations An agent only needs a few at a time Each situation has a different set of local behaviors Plugs information into agents for local behaviors Scalable authoring and scalable behaviors

Store Window

In front of Store WindowPossibly: stop to window shop

Friends Together:Possibly: Stop to talkProbably: Have same goal

Doorway

In front of DoorwayPossibly: open door, enterUnlikely: stand blocking door

Use crosswalk:Wait for green lightStart crossing

Sidewalk:Walk here

Solution :Two-Level crowd simulation framework

Situations(Sung et el, EG 04)

Motion SynthesisMotion Synthesis

High-level

Low-levelCollision Avoidance (Sung et al CS-TR-05-1592)

Probability Scheme

(Sung et al EG 04)

ConstrainedMotion Synthesis

(Sung et al SCA 05)

Low-level : Two motion synthesis techniques

Probability Scheme (Sung et al EG 04)

• Stochastic crowd control• Short term view of agents

Constrained Motion Synthesis(Sung et al SCA 05)

• Planning-based behaviors• Have constraints for individuals

Use Snap-Together-Motion(Gleicher et al I3D 03)

Related Work:Inspirations from Research Crowd Modelling

(e.g. Reynolds 1987, Musse 2001, Blue 1998, Henderson, 1974, Bouvier et al. 1997, Helbing et al 2000, …) Simple agents, simple rules Fast, scalable, emergent patterns

but…difficult to generalize to complex behaviors

Smart Environments(e.g. Kallman et al 1998, Farenc et al. 1999, Thomas et al 2000, Michael et al 2003, …) Objects tell agents what to do Scalable authoring, complex behaviors

but… difficult to compose or get variability

Related WorkInspirations from Systems The Sims (Maxis/EA Games)

Smart environment/ Smart object Scalable authoring Does not consider the social relations

Commercial softwares Massive, SoftImage/Behavior, Character

Studio, AI-Implant…• Author scenes, not environments• Similar means, Different ends• Agent-centric approach

Solution : Probability Scheme

Situations(Sung et el, EG 04)

Collision Avoidance (Sung et al CS-TR-05-1529)Collision Avoidance (Sung et al CS-TR-05-1529)

High-level

Low-level

Probability Scheme

(Sung et al EG 04)

ConstrainedMotion Synthesis

(Sung et al SCA 05)

Probability Scheme

(Sung et al EG 04)

Probability scheme : Agent Architecture Agents:

Discrete set of actions (from mograph) Randomly choose from distributions Behavior functions provide distributions

All aspects of agents can be updated dynamically

Probability Scheme:Simple Default Agents Default agents very simple

Wander, don’t bump into things, …

Extend agents as necessary to achieve complex behaviors

Situations:Extensible Agent Situations extend agents

Add Actions Add Behavior Functions Add Sensors and Rules that inform

Behavior Functions

Example Default agent can’t cross the street. How an agent crosses the street…

Enters a Crosswalk Situation Crosswalk situation extends agent

• Sensor to see traffic lights• Behavior Functions to cross a street• Behavior Functions to stop• Rules to wait for the light to change

Remove extensions when done

Composing Behaviors:Action Selection

Agent

Left

Right

Straight

?

Composing Behaviors:Probability Scheme

BehaviorFunction A

Agent

Left

Right

Straight

.5

.3

.2

Composing Behaviors:Probability Scheme

BehaviorFunction A

Agent

Left

Right

Straight

.5

.3

.2

BehaviorFunction B

.1

.1

.2

.4

.25

.33

Situations Compose Agent can be in multiple situations Agent has union of all the things that

different situations put in

Authoring: Painting Interface

Author environments (not characters)

Set of situation types Paint into environments Mix situations to make

complex/compound ones

Experiments

Gallery environment

Street environment

Solution :Collision Avoidance

Situations(Sung et al, EG 04)

Collision Avoidance (Sung et al CS-TR-05-1529)Collision Avoidance (Sung et al CS-TR-05-1529)

High-level

Low-level

Probability Scheme

(Sung et al EG 04)

ConstrainedMotion Synthesis

(Sung et al SCA 05)

Collision Detection :MOBB(Motion Oriented Bounding Box)

Problem : Collision test between motions A motion is a sequence of poses (frames)

Solution : 3D Bounding box of motion trajectory (X, Y, time)

x

y

t

xy

t

MOBB Trees :Hierarchical subdivision of motions

Level 1 Level 2 Level 3

root

level1 level1

level2 level2 level2 level2

MOBB Trees :Construction of MOBB trees Local Axis-Aligned Method

Motion trajectory

Bounding Sylinder

frame_start

frame_end

d

v1

v2

v3

v4

MOBB Trees : Intersection tests

Given two trees, intersections are tested from the top node to the bottom node until they reach leaf nodes

A00

A10 A11

B00

B10 B11

MOBB(A) MOBB(B)

A10

A11

B11

B10

Experiments

Frame-by-frame MOBB

Short motions(2.1 sec)

4.6 x 10-6 sec 2.3 x 10-6 sec

Long motions(42 sec)

68.3 x 10-6 sec 5.9 x 10-6 sec

Solution :Constrained Motion Synthesis

Situations(Sung et al, EG 04)

Collision Avoidance (Sung et al CS-TR-05-1529)Collision Avoidance (Sung et al CS-TR-05-1529)

High-level

Low-level

Probability Scheme

(Sung et al EG 04)

ConstrainedMotion Synthesis

(Sung et al SCA 05)

ConstrainedMotion Synthesis

(Sung et al SCA 05)

The Goal:Motion Synthesis for Crowds

Problem : Constrained motion synthesis Positions, Orientation, Poses,

Time duration

Requirement Fast performance Accurate meeting constraints High quality motions Complicated environment

Pose

Orientation

Position

Time duration

Target

Initial

Our approach : Synthesize crowds one individual at a time

Structured Motion graphs for low-level synthesis

(Gleicher et al. ’03)

Must adapt to crowds Individual motions must be found

very quickly Pure discrete synthesis cannot

meet continuous constraints

Adapting Graph based Synthesis :

Two-level synthesis• Coarse search for global path planning• Finer search for detailed motion synthesis• Quickly find long motions in complex

environments

Incorporate continuous motion adjustment• Discrete search to roughly satisfy constraints• Additional displacements for precision• Improves speed and accuracy

Related Work (1) Graph based motion synthesis

(e.g. Arikan 2002, Arikan 2003, Gleicher 2003, Kovar 2002, Hue 2004, Lee 2002, Lee 2004, Reitsma 2004) Connecting discrete finite clips with simple

interpolation or displacement mapping

-Create new motions strictly by attaching clips → Hard to satisfy constraints exactly- Do not consider crowds

Related Work (2) Procedural Motion Synthesis (Bouvier 1997,

Boulic 1990, Sun 2001, Boulic 2004)

Controllable but not as realistic as motion capture data

Motion Blending (Guo 1996, Park 2004, Petteré 2003)

Continuous control over trajectory Limited and computationally costly

Algorithm

1. Rough planning1. PRM query

2. Fine planning1. Greedy search2. Create seed motions3. If distance > ε

Randomly select and replace a clip

4. Joining with adjustment

Example

Initial

ObstacleTarget

waypoints

Algorithm

1. Rough planning1. PRM query

2. Fine planning1. Greedy search2. Create seed motions3. If distance > ε

Randomly select and replace a clip

4. Joining with adjustment

Example

Obstacle

Initial

Target

waypoints

1

2 3

Algorithm

1. Rough planning1. PRM query

2. Fine planning1. Greedy search2. Create seed motions3. If distance > ε

Randomly select and replace a clip

4. Joining with adjustment

Example

Obstacle

Initial

Target

BackwardMotion(Mb)

ForwardMotion(Mf)

32

1

Initial’

Algorithm

1. Rough planning1. PRM query

2. Fine planning1. Greedy search2. Create seed motions3. If distance > ε

Randomly select and replace a clip

4. Joining with adjustment

Forward motions

Backward motions

> ε

Compare all pair of motions and return minimum cost

Cost function : How close are they? C(Mf, Mb)

Algorithm

1. Rough planning1. PRM query

2. Fine planning1. Greedy search2. Create seed motions3. If distance > ε

Randomly select and replace a clip

4. Joining with adjustment

Old MotionscOld Motions

New motions

Random select andReplace a clip

< ε

Algorithm

1. Rough planning1. PRM query

2. Fine planning1. Greedy search2. Create seed paths3. If distance > ε

Randomly select and replace a clip

4. Joining with adjustment

Example

Obstacle

Initial

Target

waypoints

Joining

Demos

Time constrained demo Spelling out SCA

Box delivery Big crowds

Performance resultsExample # of

agentDuration

(sec)AVG

Time (sec)Total

Time (sec)

Time constrained

20 14.8 0.21 4.2

Spelling out “SCA”

70 21.3 0.18 12.6

Box delivery 40 30.5 0.15 6

Big crowds 500 25.6 0.035 17.5

Validation : A Virtual City

A city

A block# of characters : 2,000

Validation :Demand Satisfaction : Scalability

Scalable memory

Scalable Authoring• Divide the whole

city simulation into many small situations and reuse them through copy and paste

Validation :Demand Satisfaction : Controllability

Situations show particular behaviors Can put benches any places with

random orientations Able to control crowd flow (crossing the

street) by connecting two opposite situations

Validation :Demand Satisfaction : Efficiency

Fixed environment Fixed density

Crowd simulation performance (excluding rendering)

Validation :Demand Satisfaction : Efficiency

Average speed

Collision detection through MOBB

1.7065e-005 sec

Constrained motion synthesis

0.106 sec

Validation :Demand Satisfaction : Convincingness

Visual Convincingness• No collisions• No noticeable discontinuity on motions

Semantical Convincingness• Depending on situations, crowd show

different specific behaviors

Contributions Scalable, controllable and convincing two-

level crowd simulation framework (Sung et al, EG 04) Situation-based approach Probability scheme

Fast collision detection through MOBB trees (Sung et al, CS-TR-05-1529)

Fast constrained motion synthesis (Sung et al, SCA 05)

Limitation and Future Work Limits to motion data

• Behaviors depend on available motion data

Offline constrained motion synthesis• Searching time is needed

Crowd rendering• Fast skinning of character is needed

User interfaces• More intuitive user interface is required

Thanks

House of Moves

Collaborators

Motion Data

Funding

Michael Gleicher

Stephen Chenney

Demian Gordon Ohio State University

Lucas Kovar

HyunJoon Shin Aaron San Filippo

NSF CCR-9984506 and CCR-0204372

MIC(Ministry of Information and Communication of Korea)