BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by...

BraMBLe: The Bayesian Multiple-BLob Tracker

By Michael Isard and John MacCormick

Presented by Kristin Branson

CSE 252C, Fall 2003

Problem The goal is to track an unknown number of

blobs from static camera video.

Solution The Bayesian Multiple-BLob (BraMBLe)

tracker is a Bayesian solution. It estimates

),...,,|p( 11 ZZZX ttt

State at frame t Image Sequence

Number,Positions,Shapes,Velocities,…

Bayes Rule

p(p( )) p)| (| Z XX XZ

Posterior State Distribution Observation

Likelihood

Prior

Sequential Bayes

Posterior State Distribution Observation

Likelihood

Prior

Instead of modeling directly, BraMBLe models and .

)|p( :1 tt IX)|p( tt XI )|p( 1tt XX

1 1 1: 1 11: p( | ) p(p( | ) ) | )p( | t tt t tt tt t d X X X Z XZ XX Z

)|p( 1:11 tt ZX

Update Algorithm

)|p( :1 tt ZX

)|p( :1 tt ZX

)|p( 1:1 tt ZX

)|p( 1tt xx

)|p( tt XZ

)|p( :11 tt ZX

)|p( 11 tt XZ)|p( 1 tt XX

1 1 1: 1 11: p( | ) p(p( | ) ) | )p( | t tt t tt tt t d X X X Z XZ XX Z

Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.

)|p( tt XZ)|p( 1tt XX

)|p( :1 tt ZX

Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.

)|p( tt XZ)|p( 1tt XX

)|p( :1 tt ZX

Image Observations We want to choose our observations so

that we can compute quickly:

Z

)|p( XZ

g gz )|p()|p( XXZ

Individual observations are conditionally independent.

Image Observations We want . A bank of filters is applied at each grid point.

Z

g gz )|p()|p( XXZ

Y

Cr

Cb

Filter plots from http://www.cs.jhu.edu/~wolff/course600.461/week3.2/sld012.htm

Mexican Hat

Gaussian

Image Observations We want to choose our model so that we

can compute quickly:

Z

)|p( Xgz

g ggg g lzz )|p()|p()|p( XXZ

Observation depends on membership of grid point.

0gl

2gl1gl

3gl

Image Observations We want to choose our model so that we

can compute quickly:

We can precompute and quickly evaluate any state x.

Z

)|p( Xgz

g ggg g lzz )|p()|p()|p( XXZ

ggg llz )|p(

Appearance Models The appearance models are learned from

training data.

forek

foref

forefpfpf G

Klz ),(

1)0|p(

Training Data

backk backbackgk

backgkgg G

Klz ),(

1)0|p(

Observation Likelihood

)0|p(

)0|p(log

gg

gg

lz

lz

Observation Likelihood Review We defined our image observations

so that We defined our observation model

so that

We can precompute and quickly evaluate for many choices of x.

Z.)|p()|p(

g gz XXZ

)|p( Xgz

.)|p()|p()|p( g ggg g lzz XXZ

ggg lglz ,)|p( )|p( XZ

Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.

)|p( tt XZ)|p( 1tt XX

)|p( :1 tt ZX

Object Model The blob configuration is

mm xxX ...,,, 1Number of objects

ObjectState

1x

2x

3x

3m

X

Object Model The blob configuration is

The object state is

svlx ,,,i

mm xxX ...,,, 1

IdentityLocation

VelocityShape

X

Person Model

Generalized-Cylinder Model

CalibratedCamera

Prediction Model The number of objects can change:

Each object has a constant probability of remaining in the scene.

There is a constant probability that a new object will enter the scene.

)|p( 1tt XX

r

i

In this formulation, hypotheses with different numbers of objects can be compared directly.

Prediction Model Damped constant location velocity:

)|p( 1tt XX

11 1 12p( | , ) ( 0 8 ). ,t tt tt xx x xG xx

1tx 1tx

1 10.8t tx x

Prediction Model Damped constant location velocity:

Auto-regressive shape:

)|p( 1tt XX

11 1 12p( | , ) ( 0 8 ). ,t tt tt xx x xG xx

12

1 1 10.2(p( | ) ( ,) )t t tt t sss sG s s

1ts

1 1 10.2( )t t ts s s

1ts

Model Review

The observation likelihood is fast to compute for different hypotheses .

The prediction model allows generation of from

Estimating requires an efficient way ofRepresenting .Computing the multiplications and integrations.

tX

tX 1.t X

)|p( tt XZ

)|p( 1tt XX

1 1 1: 1 11: p( | ) p(p( | ) ) | )p( | t tt t tt tt t d X X X Z XZ XX Z

)|p( :1 tt ZX

)|p( :1 tt ZX

Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.

)|p( tt XZ)|p( 1tt XX

)|p( :1 tt ZX

Efficient Representation is represented by a set of particles,

:

Sampling from the set using the weights approximates sampling from

1tX 2

tX 1Nt

X NtX...

N Points:

N Weights: 1t 2

t 1Nt N

t

)|p( :1 tt ZX

1:p( | ).t tX Z

{( , )}i it tX

Efficient Representation is represented by a set of particles,

:

Sampling from the set using the weights approximates sampling from

)|p( :1 tt ZX

1:p( | ).t tX Z

{( , )}i it tX

Efficient Computation The particle set representing is

computed from by CONDENSATION:

1. Apply dynamics to the particle set:

2. Multiply by the observation likelihood:

)|p( :1 tt ZX

1 1: 1p( | )t t X Z

: 1: 11 p( | p(p( | ) ) | )t tt tt t X ZX XZ Z

1 1 1: 1 11: p( | ) p(p( | ) ) | )p( | t tt t tt tt t d X X X Z XZ XX Z

1: 1 1 1 11: 1p( |p( p( )| ) | ) tt t t t tt d X ZX XXZ X

Applying Dynamics Given particle set , compute1 1: 1p( | )t t X Z

Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.018.gif

1 1: 1p( | )t t X Z

1: 1p( | )t t X Z

)|p( 1tt XX

1: 1 1 1 11: 1p( |p( p( )| ) | ) tt t t t tt d X ZX XXZ X

Applying Dynamics Given particle set , compute

1. Resample into

1 1: 1p( | )t t X Z

1 1{( , )}i it t X

1{( ' ,1/ )}it NX

1 1{( , )}i it t X

1{( ' ,1/ )}it NX

Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.020.gif

1: 1 1 1 11: 1p( |p( p( )| ) | ) tt t t t tt d X ZX XXZ X

Applying Dynamics Given particle set , compute

1. Resample into

2. Predict, generating to give 1p( | ' )i i

t t t X X X{( ,1/ )}.i

t NX

1{( ' ,1/ )}it NX

{( ,1/ )}it NX

1 1: 1p( | )t t X Z

1 1{( , )}i it t X

1{( ' ,1/ )}it NX

1: 1 1 1 11: 1p( |p( p( )| ) | ) tt t t t tt d X ZX XXZ X

Multiplication by Likelihood Given particle set , compute

Weight particles, setting

1: 1p( | )t t X Z

:1 1 1:p( | ) p( | )p( | .) t tt tt t Z X X ZX Z

1: 1p( | )t t X Z

)|p( tt XZ

)|p( :1 tt ZX

{( ,1/ )}it NX

{( , )}i it tX

p( | ).i it t t Z X

Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.021.gif

Efficient Computation Review The particle set representing is

computed from by CONDENSATION:

)|p( :1 tt ZX

1 1: 1p( | )t t X Z

1 1: 1p( | )t t X Z

1 1{( , )}i it t X

1 1: 1p( | )t t X Z

1{( ' ,1/ )}it NX {( ,1/ )}i

t NX

1: 1p( | )t t X Z )|p( :1 tt ZX

{( , )}i it tX

Resample Predict Reweight

Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.022.gif

Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.

)|p( tt XZ)|p( 1tt XX

)|p( :1 tt ZX

People Tracking

People Tracking Tracking was successful in real time on

this 53s clip except when two people crossed in front of a third.

Outline Observation likelihood model . Prediction model . Estimation of posterior . Results. Discussion.

)|p( tt XZ)|p( 1tt XX

)|p( :1 tt ZX

Algorithm Summary The models chosen

Are a smooth integration of foreground and background models.

Allow hypotheses with differing numbers of objects to be compared directly.

Can be quickly evaluated in a particle filtering implementation.

Relationship to Previous Work Static camera blob tracking:

Classifies pixels as foreground or background.

Application of Stauffer and Grimson’s Adaptive Background Subtraction to video with compression artifacts

Video from http://image.pirl.umd.edu/knkim/research/BGS/compressed_video.htm

Relationship to Previous Work Static camera blob tracking:

Classifies pixels as foreground or background. Static camera blob tracking:

Finds the position in the search area Made up of foreground pixels. Matching the blob in the previous frame.

Frame t - 1 Frame t Frame t + 1

Predicted position in frame t + 1

Search area

Relationship to Previous Work Improvements over blob tracking:

Integrates the foreground and background modeling.

Multiple objects can be tracked through occlusions.

Video from http://robotics.stanford.edu/~birch/headtracker/Video from http://www.robots.ox.ac.uk/~jmac/research/thesis/thesis.html

Weaknesses The algorithm is sensitive to reflections.

The algorithm sometimes switches the labels when one object passes in front of another.

There are a lot of parameters to assign.