Sparse Bayesian Learning for Efficient Visual Tracking

O. Williams, A. Blake & R. Cipolloa

PAMI, Aug. 2005.

Presented by Yuting Qi

Machine Learning Reading Group

Duke University

06/24/2005

Overview

• Motivations - an extension of SVT

• Bayesian tracking with RVM

• Overall system

• Experimental results

Motivations• Support vector tracking (SVT) [1]

– Training a SVM classifier through the labeled image database;

– For a given test image, the tracked object region is located by maximizing the SVM score.

– Using first-order Taylor expansion, Ifinal is the linear transformation of image gradient, Ix & Iy.

[1] Shai Avidan, “Support vector tracking”, IEEE Tran. On PAMI, Aug, 2004

l

jjjj

vuvukyvu

1yxinit

,),(max*]*,[ xIII

[u,v]: motion vector

Ifinal: correct object region;

I: all possible regions;

Motivations

• Limitations of SVT– Is the optimization efficient using different kernels?– Is the optimization function suitable?– Smoothing image gradient may decrease performance;

• Properties of RVM Tracker– Fully probabilistic regression for displacement;– Observations of displacement are fused temporally with

motion prediction;– Online tracking;

Bayesian Tracking with RVM• Building a displacement expert-RVM

– Train an RVM to learn the relationship between images and motion. For a test image region x, RVM returns the displacement :

– Mapping from image space to state space.– 4 dimensional state space:

• Translation in x, y, rotation, scaling• Each dimension building one RVM

u

01

),()( wkwgN

iii

zxxu

• Creating training dataset– Given a seed image I containing the labeled ROI λ;– Generating training examples {z} from I:

• Sampling random displacements from a uniform distribution:

• Corresponding state:

• Generating example zi from state u

– Real training examples:

• Learning the expert– Given N training examples: {zi, ti}, i=1,…,N. – The relationship between subimages zi and displacem

ent ti is

– Considering additive processing noise:

– Learning

– Posterior is also Gaussian:

01

),()( wkwgN

jjiii

zzz iii gt )(z

)),(;(}){|( 2iiii gtNtp zz

),0;(~ 2 Ni

)ˆ,ˆ;(),},,{|( 2 wwαtzw Np ii

),0;()( iii wNwp

}),|,({logmax* 2αtzαα

iip

• Tracking with the expert– Given the test image I, initial state u0

– Get ROI x by sampling I around u0.

– The expert outputs the probability distribution of the corresponding displacement

– Assume the state transition probability is:

– Plug those into Kalman-Bucy filter for tracking

),ˆ|(~),ˆ,|( SuuSuxu Np

u

01

),()( wkwgN

iii

zxxu

Gaussian innovation

State predict

Innovation

State update

• Tracking algorithm

Overall System

• A validator is adopted to achieve the tracking robustness.• Absence of the verification of the tracked object triggers

a exhaustive search over the input image by the classifiers.

Face Tracking Results

• Row (1): deformation• Row (2): occlusion (lost track in the last frame)• Row (3): lighting variation

(1)

(2)

(3)

Hand Tracking Results

Cars Tracking Results

Long-term Tracking Results

Conclusions

• Develop a tracker using sparse probabilistic regression by RVMs.

• RVM can be trained from a single image (generating training set).

• Robustness is obtained by the object verification.