Foreground Detection : Combining Background

Subspace Learning with Object Smoothing Model

Gengjian Xue, Li Song, Jun Sun, Jun Zhou

Shanghai Jiao Tong University

Outline

Introduction 1

Our method 2

Experiments 3

Conclusion 4

Introduction

Foreground detection

detecting moving objects from a video sequences of a

fixed camera

Background: static scene,

Foreground: moving objects

Basic Approach: detect the moving objects as the

difference between the current frame and the image of

the scene background.

Introduction

Challenges

Illumination changes (gradual and sudden)

Dynamic background (swaying tree, ocean waves…)

Scene changes (parked car)

Shadows, Bad weathers, …

Gaussian Mixture Models (GMM)

Some Representative Methods

Kernel Density Estimation (KDE)

signals separation methods

Introduction

Basic Form:

FDY

observed signals Y

F,D background and foreground signals

Introduction

Typical methods

Robust Principal Component Analysis (RPCA)

PCA or ICA

Sparse method(Sparse)

Introduction

Emphasize the D, which is from low subspace

Emphasize the F, which is sparse

Constraining both D (low rank) and F(sparse)

argmin ||D||* + ||F||1

Outline

Introduction 1

Our method 2

Experiments 3

Conclusion 4

Motivation

subspace learning for D 1

spatial clustered

property in F 2

Contribution

A novel framework for foreground detection

simultaneously uses the properties of D and F

An effective solution method

subspace learning for D

an object smoothing model on F

Background subspace learning

The PCA based method computationally intensive

The 2D PCA based method

① J. Yang, etc., “2D PCA: A New Approach to Appearance-Based Face Representation and

Recognition”, PAMI 2004.

② D. Zhang, Z. Zhou, “(2D)2PCA: Two-directional two-dimensional PCA for efficient face

representation and recognition”, Neurocomputing, 2005.

The based method PCAD 2)2(

1. mean image computation

N

i

iAN

A1

1

2. covariance matrices construction

N

i

i

T

i AAAAN 1

row )()(1

C

N

i

T

ii AAAAN 1

column ))((1

C

Background subspace learning

3. projection matrices construction and column

row respectively select M eigenvectors

4. new image projection ))(()(Z row

t

Tcolumn AA

5. new image reconstruction AZ Trowcolumn )(At

6. getting the difference matrix tAAG t

Illustration results by thresholding the matrix G

(a) (b) (c) (d)

(a) ： 1 eigenvector (b) ： 10 eigenvectors

(c) ： 30 eigenvectors (d) ： 40 eigenvectors

Background subspace learning

Foreground refinement

Properties in G

1

Foreground

clustered

3

isolated

noises exist

2

number,

position, and

size of

clusters are

not known

Foreground refinement

Object smoothing model

r

i

c

i

iiii EHG1 1'

11

2

',',H

)(2

1minargH

r

i

r

i

c

i

iiii

c

i

iiii HHHHE2 1 2'

1',',

1'

',1',1 ||||

(1) H is a numerical matrix instead of binary one

(2) It’s convex optimization problem- 2D fused lasso

(3) It’s flexible to impose more constraints

It is more than post-processing approach

Final results:：

Foreground refinement

22

1 1'

11

2

',',H

)(2

1minargH EEHG

r

i

c

i

iiii

r

i

c

i

iiHE1 1'

',2 ||

Th|H|

But we only use the spatial smoothing constraint E1

Outline

Introduction 1

Our method 2

Experiments 3

Conclusion 4

Experiments

FPFNTP

TPscore

2

2F

Three public sequences for testing

GMM , KDE, Sparse methods for comparison

F-score metric is used for evaluation

Waving trees, rippling water and Campus

20 training frames, 3 eigenvectors

, Th = 25

20 training frames, Th = 25

351

7.0,3 bTK

3.0,100thWindowLeng bT

Experiments

Our method:

Sparse method:

GMM method:

KDE method:

Detailed Parameters selection

Experiments

wavingtrees

ripplingwater

campus

Results comparisons

GMM KDE Sparse Ours

Experiments

F-score evaluation Method GMM KDE Sparse Ours

wavingtrees 68.07 73.08 76.31 86.81 ripplingwater 75.24 67.17 78.12 80.88

campus 34.42 51.05 41.93 68.37

based background subspace learning

Conclusion

PCAD 2)2(

A framework coming subspace learning and object smoothing model

A flexible object smoothing model for foreground refinement