Real-time foreground object detection & tracking with moving camera

P93922005 Martin Chang

Motivation More and more moving cameras

Handheld devices Cell phone PDA

Hard to track object with moving camera

Hard to learn background with moving camera

Previous Work Thompson, W.B. and Pong, T.C. Detecting moving objects. Int

ernational Journal of Computer Vision, 4(1):39-57. (January 1990).

Stationary Camera K Daniilidis, C Krauss, M Hansen, G Sommer. Real-Time Track

ing of Moving Objects with an Active Camera. Real-Time Imaging, 1998.

Two degrees of freedom of a camera platform E Hayman, JO Eklundh - Procs. Statistical Background Subtra

ction for a Mobile Observer. IEEE Intl. Conf. on Computer Vision.

Moving foreground object static background Mobile observer

Steps

1. Find good feature to track2. Track features3. Classify foreground and

background features 4. Decide region of foreground

object5. Track foreground object

Video Demo

Step 1: Find good feature to track

Finding good feature to track Shi and Tomasi ‘s method

Step 2: Track features

Optical flow

Step 3: Classify foreground and background features

Classify feature points Optical flow

Moving direction of feature Length of moving direction

MTF of neighbor image patch Doesn’t work, due to

With cheap camera Low resolution video

Idea: how to identify foreground features? 1/3

Background Object

Case 1: The camera rotates The background image moves more

Idea: how to identify foreground features? 2/3

Case 2: The background moves The background image moves more

Background object

Idea: how to identify foreground features? 3/3

Background object

Case 3: The object moves The foreground image moves more

Classify Features

KMeans Hard to separate them well

Marginal KMeans Filter unreliable features

Angle issue 1° is similar to 359 °

KMeans

Marginal KMeans (Margin=1/2)

Marginal KMeans (Margin=1/4)

Step 4: Foreground Object Detection 1/2

Two two-class problems Classify foreground and background

features Cluster features

Calculate the occlusion rate The region of foreground object should be

Compact Less noise (background features)

Foreground Object Detection 2/2

Measure our confidence Geometry approach Check foreground and background

regions

Step 5: Foreground Object Tracking

1. Object detection2. If foreground object is never

detectedGo to Step 1

3. Object tracking4. Go to Step 1

Development Platform

Microsoft Visual C++ .NET 2003 Cheap webcam (USB 1.1) OpenCV

Future Work

Find parameters by machine learning

Detect finite candidate objects Cue: color moment

Multiple object detection(!)

Conclusion

The bottleneck is camera’s data transporting speed (USB 1.1)

Real time is possible OpenCV is useful

Demo

Thanks!