Video Segmentation Based on Image Change Detection

for Surveillance Systems

Tung-Chien Chen

(djchen@soe.ucsc.edu)

EE 264: Image Processing and Reconstruction

Outline

• Background – Image Change Detection– Video Surveillance Systems

• Implementation– Block diagram and algorithm description

• Demo

• Comment

Image Change Detection

Image/Video Sequences

Change Detection Processes

Change Mask

Change Understanding and

Applications

• Differencing• Significance and hypothesis tests• Predictive models• Shading Models• Background Models• Change mask consistency and post processing• …..

• Video surveillance• Remote sensing• Medical diagnosis and treatment,• Civil infrastructure,• Underwater sensing,• Driver assistance systems• ……

In My Project

Image/Video Sequences

Change Detection Processes

Change Mask

Change Understanding and

Applications

• Differencing• Significance and hypothesis tests• Predictive models• Shading Models• Background Models• Change mask consistency and post processing• …..

• Video surveillance• Remote sensing• Medical diagnosis and treatment,• Civil infrastructure,• Underwater sensing,• Driver assistance systems• ……

Video Surveillance Systems• A technological tool that assists humans by

providing an extended perception and reasoning capability about situations of interest that occur in the monitored environments

Video Surveillance Systems• A technological tool that assists humans by

providing an extended perception and reasoning capability about situations of interest that occur in the monitored environments

Reference Paper

• Efficient moving object Segmentation Algorithm Using Background Registration Technique

S-Y Chien, S-Y Ma, and L-G Chen, IEEE Fellow

@ National Taiwan University

IEEE TRANSACTIONS ON CIRCUITSAND SYSTEMS FOR VIDEO TECHNOLOGY,

2002

Block Diagram of the Framework

Previous Frames (1) Diferencing

…

Background

Current Frame

(2) Background Registration

(3) Object Detection and

Initial Mask Generation

(4) Post Processing

Step1 – Differencing (1/2)

• Frame difference and thresholding– Difference between current frame and previous frame

• FD: frame difference• FDM: frame difference mask

Step1 – Differencing (2/2)

• Background differencing and thresholding– Difference between current frame and background

• BD: background difference• BDM: background difference mask

Step2 – Background Registration

• According to FDM, pixels not moving for a long time are considered as reliable background pixels

• SI: Stationary index• BI: Background indicator• BG: Background information

Example of Background Registration (1/2)

Weather #0 Weather #150 Weather #300

CF(Current Frame)

BG(Back-

ground)

Example of Background Registration (2/2)

• Include the function of background updating

Step2- Object Detection and Initial Object Mask Generation

• Object detection– Produce “Initial object mask” (IOM)

BG(Back-ground)

IOM(Initial-Object Mask)

Object Detection

• Look up table for object detection

Step4- Post-processing

• Two main parts in post-processing: – Noise region elimination and boundary

smoothing

• Connected component algorithm to eliminate small regions

• Morphological close–open operations are applied to smooth the object boundary

Example of Post Operation

Initial Object Mask After Connect Component

After Close-open Operation Final Object

Results and Demo

CF(Current Frame)

Frame 0 Frame 75 Frame 300Frame 150 Frame 225

FG(Foreground)

BG(background)

IOM (Initial Object Mask)

Results and Demo

CF(Current Frame)

Frame 0 Frame 75 Frame 300Frame 150 Frame 225

FG(Foreground)

BG(background)

IOM (Initial Object Mask)

Result Demo

Comments (1/2)• For change detection based segmentation

algorithm for surveillance system

– Speed is high, but not robust

– Performance degrade with the uncovered background situation, still object situation, light changing, shadow, and noise

– Post-process can promote, but lose efficiency

– Should automatically decide the thresholds– Some limitations:

• strong change in light source, difference luminance between background foreground, camera moving/zoom/rotation, foreground object should move

Reference[1] R. J. Radke, S. Andra, O. Al-Kofahi, and B. Roysam “Image Change Detection Algorithms: A

Systematic Survey,” IEEE Trans. Image Processing, vol. 14, no. 3, pp. 294–303, March. 2005.

[2] R. Collins, A. Lipton, and T. Kanade, “Introduction to the special section on video surveillance,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 745–746, Aug. 2000.

[3] C. Stauffer and W. E. L. Grimson, “Learning patterns of activity using real-time tracking,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 747–757, Aug. 2000.

[4] C. R. Wren, A. Azarbayejani, T. Darrell, and A. Pentland, “Pfinder: Real-time tracking of the human body,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 780–785, Jul. 1997.

[5] R. Mech and M. Wollborn, “A noise robust method for 2D shape estimation of moving objects in video sequences considering a moving camera,” Signal Process., vol. 66, 1998.

[6] S.-Y.Ma, S.-Y. Chien, and L.-G. Chen, “An efficient moving object segmentation algorithm for MPEG-4 encoding systems,” in Proc. Int. Symp. Intelligent Signal Processing and Communication Systems 2000, 2000.

[7] S. Y. Chien, S. Y. Ma, and L. G. Chen “Efficient Moving Object Segmentation Algorithm Using Background Registration Technique,” IEEE Trans. on circuits and system for video technology, vol. 12, no. 7, pp. 577–586, JULY. 2002.

[8] R. M. Haralick and L. G. Shapiro, Computer and Robot Vision. Reading, MA: Addison-Wesley, 1992.

[9] J. Serra, Image Analysis and Mathematical Morphology. London, U.K.: Academic, 1982.

Thanks for listening !!

Questions ?