Robust Object SegmentationUsing Adaptive Thresholding

Xiaxi Huang and

Nikolaos V. Boulgouris

International Conference on Image Processing 2007

Outline

Introduction Proposed algorithm Experimental results Conclusions

Introduction(1/2) The extraction of moving objects from video se

quences is important!! Object detection methods

SGM ( Single Guassian Model ) MGM ( Mixed Guassian Model ) BG substaction combined color and edge inform

ation. (Aug. 2000)

Serious drawbacks

Poor performance for indoor shadow, light reflection, and high similarity of FG and BG.

Introduction(2/2)

Proposed algorithm Adaptive thresholding detection Shadow removal method

Proposed AlgorithmProcedures Of Algorithm

UpdatingBG Image

Initial MaskEstimate FG Areas

EstimateBG Areas

ConfidenceMap From Detection

Of RGB

ConfidenceMap From Detection Of Edge

Maximum Of The

ConfidenceMaps

Minimum Of The

ConfidenceMaps

Combined Confidence

Map

HysteresisThresholding

FG Map(Object andShadows)

Edge Map(BoundaryOf objects)

Post-processing

Final ObjectMap

Next Frame

BackgroundSubstraction

ShadowRemoval

Proposed AlgorithmBackground Updating

Why? In many background substraction method up

date all pixels in a frame. A serious drawback

To avoid this condition

Misclassfied a stop moving object.

11 1 ttt BIB

Proposed AlgorithmInitial Block-size Mask(1/5)

Roughly determine the foreground areas. Lower threshold

Calculate average different between the current frame and background frame in a block

Threshold it with T

BIdavg

1

Proposed AlgorithmInitial Block-size Mask(2/5)

Divide the blocks with larger difference which are assumed to contain foreground pixels into smaller size

Apply a higher threshold and detect sub-blocks

41 /nn

addednn TTT 1

Proposed AlgorithmInitial Block-size Mask(3/5)

The block with larger difference

Using a higher threshold

…

Proposed AlgorithmInitial Block-size Mask(4/5)

How to get a foreground blocks map? Median filter

Edge pixels of the objects might be lost Apply two initial block-size, and combine

their foreground map

' MapMapMapblock

Proposed AlgorithmInitial Block-size Mask(5/5)

Proposed AlgorithmColor Change Detection With Adaptive Threshold(1/2)

Adaptive threshold

D : difference frame ( D = | I - B|)

202

2

002

0

22

21

1

1

1

/

currcurr

diffdiff

diff

diffdiffcurradapt

I

D

D

kkT

Local variance in the current frame

Local variance in the difference frame

Local mean value in the difference frame

Proposed AlgorithmColor Change Detection With Adaptive Threshold(2/2)

Get a threshold T by setting k1, k2

Create confidence maps in three color channels respectively Maximum the confidence map CMapcolor

Proposed AlgorithmEdge Detection(1/2)

In order to － Extraction of the foreground Removal of shadow

Compute edge magnitude

Gx and Gy are the horizontal and vertical difference in the difference frame D.

Sobel mask

22yxmagnitude GGG

121

000

121

101

202

101

yx HH

Proposed AlgorithmEdge Detection(2/2)

Proposed AlgorithmCombination(1/3)

Combine two confidence maps. Estimate foreground area

Maximum the confidence map

Estimate background area Minimum the confidence map

blockedgecolor MapCMapCMapCMap ,max1

blockedgecolor MapCMapCMapCMap ,min2

Proposed AlgorithmCombination(2/3)

Combination

21 CMapCMapCMap

Proposed AlgorithmCombination(3/3)

(a) (b)

(c) (d)

Fig. (a) Original image, (b) Confidence map Of RGB change Detection with adaptive threshold, (c) Confidence map of Sobel edge detection, (d) Combined confidence Map.

Proposed AlgorithmHysteresis thresholding

Remove false positive Set two thresholds T0, T1 ; T1/T0 is about 2

or 3 C(x) > T1 : High confidence region T0 < C(x) < T1 : Check neighbors

Hysteresisthresholding

Proposed AlgorithmShadow Removal And Post-Processing(1/4)

Indoor environment Soft colored illumination, light-reflect

floor, shadow Hard to distinguish shadows from objects

by using color information. How to solve this problem?

Combine FG and edge confidence map

Proposed AlgorithmShadow Removal And Post-Processing(2/4)

Apply hysteresis thresholding to edge confidence map

Set bounding boxes Remove pixels out of bounding boxes

Proposed AlgorithmShadow Removal And Post-Processing(3/4)

(a) (b)

(d)(c)

Fig. (a) Foreground map before shadow removal, (b) Hysteresis thresholdingresult of edge confidence Map, (c) Foreground mapafter shadow removal, (d) Binary map of extracted objects.

Proposed AlgorithmShadow Removal And Post-Processing(4/4)

Some temporal filters of offline detection

To achieve above －

11

11

tttt

TF

ttttTF

MapMapMapMap

MapMapMapMap

11 ,, ttttTF MapMapMapmedianMap

Eliminate spurious points

Retrieve missed FG pixels

Experimental ResultsFig. (a) Origin image, (b) Foreground maps createdby MGM plus HSV method, (c) Foreground maps created by mixture GaussianModel, (d) Foreground mapscreated by proposed algori-thm.

(a) (b)

(d)(c)

Experimental ResultsFig. (a) Origin image, (b) Foreground maps createdby MGM plus HSV method, (c) Foreground maps created by mixture GaussianModel, (d) Foreground mapscreated by proposed algori-thm.

(a) (b)

(d)(c)

Experimental ResultsFig. (a) Origin image, (b) Foreground maps createdby MGM plus HSV method, (c) Foreground maps created by mixture GaussianModel, (d) Foreground mapscreated by proposed algori-thm.

(a) (b)

(d)(c)

Experimental ResultsFig. (a) Origin image, (b) Foreground maps createdby MGM plus HSV method, (c) Foreground maps created by mixture GaussianModel, (d) Foreground mapscreated by proposed algori-thm.

(a) (b)

(d)(c)

Conclusions Compared with the popular MGM object

segmentation method and the HSV shadow removal method, proposed method achieves more robust performance

Considering that the proposed algorithm does not involve future frames, it can be used in real-time processing applications.

Furthermore, if it is used offline, a temporal filter can be applied to further improve the performance of the algorithm.