Foreground Foreground Background Background

detection from detection from videovideo           

מאת:אבישג אנגרמןמאת:אבישג אנגרמן

The goal: dection of moving The goal: dection of moving objectobject

Why we need thisWhy we need this??

video surveillance. traffic monitoring. Human detection. video editing.

Fusing Complementary Operators to Enhance

Foreground/Background Segmentation

combine two probabilistic approaches:

1. Mixture of Gaussians Algorithm.

2. Statistical Background Disturbance Technique

Mixture of Gaussians Algorithm

Pixel processes –

At any time, t, what is known about a particularpixel, {x0; y0}, is its history

Scatter plots of the red and green values of

a single pixel from the image over time.

The AlgoritemThe Algoritem Model the values of a particular pixel as a

mixture of Gaussians. We determine which Gaussians may

correspond to background colors-Based on the persistence and the variance of each of the Gaussians.

Pixel values that do not fit the background distributions are considered foreground until there is a Gaussian that includes them.

Update the Gaussians. Pixel values that do not match one of the

pixel's “background” Gaussians are grouped using connected components.

Model the values of a particular pixel as a mixture of Gaussians.

At time t we have k distributions of Gaussian for each pixel- determined by the available memory and computational power (Currently, 3-5 are used).

For each Gaussian we have:

-is an estimate of the weight of the ith Gaussian

in the mixture at time t -(the portion of the data

accounted for by this Gaussian)

. - is the mean value of the ith Gaussian in the

mixture, at time t.

- covariance matrix of the

ith Gaussian in the mixture at

time t.

This assumes that the red, green, and blue pixel valuesm areindependent and have the same variances.

Gaussian probability

density function.

The probability of observing the current pixel value is:

Update the mixture model

Stage 1 Every new pixel value, Xt, is checked

against the existing K Gaussian distributions until a match is found.

A match is defined as a pixel value within 2.5 standard deviations of a distribution.

Stage 2-No match If none of the K distributions match

the current pixel value, the least probable distribution is go out.

A new distribution with the current value as its mean value, an initially high variance, and low prior weight, is enter .

Stage 3 The prior weights of the K

distributions at time t are adjusted as follows:

1 for the model which matched and 0 for the remaining models.

The learning

rate

Stage 4 The and parameters for

unmatched distributions remain the same.

The parameters of the distribution which matches the new observation are updated as follows:

The last mean

The value of the new pixel

The last variance

The distance of the new pixel from the updated mean.

Background Model Estimation

determine which of the Gaussians of the mixture are most likely produced by background processes.

We are interested in the Gaussian distributions which have the most supporting evidence = and the least variance.

Why?Why?

For “background” distributions when a static, persistent object is visible hige weight and relatively low variance.

New object occludes the background object creation of a distribution or the increase in the variance of an existing distribution the variance of the moving object is expected to remain larger than a background pixel until the moving object stops

low weight and relatively hige variance.

Background Model Estimation

the Gaussians are ordered by the value of

Then, the first B distributions are chosen as the background model, where

T is a measure of the minimum portion of the data that should be accounted for by the background.

Statistical Background Disturbance Technique

The idea- separates the brightness from the chromaticity component.

Ei represents an expected color of a given ith pixel, in the reference or background image.

Ii represents the color value of the pixel in a current image.

The line OEi called expected chromaticity line.

-brightness distortion- obtained by minimizing

represents the pixel's strength of brightness with respect to the expected value.

1 if the brightness of the pixel in the current image is the same as in the reference image

Less than 1 if it is darker

greater than 1 if it brighter

CD-Color Distortion- The distance between the observed color and the expected chromaticity line.

Background Subtraction

Background modeling - constructs a reference image representing the background.

Threshold selection - determines appropriate threshold values used in the subtraction operation to obtain a desired detection rate.

pixel classication - classies the type of a given pixel, i.e., the pixel is the part of background (including ordinary background and shaded background), or it is a moving object.

Background Modeling

A reference background image computed over a number of static background frames.

Each pixel is modeled by a 4-tuple <Ei; si; ai; bi>

the expected color value.

Are the arithmetic means of the ith pixel's red, green, blue values computed over N background frames.

standard deviation of the ith pixel's red, green, blue values computed over N frame of the background frame.

Cameras typically have different sensitivities to different colors, in order to make the balance weights on the three color bands we normalized the pixel color by its standard deviation.

Pixel Classication or Subtraction Operation

Original background (B): Brightness and chromaticity similar to those of the same pixel in the background image.

Shaded background (S): Similar chromaticity but lower brightness.

Highlighted background (H): Similar chromaticity but higher brightness.

Moving foreground object (F): Chromaticity different from the expected values in the background image.

the variation of the chromaticity distortion of the ith pixel

the variation of the brightness distortion of ith pixel.

Different pixels yield different distributions of and CDi

In order to use a single threshold for all of the pixels, we need to rescale the and CDi

Pixel Classication

0

BS H

0

BSF H

If pixel from a moving object contains very low RGB values, This dark pixel will always be misclassied as a shadow.

Automatic Threshold Selection

First, a histogram of the and are constructed.

The histograms are built during background learning period.

The total sample would be NXY.

The thresholds are now automatically selected according to the desired detection rate r.

A threshold for chromaticity distortion, , is the normalized chromaticity distortion value at the detection rate of r.

In brightness distortion is the value at that detection rate, and is the value at the (1-r) detection rate.

Clustering Detection Elimination

The problem- The problem- those pixels have very small variation in chromaticity distortion i.e. , very small bi.

Hence, limit the value of bi to default minimum bi.

an optimization process is performed.

too big, and likely to exceed the threshold

The process: 1. assigning a default minimum bi value to all

bi that are smaller than the default.2. performing the detection on all frames.

3. compare the error rate of pixels that have bi

bigger than the default value against the error rates of those pixels that have the default bi value.

4. A search is performed to find the default minimum bi that yields a balanced error rates.

Results

http://lecturer.it.kmitl.ac.th/thanarat/research/bgs.html

DemoDemo

Speed-Up Technique Used in Our Implementation

Reduce number of operations at run-time:

Global S vs local si:

Screening Test:

Parallel Processing: By dividing the images into segments

and performing the operations independently on each segment.

ProblemsProblems22

New objects deposited into the scene and become part of the background scene-can be coped with by adaptively update the background model.

Highly specular surfaces (such as mirror, steel, or water surface) when the color of a point on such surfaces can change non-deterministically

Fusing Complementary Techniques

Extending the Mixture of Gaussians to Remove Shadows and Highlights:

Our first improvement is to extend the mixture of Gaussians approach to remove highlights and shadows.

The Enhanced Foreground/Background SelectorThe extended mixture of Gaussians algorithm is joined with the SBD technique by ANDing the results of both operators.

Dynamic Gaussian Background DistributionsLarge size for the background distributions

More background pixels but less accuracy of identifying the moving object pixels.

Small background distribution sizes

Maintain the accuracy of the extracted moving object but will not have the ability to contain all the background pixels

The algorithm first specifies two distribution sizes : small and large.

Used with the small distribution size. The pixels identified as moving

objects are tested using the statistical background disturbance technique.

If the algorithm doesn’t agrees ,the background distributions will be enlarged

REFERENCES

Al-Mazeed, A. H., Nixon, M. S. and Gunn, S. R. (2003) Al-Mazeed, A. H., Nixon, M. S. and Gunn, S. R. (2003) Fusing Complementary Operators to Enhance Fusing Complementary Operators to Enhance Foreground/Background Segmentation. Foreground/Background Segmentation. In: British Machine Vision Conference 2003, 2003, Norwich.In: British Machine Vision Conference 2003, 2003, Norwich.

Learning Patterns of Activity Using Real-Time Tracking C. Learning Patterns of Activity Using Real-Time Tracking C. Stauffer and W. Grimson, IEEE TPAMI, 22(8):747–757, 2000.Stauffer and W. Grimson, IEEE TPAMI, 22(8):747–757, 2000.

A statistical approach for real-time robustbackground subtraction A statistical approach for real-time robustbackground subtraction and shadow detection. T. Horprasert, D. Harwood, and L. Davis. and shadow detection. T. Horprasert, D. Harwood, and L. Davis. In Proceedings IEEE ICCV’99, pages 1–19, 1999.In Proceedings IEEE ICCV’99, pages 1–19, 1999.

The endThe end……