Upload
blake-goodwin
View
237
Download
1
Tags:
Embed Size (px)
Citation preview
Introduction
Object tracking: An important computer vision problem Security and surveillance Medical therapy Retail space instrumentation Video abstraction Traffic Management Video editing
Template-Based Tracking A classic technique Idea of template-based tracker
Related Work
Jepson et al. Robust Online Appearance Models for Visual Tracking, CVPR 2001
Ho et al. Visual Tracking Using Learned Subspaces, CVPR 2004
Davis et al. Tracking Rigid Motion using a Compact-Structure Constraint, ICCV 1999
Avidan et al. Ensemble Tracking, CVPR 2005
Overview of the Approach
Next Frame
Forward Correlation
Module
Backward Correlation Module
Textured Background?
Gradient Module
Update Template
YesNo
Template Selection: first frame vs. previous frame Motion Model:
Similarity transformation
Template Selection: first frame vs. previous frame Motion Model:
Similarity transformation
Template-Based Tracking
scaling displacement
Template-Based Tracking
Cross Correlation: SSD Cross Correlation: SSD
displacement reference imagesearch image
Template-Based Tracking
Similarity measure: s(Δx, Δy) Correlation Coefficient: c(Δx, Δy)
Similarity measure: s(Δx, Δy) Correlation Coefficient: c(Δx, Δy)
Mean of templateMean of image region
Forward Correlation
Forward Correlation: Reference frame: previous frame Goal: find transformation vector (dx, dy, α) Approach: cross-correlation
Forward Correlation: Reference frame: previous frame Goal: find transformation vector (dx, dy, α) Approach: cross-correlation
Template Update:Template Update:
Put into correlation coefficient framework
Forward Correlation
Drifting Problem: Forward correlation
approximates rotation with translation.
Forward correlation does not check the reliability of the template.
We need a mechanism to question the assumption of forward correlation.
Drifting Problem: Forward correlation
approximates rotation with translation.
Forward correlation does not check the reliability of the template.
We need a mechanism to question the assumption of forward correlation.
Out-of-plane rotationOut-of-plane rotation
Previous frame
Current Frame
Backward Correlation
Consider our problem as motion segmentation Goal of motion segmentation Why is motion segmentation of video sequences difficult?
Under-constrained Occlusion & Disocclusion Image noise
A two-step procedure: Determine the motion vectors associate with each pixel or feature point. Group pixels or feature points that perform common motion.
Backward Correlation
Kanade-Lucas-Tomasi (KLT) feature tracker Idea: minimize the dissimilarity of feature windows in
two images Assumption: mutual correspondence
Kanade-Lucas-Tomasi (KLT) feature tracker Idea: minimize the dissimilarity of feature windows in
two images Assumption: mutual correspondence
Backward Correlation
Now consider the dissimilarity under the template window. Decompose the template window into 2 partitions:
Now consider the dissimilarity under the template window. Decompose the template window into 2 partitions:
Rewrite dissimilarity as:Rewrite dissimilarity as:
foreground background
highlow
Backward Correlation
Background is moving at a different velocity than the foreground. Foreground pixels have similar velocity and generate low SSD error. Correlation between background pixels using foreground velocity
generates high SSD error. Goal: group foreground pixels which are moving at similar velocities
Background is moving at a different velocity than the foreground. Foreground pixels have similar velocity and generate low SSD error. Correlation between background pixels using foreground velocity
generates high SSD error. Goal: group foreground pixels which are moving at similar velocities
Reference frame I(x)Reference frame I(x)Current image J(x)Current image J(x) Difference image
D(x)=[I(x)-J(x+d)]2
Difference image
D(x)=[I(x)-J(x+d)]2
Backward Correlation
Formulations for backward correlation Formulations for backward correlation
Set of template candidates
Correlation coefficient (likelihood)
Untextured Backgrounds
Limitation of backward correlation: Fails if background has little texture. Why? --- Examine the assumption. Backward correlation has no reason to prefer the
foreground to the background which is untextured.
Limitation of backward correlation: Fails if background has little texture. Why? --- Examine the assumption. Backward correlation has no reason to prefer the
foreground to the background which is untextured.
Also low if untexturedlow
Untextured Backgrounds
Likelihood of backward correlation: textured vs. untextured Likelihood of backward correlation: textured vs. untextured
Textured backgroundTextured background Untextured backgroundUntextured background
Foreground
Template containing background pixels
Gradient Module
Motivation: Seek a module focusing on the boundary of the target being tracked.
An edge-based segmentation problem. Prior information: an ellipse model.
Gradient Module:
Motivation: Seek a module focusing on the boundary of the target being tracked.
An edge-based segmentation problem. Prior information: an ellipse model.
Gradient Module:
Unit vector normal at pixel i Intensity gradient
Combining Modules
Gradient module and backward correlation module have orthogonal failure modes.
Textured or Untextured? Use sum of the gradient magnitude of the
neighborhood region. Combination of forward correlation module and
backward correlation module is straightforward. Combination of forward correlation module and
gradient module requires the normalization of the matching scores.
Combining Modules
Normalize the matching score (likelihood): Normalize the matching score (likelihood):
Finial state is decided by:Finial state is decided by:
Adaptive Scale
Vary the scale by ± 10 percent during search process. Filter the result to avoid oversensitive scale adaptation.
Comaniciu et al. Kernel-based object tracking, TPAMI 2003
Vary the scale by ± 10 percent during search process. Filter the result to avoid oversensitive scale adaptation.
Comaniciu et al. Kernel-based object tracking, TPAMI 2003
Size of the best state given by the alg.
Size of the object in the previous frame
Experimental Results:Cluttered Background
Traditional template-based tracker slides off target: Traditional template-based tracker slides off target:
Experimental Results:Cluttered Background
Our algorithm remains locked onto target: Our algorithm remains locked onto target:
Experimental Results:Cluttered Background Tracking error plot:
Our algorithm (blue, solid) vs. traditional template-based tracker (red, dashed)
Tracking error plot: Our algorithm (blue, solid) vs. traditional template-based tracker (red, dashed)
Error in x direction Error in y direction
Experimental Results:Untextured Background
Tracking results of traditional template-based tracker: Tracking results of traditional template-based tracker:
Experimental Results:Untextured Background
Tracking results of our algorithm: Tracking results of our algorithm:
Conclusion
Presented an extension to template-based tracking. Achieved robustness to out-of-plane rotation. Effective tracking in both textured and untextured
environment. Remaining challenges:
Robustness when scale changes. Use motion discontinuities to improve performance. Analysis of parameter sensitivity for untextured
backgrounds.