Upload
osborn
View
47
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Methods of Video Object Segmentation in Compressed Domain. Cheng Quan Jia. Presentation Outline. Features for Segmentation in Compressed Domain Using Motion Vectors in Segmentation Confidence Measure Conclusion Q & A. Features for Segmentation in Compressed Domain. - PowerPoint PPT Presentation
Citation preview
Methods of Video Object Segmentation in Compressed Domain
Cheng Quan Jia
Presentation Outline
• Features for Segmentation in Compressed Domain
• Using Motion Vectors in Segmentation
• Confidence Measure• Conclusion• Q & A
Features for Segmentation in Compressed Domain
An introduction to Compressed Domain
Compressed Domain: Definition
• Compressed Domain refers to video compression techniques that expliots Spatial and Temporal Redundancy using – DCT & Quantization– Motion Compensation
• Examples include MPEG-1/-2/-4, H.261 and H.263
Compressed Domain: Definition
• Opreations in the Compressed Domain involves processing of
– DCT coefficients (from I-macroblocks)– Motion Vectors (from P-/B-macroblocks)
Compressed Domain: Parsing
• Unlike pixel domain, operations in the compressed domain do not require the input bitstream to be decoded
• Instead, they are Parsed
Compressed Domain: Parsing
Features for Segmentation
• After Parsing, we have– DCT coefficients (from I-macroblocks)– Motion Vectors (from P-/B-macroblocks)
• Which coresspond to – Frequencies of texture change– Motion of the macroblock
Using Motion Vectors in Segmentation
Acquiring Dense Motion Field
• Many video object segmentation methods attempt to acquire a dense smooth motion field in order to create object masks
• For this end spatial interpolation and motion accumulation are employed
Motion Accumulation
Motion Accumulation
• Due to the different magnitude and signs of motion vectors, the obtained MVs are normalized, e.g. MVs in B-macroblocks would have their signs reversed
• Filtering is applied to remove non-uniform MV and smooth the motion field
Motion Accumulation
• Chen and Bajic [chen2009] employs MV Integration block-wise and pixel-wise to enhance the Motion Field
Motion Accumulation
Chen and Bajic [chen2009] Babu et al. [babu2004]
Porikli et al.’s Investigation
• The Compression Domain segmentation system published by Porikli et al. [porikli2010] experimented the effect of DCT coefficients and MV on segmentation performance– The DC parameters(for Y, U, V channels) of the I-
frame– Low vertical and horizontal frequency AC values– A spatial energy term– Aggregated motion flow of the corresponding
macroblock
Porikli et al.’s Investigation
• They create a Frequency-temporal data structure for each macroblock with the features and perform volume segmentation
• Their results show that using DCT terms in FT segmentation and using MV in the hierarchical clustering, on average, gives better results.
Porikli et al.’s Investigation
• The Block Matching Process in encoding stage looks for only the best match for a macroblock rather than object motion
Porikli et al.’s Investigation
Confidence Measure of Motion Vectors
• Coimbra and Davies [coimbras2005] try to approximate Lucas–Kanade optical flow in MPEG-2 Compressed Domain
Approximating Optical Flow
• They argue that AC[1] and AC[8] in an I-macroblock can be used as confidence measure
• The confidence update step will have a 8×8 macroblock referencing a 16×16 image block in the I-frame, and the confidence of the motion vector of the macroblock is the weighted average of confidence in the 16×16 window
Confidence Measure
Confidence Measure
Original image MPEG-2 smooth motion field afterconfidence threshold
Conclusion
• Due to block matching process, motion vectors in P-/B- frames do not necessary relate to object motion
• To ensure a motion vector is correlated to object motion, some sort of confidence measure is required
• [coimbras2005] demonstrated that edge strength can be an effective measure
Conclusion
• Problems not discussed here – Camera motion– Changes in illumination– Occlusions
Conclusion
References1. R. V. Babu, K. R. Ramakrishnan, and S. H. Srinivasan.
Video Object Segmentation: A Compressed Domain Approach. IEEE Transactions on Circuits and Systems for Video Technology, 14(4):462–473, April 2004.
2. Y.-M. Chen and I. V. Bajic. Compressed-Domain Moving Region Segmentation with Pixel Precision using Motion Integration. In IEEE Pacific Rim Conference on Computers and Signal Processing, 2009, pages 442 – 447, August 2009.
3. M. T. Coimbra and M. Davies. Approximating Optical Flow Within the MPEG-2 Compressed Domain. IEEE Transactions on Circuits and Systems for Video Technology, 15(1):103–107, January 2005.
4. F. Porikli, F. Bashir, and H. Sun. Compressed Domain Video Object Segmentation. IEEE Transactions on Circuits and Systems for Video Technology, 20(1):2–14, January 2010.
Q & A SECTION
Comments and Suggestions