MultiHypothesis PicMultiHypothesis Pictures For H.26Ltures For H.26L

Markus FlierlTelecommunications Laboratory

University of Erlangen-Nuremberg

Erlangen, Germany

mflierl@stanford.edu

Thomas WiegandImage Processing Department

Heinrich Hertz Institute

Berlin, Germany

wiegand@hhi.de

Bernd GirodInformation Systems Laborator

y

Stanford University

Stanford, CA

girod@ee.stanford.edu

To be published in Proc. ICIP, Thessaloniki, Greece, Oct. 2001To be published in Proc. ICIP, Thessaloniki, Greece, Oct. 2001

OutlineOutline• Motion Estimation in H.26L• Introdution to MultiHypothesis Pictures• Coding of MultiHypothesis Pictures• Experimental Results

Motion Estimation in H.26LMotion Estimation in H.26L• Low Complexity

– MC_Range for 16x16 Macro Blocks– Range = ½ MC_Range for Other Sub Blocks– Range = ½ Range for Search in the older ref.

pictures– MVs must be inside of the ref pictures– Choose the results which SA(T)D is minimal.

Motion Estimation in H.26LMotion Estimation in H.26L• High Complexity

– One MC_Range for all Inter Mode and Ref. Pictures– Sub Block Matching starts at the result of 16x16 Ma

cro Block Matching results– Not Forced to contain the MV (0,0)– MVs could be out of the boundaries of the Ref. Pictu

re– Choose the results which much matching the RD co

nstrain.

MultiHypothesis PicturesMultiHypothesis Pictures

MultiHypothesis PicturesMultiHypothesis Pictures• An extension of P pictures.• Each macroblock can be compensated by a linear co

mbi-nation of two motion-compensated macroblocks.

Modified in TML 6Modified in TML 6• Add a individual ref. parameter for

each block.• It’s efficient for H.263, but not for

H.26L– In H.263, 16x16 and 8x8

MacroHypothesis TypesMacroHypothesis Types

MacroHypotheses CodingMacroHypotheses Coding• In the multihypothesis mode

– Two macrohypotheses– For Each macrohypothesis

• Picture ref. Parameter• A Macrohhypothesis type• The corresponding motion vector for each sub-b

lock

EncoderEncoder• A Lagrangian cost function is used for coding

mode decisions.– D: Multihypothesis prediction error– R: Bit rate for two picture ref. Parameterstwo picture ref. Parameters, two mactwo mac

rohypotheses typesrohypotheses types, and the associated motion vecthe associated motion vectorstors.

• For multihypothesis mode– Rate-Constrained multihypothesis motion estimati

on.– Performed by the macrohypothesis selection algo.

Iterative algo. for conditional rate-coIterative algo. for conditional rate-contrained motion estimationntrained motion estimation

• Initial macrohypothesis– The best macroblock type for long-term memory m

otion-compensated prediction.• Steps

1. One macrohypothesis is fixed and conditional rate-constrained motion estimation is applied to the complementary macrohypothesis such that the multihypothesis costs are minimized.

2. The complementary macrohypothesis is fixed and the first macrohypothesis is optimized.

3. To repeat step 1, 2 until convergence.

Rate-Constrained Multi-Hypothesis Rate-Constrained Multi-Hypothesis Motion-Compensated Prediction for Motion-Compensated Prediction for Video CodingVideo Coding

M. Flierl, T. Wiegand, and B. Girod, “Rate-ConstrainedMulti-Hypothesis Motion-Compensated Prediction for VideoCoding,” in Proceedings of the IEEE International Conferenceon Image Processing, Vancouver, Canada, Sept. 2000,vol. III, pp. 150–153.

Conditional rate-constrained Conditional rate-constrained motion estimationmotion estimation

• For the current macrohypothesis, conditional rate-constrained motion estimation determines the conditional optimal picture reference pconditional optimal picture reference parameterarameter, macrohypothesis typemacrohypothesis type, and associatassociated motion vectorsed motion vectors.

• For the conditional motion vectors, an integer-pel accurate estimate is refined to half-pel and quarter-pel accuracy.

Additional MBtypeAdditional MBtype• A joint optimization of multihypothesis motion estima

tion and prediction error coding is far too demanding.• But multihypothesis motion estimation independent

of prediction error encoding is an efficient and practical solution.

Code number MB type 0 Skip 1 16x16 2 8x8 3 16x8 4 8x16 5 8x4 6 4x8 7 4x4 8 MH 9 Intra4x4 ...

Ref_frame Blk_size MVD

Experimental ResultsExperimental Results• Our coder is based on the H.26L test mo

del TML-6. For our experiments, the CIF sequences Mobile & CalendarMobile & Calendar and TempeTempetete are coded at 30 fps30 fps. We investigate the rate-distortion performance of multihypothesis pictures with respect to H.26L P pictures for various long-term memory buffer sizes.