1/23/2005page1 11/11/2004

MPEG4 Codec for Access GridsNational Center for High Performance Computing

Speaker: Barz Hsu

Email: barz@nchc.org.tw

1/23/2005page2

Outline

• Definition of HDTV• Brief Introduction to MPEG-4 Standard• History and Problems of VIC• Contributions

1/23/2005page3

What’s High Definition Video?

• Frame Resolution: >= 720x480 or 640x480

• Frame Rate >= 20 fps

• Peak Signal to Noise Ratio (PSNR)>= 30 dB

1/23/2005page4

Another Key Factors for Access Grids

• Real-Time– Minimize latency & jitter

• Error Resilience– RTP over UDP is unreliable

• Large Scale Video Conferencing– Selection of suitable coding tools– Optimization

• Adaptive– network bandwidth– computing resources

1/23/2005page5

Status of MPEG4

• Part 2: Advanced Simple Profile (ASP)• Part 10: Advanced Video Coding (AVC or H.264)

• Match MPEG-2 quality at up to half data rate• Deliver excellent video quality across entire

bandwidth spectrum – from 3G to HD– (from 40Kbps to 10MBps)

1/23/2005page6

Performance Comparison

1/23/2005page7

Interoperability Issues

• Communicate with Polycom, iChat AV (Tiger)• Transport Stream

– RFC 3016: RTP Payload Format for MPEG-4 Audio/Visual Streams.– RFC 3640: RTP Payload Format for Transport of MPEG-4

Elementary Streams– MPEG2TS

1/23/2005page8

VIC: Video Conferencing Tool

• Developed by Network Research Group in U.C. Berkeley

• Mbone Tools used by Access Grid, inSORS, and VRVS– vic for video and rat for audio

• Based on RTP/RTCP to provide real-time video• Support Multicast/Unicast Protocol• Hybrid of Tcl/Tk and C++ • Supported codecs

– H.261, H.263, H.263++, MJPEG, nv, and etc.

1/23/2005page9

The Problems of VIC

• Hard to maintain

• Lack of high-quality video codec– only H.261 is relative useable

• Limitation on frame resolution– Only QCIF (176x144), CIF (352x288)

• Limitation on viewing resolution• Lack of exhaustive optimization on codec, render, grabber,

and etc

1/23/2005page10

Our Goal

• Broadcast in Television Quality– Leverage with Existing Open Source Resources

• VLC, MPlayer, and FFmpeg– Providing “Sense of Presence”– As Real-Time As Possible– High-Definition, High Quality– Incorporate with up-to-date codecs such as MPEG2, MPEG4

(ASP), H.264– Less Compression Artifact– Error Resilience/Concealment

1/23/2005page11

MPEG4 Codec

• Related Open Source Project– FFmpeg, xvid, and sklmp4

• FFmpeg/libavcodec– High performance of CPU usage– Support various codec

• MPEG4, MPEG2, MPEG1• H.263++, H.263, H.261

– Provide error resilience tools• slices, data partition

– Cross Platform• Linux, Windows, and Mac OS

1/23/2005page12

Benefits from MPEG4 codec

• Support arbitrary sizes of a video stream – in contrast to H.261– the largest size depends on your video grabber card and camera

• Less obvious block artifacts

1/23/2005page13

MPEG4 Performance

• Typical bit-rate is 1Mbps– 720x480 (NTSC) with 25 frames/sec and PSNR value is about 40 dB

• Consistent bit-rate– adjusting bit-rate only affects to the quality; not influence the frame-

rate• Up to 40 CIF, or 10 NTSC video streams.

1/23/2005page14

Deinterlace

• Convert interlaced video into progressive one

1/23/2005page15

Efficient Color Conversion

• VIC didn’t give correct color representation.

• Efficient Color Conversion– Grabber (YUYV) to Encoder (YUV420)– Decoder (YUV420) to Display (RGB15/RGB16/RGB24)

1/23/2005page16

Scaling Viewing Windows

• Original VIC– only support limited sizes of viewing windows– based on unrealistic pixel replication method

• NCHC-VIC– scaling viewing windows arbitrarily without too much overhead– interpolation by 9 tap FIR filter– hardware assisted by Xvideo extension in XFree86– full screen mode

1/23/2005page17

Full Screen Snapshot

1/23/2005page18

Real-Time Video Streaming

• Real-time– the compressed data are transferred at a speed that matches the

coded video source rate.

• Latency, Jitter– average end-to-end delay– delay variation

1/23/2005page19

Error Resilience

• Error Progratation

1/23/2005page20

Error Resilience (cont’d)

• Resynchronization marker– the data between the error location and the first following

resynchronization marker would have to be discarded.

• Data Partition– motion vectors and related syntactic elements are transmitted first,

followed by syntactic elements like MB headers, motion vectors, and DCT coefficients.

1/23/2005page21

Minimize Latency

• Multithread Encoding– Take advantage of symmetric multiple processor architecture

• Pipeline the latency of transmission and encoding

• SIMD Instruction Ses– Optimized by SSE2/SSE, MMX

1/23/2005page22

Future Work

• Support H.264 codec• Support MPEG-4 High Efficient AAC (HE-AAC)

1/23/2005page23

Conclusion

• Project Website:

• http://vic.nchc.org.tw

1/23/2005page24

Thanks for

your attendance