VIC Development

National Center of High-performance ComputingBarz Hsu (barz@nchc.org.tw)

15 January 2007

Start Point

Intend to develop high visual quality video tool

Based on AccessGrid vic-2.8ucl1.1.3-3 Initial release of NCHC-VIC with

MPEG4 codec Up to NTSC/PAL resolution High-efficient render Fullscreen mode Linux version only

Timeline

Develop NCHC-VIC from January 2004 to August 2005

Develop GStreamer-based media tools from June to August 2006

Merge NCHC-VIC to UCL code base from September to December 2006

Outlines

Review NCHC-VIC Porting Issues Demos GStreamer based new media tools

NCHC-VIC Features

Codec MPEG4 H.264

Render Xvideo Swscale Fullscreen

Postprocess Deinterlace Deblocking Colorspace conversion

MPEG4

FFmpeg encoder/decoder ME (motion estimation)

EPZS algorithm QPEL

CBR (constant bit rate) GOP (group of picture): 15 Low latency

No B-frame rtp_callback Old_latency = encoding_latency + tramsmission_latency New_latency =~ encoding_latency/5 + tramsmission_latency

Low CPU usage

MPEG4 (cont’d)

IETF RFC 3640 RTP payload type 45 Rearrangement RTP packet buffer for only on

e frame Error resilience/concealment Compatible with VP Handy to use other codec such as

MPEG2, H.261, H.263, H.263+, DV, MJPEG, Theora

H.264

X264 encoder Turn off CABAC

save computation up to 30%, increasing coding bit rate around 15%

Inter-ME X264_ANALYSE_PSUB16x16 RC_METHOD X264_RC_ABR i_direct_mv_pred X264_DIRECT_PRED_NONE Deblocking filter No B-frame GOP is 20~50 frames

H.264 (cont’d)

FFmpeg encoder FFmpeg decoder Each RTP packet is a collection of NAL units IETF RFC 3894? Compatibility?

Quicktime、 VLC、 inSORS Contributed by Champ Yeh and Barz Hsu fro

m NCHC

Advanced UI

Contributed by Morris Lin from NCHC

Accelerated Render

XVideo+Shared memory Software scaling CPU capability runtime detection Win32/Win64, Linux, MacOSX Colorspace Conversion

SWSCALE: Software Scaling

Libswscale Shipping with FFmpeg and MPlayer

Arbitrary resizing Bilinear interpolation Colorspace conversion

YUV420 to RGB32/RGB24/RGB16 for render YUYV to YUV420 for grabber

CPU multimedia instructions Intel: MMX/MMX2/SSE/SSE2 AMD: 3D Now/3D Now2 PowerPC: Altivec

Runtime CPU capability detection Speed up to 10X than pure C code

Xvideo Extension: Graphic Accelerated Render XFree86 X11 Extension Supported by Nvidia and ATI graphic card Direct YUV420 render

No colorspace conversion needed Perform image scaling and colorspace conversion b

y graphics Most efficient way Up to 40~100 videos at same time Automatically use swscale if xvideo no longer availa

ble

Fullscreen

Boardless Light weight If graphics accelerated Dual head is fine Only for Windows and Linux

Fullscreen

Postprocess

Provide better visual quality Remove coding artifact by specific encoders Deinterlace Deblocking Dering Automatic level

Use higher level of postprocessing if CPU usage is not full

Linux Port

Nothing special! File size of statically linking binary is around

10MB Most robust and efficient version V4l grabber bugfix

Windows Port

Visual Studio 2003 Inline assembly syntax different from GCC

AT&T v.s. INTEL Create DLLs and LIBs (import library) under

Mingw/Cygwin ffmpeg, x264, libswscale, libpostproc Is it possbile to use sttatical linking?

S-video, Composite input support correct framerate for win32 DS grabber

MacOSX Port

Merge AG Aqua branch http://www.vislab.uq.edu.au/research/accessgrid/software/aquavic/

Aqua and Quicktime framework Without X11 Use Aqua Tcl/Tk 8.4 shipped with MacOSX Native look Grabber-macosx

iSight, QuickCam support More robust than AG Aqua branch

MacOSX Port (cont’d)

Compilation Steps (Linux)

Check out source code svn co https://frostie.cs.ucl.ac.uk/repos/mmedia/vic/branches/mpeg4

Configuration ./configure

Check out FFmpeg with revision 7110 (Dec, 2007) svn co -r 7110 svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg ./configure --enable-gpl --enable-swscaler --enable-pp

Check out X264 with revision 600 (Dec, 2007) svn co -r 600 svn://svn.videolan.org/x264/trunk x264

Options --with-ffmpeginc --with-ffmpeglib --with-x264inc --with-x264lib

Build make

Compilation Steps (MacOSX)

Install TclTkAquaBI for Tcl/Tk 8.4.10 http://tcltkaqua.sourceforge.net/

Configuration and Building Steps are same as Linux platform

Compilation Steps (Windows) Configuration

./configure Check out FFmpeg with revision 7110 (Dec, 2007)

svn co -r 7110 svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg ./configure --enable-mingw32 --enable-memalign-hack --enable-swscaler

--enable-gpl --enable-shared --disable-static Check out X264 with revision 600 (Dec, 2007)

svn co -r 600 svn://svn.videolan.org/x264/trunk x264 ./configure --enable-shared Patch Makefile for generating DLL add the 3-6 lines in SONAME targe $(SONAME): .depend $(OBJS) $(OBJASM) $(CC) -shared -o $@ $(OBJS) $(OBJASM) -Wl,-soname,$(SONAME)

$(LDFLAGS) ifeq ($(SYS),MINGW) $(CC) -shared -Wl,--output-def,libx264.def -o libx264.dll $(OBJS) $(O

BJASM) lib /machine:i386 /def:libx264.def endif

Compilation Steps (Windows) (cont’d)

Copy LIBs for VS2003 avcodec-51.lib avutil-49.lib libx264.lib postproc-51.lib

swscale-0.lib Copy DLLs to the path of excuteable

avcodec-51.dll avutil-49.dll libx264.dll postproc-51.dll swscale-0.dll

Tcl/Tk 8.4

Multi-threading Tcl_init.tcl and tk_init.tcl files required Further work needed to embed the init scripts

to binary It’s critical to call Tk_FindExecuteable() to loc

ate init scripts for Windows platform

Open Issues?