multimedia image compression

Multimedia Image Compression

1. ObjectivesThe objective of this program is to develop first-of-a-kind MPEG-4 architectures, optimised for high performanceand low power. The main output of the program are system-on-chip MPEG-4 architectures, system-levelimplementation methodologies and tools supporting the design flow of this family of applications.

2. Background

Advanced multimedia image compression applications are characterised by media diversity and dynamicresource availability, making the use of first generation image compression techniques not appropriate. Toovercome these inefficiencies, new image compression algorithms have been developed, and are beingstandardised within the ISO/IEC MPEG-4 context.

MPEG-4 can indeed be considered as the first true multimedia standard, because it includes concepts such asobjects, scalability, scene composition, etc. As such, it will be the select standard for a variety of applicationsranging from mobile multimedia terminals to game applications and set-top boxes.

A MPEG-4 system no longer compresses images as a whole, but allows for 3-D scene composition of multipleheterogeneous objects, e.g. video sequences of arbitrary shapes, face and body animated mannequins, wavelet-based still images and wavelet-based textures. Mechanisms are provided to adapt to changing communicationbandwidth and bit error rate as well as to different (possibly changing in time) terminal resources. Thesemechanisms include advanced error resilience, spatial-, SNR-, and temporal-based scalability and gracefuldegradation operation modes. Additionally, MPEG-4 provides the necessary hooks for user interaction with the3-D scene.

The challenges for designing a MPEG-4 architecture are being able to 1) cope with the high system complexity, 2)to efficiently design the new functionality (e.g. wavelets), and 3) combine the flexibility requirements resultingfrom the heterogeneous nature of MPEG-4 and from the user interaction, with the efficiency of applicationspecific architectures. Current research proves that the main bottlenecks for a MPEG-4 system are with thememory requirements (memory size and bandwidth).

IMEC is consequently currently focusing on the development of methodologies and CAD tools for supportingthe implementation of MPEG-4 like systems, on the implementation of MPEG-4 scalability functionality, and onthe design of specific memory architectures for MPEG-4 systems.

IMEC has investigated for several years the use of microelectronics for complex digital signal processingsystems. In the field of multimedia image compression, the DESICS division has several realisations including thedesign of a hardware motion estimator for a H.261 video-conferencing encoder, the development of a low powerarchitecture for a H.263 video-conferencing decoder and a colour transformation chip for region-oriented coding,the development of a low-power architecture for a MPEG-4 video encoding, and the world first silicon for MPEG-4 specific functionality (Ozone).

IMEC’s strategy includes an active participation in the MPEG-4 standardisation committee.

Page 2 Multimedia Image Compression March 00

3. Target CustomersTarget customers for this program are system houses and multimedia component providers that want to extendtheir existing expertise with MPEG-4 specific implementation knowledge.

4. Program Outline

The program consists out of 6 work packages:

WP1: Application definition and analysis.WP2: System-level optimisation.WP3: Quality of Service and Graceful Degradation.WP4: Design of MPEG-4 IP Blocks.WP5: Advanced topics.WP6: Demonstration

In this work program, it is assumed that the partner will define a limited set of MPEG-4 application scenarios atthe start of the project. Quantitative characteristics of the data streams, data sources and user interaction in thisapplication will be needed.

Every six months there will be a meeting to review this work plan and update it, if necessary.

WP1: Application definition and analysisAt the start of this work package, high level models and implementations of MPEG-4 functionality are assumed tobe available in the IIAP program. In this work package, an initial assessment of the memory, computation andcommunication requirements for the different application scenario's is performed. For this purpose, ATOMIUMadvanced analysis tools are used.

WP2: System level optimisationBased on the quantitative outcome of WP1, an initial architecture for the defined application(s) will first beestablished. This architecture will be focussing on a selected scenario, but effort will be made to investigatewhether generic optimised architectural elements can be designed.

The main cost factors under consideration are performance and power consumption.

The ATOMIUM system level design methods and optimisation tools which are developed at IMEC will be usedto dimension and optimise the memory architecture and interconnections. Together with the definition of thearchitecture, also the mapping of the functionality of MPEG-4 onto the architecture has to be performed.

WP3: Quality of Service and Graceful DegradationOne way to eliminate some of the bottlenecks that are defined in WP2 is the use of graceful degradationtechniques. When the architecture has insufficient processing, communication or memory resources, the qualityof the decoding (e.g. frames per second, resolution, SNR) can be traded for decoding complexity. The mainchallenge in implementing the graceful degradation techniques is the treatment of the dynamic nature of thedecoding process, caused by the variable characteristics of the incoming data stream and the user interactions.

In this work package, we will first define complexity metrics and complexity prediction. Consequently, gracefuldegradation strategies will be worked out. To this end, graceful degradation will be considered both at thesystem level and the object decoder level. The developed techniques will mainly focus on 3D rendering, wherethe expected impact is the greatest. These techniques shall be based on the information found in the MPEG-4normative bitstream and on optional MPEG-4 meta-information.

WP4: Design of MPEG-4 IP BlocksIn WP 2, implementation bottlenecks are identified. Consequently, optimised implementations are necessary (inhardware or software). It is the intention to process the hardware (if any) blocks as chips, as long as they are inline with the research pipeline of the multimedia IIAP program.


The HW IP blocks will preferably be developed by means of the OCAPI design flow, which support thedevelopment of hardware from C++. The main advantage of OCAPI is the possibility to rapidly generate derivedversions (re-use) of the IP blocks.

WP5: Advanced topicsThe goal of this work package is to monitor the functionality that has a high probability to be included in futureMPEG-4 versions. Given the partner’s interests and preliminary information on novel MPEG-4 versions, inparticular, early detection of implementation bottlenecks is aimed for.

WP6: DemonstrationIn this last work package, a demonstration will be built for the results of this project. A possible implementationwould consist of a PC with a plug in card for the hardware accelerators, connected via a cable to a handheldpersonal organiser on which the received information is displayed.

The demo will show the relevance of the architectural refinements (e.g. memory optimisations, gracefuldegradation, etc.).

5. DeliverablesParticipating in this program offers the following results to the industrial partner:

• assessment of specific memory architectures for MPEG-4,

• transfer of IMEC’s system-level implementation methodologies,

• access to ATOMIUM CAD for the design of MPEG-4 systems,

• insight in MPEG-4 graceful degradation techniques,

• assessment of low power techniques,

• insight in issues concerning integration of MPEG-4 systems with other state-of-the-art developments whereIMEC has an important research activity (e.g. broadband modems and wireless communications systems),

• up-to-date information on the MPEG-4 standardisation,

• access to implementation know-how for state-of-the-art wavelet compression,

• evaluation of the performances and feasibility of next-generation multimedia algorithms (e.g. wavelet-basedcoding of video sequences).


6. Program TimingThe following is an overview of the program planning:

Year 1 Year 2 Year 3

The major results of the IIAP are:

System-level implementation methodologiesThe development of system-level implementation and optimisation methodologies bootstrap on DESICS'yearlong tradition in the field of silicon compilation. With respect to multi-media, the main focus is on thereduction of the data transfer and storage cost, which is the predominant cost factor, both with respect toperformance as well as power consumption. An overview of the developed ATOMIUM methodology can befound at http://www.imec.be/vsdm/projects/atomium/ (over 50 scientific publications).

The main results of applying ATOMIUM on designs are:1. The reduction of the (worst-case) power consumption of a H.263 videophone decoder with a factor 92. The reduction of the storage size with a factor 10 for a MPEG-4 visual texture encoder3. The reduction of the (worst case) power consumption with a factor 8 for MPEG-4 motion estimation4. The reduction of the number of memory transfers with a factor 9/8/4 (averages for low complexity/medium

complexity/high complexity video) for the MPEG-4 simple profile decoder (compared to MPEG-4 FDIS Ccode)

Robust CAD tools for multi-media system explorationTool support for the above-mentioned methodologies is now actively being developed. The objective is toimplement robust tools, supporting the application of the system-level methodologies, focusing on therequirements of multi-media applications and supporting the most error-prone and critical tasks concerningdesign exploration and optimisation. The tools operate on functionality described in the C language.

Currently, following tools are being used for MPEG-4 design:


• Dynamic pruning of unwanted MPEG-4 functionality: given the MPEG-4 reference code and a given MPEG-4profile, this tool supports the removal of redundant code. For an MPEG-4 videophone encoder application,the MPEG-4 reference code was reduced to only one fifth of its original size using dynamic pruning.

• Simulation-based instrumentation of the memory requirements (data transfer and storage) of MPEG-4• Formal memory cost analysis of C code• C-in, C-out memory compaction tools that support:− Fully automatic intra-data optimisation and window calculation− Fully automatic inter-data optimisation (placement)− Advanced lifetime analysis− The (possibility for) manual assignment of data to memory− Designer imposed constraints (e.g., fix offset)− Automatic transformation of the C code− Address optimisationsThis functionality minimises (the size of) memories and the number of capacity misses in caches

• OCAPI-1-1, to translate C descriptions of MPEG-4 functionality into VHDL

The following functionality is currently under development:• Interactive code modification• Automated support to optimise the physical memory management. Given a cycle budget, this functionality

minimises the cost (power consumption and silicon area) of the on-chip memories for custom architecturesby distributing the available cycle budget to reduce the required memory bandwidth and assigning the datato the allocated memories in a cost effective way.

Implementation of optimised MPEG-4 functionalityFollowing implementations of MPEG-4 functionality have been designed:• The Ozone, a chip for scalable encoding of wavelet images consisting of a zero-tree encoder and a high-

speed adaptive binary arithmetic encoder. The Ozone allows for real-time compression (30 fps) of CIFimages (352x288 pixels) and is the first chip specifically being designed worldwide for novel MPEG-4functionality. Higher resolutions and frame rates are achievable by using parallel Ozone chips. The Ozonehas been demonstrated on a PC plug-in card.

• Optimised VHDL and C for the zero-tree encoder of the Ozone• Optimised VHDL and C for the adaptive binary arithmetic encoder of the Ozone• Optimised software implementing some selected profiles of the MPEG-4 standard (C language), more

specifically:• Platform independent C code for the MPEG-4 simple profile decoder• Platform independent C code for the MPEG-4 simple profile encoder• PC Platform C code for the MPEG-4 simple profile decoder (achieving CIF 30 fps on a 350 MHz PC)• PC Platform C code for the MPEG-4 simple profile encoder (achieving CIF 15 fps on a 350 MHz PC)• Optimised software of the integer-to-integer lossless forward and inverse wavelet filtering (including

Pentium II MMX, SHARC multiprocessor, TI C6x and TriMedia TM1)• Optimised Local Wavelet Transform C code, allowing for a block-based computation of wavelet filter

coefficients• Optimised C and Java for the compression of artificial objects.

For more Information

Program Manager: Jan Bormans, IMEC, 75 Kapeldreef, B-3001 Leuven, Belgium, Tel.: +32 16 28 15 72,

Fax: +32 16 28 15 15, E-mail: [email protected]

IMEC web site: http://www.imec.be/

Documents

multimedia image compression