Choosing the Best IPfor CompressingImages and Video

Choosing the Best IPfor CompressingImages and Video

CAST, Inc.+1 201.391.8300info@cast-inc.comwww.cast-inc.com

Slide 2

Selecting the Best IPSelecting the Best IP

Algorithms Image Quality Formats

Hardware Area Speed Memory System Integration Application Requirements

Evaluation

Expertise

JPEGJPEG

JPEG-LSJPEG-LS

JPEG 2000JPEG 2000

Lossless JPEGLossless JPEG

H.264H.264

Motion JPEG

Motion JPEG

Slide 3

Compression AlgorithmsCompression Algorithms Lossy

discards image data but looks good enough

Losslessdiscards no image data

Near-Losslessvisually identical, ornumerically acceptable data loss

Slide 4

JPEGGood quality

< 20:1

Moderate quality 20:1 – 50:1

Poor quality > 50:1

JPEG 2000Good up to 80:1

H.264Best for > 50:1

Image Quality: LossyImage Quality: Lossy

Slide 5

LJPEG Compresses 2:1

Not good for text, computer graphics

JPEG-LS 2.3:1

Usually best

JPEG 2000 2.1:1

Flexible (lossy, too)

Image Quality: Lossless Image Quality: Lossless

Slide 6

Input Format: Image TypesInput Format: Image Types

Still images vs. motion video Still images — JPEG, LJPEG, JPEG-LS, JPEG 2000 Motion video — Motion JPEG, H.264

H.264 Motion Estimation Predicts where motion will occur Represents motion, not entire frame: fewer bits

Slide 7

CGA 320 x 20064,000 pixels

CGA 320 x 20064,000 pixels

QSXGA 2560 x 2048 5,242,880 pixels

QSXGA 2560 x 2048 5,242,880 pixels

Input Format: Data FactorsInput Format: Data Factors

Number of bits/pixelJPEG — 8LJPEG — up to 16JPEG-LS — up to 16JPEG 2000 — up to 16

Number of pixels: frame size

Representation method

Slide 8

Input Format: SubsamplingInput Format: Subsampling

Eye more sensitive to changes in brightness (luma) than color (chroma)

Can use just some of the chroma info (a subsample) to compress an image

Encoders use different subsample formats JPEG — 4:4:4, 4:2:2, 4:1:1, 4:2:0, grayscale

LJPEG — 4:4:4, grayscale

JPEG-LS — 4:4:4, 4:2:2, 4:1:1, 4:2:0, grayscale

JPEG 2000 — 4:4:4, 4:2:2, 4:1:1, 4:2:0, grayscale

H.264 — 4:2:0

Color SpaceNotation

Y:Cb:Cr

Y = luma value

Cb:Cr together

= chroma value

Can include fewer chroma values in various ways.

Another popular color space is similar:

Y:U:V

Slide 9

Hardware & Algorithm Trade-OffsHardware & Algorithm Trade-Offs

Area

Speed

Memory

System Integration

Application Requirements

Slide 10

AreaArea

JPEG 2000 Parameters

No. of Entropy Encoders

Filter Types & Levels

Quantization Tables

Block Size

Whole Frame or Tiles

Slide 11

SpeedSpeed

JPEG fastest on the market, 1 clock/sample

JPEG 20001.4 clocks/sample, depending on configuration

H.2642.5 clocks/sample

LJPEG and JPEG-LSvery fast, 1 clock/sample

Max clock

rate of the

technology

limits the

frame size

and fps

Slide 12

MemoryMemory

Dependent on frame size

H.264can use SDRAM, DDR or DDR2(more flexible than MPEG4)

JPEG 2000large demands, but can reduce by tiling (at expense of speed)

WVGA480 x 854

1024 x 768

1080p 1920 x 1080

Slide 13

IntegrationIntegration

Latency Time from first input sample to first compressed output

CPU required? Core may have a CPU for configuration,

or read from input stream

Memory Interface Best not embedded in compression core to allow flexibility

Slide 14

Application RequirementsApplication Requirements

Rate Control

Error Resilience

Special FeaturesMultiple ResolutionsQuality LayersRectangular Regions of Interest

Slide 15

Rate ControlRate Control

Trades off transmission bit rate and video quality

Essential for applications with constrained channel bandwidth or storage capacity

Must: Guarantee channel will never be overflowed Maximize quality for even very low bit rates Be adaptable to any type of scene or motion Provide a fast response to changes in video

characteristics Allow simple system integration

Hardware Bit Rate Control is best

Slide 16

Multiple Resolution LevelsMultiple Resolution Levels

JPEG 2000 can store Images at multiple resolutions

Thumbnails quickly pulled from storage and analyzed

Higher resolution images then requested from storage for in-depth analysis

Excellent for reconnaissance& surveillance

From: http://www.centurionalarm.com/j2kvsmpeg4vsjpegvsh264.pdf

Slide 17

Error ResilienceError Resilience

JPEGRestart markers, but works on blocks

JPEG-LSNone

JPEG 2000Packet headers in stream, other features

H.264Reference frame errors extend to dependent frames; CAST adds multiple slices to increase error resilience

Slide 18

Original Lossless710 Kbytes20 Quality Layers

Original Lossless710 Kbytes20 Quality Layers

8:1 Compression88.76Kbytes10 Quality Layers

8:1 Compression88.76Kbytes10 Quality Layers

16:1 Compression44.28 Kbytes5 Quality Layers

16:1 Compression44.28 Kbytes5 Quality Layers

256:1 Compression2774 bytes1 Quality Layer

256:1 Compression2774 bytes1 Quality Layer

Quality Layers with JPEG 2000Quality Layers with JPEG 2000

Military applications: thermal IR & satellite imagery

Multiple layers with different compression ratios

Slide 19

Rectangular Regions of InterestRectangular Regions of Interest

Specify an area is more important than the rest of the image

Vary quality of the region and the background per: Desired

compression ratio Relative

importance of the region

50:1 CompressionRegion: no special importance50:1 CompressionRegion: no special importance

50:1 CompressionRegion: slightly important50:1 CompressionRegion: slightly important

50:1 CompressionRegion: moderately important50:1 CompressionRegion: moderately important

50:1 CompressionRegion: extremely important50:1 CompressionRegion: extremely important

Slide 20

Image & Video IP EvaluationImage & Video IP Evaluation

Get a Bit Accurate Model (BAM) to run your images

Try a Prototype Board with the IP in hardware

Get references in your market sector (military is tough)

Study representative synthesis results for area and max operating frequency

Make sure you will have access to compression engineers

Slide 21

CAST Compression Reference Design SystemCAST Compression Reference Design System

Complete FPGA System Board: Learn compression Evaluate our cores with

your media Get a system head start

Combines multiple cores and essential software

Slide 22

Vendor ExpertiseVendor Expertise

CAST:

Delivering IP for 16 Years Privately-held, independent 1,000 sales to 600 customers

Expert Development Partners

Extreme Customer Focus 24/7, fast-response culture Global team of 100 people

Airborne Reconnaissance

Audio/Video Platform

Broadcast Equipment

Camera Module

Cell Phone Camera Image Processing Chip

Digital Still Camera

Digital Video Recorder

Earth Observation Satellite

High-Definition Monitor

High-Resolution Airborne Reconnaissance Sensor

High-Speed Video Camera

Imaging Chips

Military - Classified

Mobile Camera SoC

Mobile Digital Video Recorder Platform

Mobile Phone SIM Card

Multiple Kill Vehicle

Office Class Copier/printer

Prototype 3D Camera

Satellite - Classified

Satellite Camera and Measurement Device

Security and Surveillance Camera

Security LCD Monitor

SoC-on-panel Display

Surveillance Camera

Technology Proof-of-Concept Camera

TV Surveillance Camera Display

USB Module

Video Compression

Video Conferencing System

Video Image Acquisition System for the Military

Video Surveillance for Road Tolling

Weather Satellites

Wireless Rear-view Camera

X-ray Dental Project

Some Customer Multimedia Applications

Slide 23

CAST Broad IP Cores LineCAST Broad IP Cores Line

MultimediaJPEGJPEG Scalado SpeedtagsJPEG Ext. 12-bitJPEG LosslessJPEG-LS JPEG 2000 H.264 VideoAudio: I2S, SPDIFImage Conversion

System IPSubsystems

8051/USB, 8051/MACSoC Platforms

ARM, eASICAMBA Infrastructure

ReplacementsProcessors, Bit SlicesControllers, Peripherals, Timers, UARTs, TTL Parts

EncryptionAES DES, Triple DES MD5 SHA-1, SHA-256

CommunicationsConsumer IR ControllersSynchronous UARTS SDLC, HDLC

InterfacesPCI & PCI ExpressEthernet MACs USB & OTG CAN, LIN, I2C, SPIECP

Memory & PeripheralsMemory Controllers

DDR1/2 SDRAMSDIO HostSDR mobile SDRAMNAND flash

Serial flashStorage Controllers

ATA/IDE Interface Device Controllers

Smart Card Reader, TV Display, Hi-Res

Processors8-bit: 8051 fast, tiny, entry

Z80, C650216-bit: 68000, 80186xl,

80186ec, 387L32-bit 68000 AHBDSP: 32025, 32025TX

See more at www.cast-inc.com

Slide 24

CAST Image & Video IP CoresCAST Image & Video IP Cores

MultimediaJPEGJPEG Scalado SpeedtagsJPEG Ext. 12-bitJPEG LosslessJPEG-LS JPEG 2000 H.264 VideoAudio: I2S, SPDIFImage Conversion

Cameras, Printers, Copiers, etc.

Pro Video,Medical Imaging

Best Lossless

Surveillance,Medical Imaging

Replacing MPEG4

Fast Cameras

High QualityImaging

Block to RasterRaster to BlockDownscalingColor Space Conversion