Implementing Video Systems Using the Altera Video Framework

© 2011 Altera Corporation—Internal

Implementing Video Systems Using the Altera Video FrameworkImplementing Video Systems Using the Altera Video Framework


2

AGENDA

What is the Video Framework?- Overview

VIP Core Details

Boards/Reference Designs


3

Altera Video Framework

Building block IP cores to speed up development (VIP suite)

Open, low-overhead interface standard to mix-and-match custom or off-the shelf IP blocks

Format conversion reference designs that showcase silicon capability, provide a starting point for design

System Level Tools and design methodology

Variety of development boards/kits for rapid design prototyping


4

Building block IP to speed up development- Interfaces (SDI, DVI, etc), High Performance Memory Controllers, Nios- Video IP suite: library of interoperable Video IP blocks using common standards

and protocols

Open interface standard to mix-and-match custom or off-the shelf IP- Avalon Memory-Mapped and Avalon-Streaming Video protocol- Open Source HDL Template to facilitate integration of custom IP into framework

The Video IP framework – The IP

Custom IP

Input

interface

Clocked Video Input

Output

interface

Video in Video out

Clocked Video Input

Altera VIP 1

Altera VIP 2

Avalon-ST Video (over Avalon-ST) Avalon-MM

Nios Memory controller


5

The Video IP framework – Design Tools

Pin mapping,Interface IP,

Top-level HDL

Quartus II

Software control andconfigurationDebug (Nios II & SignalTap)

Nios II IDE

SOPC Builder

Nios II IDE / Quartus II

Development boards/kits for rapid design prototyping

Stratix II GX, Cyclone IIIStratix IV GX PCIe, Arria II GX

Abstract design captureSwitch fabric generation

IP Catalogue tool

Software library to control the cores of the VIP Suite and for interrupt servicing

1080p reference designs: out-of-the-box ASSP-level capabilities

and/or great starting pointsV-series, M-series, partner designs


6

AGENDA


VIP Core Details


© 2011 Altera Corporation - Public

7

Video Image Processing (VIP) Suite

BT 656

Avalon ST video

Scaler

Deinterlacer

Chromaresampler

Alphablending mixer

Color planesequencer

Frame buffer

Image clipper

Gammacorrection

Test patterngenerator

2D FIR filter

2D median filter

Color spaceconverter

Avalon ST video

BT 656

Half a dozen hardware verified video processingreference designs using these functions

Available TODAY!

Switch

Control Sync

Frame ReaderScaler II

v11.0Deinterlacer II

v10.1


New Cores in VIP Suite (2011)

Scaler II- Reduces area and improves performance

Compared to first-generation scaler- Supports 4:2:2 chroma data sampling rate

Further reduces area used

Deinterlacer II- Supports 3:2 cadence detection- Supports low-angle edge detection- Delivers lower latency

8


9

VIP Function Details Chroma Resampler

This function allows you to change between 4:4:4, 4:2:2 and 4:2:0 sampling rates

For 4:2:2 4:4:4, the filtered algorithm uses a 4-tap filter with fixed Lanczos-2 coefficients

For 4:4:4 4:2:2, the filtered algorithm uses a 7-tap filter with fixed Lanczos-2 coefficients


10

VIP Function Details Color Space Converter

This function provides a flexible and efficient means to convert image data from one color space to another

You can select from a list of common color space conversion

Or choose a custom color space


11

VIP Function Details Deinterlacer / Deinterlacer II (motion adaptive)

Select the input image height and width Choose the deinterlacing algorithm –

bob, weave, motion adaptive

The weave and motion adaptive algorithms require external frame buffering


12

Understanding Algorithmic Constraints Deinterlacer (motion adaptive) Motion adaptive deinterlacer requires five

(master) accesses to the DDR memory- 1 field write- 2 field reads- 1 motion vector write- 1 motion vector read

Calculating DDR memory bandwidth- Input format: 1080i, 60 fields/sec, 10-bit color

1920 × 1080 × 30bits × 60/2 = 1.866Gbit/s- Output format: 1080p, 60 frames/sec, 10-bit color

1920 × 1080 × 30bits × 60 = 3.732Gbit/s- Motion format: Only use 10bits for the motion values

1920 × 1080 × 10bits × 60/2 = 0.622Gbit/s - Memory access:

1 × write at input rate: 1.866Gbit/s 1 × write at motion rate: 0.622Gbit/s 1 × read at motion rate: 0.622Gbit/s 2 × read at output rate: 7.464Gbit/s

Total: 10.574Gbit/s

Deinterlacer (MA)

DDR2

4:4:4 Mode, Motion Bleed is ON

4:4:4 Mode


13

Understanding Algorithmic Constraints Deinterlacer (motion adaptive) Motion adaptive deinterlacer requires five

(master) accesses to the DDR memory- 1 field write- 2 field reads- 1 motion vector write- 1 motion vector read

Calculating DDR memory bandwidth- Input format: 1080i, 60 fields/sec, 10-bit color

1920 × 1080 × 20bits × 60/2 = 1.24Gbit/s- Output format: 1080p, 60 frames/sec, 10-bit color

1920 × 1080 × 20bits × 60 = 2.48Gbit/s- Motion format: Only use 10bits for the motion values

1920 × 1080 × 10bits × 60/2 = 0.622Gbit/s - Memory access:

1 × write at input rate: 1.24Gbit/s 1 × write at motion rate: 0.622Gbit/s 1 × read at motion rate: 0.622Gbit/s 2 × read at output rate: 4.96Gbit/s

Total: 7.44Gbit/s

Deinterlacer (MA)

DDR2

4:2:2 Mode, Motion Bleed is ON

4:2:2 Mode, 30% reduction in DDR bandwidth


14

VIP Function Details Scaler

Choose the scaling algorithm – number of taps/phases

Select the input and output image sizes

Choose from a selection of pre-built filter functions or choose a custom function

Requires progressive 4:4:4 input


15

VIP Function Details Scaler II – Utilizes Fewer Resources

Reduces area and improves performance- Compared to first-generation scaler

Supports 4:2:2 chroma data sampling rate- Further reduces area used

Algorithm supports - Linear and polyphase with

run-time coefficient load

New core in HDL- More efficient in using resources


16

VIP Function Details Alpha Blending Mixer

Select the number of layers to be mixed

The level of blending is controlled by the number of alpha bits (up to 8)

Select the size of the image

http://upload.wikimedia.org/wikipedia/commons/6/62/Picture_in_Picture_(Photomontage).jpg


17

VIP Function Details Generate a color bar test pattern


18

BT656 Avalon ST Video

The Clocked Video Input MegaCore function converts from clocked video formats (such as BT656 and DVI) to Avalon-ST Video

It strips the incoming clocked video of horizontal and vertical blanking, leaving only active picture data

No conversion is done to the active picture data- The color plane information

remains the same as in the clocked video format.


19

BT656 Avalon ST Video

The Clocked Video Output MegaCore function converts from Avalon-ST Video to clocked video formats (such as BT656 and DVI)

It formats Avalon-ST Video into clocked video by inserting horizontal and vertical blanking and generating horizontal and vertical sync information

No conversion is done to the active picture data- The color plane information

remains the same as in the Avalon-ST Video format.


20

Frame Buffer


21

Run-time Control of the Video Chain

Build video signal chains that can be updated on-the-fly without changing the HDL or bit-stream- Using the streaming interface protocol: Avalon ST Video- Using an embedded processor


22

Run-time Control of the Video Chain Avalon-ST Video protocol is

a packet-oriented way to send video and control data- First packet contains a frame of

video data - Second packet contains control

information that applies to the subsequent video packet/frame

Video function 1 Video function 1

DATAVALID

SOPEOP

READY

Avalon ST

Examples of Avalon-ST Video Control Packets


23

Run-time Control of the Video Chain

Most video functions permit run-time control of some aspects of their behavior, using a common type of Avalon-MM slave interface

Each slave interface provides access to a set of control registers which must be set by an embedded processor/logic

Generally, a function can be updated on a frame boundary

Processor

Avalon MMMaster port

Video function 1

Avalon MM control plane


24

NIOS Control: Cyclone III Example DesignInitialize daughter card Initialize daughter card

Start the IP cores

Control the mixer and Scaler parameters at run-

time


25

AGENDA


VIP Core Details



26

Video Development Kits

Cyclone III FPGA, $1,895

Stratix IV FPGA, $4,995

Cyclone III FPGA, $2,995

Arria II GX FPGA, $2,995

$495

$795

Cyclone IV FPGA, $1,295

http://www.omnitek.tv/


New Demo in 2011 (VEEK)

Demo # 1 – Simple Video Format Conversion

Demo # 2 – Camera Input Scaling

27

Input Processing Output

composite

de-interlaced video /w txt overlay (NTSC to 800x600 with real-time

resizing, zoom and panning) VEEK display VGA out

Download Design Files on Altera Video Wikiwww.alterawiki.com/wiki/Videoframework

Input Processing Output

Camera on VEEK

Camera Input/w txt overlay (Camera input scaling with real-time

resizing, zoom and panning) VEEK display VGA out


New Demo in 2011 (VEEK)

Demo # 3 – Picture in Picture (PiP)

Demo # 4 - Multi-view

28

Input Processing Output composite de-interlaced video /w txt overlay VEEK display

camera on VEEK PIP - (camera source) VGA out

Input Processing Output composite CH 1 / 3: de-interlaced video DVI (HSMC) @ 800x600

camera (GPIO) CH 2 : camera videoDVI (HSMC) CH 4: DVI video (800x600 down scaling)

Watch Demo of VIP Suite


New Low Cost UDX Reference Design

C4GX150 board- Develop in Quartus II software v11.0 using Qsys- Available at end of July

1 channel UDX design- SDI / DVI input switch- SDI / DVI output- Resolution supported

1920 x 1080P 50/59.94/60Hz1280 x 720P 50/59.94/60Hz

- Text overlay and testing pattern

29


Video Reference DesignsDesign Device Features I/Os

Video and image processing (VIP) demo

Cyclone III, Cyclone IV

National Television System Committee (NTSC)-to-480p scaling, 720p camera input

scaling, mixing for PiP, multiview , 1-channel UDX design

Analog, digital video interface (DVI)

Cyclone® III FPGA design

Cyclone IIINTSC-to-720p scaling, MA deinterlacer,

scaler, mixing for PiPAnalog, DVI

UDX2 Arria® II GX Same as UDX4Serial digital interface

(SDI), DVI

UDX3/4 Stratix® IV GX

V1, on-screen display (OSD), switch, active format description (AFD),

mixing, multiport front-end (MPFE), 1080p, 720p, 1080i, ntsc/pal, 4:2:2 scaler, MA

deinterlacer with low-angle edge

SDI, DVI, high-definition multimedia interface

(HDMI)

4K scaling Stratix IV GX 1080p 4K scaling, component design methodology

SDI4x SDI for output

Broad Portfolio of Reference Designs to Help You Get Started

30


31

Summary

Altera video framework enables rapid development- Mix and match existing IP – Leverage Altera’s open interface

standard - Automatically integrate embedded processors and arbitration

logic- Leverage building block IP provided by Altera- Use existing reference designs as starting points

And rapid prototyping- Implement design using the appropriate development boards- Test the design with actual video signals


32

Next Step

Coming Video Framework Workshop, please contact Altera Sales orDistributors for more information

Provide an introduction to Video and Image Processing suite including the cores, reference designs, roadmap, video framework, etc.

For information on:- DDR-based Memory controllers

http://www.altera.com/education/training/courses/IMEM210 http://www.altera.com/technology/memory/mem-index.jsp

- SOPC Builder http://www.altera.com/education/training/courses/OEMB1115 http://www.altera.com/literature/lit-sop.jsp

- System Console http://www.altera.com/education/training/courses/OEMB1117 http://www.altera.com/literature/ug/ug_system_console.pdf

- NIOS-II Flow http://www.altera.com/education/training/courses/IEMB115 http://www.altera.com/literature/lit-nio2.jsp

- Video and Image Processing Suite http://www.altera.com/products/ip/dsp/image_video_processing/m-alt-vipsuite.html http://www.altera.com/end-markets/broadcast/1080p/bro-1080p.html http://www.altera.com/support/refdesigns/sys-sol/broadcast/ref-format-conversion.html

- On-chip Debugging, including SignalTap-II http://www.altera.com/education/training/courses/ODSW1164 http://www.altera.com/support/software/debugging/sof-qts-debugging.html

http://www.altera.com/education/training/courses/IMEM210

http://www.altera.com/technology/memory/mem-index.jsp

http://www.altera.com/education/training/courses/OEMB1115

http://www.altera.com/literature/lit-sop.jsp

http://www.altera.com/education/training/courses/OEMB1117

http://www.altera.com/literature/ug/ug_system_console.pdf

http://www.altera.com/education/training/courses/IEMB115

http://www.altera.com/literature/lit-nio2.jsp

http://www.altera.com/products/ip/dsp/image_video_processing/m-alt-vipsuite.html

http://www.altera.com/end-markets/broadcast/1080p/bro-1080p.html

http://www.altera.com/support/refdesigns/sys-sol/broadcast/ref-format-conversion.html

http://www.altera.com/education/training/courses/ODSW1164

http://www.altera.com/support/software/debugging/sof-qts-debugging.html


Backup

33

Documents

Implementing Video Systems Using the Altera Video Framework