Multimedia SystemsMultimedia Systems

Lecture 01 Introduction

Saeid Nooshabadi

APPLICATIONS OF VLSI SIGNALPROCESSINGPROCESSING

• Embedded digital signal processing isEmbedded digital signal processing is everywhere!

• Examples:Examples:– Speech– AudioAudio– Video– Radio/wireless– Any application that processes signals in the

digital domain

TYPICAL ALGORITHMSTYPICAL ALGORITHMS

• Filtering: FIR IIR with fixed coefficients orFiltering: FIR, IIR, with fixed coefficients or adaptive

• Encoding/decoding• Encoding/decoding• Compression/decompression• Frequency-domain processing• Etc.

TYPICAL NUMBERSTYPICAL NUMBERS

• Speech: 8 kHz 12-16 bitsSpeech: 8 kHz, 12 16 bits• Audio: 44 kHz, 16-24 bits, two channels

(stereo)(stereo)• Video, various formats, e.g.:

– HDTV approx. 2000 by 1000 pixels at 50 frames per second resulting in data rates of 100 MHz 3 colors of 8 12 bits each100 MHz, 3 colors of 8-12 bits each

STREAMING VS. BLOCK-BASEDSTREAMING VS. BLOCK BASED

• Streaming data:Streaming data:– Data samples are processed as they arrive– Requires little local storage– Time-domain processing

• Block-based processing:– Stores incoming data until some block size is

filledProcesses entire block– Processes entire block

– Think e.g. of an FFT (Fast Fourier Transform) or DCT (Discrete Cosine Transform)

IMPLEMENTATION PLATFORMSIMPLEMENTATION PLATFORMS• General-purpose processor, such as a Pentium• Digital signal processor:

– Much better suited (parallel arithmetic in data path, support for “multiply-accumulate” operation, Harvard architecture f )for parallel access to data and program memory, etc.)

• Very large instruction word (VLIW) processor:– Many parallel arithmetic units in data path, each controlled

by appropriate bits in instruction word• Processor arrays• User-defined architectures• Dedicated logic (implemented by standard cells or on

FPGA)

TEXAS INSTRUMENTS DSP TMS320C 6455 (1 GH )TMS320C- 6455 (1 GHz)

VLIW ARCHITECTURE(e.g. PHILIPS/NXP TRIMEDIA)

• long instruction words e g (3*7+4)*25=625 bits• long instruction words e.g. (3*7+4)*25=625 bits• many ports on the register file e.g. 75

MAPPING PROBLEMMAPPING PROBLEM• How do we get the most efficientHow do we get the most efficient

implementations of DSP algorithms on our platforms?

• Optimization criteria:– Fastest– Smallest– Minimal energy

Shortest design time– Shortest design time• In general, flexibility comes at the expense of

efficiencyefficiency.

EXAMPLE: SITEL’S SC14480 SINGLECHIP DECT PROCESSORSINGLECHIP DECT PROCESSOR

VVLS

I Siggnal P

roocessingg

VLSI DSP IssuesVLSI DSP Issues

• Non‐Terminating Programs Require Real‐TimeNon Terminating Programs Require Real Time Operations

• Applications dictate different speed constraints• Applications dictate different speed constraints– (e.g., voice, audio, cable modem, settop box, Gigabit ethernet 3 D Graphics)Gigabit ethernet, 3‐D Graphics)

• Need to design Families of Architectures for ifi d l ith l it d dspecified algorithm complexity and speed 

constraints• Methods of representations of DSP Algorithms

Typical DSP ProgramsTypical DSP Programs

• Usually highly real‐time design hardwareUsually highly real time, design hardware and/or software to meet the application speed constraintspeed constraint

N i i E l• Non‐terminating Examples

Area‐Speed‐Power TradeoffsArea Speed Power Tradeoffs

• 3‐Dimensional Optimization (Area, Speed, Power)3 Dimensional Optimization (Area, Speed, Power)• Achieve Required Speed, Area‐Power Tradeoffs• Power Consumption (P = C ×V 2 × f)Power Consumption (P = C ×V × f)• Latency reduction Techniques  Increase in speed or power reduction through lower supply voltageor power reduction through lower supply voltage operation

• Since the capacitance of the multiplier is usually p p ydominant reduction of the number of multiplications is important (this is possible through strength reduction)

Representation Methods of DSP systems

Example: y(n)=a*x(n)+b*x(n‐1)+c*x(n‐2)Example: y(n)=a x(n)+b x(n 1)+c x(n 2)• Graphical Representation Method 1: Block DiagramDiagram– Consists of functional blocks connected with directed edges which represent data flow from itsdirected edges, which represent data flow from its input block to its output block

Graphical Representation Method 2: Signal‐Flow Graph

• SFG: a collection of nodes and directed edgesg• Nodes: represent computations and/or task, sum all incoming signals• Directed edge (j, k): denotes a linear transformation from the output 

signal at node j to the input signal at node k• Linear SFGs can be transformed into different forms without changing 

the system functions For example Flow graph reversal orthe system functions. For example, Flow graph reversal or transposition is one of these transformations (Note: only applicable to single‐input‐single‐output systems)

• Usually used for linear time‐invariant DSP systems representation

Graphical Representation Method 3: Data‐Flow Graph

• DFG: nodes represent computations (or functions or subtasks), while the directed edges represent data paths (data communications between nodes), each edge has a nonnegative number of delays.

• DFG captures the data‐driven property of DSP algorithm: any nodeDFG captures the data driven property of DSP algorithm: any node can perform its computation whenever all its input data are available.

• Each edge describes a precedence constraint between two nodes in DFG:– Intra‐iteration precedence constraint: if the edge has zero delays– Inter‐iteration precedence constraint: if the edge has one or more delaysInter iteration precedence constraint: if the edge has one or more delays– DFGs and Block Diagrams can be used to describe both linear single‐rate and 

nonlinear multi‐rate DSP systems Fine Grain DFG– Fine‐Grain DFG

Examples of DFGExamples of DFG

• Nodes are complex blocks (in Coarse‐GrainNodes are complex blocks (in Coarse Grain DFGs)

• Nodes can describe expanders/decimators in M l i R DFGMulti‐Rate DFGs

Gourse GradingGourse Grading

• Homework (10)Homework (10)• Project (25)

id S (20)• Mid Sem (20)• Final (45)

DE2‐70 Development BoardDE2 70  Development Board

The DE2-70 board has many features that allow the user to yimplement a wide range of designed circuits, from simple circuits to various multimedia projects.

DE2‐70 Features (#1/2)DE2 70 Features (#1/2)

• The Altera DE2‐70  • 9 green user LEDsdevelopment and education board, equipped with almost 70 000 LEs of Altera

g• 50‐Mhz and 28.63‐Mhz 

oscillators for clock sourcesalmost 70,000 LEs of Altera Cyclone® II 2C70, 

• 2‐Mbyte SSRAM

• 24‐bit CD‐quality audio CODEC with line‐in, line‐out, and microphone‐in jacks

• Two 32‐Mbyte SDRAM• 8‐Mbyte Flash memory

p j• VGA DAC (10‐bit high‐speed 

triple DACs) with VGA‐out • SD Card socket• 4 pushbutton switches

18 t l it h

connector• 2 TV Decoder (NTSC/PAL) 

and TV‐in connector• 18 toggle switches• 18 red user LEDs

and TV in connector

DE2‐70 Features (#2/2)DE2 70 Features (#2/2)

• 10/100 Ethernet Controller  • Two 40‐pin Expansion /with a connector

• USB Host/Slave Controller ith USB t A d t B

p pHeaders with diode protectionSi 153*203with USB type A and type B 

connectors• RS‐232 transceiver and 9‐

• Size：153*203 mm

pin connector• PS/2 mouse/keyboard 

connector• IrDA transceiver• 1 SMA connector• 1 SMA connectorhttp://www.terasic.com.tw/cgi-bin/page/archive.pl?Language=English&CategoryNo=39&No=226

DE2‐70 LayoutDE2 70 Layout

NIOS‐Processor on DE2‐70NIOS Processor on DE2 70

SOPC Builder ToolSOPC Builder Tool 

TopicsTopics

• Topics include; introduction to custom digitalTopics include; introduction to custom digital processors including DSP hardware, high‐speed digital design techniques modern chipspeed digital design techniques, modern chip design methodologies, hardware and software co‐design advanced programming paradigmsco design, advanced programming paradigms including state machines and concurrent processes real‐time programming andprocesses, real time programming and operating systems.

ReferencesReferences

• K. Pahri, "VLSI digital signal processing systems,” JohnK. Pahri,  VLSI digital signal processing systems,  John Wiley. &. Sons,. 1999

• J. Ganssle: The Art of Designing Embedded Systems, 1st g g y ,ed., 1999, Newnes Press, Butterworth‐Heinemann.

• Additionally, you may find the following reference books y y y ghelpful:– Embedded Systems Architecture: A Comprehensive Guide for 

E i d P (E b dd d T h l ) b TEngineers and Programmers (Embedded Technology) by Tammy Noergaard

– Embedded Systems: A Contemporary Design Tool by James K. Peckol

– Programming Embedded Systems: With C and GNU Development Tools, 2nd Edition by Michael Barr and Anthony Massa

ConclusionConclusion

• Digital architectures are used for DSPDigital architectures are used for DSP applications

Speech– Speech– Audio

Video– Video– Radio/wireless

A li ti th t i l i th– Any application that processes signals in the digital domain