TSEA26 Design of Embedded DSP Processors · – Learn firmware (kernel) design basic skills • After completing the course we will – Design a simple processor, know embedded systems,

by Dake Liu: [email protected]© Copyright of Linköping University, all rights reserved

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 1

TSEA26

Design of Embedded

DSP Processors

Unit 1: Introduction

Dake Liu

mailto:[email protected]


9/11/2017 Unit 1 of TSEA26 – 2017 –H1 2

Introduction to

ASIP



Hardware design

• Functional (hardware) design

– Integrated circuit design can be

• 1. Design of functional modules, 2. SoC design (integration)

– Design using COTS can be

• 1. Design on FPGA, 2. PCB design + assembly coding

• Physical design (performance)

– Analog, RF, optical, sensor, actuator, and I/O design

• Non functional design (DFT, reliability, thermal)

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 3



Design of functional modules

• We learned digital logic circuit design

1. Combinational logic design and optimization

2. Sequential synchrounous logic circuit design

• We might learned functional module design

1. Function mapping on hardware (datapath)

2. Expose and design for storage and data access

3. Design a FSM, the module can work by itself!

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 4



A simplest example of designing

a functional ASIC module

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 5



A typical example: FFT ASIC module

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 6

writes

DIT datapathData addresses W address

Finite state machine and control signal decoding

4 Data memories W memory



ASIC advantages and drawbacks

• Advantages

– Simple

– Fast

– Low cost

– Low power

consumption

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 7

• Drawbacks

– Not flexibile

– Short product life

time

– Not suitable in a

modern SoC when

its NRE cost is up

to 100 million SEK



Introduction to ASIP

• ASIP: Application Specific Instruction-set

processor

– A programmable ASIC designed only for an

application domain, keeps life time longer.

– For example radio baseband processor, camera

processor, video CODEC, deep learning engine

– Programmable and flexible in an application

domain, very low HW overheads (low power,

and low silicon cost), ASIC performance.

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 8



When CPU, ASIP, and ASIC

CPU

• All general applications

• x86, ARM，

• High end, Very high design cost

• Strong SW ecologicalsupport

ASIP

• For volume embedded applications

• Low SW eco requirement

• Performance in an application domain

ASIC

• A function module not progrmmable

• Performance

• High design cost & Short product life time



2017/9/11 ASIP，刘大可，北理工 10

Requirements on embedded computing needs ASIP 0.1

GOPS 1 10 100 1000

Video

Audio

Communication

Graphics

MPEG1 Extraction

JPEG

MPEG2/4 compression

MPEG Extraction

Dolby-AC3

MP3

Sentence translation

Word recognition

Real-time voice recognition

2D graphics

3D video games

Modem FAX

LTE/HSPA eNodeB

LTE/HSPA BB in MS

Moving objective recognition Recognition

QCIF P1080

QCIF HDTV

Voice AMR

Voice gateway for VoIP / cellular

CT

Radar

EDGE+, CDMA

Enhancement

xDSL, DOCSIS

Ultrasonic array

Image/video analysis/process

3DTV



9/11/2017 Unit 1 of TSEA26 – 2017 –H1 11

The great history of

our division, our spin-

off company offered

20% global BB-ASIP

IP for mobile phones!



9/11/2017 Unit 1 of TSEA26 – 2017 –H1 12

Introduction to the

course and the staff



9/11/2017 Unit 1 of TSEA26 – 2017 –H1 13

Self introduction

Dake Liu Professor，

[email protected]

Phone 281256

Datorteknik

Oscar Gustafsson Docent

Erik Bertilsson PhD

http://www.da.isy.liu.se/courses/tsea26/



The goal of the course

• The course will be challenging and rewarding

• To learn industrial (warning) design experiences

– Learn (application specific) processor design skills

– Focus on efficient HW microarchitecture design

– Learn firmware (kernel) design basic skills

• After completing the course we will

– Design a simple processor, know embedded systems, DSP

implementations, DSP processor architectures.

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 14



You can do sth after the course

• Design an application specific instruction set or map

functions to hardware

– Code analysis, instruction set specification, SW-HW co-design

• Design a processor microarchitecture

– Datapath, Data access and memory, control path, and peripherals

• Be able to write efficient firmware for

– Quality computation kernels (parallelization, finite precision etc)

• have knowledge in firmware development toolchain

• have knowledge in processor integration and verification.

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 15



The scope and the backgrond

• It is about implementation, we learn methods

• Literature and references

– My book, Andreas Ehliar and Dake Liu: compedium

– Synopsys ASIP Designer (to know how to synthesis)

• Background knowledge:

– Logic circuit (Digitalteknik)

– Processor fundamentals (Datorteknik)

– DSP fundamentals (Signalbehandling)

– Programming skill MATLAB, C, ASM, VHDL/Verilog

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 16



For special international students

• Diplomatically speaking, if you don't have the

pre-knowledge, you will have gained a lot of

knowledge when passing this course. . .

– Students have passed this course before, without

having all of the prerequisites, but they probably

had to work pretty hard.

– Talk to me during the break if you have questions.

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 17



Book reading guide

• Following chapters / sections you can skip

• They will not be essential and will not be examed

– Skip 1.7.3, 2.1.6, 2.1.8, 3.1, 3.2, 3.4, 3.5.1, 3.5.2, 3.5.4, 3.5.5, 4.6

– chapter 5, 6, 9, 16, and 17 will not be examed

– Skip 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.4, 7.5, 8.2, 8.3, 8.4, 8.9, 10.1, 10.5, 11.3

– Do not read chapter 14. Read my compendium instead

– Skip 18.2.5

– 19.1 is rather old. It is enough to carefully follow my lecture/slides

– 19.2 is OK to read. Chapter 20 is rather old, try to follow my slides

• You are suggested to read through the book if you really want

to design a processor in the future.

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 18



9/11/2017 Unit 1 of TSEA26 – 2017 –H1 19

HW Design methodology

SoC

Micro arc

RTL

layout

behavior

function structure

physical

Gajski-Kuhn chart

(1983) from wikipedia



The method to learn the course

• Divide & conquer

– Refine & simulate

• We will use Daniel Gajski Y-Chart

• During HW design and spec, we describe

behavior, architecture, and phy reqs

– Describe & synthesis

• We will not teach ASIP synthesis

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 20



Summarize what/how to learn

System

understanding

Plan HW

schematic

HW

codingFW coding

Integration

verification

Finite precision Just enough quality Where/what Sat/rnd Gain strl Corner cases

Micro architecture Functions to map Sharing sharing ------ Balance

Register file Write conflict Critical path Fanout Life time Fanin fanout

ALU: Arithmetic & Logic HW sharing Reuse skill IP code precision corner

MAC: MUL and ACC MAC/LALU/MLU Reuse skill IP code Use MAC corner

Memory and data access Modulo Pipeline pipeline D-allocate IP coding

Program flow control PC and I-decoder I-decoders PC C-hazard Pipeline

Assembly coding tools Behavior/arch SIM ------ ------ debug Verification

Firmware plan & design Bit/mem/cycle ------ ------ plan vs code SW v.s. HW

Survey of Different ASIP Efficient VPU Tool limited critical Kernels

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 21

Skills

Con

cep

ts

5% 15% 10%20% 50%

10%

10%

10%

10%

10%

10%

10%

10%

10%

10%



The way to learn/reach TSEA26

• Based on Y-chart

– Focusing on specification / description

– Behavior, Architecture, Phy constraint

• Based on the 2D what/how table

– To fill in the table step by step

– To check, if you got enough knowledge

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 22



Education process and the

Examination

• Fö: 11 Lectures, ASIP design process

• Le: 6 Tutorials, follow me step by step

• La: 4 Labs, design a processor by filling in

• Exam

– Written examination 3ECTS

– Laboratory assignments 3ECTS

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 23



TSEA26 staff

• Lectures/examiner: Dake Liu

• Tutorial: Oscar, Dake

• Labs: Oscar, Eric

• Course homepage:

http://www.isy.liu.se/edu/kurs/TSEA26/

• About the Lectures, Tutorials, The labs

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 24



9/11/2017 Unit 1 of TSEA26 – 2017 –H1 25

Motivation to learn

ASIP design



9/11/2017 Unit 1 of TSEA26 – 2017 –H1 26

Application example: Communication

Transmitter

Transmitted

Signal Channel

Received

Signal Receiver

Estimated

message signal

Training signal … Normal signal …

A streaming period

… Normal signal Training …

Known

data Synchronization and

Channel estimator

Channel equalizer

and Receiver

Received

data

Channel

behavior

Streaming signal between a transceiver pair

Recovering data from noise channel

We need 100 times x86 performance and 1/100 x86 power and cost



9/11/2017 Unit 1 of TSEA26 – 2017 –H1 27

Application example:

Image and video compression

B-f

ram

e

G-f

ram

e

R-f

ram

e

RG

B t

o Y

UV

co

mp

ress

ion

V-f

ram

e

U-f

ram

e

Y-f

ram

e

F-domain frame compression

Lo

ssle

ss c

om

pre

ssio

n

Inter-frame compression

Intra-frame compression

F-d

om

ain

resi

du

e

co

mp

ress

ion

We need 10 times x86 performance and 1/200 x86 power and cost



9/11/2017 Unit 1 of TSEA26 – 2017 –H1 28

DSP implementation



9/11/2017 Unit 1 of TSEA26 – 2017 –H1 29

Implement DSP on general processor

• When you have a computer, run software on it

• Video audio player, image viewer etc. based on SIMD (Single Instruction Multiple Data)

– SSE (Intel), AltiVec (IBM), NEON (ARM)

• You do not want to design a video player using a 6000$ x86 Xeron E7 (a joke)– You need to design an Application Specific DSP

for high volume and long life time product



We thus need embedded system

• A computer system for dedicated functions

within a larger mechanical or electrical

system, often with real-time computing

constraints

• 98% of microprocessors, are manufactured

as components of embedded systems.

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 30



9/11/2017 Unit 1 of TSEA26 – 2017 –H1 31

ASIP• Application Specific Instruction-set Processor

– A processor dedicated for a certain kind of

application domain

– Instruction set optimized for specific applications

– ASIP are programmable accelerators for very

demanding applications

– Low power, high performance, low cost

• The focus of this course



9/11/2017 Unit 1 of TSEA26 – 2017 –H1 32

ASIC• Application Specific Integrated Circuit

– Lowest cost (in high volume)

– Lowest power

– Highest performance

• In industry:

– Very high development cost

– Long development time

– May reduce a product lifetime, induce another

chip design with huge NRE cost



The future

Any applications, when functions can be predicted and the volume is high, it will be implemented on ASIP

Classical highend embedded system consists of ARM + ASIC accelerators, the future one will be ARM + ASIP



9/11/2017 Unit 1 of TSEA26 – 2009 –H1 34

WiMax

HDTV STB

WiFi

WiFi

UWB

DVB

HSDPA

MediaBaseband

Network

Car

Entertainment

Automotive

Driving-

assistant

Satellite

More will be programmable

no more cost acceptable

Where are ASIPs?



9/11/2017 Unit 1 of TSEA26 – 2009 –H1 35

Let us learn it!



9/11/2017 Unit 1 of TSEA26 – 2017 –H1 36

ASIP fundamentals



ASIP (IP) in an Y-chart system

9/11/2017 37

Behavior Structure

Physics

SoCASIPmoduleRTL

micr



2017/9/11 Unit 1 of TSEA26 – 2017 –H1 38

ASIP design flow

Source code analysis, Decision for ISA of ASIP

Design instruction set and toolchain for prototyping

Benchmark (kernel), evaluate microarchitecturte

Microarchitecture design, VLSI design, Verifications

Change

ISA?Satisfied?

Yes

No



To design an efficient ASIP, we need

1. Application specific datapath and data types

– Deep understanding of data types /corner cases

– Data types / finite data precision (audio example)

2. Application specific memory access

– Deep understanding parallel data/access features

3. Application specific program flow

– Deep understanding of control complexities

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 39



What is (binary) data type

and why to discuss on it

• Data type defines implements & uses of data

– Standard data types: Integers, floating-points,

booleans, characters, alphanumeric strings

– Custom data type for ASIP: fractional, sufficient

precision, low cost, and minimized run time.

• Skills to handle low cost data type via trade

offs of low cost HW and high quality FW.

– Going through and to learn the course TSEA26

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 40



Custom data precision: Fixed

point numerical representation

• What is fixed point numerical representation?

– Real data type that has a fixed number of digits

• Integer or fractional representation for DSP

– Integer: Between -2n-1 and + 2n-1 -1

– Fractional: Between -1 and +1 – 2-n+1

– (Where n is the number of bits)

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 41



Frequently used Fixed Point number

Decimal Value Unsigned Two’s Complement

15 1111 ----

8 1000 ----

7 0111 0111

1 0001 0001

+0 0000 0000

-0 ---- ----

-1 ---- 1111

-7 ---- 1001

-8 ---- 1000



Precision

• The distance between the smallest values

that can be represented using a certain

number format

• Example:

– 16 bits fractional numbers, the precision of the

data is

0:000-0000-0000-00012 ≈ 0.00030510

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 43



Dynamic range (of a digital signal)

• The ratio between the largest number can be

represent and the precision.

• E.g., the dynamic range of a 16 bit fixed point

number format is 65535/1

• Commonly measured in dB. Example:

– 20 log1065535 ≈ 96dB.

– (Each extra bit adds 6dB)

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 44



Dynamic range and resolution

examples

Unsigned integer Signed Integer

Smallest Value:0000(0)

Largest value :1111(15)

Most Positive Value:0111= (+7)

Least negative value:1000 =(-8)

Unsigned Fraction Signed Fraction

Smallest Value:.0000(0)

Largest value :.1111(0.9375)

Most Positive Value:0.111= (+0.875)

Least negative value:1.000 =(-1)

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 45



Number

format

Dynamic

Range

Dynamic Range

in dBPrecision

Unsigned

Integer0 to 65536 20log10(2^16)= 96 dB 1

signed

Integer

-32768 to

3276720log10(2^15)= 90 dB 1

Unsigned

Fraction

0 to

0.9999847496 dB 2-16

signed

Fraction

-1 to

0.9999694890 dB 2-15

Dynamic range and resolution

examples, 16-bit data

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 46



Fractional vs integer (multiplication)

9/11/2017 Unit 1 of TSEA26 – 2016 –H1 47



Drawbacks of Fixed Point

• Sometimes it is not possible to separate

dynamic range and precision

• Higher firmware design costs (E.g., when

using a Matlab model as a reference)

• We thus may use floating point data for

applications requiring higher dynamic range

and very short software development time

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 48



Let us review what did we do

• If you understood, congratulations!

• If you do not understand, find out the reason?

• What is ASIP, why design it?

• What is microarchitecture? How to design?

• How many kinds of DSP implementations?

• We already have the 1st step: custom data types.

• To think when you are home:

– Microarchitecture design, Y-chart, and 8p FFT ASIC

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 49



ASIP on markets (mostly in SoC as IP)

2017/9/11 Unit 1 of TSEA26 – 2017 –H1 50

ASIP Applications IP/year $/year

SDR baseband Handset, base station 1B 3-5B

ISP for image and video Handset, video, camera 1B 1-2B

Video codec Handset, survaillence 0.5B 2-4B

Storage SSD、M-cards >100M ~500M

Gateways Gateway, home gateway >50M ~500M

Network processors ISP、router >10M ~100M

Industrial control Motion and motor control 100M 300M

Robots Vision, control 10M 30M

IoT Communication, sensing 50B 50B

Deep learning Server, terminal ? ?

Defense DFE Baseband, sensing, ISP ? ?

Defense AP Recognition, decision ? ?

……Video application, VR, AR, medical, toys, commercial, and much more……



Self reading after the lecture

• Pick up related knowledge in the pre-

requirement list

• Chapter 1

• Integer part in Chapter 2

• Think about: How to reach the best

data quality via FW coding on low cost

fixed point processor

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 51



Exciting time now!

Let us discuss• Whatever you want to discuss and

related to HW

• You will have the chance after each

lecture (Fö), do take the chance!

• Prepare your Qs for the next time

9/11/2017 Unit 1 of TSEA26 – 2017 –H1 52



LOGO

Dake Liu, Room 556 coridoor B, Hus-B, phone 281256, [email protected]

Welcome to ask any

questions you want to

• I can answer

• Or discuss together

• I want to know what you want


Documents

TSEA26 Design of Embedded DSP Processors · – Learn firmware (kernel) design basic skills • After completing the course we will – Design a simple processor, know embedded systems,