Upload
vuongminh
View
237
Download
0
Embed Size (px)
Citation preview
1
Youngsu Kwon
Processor Research Group
ICT Materials and Components Laboratory, ETRI
Trend and Analysis of Vision Intelligence and
ANN Intelligence Processor
Sep. 26, 2016
2 Aldebaran: Automotive ADAS Fault-Tolerant Processor
Vision Perception is Crucial
2
AUDI zFAS with NVIDIA K1
Mobileye EyeQ4
Google: 3D Scanning with Ridar
and map-based self-locating
Tesla: Single-lens object
detection with Mobileye
New era of “fast” Automotive Processor...
but, is our current technology sufficient to
provide Safety in Self-Driving?
3 Aldebaran: Automotive ADAS Fault-Tolerant Processor
Aldebaran3: ISO26262-Compliant Vision Processor
3
8mm
4 Aldebaran: Automotive ADAS Fault-Tolerant Processor 4
8mm
Aldebaran3 Features
Quad-Core
ABC_CT with Cache-ECC
and Recovery
CAN 1.0A/B and FD
for fast auto network
Recognition Engine
(Now transitioning to ANN)
H.265 Video Encoder and
Decoder for Video Storage
5 Aldebaran: Automotive ADAS Fault-Tolerant Processor
Aldebaran3: ISO26262-Compliant 1GHz Processor
5
ABC_CT3 @c3
m2
ABC_CT2 @c2
m3
m4
@b1
IR
s0 s1
@ddr3c_ui
DDR3C_MC @sdr
SDR_MC
s8 s9
@b1
LP
SW0 128bit @b0
RES @re
m5 m6
VOM @b0
VIM_128 @b0
SW3 32bit @b1
m0 m1
ABC_CT1 @c1
ABC_CT0 @c0
32bit 32bit 32bit 32bit
128bit
m9
128bit 32bit 128bit 128bit
DMA @b0
m8
32bit
HPN @hpn
32bit
m10
SPN @spn
128bit
32bit 32bit
PMU GPIO
VC FMC
UART TIMER
PWM WDT
RTC I2C
AC97 INTC
SDIO VIM
MOTOR JTAG
@b1
MP
s10
32bit
@vim_e @vom_e
@can_e
SW1 64bit @b0
SJTAG @b1
m12 m13
SDC0 @sdc
s5
SW2 32bit @b1
s7
EMP @emp
32bit
m11
s6
32bit 32bit
32bit
32bit
32bit 32bit
s4
@b1
FMC @b1
SMC @b1
USBHS
@usb_e
iROM
iRAM
CAN
SPN
HPN
abts3_noc128
abts3_noc128
abts3_noc128
abts3_noc128
p_osc_clki PLL
clkpll0~6
CSG
CSEL
clk00 ~ clk16 clkpll0~6
AXI
AHB
APB
RESET FTM
PMU
RE0 @re
32bit
m14
RE1 @re
32bit
m15
SW5 32bit @b0
m16
VIM_32 @b0
32bit
@vim_e
SW4 128bit @b0
AB3
6 Aldebaran: Automotive ADAS Fault-Tolerant Processor
Aldebaran3: Automotive Vision Processor
6
Vertical Integration :
ECU-SW-System-Chip
7 Aldebaran: Automotive ADAS Fault-Tolerant Processor
Aldebaran5
ABC_ST ABC_ST ABC_ST
Aldebaran5: Automotive ANN Processor
7
ABC_ST ABC_ST ABC_ST ABC_ST ABC_ST
ABC_ST ABC_ST ABC_ST ABC_ST ABC_ST ABC_ST ABC_ST ABC_ST
ABNC
(x1024 Convolver)
NOVA
(4K Enc/Dec)
Ethernet Cam/ISP CAN-FD
QEI
Legacy
AB FT-NoC
ISO26262 ASIL D
Compliant Multi-Core
H.265 Video Recording
* AI Processor for Autonomous Vehicle & Driver Assistance
ANN for
Real-time Recognition
with Power Efficiency
AI processor for
Vision in EVs AB-NZVC
Near-Zero Voltage Core
for Power-off missions
8 Aldebaran: Automotive ADAS Fault-Tolerant Processor
Searching for a Practical Way to ANN Processor
8
Neuromorphic Research
Large Scale Neuromorphic
Sytem
Memristive Device
IBM C 2 S 2 Synapse Project
Carbon nanotube
Sensor Application
Silicon Retina
Electronic Nose
Silicon Cochleas
Sensorimotor System
Stanford Brainstorm
Univ . of Hidelberg Spikey
USCD NeuroDyn
Small Scale Neuromorphic
System
Digital
Analog
UCSD HiAER IFAT Project
Stanford Neurogrid
Univ . of Hidelberg FACETS and
BrainScale Project
Univ . of Manchester
SpiNNaker Project
Neuro Computing System
Spin Device
HRL Neuromorphic
Aldebaran-ANN Project in ETRI
9 Aldebaran: Automotive ADAS Fault-Tolerant Processor
Practicality
9
It's nice to imitate Nature,
But we also need to understand
How do we know which
details are important?
Which details are merely
the result of evolution, and
the constraints of
biochemistry?
For airplanes, we developed
aerodynamics and fluid
dynamics.
We figured out that feathers
and wing flapping weren't
crucial.
Question: What is the
equivalent of aerodynamics for
understanding intelligence?
Excerpts from Yann LeCun's
L'Avion III de Clement Ader, 1897
His Eole(Avion) took off from the ground in 1890,
13 years before the Wright Brothers, but you may
have never heard of it.
10 Aldebaran: Automotive ADAS Fault-Tolerant Processor
Neuromorphic Researches in Korea
10
Institu
te
Circuit
Sensor/
Circuit
Memristor
ExoBrain
(SW)
DeepView
(SW)
Neurmorphic
Vision
Processor
(Chip)
Conte
nts
Low Power
Neural Cell and
STDP-based
512 Neuron
Core
Memristor-
Synapse
integrated into
image sensor
Memristor
synapse for
neuromorphic
device
Exobrain is a
SW for
Knowledge
learning and
sharing
Real-time video
recognition for
vision
intelligence
Mobile ANN
Processor for
Vision
intelligence
11 Aldebaran: Automotive ADAS Fault-Tolerant Processor
Huddles ahead
11
State-of-the-Arts Hardware Configuration Power/Energy
Google,
Andrew Ng,
ICML 2012
1000 server machines
Total 16,000 Processor
1 billion connections
10 million images (200x200)
3 days Training
1.6 MW
Google,
Andrew Ng,
ICML 2013
3 Server Machine
[4GPUs, GTX680, 3Tflops, 300W]+
2xQuadCore CPU]/server 3-5 days Training
1.5 kW (36Tflops)
Oxford, VGG
ILSVRC 2014
1 server machine
[4GPUs, Titan Black, 5Tflops, 300w]
16-19 layers
1.2 million images 2-3 weeks Training
1.2 kW (20Tflops)
Over 15x109 Neuron computation per second requires
specialized computing techniques for single-chip applications.
12 Aldebaran: Automotive ADAS Fault-Tolerant Processor
Synapse Compiler
12
Synapse Compiler
Re-Configuration
Synapse Optimization
Synaptic Op. Scheduling
Neural Net. Optimization
Neuron 1
Neuron 2
Neuron 3
Dendrite
Axon
Synapse
NN Architecture Model
SNN, CNN/FCN circuits u-Neural Core
Hardware Neural Net. (HNN)
Localization (Obj. Region Proposal)
Quantized Synapse Neural Net
Learning/Recognition
Algorithm
AlexNet, VGG, GoogleNet, ResNet,
…etc.,
Neural Network Model
Memory Capacity & Bandwidth
Synapse Capacity
Processing Performance (GSOPS)
Target Perception Performance
Optimization
CONSTRAINTs
Neuro-Synaptic
Code Network Connection
Synapse Weight
Operation Scheduling
Internal NC Network
uNC uNC
uNC uNC
uNC uNC
uNC uNC
uNC uNC
uNC uNC
uNC uNC
uNC uNC
uNC uNC
uNC uNC
uNC uNC
uNC uNC
Synapse Weights
Neural Op. Scheduling
Sensor
(ISP)
Aldebaran5 ANN Processor
Synapse Compiler produces a Neural Network by
restructuring, conversion, compression of state-of-the-art NNs
stream-in
13 Aldebaran: Automotive ADAS Fault-Tolerant Processor
Aldebaran5: A Practical ANN Processor
13
mdm
@b1clk
IR
sdd
@ddr3c_uiclk
DDR3C_MC
sir smp
@b1clk
LP
SW00 128bit @b0clk
NOVFMC @novclk
mvd mvi
VOM @b0clk
VIM_128 @b0clk
SW40 32bit @b1clk
mc0
ABC_CT @c0clk
$32K
32bit
128bit
128bit 128bit 128bit
DMA @b0clk
mnv
32bit
32bit 32bit
BootMode INTC
UART GIOCAP
PWM TIMER/WDT
RTC I2C
SPI
@b1clk
MP
slp
32bit
SW10 32bit @b0clk
SJTAG @b1clk
msi mjt
SDC0 @sdcclk
SW30 32bit @b1clk
32bit
32bit 32bit
sfm
@b1clk
FMC
iROM
iRAM
CAN
@can_e
QEI
p_osc_clki ABSYSF clk00-clk34
AXI
AHB
APB
PMURST
FTM
ISP @isp (clknet)
64bit
mip02
SW20 64bit @isp_busclk
64bit
mip03
32bit
sip26
mc1
ABC_CT @c1clk
32bit
mc3
ABC_CT @c3clk
32bit
mc4
ABC_CT @c4clk
32bit
mc5
ABC_ST @c5clk
32bit
SW11 32bit @b0clk
mc6
ABC_ST @c6clk
32bit
mc7
ABC_ST @c7clk
32bit
mc8
ABC_ST @c8clk
32bit
ADAS_IP @adas
32bit
mad0
SW21 32bit @adaclk
32bit
mad1
32bit
mad2
ABNC @ncclk
mnc
mc2
ABC_CT @c2clk
32bit
FT
M
FT
M
64bit
mip04
64bit
mip05
64bit
mip06
64bit
mip07
64bit
mip08
64bit
mip09
64bit
mip10
64bit
mip11
CDC
32bit
sspi
32bit
SW50 32bit @b0clk
mep
EMP @empclk
sep
14 Aldebaran: Automotive ADAS Fault-Tolerant Processor 14