View
1
Download
0
Category
Preview:
Citation preview
eFPGA for AI and IoT ApplicationsTim Saxe
CTO
213/04/2018Copyright © 2018 QuickLogic, Inc. All rights reserved.
IoT Technology Stack-Up – Overview
Vision Sound Motion etc
Transistor-level (process and foundry)
Gate-level (ASIC, eFPGA, memory)
H/W accelerators/off-load engines
SoC (QUIK, NXP, ST, QCOM, etc)
Basic S/W (HAL, drivers, etc)
S/WEmbedded
CloudS/W
CloudS/W
CloudS/W
CloudS/WS/W
EmbeddedS/WEmbedded
S/WEmbedded
AI Applications
Tech
nolo
gy
FPGA IP (Data Control & Transform)
Pre-Fab(Time consuming and expensive to change)
Post-Fab(Fast cycle and relatively inexpensive to change)
313/04/2018Copyright © 2018 QuickLogic, Inc. All rights reserved.
AI is like a power tool – theoretically it can do anything
Build a house – much easier with tool
Put together a sofa – easier with tool
413/04/2018Copyright © 2018 QuickLogic, Inc. All rights reserved.
If you can connect it to the real world
Build a house – much easier with tool
Put together a sofa – easier with tool
513/04/2018Copyright © 2018 QuickLogic, Inc. All rights reserved.
If you can connect it to the real world
Build a house – much easier with tool
Put together a sofa – easier with tool
eFPGACan bridge the gap
613/04/2018Copyright © 2018 QuickLogic, Inc. All rights reserved.
And exactly what size problem are you trying to solve?
Data center problemIoT problem
eFPGACan help here, too
eFPGACan help here
713/04/2018Copyright © 2018 QuickLogic, Inc. All rights reserved.
Deep Learning Image Classification Network
AI Chip
rgb
3 planes32 x 32
128 planes32 x 32
convolve
128 planes32 x 32
convolve
128 planes16 x 16maxpool
256 planes16 x 16
convolve
256 planes16 x 16 convolve
256 planes8 x 8
maxpool
512 planes8 x 8
convolve
512 planes8 x 8
convolve
512 planes4 x 4
maxpool
1024x1 10x1Fully
connected
Requires 128KB scratch memory and 6MB coefficient memory
813/04/2018Copyright © 2018 QuickLogic, Inc. All rights reserved.
Deep Learning Image Classification Network
AI Chip
rgb
3 planes32 x 32
128 planes32 x 32
convolve
128 planes32 x 32
convolve
128 planes16 x 16maxpool
256 planes16 x 16
convolve
256 planes16 x 16 convolve
256 planes8 x 8
maxpool
512 planes8 x 8
convolve
512 planes8 x 8
convolve
512 planes4 x 4
maxpool
1024x1 10x1Fully
connected
Requires 128KB scratch memory and 6MB coefficient memory
And somehow the image miraculously appears at the start
CameraI/F
913/04/2018Copyright © 2018 QuickLogic, Inc. All rights reserved.
AI SoCeFPGA
Deep Learning Image Classification Network
rgb
3 planes32 x 32
128 planes32 x 32
convolve
128 planes32 x 32
convolve
128 planes16 x 16maxpool
256 planes16 x 16
convolve
256 planes16 x 16 convolve
256 planes8 x 8
maxpool
512 planes8 x 8
convolve
512 planes8 x 8
convolve
512 planes4 x 4
maxpool
1024x1 10x1Fully
connected
Camera I/F IP
Requires 128KB scratch memory and 6MB coefficient memory
eFPGA can manage the sensor and format the image
1013/04/2018Copyright © 2018 QuickLogic, Inc. All rights reserved.
Edge Friendly Deep Learning Image Classification Network
Requires 8KB scratch memory and 150KB coefficient memoryFits a small eFPGA with scratch memory
eFPGA
Neural Net IP
rgb
3 planes32 x 32
64 planes32 x 32
convolve
64 planes32 x 32
convolve
64 planes16 x 16maxpool
64 planes16 x 16
convolve
64 planes16 x 16 convolve
64 planes8 x 8
maxpool
128 planes8 x 8
convolve
128 planes8 x 8
convolve
128 planes4 x 4
maxpool
256x1 2x1Fully
connected
Camera I/FIP
1113/04/2018Copyright © 2018 QuickLogic, Inc. All rights reserved.
Radial Basis Function Image Classification Network
AI Chip
rgb
3 planes32 x 32
128 planes32 x 32
convolve
128 planes32 x 32
convolve
128 planes16 x 16maxpool
256 planes16 x 16
convolve
256 planes16 x 16 convolve
256 planes8 x 8
maxpool
512 planes8 x 8
convolve
512 planes8 x 8
convolve
512 planes4 x 4
maxpool
1024x1 10x1Fully
connected
DNN requires 128KB scratch memory and 6MB coefficient memory
CameraI/F
Feature Extraction Classification
RBF IP(65KB)
eFPGA
Camera I/F IP
Feature Extraction IP
1213/04/2018Copyright © 2018 QuickLogic, Inc. All rights reserved.
eFPGA provides sensor flexibility
AI SoC
eFPGA
Camera I/F IP AI EngineROI/Subsample IP
AI SoC
eFPGA
Sensor I/F IP AI EngineScale & Segmet IP
AI SoC
eFPGA
Microphone I/F IP AI EngineMFCC IP
1313/04/2018Copyright © 2018 QuickLogic, Inc. All rights reserved.
eFPGA provides real-time control
AI SoC
eFPGA
Camera I/F IPAI Engine
ROI/Subsample IP
Model Update IPServo Control IP
1413/04/2018Copyright © 2018 QuickLogic, Inc. All rights reserved.
Conclusions
Big Iron AI uses powerful eFPGAs to further accelerate powerful data center CPUs
IoT AI uses eFPGAs as part of an SoC to bring the benefits of hardware: Real-time operation Low-power operation
into the post-fab environment
IoT AI uses eFPGAs as part of an SoC to: Manage sensors Preprocess and format data Provide real-time control based on AI outputs
Recommended