Deep Learning with Coherent Nanophotonic Circuits Neural Networks (ANN) 11/8/2017 3 Deep Learning with Coherent Nanophotonic Circuits Breakthroughs in deep learning: • Natural Language

Deep Learning with Coherent Nanophotonic Circuits

Yichen Shen, Nicholas Harris, Dirk Englund, Marin Soljacic

Massachusetts Institute of Technology

@ Berkeley, Oct. 2017

11/8/2017 1Deep Learning with Coherent Nanophotonic Circuits

11/8/2017Deep Learning with Coherent

Nanophotonic Circuits2

Neuromorphic Computing

Biological Neural Networks Artificial Neural Networks

Artificial Neural Networks (ANN)

11/8/2017 3Deep Learning with Coherent

Nanophotonic Circuits

Breakthroughs in deep learning: • Natural Language

Processing (NLP)• Game Playing (Go, Atari)• Autonomous Vehicles• Control• Ad Placement• Researches (drug discovery,

material study)• Etc.

Basic Algorithm of ANN

Deep Learning with Coherent

Nanophotonic Circuits411/8/2017

Matrix Multiplication:

Nonlinear Activation:

…

… …

…

1

2

9

Hardware and Data Enable Deep Learning



The Need for SpeedMore Data Bigger Models More need for Computation



But Moore’s Law is no-longer providing more computation…

The Market:

On clouds:Millions of high power AI processors ($10,000 each) in data centers by 2020

On premise:Billions of compact AI processors needed due to the rise of autonomouse driving, AR and IoT.

Von Neumann ASIC/FPGA Optical Processing



Optical AI Computing

In Deep Learning

Key Operation is dense M x V



ajbi

Wij= x

In Optics, Matrix Multiplication

is very common & (usually)

consumes no energy !

Convolution / FFT

Matrix

Multiplication

Programmable Nanophotonic Processors



100µm

S U ( 4 ) C o r e D M M C

a

b

c

Phase Shifter

Waveguide

Transm

issio

n

Voltage2

O p t i c a l I n t e r f e r e n c e U n i t ( O I U )

MZI

Detectors

Input Modes

φi✓i

J. Mower et al, Physical Reviews A, 92, 032322 (2015)Carolan, Jacques, et al. "Universal linear optics." Science 349.6249 (2015): 711-716.

ANN does NOT require high resolution



Sze et al, arXiv:1703.09039 (2017)

Deep Learning Inference is “Passive”



Once the Optical Neural Network is trained, no

need to update the weights frequently…

Deep Learning is very parallelizable



Multiple wavelengths can be used to

simultaneously execute batch of data



Coherent Optical Neural Networks (ONN)

X1

X2

X3

X4

h1(1)

h2(1)

h3(1)

h4(1)

Z (1) = W0X

h1(i)

h2(i)

h3(i)

h4(i)

h1(n)

h2(n)

h3(n)

h4(n)

Y1

Y2

Y3

Y4

h( i ) = f (Z ( i ) ) Y = Wn h(n )

Input LayerHidden Layers

Output Layer

Layer i Layer n

Optical Input Optical Output

Layer 1X Y

a

b

c

x i n xou t

Waveguide

0

0

Optical Interference Unit Optical Nonlinearity Unit

0

0

d

S(n)V (n) U (n)

Vowel X

M (n)M (1) = U (1)S(1)V (1) M (2) M (n− 1)

fNL()fNL()

Photonic Integrated Circuit

fNL

Optical Vowel Recognition

(4d 4 classes)



11/8/2017Deep Learning with Coherent Nanophotonic Circuits 15

Y. Shen and N. Harris et al, Nature Photonics, 11, 441-446 (2017)

da

bLaser OIU Detectors Computer

U1S1V1

Tra

nsm

issio

nOIU1

OIU2 CPU OIU

3OIU

4

U1S1V1fSA ( Iin)

Input Output

Instance

Insta

nce

60µm

Optical Interference Unit

SU(4) Core DMMC

Dq Df

c

V2Dq

18µm

Directional Coupler

Phase ShifterSimulated Optical Nonlinearity

Experimental Result



Simulation Result: 165/180=91.7%Experiment Result: 138/180=76.7%

VowelIdentified

VowelIdentified

Vo

wel

Sp

ok

en

Vo

wel

Sp

ok

en



Fully Connected Neural Networks

Recurrent Neural Networks Convolutional Neural Networks

Recurrent Neural Networks



Commonly used for Speech Recognition and Language Processing

Convolution Neural Networks



Scott Skirlo and Yichen Shen et al, Manuscript in Preparation

Optical Convolutional Neural Network






Unified Buffer (localStorage SRAM)

ADC arrays

*modified block diagram from TPU architecture

Speed and Energy Efficiency Comparison with Electrical ANN



NVIDIA TITAN X ONN (with thermal PS)

Architecture Von Neumann Neuromorphic

Power Consumption 1 kW 1-2 kW

Operation Speed 10 TFLOP 10,000 TFLOP

Y. Shen and N. Harris et al, Nature Photonics 11, 441 (2017)

11/8/2017Deep Learning with Coherent Nanophotonic Circuits 23

Some History on Optical Neural Networks



Acknowledgement



Nicholas HarrisPhD, EECSMIT

Scott SkirloPhD, Physics MIT

Prof. Marin SoljacicPhysics, MIT

Prof. Dirk EnglundEECS, MIT

Li JingPhD, Physics MIT



Optical Convolutional Neural Network


Nonlinearity



A. Selden, British Journal of Applied Physics 18 , 743 (1967)M. Soljacic, Physical Review E 66, 055601 (2002)Z. Cheng et al, IEEE Journal of Selected Topics in Quantum Electronics 20.1 (2014): 43-48.

Nonlinear Photonic System

Iin Iout=f(Iin)

Saturable Absorption Photodiode

For Deep Learning, the constraint on nonlinearity is weak

Documents

Deep Learning with Coherent Nanophotonic Circuits Neural Networks (ANN) 11/8/2017 3 Deep Learning with Coherent Nanophotonic Circuits Breakthroughs in deep learning: • Natural Language