1/25

http://www.deeplearning.net/software/theano

http://groups.google.com/group/theano­announce

theano

2/25

Theano: A CPU and GPUMath Expression Compiler

James Bergstra, Olivier Breuleux, Frederic Bastien,Pascal Lamblin, Razvan Pascanu, GuillaumeDesjardins, Joseph Turian, Yoshua Bengio

3/25

numpynumexpr

SymPy

scipy.linalgscipy.weavescipy.signal

pycuda

pyopencl

cythonthreading

multiprocessing

4/25

>>>

5/25

6/25

7/25

No pow() ??

8/25

A real example: Logistic Regression­ GPU ­ready­ symbolic differentiation­ speed optimizations­ stability optimizations

9/25

10/25

11/25

12/25

13/25

log1p(exp(x))

DGEMV(x.T,<>,0.98,W)

Loop Fusion logistic sigmoid

14/25

Some Benchmarks­ Multi­layer Perceptron­ Convolutional Neural Network­ Misc. Elementwise

Competitors­ NumPy+SciPy, MATLAB­ EBLearn, Torch5­ numexpr

15/25

Multi­Layer Perceptron: 60x784 matrix times 784x500matrix, tanh, times 500x10 matrix, elemwise, then all inreverse for backpropagation.

16/25

Convolutional Network: 256x256 images convolved with6 7x7 filters, downsampling to 6x50x50, tanh,convolution with 16 6x7x7 filters, tanh, matrix multiply,elemwise, then all in reverse for backpropagation.

17/25

Solid blue: TheanoDashed red: numexpr (without MKL)

18/25

Theano in Summary• High Level functional description of calculation• Rearranges expressions for speed and stability• Compiles many expressions to machine language• Uses GPU implementations (where possible)

http://www.deeplearning.net/software/theanohttp://groups.google.com/group/theano­announce

19/25

Theano Sprint

Ideas:­ Get Theano running on your computer­ Get an algorithm running with Theano­ Improve PyCUDA integration?­ Translate to/from SymPy graphs?­ Improve linear algebra support?­ Implement a conjugate solver ?­ Advanced Indexing with gradient ??

20/25

21/25

22/25

23/25

24/25

25/25