Introduction to Deep Learning

Google Brain

Copied from google image search

Project Adam

• Microsoft built a highly efficient, highly scalable distributed system from commodity PCs to train deep neural networks

• They use 14 million images from ImageNet to train the deep neural network

http://research.microsoft.com/en-us/news/features/dnnvision-071414.aspx

• Project Adam can identify dogs

• Project Adam can identify kinds of dogs - Corgi

• Project Adam can identify particular breeds of Corgi: Pembrake vs Cardigan Corgi

Demo Systems

• http://deeplearning.cs.toronto.edu/

Highway Perception

Copied from “An Empirical Evaluation of Deep Learning on Highway Driving”http://arxiv.org/pdf/1504.01716v2.pdf

Microsoft Speech Translation

• Chief Research Officer of Microsoft Research Rick Rashid demonstrates a speech recognition breakthrough patterned after deep neural networks

• https://www.youtube.com/watch?v=Nu-nlQqFCKg

Deep Learning in IQ Test

Isotherm is to temperature as isobar is to? (i) atmosphere, (ii) wind, (iii) pressure, (iv) latitude, (v) current.

Which is the odd one out? (i) calm, (ii) quiet, (iii) relaxed, (iv) serene, (v) unruffled.

Which word is most opposite to MUSICAL? (i) discordant, (ii) loud, (iii) lyrical, (iv) verbal, (v) euphonious.

Copied from http://www.technologyreview.com/view/538431/deep-learning-machine-beats-humans-in-iq-test/

Copied from http://www.technologyreview.com/view/538431/deep-learning-machine-beats-humans-in-iq-test/

Medical Diagnosis

• Deep learning technology has been applied to medical diagnosis based on a large amount of accumulated X-rays, CT scans, lab data and MRIs

• Diagnosis by deep learning are typically more objective and accurate according to some studies.

Deep Genomics

A new startup led by University of Toronto professor Brendan Frey use deep learning to identify gene variants and mutations never before observed or studied and find how these link to various diseases

Copied from http://www.gizmag.com/deep-genomics-medicine-machine-learning/38623/Image credit: Shutterstock

A Little History of Neural Network

• 1958: Frank Rosenblatt (1958) created the perceptron.

• 1969: Marvin Minsky and Seymour Papert discovered that perceptron failed to solve exclusive-or problem.

• 1975: backpropagation learning algorithm was proposed by Werbos

• 1992: Kernel Machines were proposed for nonlinear classification

• Between 2009 and 2012: Deep Learning methods achieved great success

- Deep learning was shown achieving human-competitive or even superhuman performance on important benchmarks - A team from Georff Hinton’s lab at University of Toronto won a 2012 contest sponsored by Merck to design software to help find molecules that might lead to new drugs.[29]

Deep Learning Vs Traditional Machine Learning

• Deep learning: don’t need to provide features ahead of time, it learns features at different level by itself.

• The same deep learning architecture can be trained to accomplish different tasks.

• Deep learning is the solution to big data

Deep Learning Vs IBM Watson

Copied from google image search