Lecture: Introduction to Computer Visionvision.stanford.edu/teaching/cs131_fall1819/files/01_introduction.pdf · Lecture: Introduction to Computer Vision Juan Carlos Niebles and Ranjay

Introduction25-Sep-2018

1

Lecture: Introduction to Computer Vision

Juan Carlos Niebles and Ranjay KrishnaStanford Vision and Learning Lab


2Welcome to CS131


3Jason SalavonGAN experiment on Twitter


4Mario Klingemann, GAN experiment on Twitter


5Memo Akten


6

What you should expect to learn in this class.• What is computer vision?• Which problems fall under this umbrella?• Which applications are possible today and in the near future?• What are common research questions?• What is the history behind these problems and how did it lead to

deep leading?• What tools will help you develop a framework to solve these

problems?

Introduction2

5-S

ep

-20

18

7

CS131 is the introductory course for computer vision

• CS131 (fall, 2017):

– Overview of computer vision and all its applications

–Will prepare you for an industry job in vision

• CS231a (winter, 2018, Prof. Silvio Savarese)

– Advanced Computer Vision

– focusing on 3D vision

• CS231n (spring, 2018):

– Convolutional Neural Networks


8

Mathematics

ComputerScience

Biology

Engineering

Physics

Psychology

Computer Vision

Neuroscience

Machine learning

Speech, NLP

Information retrieval

Robotics

Cognitive sciences

Algorithms, theory,…

Imageprocessing

Systems, architecture, …

optics


9

Today’s agenda• Introduction to computer vision• Course overview


10

Quiz?


11

What about this?


12

Image (or video) Sensing device Interpreting device Interpretations

garden, spring,bridge, water,trees, flower,green, etc.

What is (computer) vision?


13

The goal of computer vision• To bridge the gap between pixels and “meaning”

What we see What a computer sees Sour

ce: S

. Nar

asim

han


14





15

1981: Nobel Prize in medicine

Hubel & Wiesel


16

Potter, Biederman, etc. 1970s

Human vision is superbly efficient


17Thorpe, et al. Nature, 1996


18Thorpe, et al. Nature, 1996

150 ms !!


19

Rensink, O’regan, Simon, etc.

Change Blindness


20

Rensink, O’regan, Simon, etc.

Change Blindness


21

Segmentation


22

Perception


23


24


25





26

Paintings in 1838


27

1812: Jacques-Louis-DavidThe Emperor Napoleon at his Study at the Tuileries


281808: Ingres, La grande baigneuse


29

“From today, painting is dead”— painter Paul Delaroche

at a demonstration of the Daguerreotype, 1839


301837: Niépce, First photo of one’s meal


31

1838

: Bou

leva

rd d

u Te

mpl

e, D

ague

rre


321838: First selfie, Robert Cornelius


33DeepDreams[Mordvintsev et al. 2015]


34

Neural Style Transfer[Gatys et al. 2015]


35Neural Style Transfer[Gatys et al. 2015]


36

CycleGAN [Zhu et al. 2017]


37

The goal of computer vision• To bridge the gap between pixels and “meaning”

What we see What a computer sees

Sour

ce: S

. Nar

asim

han


38

Origins of computer vision:an MIT undergraduate summer project

L. G. Roberts, Machine Perception of Three Dimensional Solids, Ph.D. thesis, MIT Department of Electrical Engineering, 1963.


39

What kind of information can we extract from an image?• Metric 3D information• Semantic information


40

Vision as measurement deviceReal-time stereo Structure from motion

NASA Mars Rover

Pollefeys et al.

Reconstruction fromInternet photo collections

Goesele et al.


41

sky

water

Ferris wheel

amusement park

Cedar Point

12 E

tree

tree

tree

carouseldeck

people waiting in line

ride

ride

ride

umbrellas

pedestrians

maxair

bench

tree

Lake Erie

people sitting on ride

ObjectsActivitiesScenesLocationsText / writingFacesGesturesMotionsEmotions…

The Wicked Twister

Vision as a source of semantic information

Slid

e c

red

it:

Kri

ste

n G

rau

man


42

Why study computer vision?• Vision is useful: Images and video are everywhere!

Personal photo albums

Surveillance and security

Movies, news, sports

Medical and scientific images


43

Special effects: shape and motion capture

Sour

ce: S

. Sei

tz


44

3D urban modeling

Bing maps, Google Streetview

Source: S. Seitz

http://www.bing.com/maps/explore/


45

3D urban modeling: Microsoft Photosynth

http://photosynth.netSource: S. Seitz

http://photosynth.net


46

Face detection

• Many digital cameras now detect faces– Canon, Sony, Fuji, …

Source: S. Seitz


47

Smile detection

Sony Cyber-shot® T70 Digital Still Camera Source: S. Seitz

http://www.sonystyle.com/webapp/wcs/stores/servlet/ProductDisplay?catalogId=10551&storeId=10151&productId=8198552921665200469&langId=-1


48

Face recognition: Apple iPhoto software

http://www.apple.com/ilife/iphoto/

http://www.apple.com/ilife/iphoto/


49

Biometrics

How the Afghan Girl was Identified by Her Iris Patterns

Source: S. Seitz

http://www.cl.cam.ac.uk/~jgd1000/afghan.html


50

Biometrics

Fingerprint scanners on many new laptops, other devices

Face recognition systems now beginning to appear more widelyiphone X just introduced face recognition


51

Optical character recognition (OCR)

Digit recognition, AT&T labs

Technology to convert scanned docs to text• If you have a scanner, it probably came with OCR software

License plate readershttp://en.wikipedia.org/wiki/Automatic_number_plate_recognition

Source: S. Seitz

http://en.wikipedia.org/wiki/Automatic_number_plate_recognition


52

Google maps: Annotate all houses and streets

Goodfellow et al. 2014


53

Toys and Robots


54

Mobile visual search: iPhone Apps


55

Snapstacles and Google glasses

• That’s Ranjay in undergrad ->


56

Automotive safety

• Mobileye: Vision systems in high-end BMW, GM, Volvo models – “In mid 2010 Mobileye will launch a world's first application of full

emergency braking for collision mitigation for pedestrians where vision is the key technology for detecting pedestrians.”

Source: A. Shashua, S. Seitz

http://www.mobileye.com/


57

Vision in supermarkets

LaneHawk by EvolutionRobotics“A smart camera is flush-mounted in the checkout lane, continuously watching for items. When an item is detected and recognized, the cashier verifies the quantity of items that were found under the basket, and continues to close the transaction. The item can remain under the basket, and with LaneHawk, you are assured to get paid for it… “

Source: S. Seitz

http://www.evolution.com/products/lanehawk/


58

Amazon Go


59

Vision-based interaction (and games)

Microsoft’s Kinect

Source: S. Seitz

Assistive technologies

Sony EyeToy


60

Augmented Reality


61

Virtual Reality

Introduction2

5-S

ep

-20

18

62

Vision for robotics, space exploration

Vision systems (JPL) used for several tasks• Panorama stitching

• 3D terrain modeling

• Obstacle detection, position tracking

• For more, read “Computer Vision on Mars” by Matthies et al.

NASA'S Mars Exploration Rover Spirit captured this westward view from atop a low plateau where Spirit spent the closing months of 2007.

Sour

ce: S

. Sei

tz

http://www.ri.cmu.edu/pubs/pub_5719.html

http://marsrovers.jpl.nasa.gov/gallery/images.html


63MGMT “When You Die”

Documents

Lecture: Introduction to Computer Visionvision.stanford.edu/teaching/cs131_fall1819/files/01_introduction.pdf · Lecture: Introduction to Computer Vision Juan Carlos Niebles and Ranjay