Upload
lefty
View
47
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Visual Object Recognition. Rob Fergus Courant Institute, New York University. http://cs.nyu.edu/~fergus/icml_tutorial/. Agenda. Introduction Bag-of-words models Visual words with spatial location Part-based models Discriminative methods Segmentation and recognition - PowerPoint PPT Presentation
Citation preview
Visual Object Recognition
Rob FergusCourant Institute,
New York University
http://cs.nyu.edu/~fergus/icml_tutorial/
AgendaAgenda• Introduction
• Bag-of-words models
• Visual words with spatial location
• Part-based models
• Discriminative methods
• Segmentation and recognition
• Recognition-based image retrieval
• Datasets & Conclusions
Li Fei-Fei, PrincetonRob Fergus, NYU
Antonio Torralba, MIT
Recognizing and Learning Recognizing and Learning Object Categories: Year 2007Object Categories: Year 2007
http://people.csail.mit.edu/torralba/shortCourseRLOC
AgendaAgenda• Introduction
• Bag-of-words models
• Visual words with spatial location
• Part-based models
• Discriminative methods
• Segmentation and recognition
• Recognition-based image retrieval
• Datasets & Conclusions
So what does object recognition involve?
Classification: are there street-lights in the image?
Detection: localize the street-lights in the image
Object categorization
mountain
buildingtree
banner
vendorpeople
street lamp
Scene and context categorization
• outdoor• city• …
Application: Assisted driving
meters
met
ers
Ped
Ped
Car
Lane detection
Pedestrian and car detection
• Collision warning systems with adaptive cruise control, • Lane departure warning systems, • Rear object detection systems,
Application:Computational photography
Application: Improving online search
Query:STREET
Organizing photo collections
Challenges 1: view point variation
Michelangelo 1475-1564
Challenges 2: scale
Challenges 3: illumination
slide credit: S. Ullman
Challenges 4: background clutter
Bruegel, 1564
Challenges 5: occlusion
http://lh5.ggpht.com/_wJc6t2hDl2M/RrL7Gh6sS7I/AAAAAAAAAYY/n3xaHc2opls/DSC00633.JPG
Challenges 6: deformation
http://img.timeinc.net/time/asia/magazine/2007/1112/racehorse_1112.jpg
History: single object recognition
Object 1 Object 2
Object 3
David Lowe [1985]
Single object recognition history: Geometric methods
Rothwell et al. [1992]
Single object recognition history: Appearance-based methods
• Murase & Nayer 1995 • Schmid & Mohr 1997• Lowe, et al. 1999, 2003• Mahamud and Herbert, 2000• Ferrari et al. 2004• Rothganger et al. 2004• Moreels and Perona, 2005• …
Instance 1 Instance 2
Instance 3
Challenges 7: intra-class variation
Shoe class
History: early object categorization
• Fischler, Elschlager, 1973 • Turk and Pentland, 1991• Belhumeur, Hespanha, & Kriegman, 1997• Rowley & Kanade, 1998• Schneiderman & Kanade 2004• Viola and Jones, 2000• Heisele et al., 2001
• Amit and Geman, 1999• LeCun et al. 1998• Belongie and Malik, 2002• DeCoste and Scholkopf, 2002• Simard et al. 2003
• Poggio et al. 1993• Argawal and Roth, 2002• Schneiderman & Kanade, 2004…..
Three main issuesThree main issues
• Representation– How to represent an object category
• Learning– How to form the classifier, given training data
• Recognition– How the classifier is to be used on novel data
Representation– Generative /
discriminative / hybrid
Representation– Generative /
discriminative / hybrid– Appearance only or
location and appearance
Representation– Generative /
discriminative / hybrid– Appearance only or
location and appearance
– Invariances• View point• Illumination• Occlusion• Scale• Deformation• Clutter• etc.
Representation– Generative /
discriminative / hybrid– Appearance only or
location and appearance
– Invariances– Part-based or global with sub-window
Representation– Generative /
discriminative / hybrid– Appearance only or
location and appearance
– Invariances– Parts or global w/sub-
window– Use set of features or
each pixel in image
– Unclear how to model categories, so learn rather than manually specify
Learning
– Unclear how to model categories, so learn rather than manually specify
– Methods of training: generative vs. discriminative
Learning
– Unclear how to model categories, so learn rather than manually specify
– Methods of training: generative vs. discriminative
– Level of supervision• Manual segmentation; bounding box; image labels; noisy labels
Learning
Contains a motorbike
Learning– Unclear how to model categories, so learn rather than manually specify
– Methods of training: generative vs. discriminative
– Level of supervision• Manual segmentation; bounding box; image labels; noisy labels
-- Training images:Issue of over-fitting (typically limited training data)Negative images for discriminative methods
Learning– Unclear how to model categories, so learn rather than manually specify
– Methods of training: generative vs. discriminative
– Level of supervision• Manual segmentation; bounding box; image labels; noisy labels
-- Training images:Issue of over-fitting (typically limited training data)Negative images for discriminative methods
-- Priors
– Scale / orientation range to search over – Speed– Context
Recognition
Hoi
em, E
fros,
Her
bert,
200
6
– Context enables pruning of detector output
Recognition