Quantifying and Transferring Contextual Information in Object
Detection Professor: S. J. Wang Student : Y. S. Wang 1
Slide 2
Outline Background Goal Difficulties in Usage of Contextual
Information Provided solutions Another method: TAS Experimental
Results and Discussion Conclusion and Future Direction 2
Slide 3
Background (I) Only the properties of target object used in the
detection task in the past. Problem: Intolerable number of false
positive 3
Slide 4
Background (I) Only the properties of target object used in the
detection task in the past. Problem: Intolerable number of false
positive 4
Slide 5
Background (II) What else??? Contextual information! 5
Slide 6
Goal Establish a model to efficiently utilize the contextual
information to boost the performance of detection accuracy. 6
Slide 7
Difficulties (I) Diversity of Contextual Information There are
may different types of context often co-existing with different
degrees of relevance to the detection for the target object(s) in
different images. Terminology: Things (e.g. cars and people) Stuffs
(e.g. roads and sky) Scene (e.g. what happen in the image)
Thing-Thing, Thing-Stuff, Stuff-Stuff and Scene-Thing 7
Slide 8
Difficulties (II) Ambiguity of Contextual Information
Contextual information can be ambiguous and unreliable, thus may
not always have a positive effect on object detection. Ex: Crowded
Scene with constant movement and occlusion among multiple objects.
8
Slide 9
Difficulties (III) Lack of Data for Context Learning Not enough
training data : Over-fitting problem Wrong degree of relevance Ex:
The contextual information of people on top of sofa can be more
useful than people on top of grass. 9
Slide 10
Training Data Preparation & Notation Representation 10 Base
Detector (HOG) Training Image Candidate windows Positive sample:
Red Negative sample: Green
Slide 11
Provided Solutions A polar geometric descriptor for contextual
representation. A maximum margin context model (MMC) for
quantifying context. A context transfer learning model for context
learning with limited data. 11
Slide 12
Polar Geometric Descriptor Instead of traditional annotation
based descriptor, here we use polar geometric descriptor to
describe two kind of contextual information (Thing-Thing,
Thing-Stuff). 12 r :orientation b+1 :radial bins r*b+1 :patches
0.5, and 2 :bin length Feature :HOG Patch representation: Bag of
Words method using K-means with K = 100
Slide 13
Provided Solutions A polar geometric descriptor for contextual
representation. A maximum margin context model (MMC) for
quantifying context. A context transfer learning model for context
learning with limited data. 13
Quantifying Context (II) Quantifying Context (II) Goal =
Minimize the Risk function 15 Minimize L equal to fulfill the
following constraint Hard to be solved, could be replaced by
Slide 16
Quantifying Context (III) Maximum Margin Context Model 16 Add
some extra variables and constraints
Slide 17
Provided Solutions A polar geometric descriptor for contextual
representation. A maximum margin context model (MMC) for
quantifying context. A context transfer learning model for context
learning with limited data. 17
Slide 18
Context Transfer Learning Context Transfer Learning Two Cases:
Similar contextual information Ex: Cars and motorbikes Little in
common in both appearance and context, but similar level of
assistance provided by contextual information. Ex: People and bikes
18
Slide 19
TMMC-I: Transferring Discriminant Contextual Information
TMMC-I: Transferring Discriminant Contextual Information Similar
context provide the assistance on the learning of w. 19
Slide 20
TMMC-I: Transferring Discriminant Contextual Information
TMMC-I: Transferring Discriminant Contextual Information New
Constraint: 20 Modified optimization function:
Slide 21
TMMC-II: Transferring the Weight of Prior Detection Score
Similar level of assistance, same weight 21
Slide 22
TMMC-II: Transferring the Weight of Prior Detection Score 22
New Constraint: Modified optimization function:
Slide 23
Another Method: TAS 23
Slide 24
Another Method: TAS (I) 24 Steps: 1.Segmenting image into
regions. 2.Use base-detector to get the candidate patches.
3.Establish the relationships between candidate patches and
regions. 4.Use the relationships to judge there is a target object
in the patch or not.
Slide 25
Another Method: TAS (II) Region clusters: 25
Slide 26
Another Method: TAS (III) Examples of experiment: 26
Slide 27
Experimental Result and Discussion Use four data sets for
testing: VOC 2005 VOC 2007 I-LIDS FORECOURT 27
Slide 28
Experimental Result and Discussion 28
Slide 29
Experimental Result and Discussion 29
Slide 30
Experimental Result and Discussion Context Transfer Learning
Models: 30
Slide 31
Experimental Result and Discussion Context Transfer Learning
Models: 31
Slide 32
Conclusion and Future Direction In this paper, the author
proposes a contextual information model to quantify and select
useful context information to boost the detection performance. What
can we do next? HOG feature not suits for stuff (e.g. sky, road)
Automatic selection between TMMC-I, TMMC-II Automatic selection
between target object category and source category 32
Slide 33
Reference Wei-Shi Zheng, Member, IEEE, Shaogang Gong, and Tao
Xiang, Quantifying and Transferring Contextual Information in
Object Detection , PAMI accepted. Geremy Heitz, Daphne Koller,
Learning Spatial Context: Using Stuff to Find Things, ECCV 2008.
Youtube Search Hard-Margin SVM 33