Upload
hinda
View
73
Download
2
Embed Size (px)
DESCRIPTION
Yuanlu Xu Advisor: Prof. Liang Lin [email protected]. Person Re-identification by Matching Compositional Template with Cluster Sampling. Problem. Person Re-identification. Identifying The Same Person Under Different Cameras. Basic Assumption : - PowerPoint PPT Presentation
Citation preview
Yuanlu XuAdvisor: Prof. Liang [email protected]
Person Re-identification by Matching Compositional Template with Cluster
Sampling
Problem
Identifying The Same Person Under Different Cameras
Person Re-identification
Basic Assumption: 1. Face is unreliable due to view, low resolution and noises.2. People's clothes should remain consistent.
Large Intra-class Variations
Difficulty
Pose/View Variation Illumination Change Occlusion
Problem
Query Person
S vs. S M vs. S
Scene
Search
Multiple Setting
Representation
1. Body into 6 parts, limbs further into 2
symmetric parts.
2. Leaf nodes contain multiple instances.
3. Contextual relations between parts:
kinematics
symmetry.
Multiple-Instance Compositional
Template (MICT)
Problem Formulation
Given the template, the problem is
formulated as
Selecting an instance for each part.
Finding the matched part in target.
…
Matching-based Formulation
Problem Formulation
(a) Query Person (b) Test Scene
11 12 13
1112
13
2221 23 2421 22 23
24
31 32 34
33
3132
41 4244
43
41
42
(24,21)
(24,22)
(24,23)
(24,24)
(12,11)
(11,12)
(13,13)
(31,31)
(31,32)
(32,32)
(31,33)
(32,33)
(32,34) (41,41)
(41,42)
Candidacy Graph:
Vertices – possible matching pairs
Solving the problem:
Labeling vertices in the graph (selecting matching pairs)
NP hard – incorporating graph edges
Problem Formulation
(a) Query Person (b) Test Scene
11 12 13
1112
13
2221 23 2421 22 23
24
31 32 34
33
3132
41 4244
43
41
42
(31,31)
(31,32)
(32,32)
(31,33)
(32,33)
(32,34)
(12,11)
(11,12)
(13,13)
(41,41)
(41,42)
(24,21)
(24,22)
(24,23)
(24,24)
Compatible Edges:
Encouraging matching pairs to activate together in matching
Defined by contextual constraints
Problem Formulation
(31,31)
(31,32)
(32,32)
(31,33)
(32,33)
(32,34)
(12,11)
(11,12)
(13,13)
(41,41)
(41,42)
(24,21)
(24,22)
(24,23)
(24,24)
(a) Query Person (b) Test Scene
11 12 13
1112
13
2221 23 2421 22 23
24
31 32 34
33
3132
41 4244
43
41
42
Problem Formulation
Competitive Edges:
Depressing conflicting matching pairs being selected at the same time
Defined by matching constraints (31,31)
(31,32)
(32,32)
(31,33)
(32,33)
(32,34)
(12,11)
(11,12)
(13,13)
(41,41)
(41,42)
(24,21)
(24,22)
(24,23)
(24,24)
(a) Query Person (b) Test Scene
11 12 13
1112
13
2221 23 2421 22 23
24
31 32 34
33
3132
41 4244
43
41
42
Inference
Using Cluster Sampling [1] for inference:1. Sampling edges in candidacy graph to generate clusters.2. Randomly selecting/deselecting the clusters.3. Decide whether to accept the new state.
[1] J. Porway et al., “C4: Exploring multiple solutions in graphical models by cluster sampling”, TPAMI 2011.
(31,31)
(31,32)
(32,32)
(31,33)
(32,33)
(32,34)
(12,11)
(11,12)
(13,13)
(41,41)
(41,42)
(24,21)
(24,22)
(24,23)
(24,24)
State A Clusters
Cluster1
Cluster2
Re-identification
(31,31)
(31,32)
(32,32)
(31,33)
(32,33)
(32,34)
(12,11)
(11,12)
(13,13)
(41,41)
(41,42)
(24,21)
(24,22)
(24,23)
(24,24)
State B Clusters
Cluster2
Cluster1
Re-identification
Dataset
VIPeR Dataset:
1. Classic ReID dataset
2. Well-segmented people, limited pose/view3. Heavy illumination changes, lack occlusion
D. Gray et al., "Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features”, ECCV 2008.
Dataset
EPFL Dataset:
1. Cross-camera tracking dataset
2. Few people, shot scene provided, various pose/view
3. Little illumination changes, limited occlusions
F. Fleuret et al., "Multiple Object Tracking using K-Shortest Paths Optimization”, TPAMI 2011.
Query Instance Video Shot Target Individual
Dataset
CAMPUS-Human Dataset:
1. Camera and annotate by us
2. Many people, shot scene provided, various pose/view
3. Limited illumination changes, heavy occlusions
Query Instance Video Shot Target People
Result
Setting 1:
Re-identify people in segmented images, i.e. targets already localized.
Result
Setting 2:
Re-identify people from scene shots without provided segmentations.
Result
Evaluating feature and constraints effectiveness
Component Analysis
Conclusion
1. A solution for a new surveillance problem.
2. A person-based model, a graph-matching-based formulation, a more complete database for evaluation.
3. Exploring robust and flexible person models [1], efficient search method [2] in future.
[1] J. B. Rothrock et al., “Integrating Grammar and Segmentation for Human Pose Estimation”, CVPR 2013.[2] J. Uijlings et al., “Selective Search for Object Recognition”, IJCV 2013.
Published Papers
1. Yuanlu Xu, Liang Lin, Wei-Shi Zheng, Xiaobai Liu. “Human Re-identification by Matching Compositional Template with Cluster Sampling”. ICCV 2013.
2. Liang Lin, Yuanlu Xu, Xiaodan Liang, Jian-Huang Lai. “Complex Background Subtraction by Pursuing Dynamic Spatio-temporal Manifolds”. IEEE TIP 2014, under revision.
3. Yuanlu Xu, Bingpeng Ma, Rui Huang, Liang Lin. “Person Search in a Scene by Jointly Modeling People Commonness and Person Uniqueness”. ACMMM 2014, submitted.
QUESTIONS?
1. Given a candidacy graph and the current matching state , we first separate graph edges into two sets: set of inconsistent edges
and set of consistent edges in the other two cases.2. Next we introduce a boolean variable to indicate an edge is being turned on or turned off. We turn off inconsistent edges deterministically and turn on every consistent edge with its edge probability .
Cluster Sampling
Generating a composite cluster
3. Afterwards, we regard candidates connected by ”on” positive edges as a cluster and collect clusters connected by ”on” negative edges to generate a composite cluster .
Cluster Sampling
Generating a composite cluster
Composite Cluster Sampling
state transition probability ratio posterior ratio
Using Metropolis-Hastings method to achieve a reversible transition between twostates and , the acceptance rate of the transition is defined as
Composite Cluster Sampling
The state transition probability ratio is computed by
edges being turned off around ,
Composite Cluster Sampling
Inference Algorithm