Upload
dolan
View
65
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work. Transitive Re-identification. Technion - I.I.T. Haifa, Israel. This research was supported by the MAGNET program in the Israeli ministry of industry - PowerPoint PPT Presentation
Citation preview
Transitive Re-identification
Technion - I.I.T. Haifa, Israel
BMVC 2013
This research was supported by the MAGNET program in the Israeli ministry of industryand commerce, by the Israeli ministry of science and by the E. and J. Bishop research fund.
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
recognizing individuals over different camera views(we focus on ReID based on appearance based cues)
β’ upper image from: A. Bialkowski, S. Denman, P. Lucey, S. Sridharan, and C. B. Fookes. βA database for person re-identification in multi-camera surveillance networks.β (DICTA 2012)
β’ lower images: courtesy of Marco Cristani
Camera A, time t Camera B, time t+βt
ReIDentification (ReID)
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
ReIDentification
β’ require a training set
β’ classification is with a classifier or a metric learned on the training set
β’ do not depend on any training set
β’ classification is based on pre-defined distance metric
Appearance-based ReIDentification methods:
perform better for βcamera-specificβ scenarios:ReID benefits from training with corresponding appearance-pairs captured by specific camerapairs, as the background, illumination, resolution and pose are camera dependent.
Learning based methods: Direct methods:
β’ always camera-invariantβ’ camera-invariant or camera-specific, depending on the used training set
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
ReIDentificationLearning based methods perform better for βcamera-specificβ scenarios:
1 2 3 4 5 6 7 8 9 10 11 1210
20
30
40
50
60
70
80
90
100
rank score
Rec
ogni
tion
%
ICT trained on 3-7 dataSDALF for cameras 3-7
ICT β learning based
SDALF - direct
β’ ICT (Implicit Camera Transfer) by Avraham et al (ECCV Re-Id 2012)β’ SDALF (Symmetry-Driven Accumulation of Local Features) by Farenzena et al
(CVPR 2010) and Bazzani et al (CVIU 2013)β’ performance tested on 2 cameras from the SAIVT-SoftBio dataset
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
ReIDentificationLearning method: ICT Direct method: SDALF
same
not same
Camera A
Camera B
F
FC
Binary SVM with RBF kernel, usingDecision values
Image A
Image B
(a) localizing meaningful body parts
(a) (b) (c) (d)
* Image from: L. Bazzani, M. Cristani, and V. Murino. Symmetry-driven accumulation of local features for human characterization and re-identification.
(b) weighted HSV histogram(c) MSCR - maximally stable color regions(d) RHSP - recurrent highly structured patches=> matching SDALF descriptors
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
Motivation
Given a site with cameras , we may ReID either by:
πͺπ
πͺπ
πͺπ
πͺπ
πͺπ΅
πͺπ
πͺπ
πͺπ
πͺπ
πͺπ
πͺπ΅
πͺπ
1. applying a direct method
2. applying a learning based method where distinct inter-camera training sets must
be collected and annotated for each camera-pair. This is impractical.
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
Motivation
We aim at reducing the number of required direct inter-camera training sets
from to by suggesting a transitive algorithm which uses inter-camera training sets
only for camera-pairs , and infers a ReID classifier for any other camera-pair in the
system.
directly trainable pairs non-directly trainable pairs
By recursively applying the transitive algorithm:
By applying the transitive algorithm:
πͺπ
πͺπ
πͺπ
πͺπ
πͺπ΅
πͺπ
πͺπ
πͺπ
πͺπ
πͺπ
πͺπ΅
πͺπ
πͺπ
πͺπ
πͺπ
πͺπ
πͺπ΅
πͺπ
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
πͺπ΅+π
πͺπ
πͺπ
πͺπ
πͺπ
πͺπ΅
πͺπ
When a new camera is added to a set of N previously installed cameras,
with the proposed transitive approach, collecting data for only one pair is
required.
Motivation
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
The Transitive Algorithmβ’ We focus on a camera triplet case .
πͺπ
πͺπ
πͺπ
AB
C
πͺπ
πͺπ
πͺπ
πͺπ
πͺπ΅
πͺπ
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
The Transitive Algorithm AB
C
β’ Given two training sets associated with camera-pairs and ,
respectively, we would like to infer a classifier for the camera-pair, for
which is missing.
Training
A B A C
= ?
Test
= ?
= ?
B C
πΊπ¨π©
πΊπ©πͺ
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
The Transitive AlgorithmExpanding the learning based method β the Naive solution:training the classifier with
1 2 3 4 5 6 7 8 9 10 11 1210
20
30
40
50
60
70
80
90
100
rank score
Rec
ogni
tion
%
ICT trained on 3-7 dataSDALF for cameras 3-7ICT trained on 3-5 and 5-7 data
ICT
SDALF
Naive ICT
Our goal is to find a more effective use of the available training sets that will decrease this performance gap.
AB
C
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
The Transitive Algorithm
π· (π π¨πͺ|π π¨ ,ππͺ )=?
π (π π¨πͺ|π π¨ ,ππͺ )= βπ π¨π©β{π ,π }
βππ©πͺβ{π ,π }
β«ππ©βπΉπ
π· (π π¨πͺ ,π π¨π©=π π¨π© ,π π©πͺ=π π©πͺ , ππ©|π π¨ , ππͺ ΒΏβ ππ©ΒΏ
π (π π¨πͺ|π π¨ ,ππͺ )= βπ π¨π©β{π ,π }
βππ©πͺβ{π ,π } [ β«
ππ©βπΉπ
π· (π π¨πͺ ,π π¨π©=π π¨π© ,π π©πͺ=π π©πͺ|π π¨ ,ππ© ,π πͺΒΏ π ππ©(ππ©)β ππ©]
The Transitive ReID algorithm (TRID) establishes a path between the non-directly trainable camera pair by marginalization over a βconnecting elementβ, the domain of possible appearances in camera .
AB
C
πΊπ¨π©
πΊπ©πͺ
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
The Transitive Algorithm
π· (π π¨πͺ|π π¨ ,ππͺ )=?
π· (π π¨πͺ|π π¨ ,ππͺ )=β«ππ©
π· (π π¨π©|ππ¨ , ππ©)π· (π π©πͺ|π π© , ππͺ ) π πΏπ© (π π©)β ππ©
πββ«ππ©
π· (π π¨π©|π π¨ , ππ© )π· (π π©πͺ|ππ© , ππͺ ) π πΏ π© (ππ© )β π π©
AB
C
.
.
.
.
The Transitive ReID algorithm (TRID) establishes a path between the non-directly trainable camera pair by marginalization over a βconnecting elementβ, the domain of possible appearances in camera .
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
The Transitive Algorithm
π· (π π¨πͺ|π π¨ ,ππͺ )=β«ππ©
π· (π π¨π©|ππ¨ , ππ©)π· (π π©πͺ|π π© , ππͺ ) π πΏπ© (π π© )β ππ©
πββ«ππ©
π· (π π¨π©|π π¨ , ππ© )π· (π π©πͺ|ππ© , ππͺ ) π πΏ π© (ππ© )β π π©
β’ - probability for a match in camera-pair (A, B).
β’ Any classifier can be used here provided that it can be modified to output probability.
β’ TRID uses the ICT algorithm by training two ICT ReID classifiers. The SVM decision values are converted to probability estimates using a sigmoid according to Plattβs method.
AB
CICT [A B] ICT [B C]
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
The Transitive Algorithm
π· (π π¨πͺ|π π¨ ,ππͺ )=β«ππ©
π· (π π¨π©|ππ¨ , ππ©)π· (π π©πͺ|π π© , ππͺ ) π πΏπ© (π π© )β ππ©
πββ«ππ©
π· (π π¨π©|π π¨ , ππ© )π· (π π©πͺ|ππ© , ππͺ ) π πΏ π© (ππ© )β π π©
π· (π π¨πͺ|π π¨ ,ππͺ )β
π|πΊπ©|
βππ©βπΊπ©
π· (π π¨π©|π π¨ ,π π© )π· (π π©πͺ|ππ© ,ππͺ )
πβ π|πΊπ©|
βππ©βπΊπ©
π· (π π¨π©|π π¨ , ππ© ) π· (π π©πͺ|ππ© , ππͺ )
β’ is a multi-dimensional probability density function.
β’ Can be estimated by methods for density estimation using all .
β’ However, estimating high-dimensional density is hard and the integration is
computationally costly.
β’ Thus, in TRID the integral is approximated by a sum relying on the
smoothness of the probability functions.
AB
C
-40 -20 0 20 40
-20
0
20
-50 0 50
-50
0
50
0 20 40 60 80 1000
50
100
rank score
Rec
ogni
tion
%
ICT trained on AC dataNaive ICTTRID
-50 0 50
-50
0
50
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
Synthetic Experiment 1
-40 -20 0 20 40
-20
0
20
-50 0 50
-50
0
50
0 20 40 60 80 1000
50
100
rank score
Rec
ogni
tion
%
ICT trained on AC dataNaive ICTTRID
-50 0 50
-50
0
50π π¨Γπ π© ππ©Γ ππͺ π π¨Γππͺ
π΅ππππ π°πͺπ» π»πΉπ°π«π°πͺπ» πππππππ ππ π¨πͺπ πππ
AB
C-40 -20 0 20 40
-20
0
20
-50 0 50
-50
0
50
0 20 40 60 80 1000
50
100
rank score
Rec
ogni
tion
%
ICT trained on AC dataNaive ICTTRID
-50 0 50
-50
0
50
-40 -20 0 20 40
-20
0
20
-50 0 50
-50
0
50
0 20 40 60 80 1000
50
100
rank score
Rec
ogni
tion
%
ICT trained on AC dataNaive ICTTRID
-50 0 50
-50
0
50
π· (π π¨πͺ|π π¨ ,ππͺ ):
π π¨Γππͺ π π¨Γππͺ π π¨Γππͺ
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
-20 0 20
-20
0
20
-50 0 50-50
0
50
-50 0 50-50
0
50
0 10 20 30 40 50 60 70 80 90 1000
50
100
Rec
ogni
tion
%
(d)
rank score
ICT trained on AC dataNaive ICTTRID
Synthetic Experiment 2π π¨Γπ π© ππ©Γ ππͺ π π¨Γππͺ
AB
C
π· (π π¨πͺ|π π¨ ,ππͺ ):
π π¨Γππͺ π π¨Γππͺ π π¨Γππͺ
-20 0 20
-20
0
20
-50 0 50-50
0
50
-50 0 50-50
0
50
0 10 20 30 40 50 60 70 80 90 1000
50
100
Rec
ogni
tion
%
(d)
rank score
ICT trained on AC dataNaive ICTTRID
-20 0 20
-20
0
20
-50 0 50-50
0
50
-50 0 50-50
0
50
0 10 20 30 40 50 60 70 80 90 1000
50
100
Rec
ogni
tion
%
(d)
rank score
ICT trained on AC dataNaive ICTTRID
π΅ππππ π°πͺπ» π»πΉπ°π«π°πͺπ» πππππππ ππ π¨πͺπ πππ
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
SAIVT-SoftBio Experimentβ’ Testing TRID requires an annotated dataset associated with at least 3 stationary
cameras. β’ Common ReID benchmark datasets (VIPeR , CAVIAR4REID, iLIDs MCTS, ETHZ) are
unsuitable for our set-up.β’ Multi-camera surveillance database named SAIVT-SoftBio presented by Bialkowski et al:
β’ image from: A. Bialkowski, S. Denman, P. Lucey, S. Sridharan, and C. B. Fookes. βA database for person re-identification in multi-camera surveillance networks.β (DICTA 2012)
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
SAIVT-SoftBio Experiment
0 2 4 6 8 10 12 14 16 18 2010
20
30
40
50
60
70
80
90
100
rank score
Rec
ogni
tion
%
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] =
0 5 10 1510
20
30
40
50
60
70
80
90
100
rank score
Reco
gniti
on %
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] =
0 2 4 6 8 10 12 1410
20
30
40
50
60
70
80
90
100
rank score
Reco
gniti
on %
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] =
0 2 4 6 8 10 1210
20
30
40
50
60
70
80
90
100
rank score
Reco
gniti
on %
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] = C5 C7C3 C5 C7C1
C1 C3 C8C5C1 C3
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
SAIVT-SoftBio Experiment
0 2 4 6 8 10 1210
20
30
40
50
60
70
80
90
100
rank score
Reco
gniti
on %
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] = C5 C7C3
ABC
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
SAIVT-SoftBio Experiment
0 2 4 6 8 10 12 14 16 18 2010
20
30
40
50
60
70
80
90
100
rank score
Rec
ogni
tion
%
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] =
0 5 10 1510
20
30
40
50
60
70
80
90
100
rank score
Reco
gniti
on %
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] =
0 2 4 6 8 10 12 1410
20
30
40
50
60
70
80
90
100
rank score
Reco
gniti
on %
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] =
0 2 4 6 8 10 1210
20
30
40
50
60
70
80
90
100
rank score
Reco
gniti
on %
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] = C5 C7C3 C5 C7C1
C1 C3 C8C5C1 C3
SAIVT-SoftBio ExperimentIntroduction Motivation The Transitive Alg. Experiments Summary & Future Work
0 5 10 1510
20
30
40
50
60
70
80
90
100
rank score
Reco
gniti
on %
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] = C1 C3 C8
A BC
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
Summaryβ’ When performing ReID in multi-camera site, we do not have to choose
between the less accurate direct method and the computationally expensive learning based method. We may perform limited learning, such that requires reasonable resources. We have shown that transitively using such learning performs well for the ReID task.
β’ A specific algorithm based on marginalization was suggested: the TRID - which presents a new approach of transitivity in ReID.
β’ The accuracy of the TRID is superior to that of a state-of-the-art non learning based approach.
β’ The TRID algorithm is in fact a general framework that may be combined with different probabilistic classifiers (not necessarily ICT) and of course different features.
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
Future Work
πͺπ
πͺπ
πͺπ
πͺπ
β’ Improve the approximation of the transitive integral.
β’ Transitively use indirect training sets to strengthen direct learning when the set of direct annotated pairs is small, but not empty.
β’ Test the effect of recursively applying the TRID algorithm, and to study the deterioration of the performance as a function of the number of cameras.
β’ Generalize the TRID algorithm beyond the scope of ReID to other domain adaptation tasks.
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
The Matlab source code of TRID as well as of the ICT are available at:http://www.cs.technion.ac.il/~tammya/Reidentification.html .
Thank-You.