Embed Size (px)
DESCRIPTION
H euristic Pre-Clustering Relevance Feedback in Attention -Based Image Retrieval. Wan-Ting Su , Wen-Sheng Chu and Jenn-Jier James Lien Speaker: Wen-Sheng Chu Robotics Lab. CSIE NCKU. Query Image. Positive Feedback. Negative Feedback. Heuristic Pre-Clustering View. - PowerPoint PPT Presentation
Citation preview
Heuristic Pre-Clustering Relevance Feedback
in Attention-Based Image RetrievalWan-Ting Su, Wen-Sheng Chu
and Jenn-Jier James Lien
Speaker: Wen-Sheng Chu
Robotics Lab. CSIE NCKU
Robotics Lab, CSIE NCKU
System InterfaceSystem Interface
Result View
Heuristic Pre-Clustering View
User can revise the clustering results manually
User can change the positive group number on his/her own
Query Image
Positive Feedback
Negative Feedback
Robotics Lab, CSIE NCKU
System Overview
Wavelet Transformation
Low-LowSubband
Attended View Extraction
Image Database
Ranking by Euclidean Distance
User Feedback
?
Query Image
END
No
Yes
Best Matches
PCA
VQUser
Re-clustering
Ranking by GBDA Learning
Offline Module : Attention-Based Image Retrieval
Online Module : Heuristic Pre-Clustering Relevance Feedback
Feature Extraction from Attended View
HeuristicPre-clustering
Robotics Lab, CSIE NCKU
Wavelet and Attended View Extraction• To reduce the computational cost
• Contrast extraction is applied to the wavelet coefficient in the LL-subband.
) ,( ,, qpdCq
jiji
contrast value of pixel p at image location (i, j)
Gaussian distance
neighborhood of pixel (i, j)
1
0
1
0,0
1
0
1
0,0
1
1
M
i
N
jji
M
N
j
M
iji
M
jCC
y
iCC
x
1
0
1
0,
M
i
N
jjiC
attention center
Got saliency
map!
Robotics Lab, CSIE NCKU
System Overview
Wavelet Transformation
Low-LowSubband
Attended View Extraction
Image Database
Ranking by Euclidean Distance
User Feedback
?
Query Image
END
No
Yes
Best Matches
PCA
VQUser
Re-clustering
Ranking by GBDA Learning
Offline Module : Attention-Based Image Retrieval
Online Module : Heuristic Pre-Clustering Relevance Feedback
Feature Extraction from Attended View
HeuristicPre-clustering
Robotics Lab, CSIE NCKU
Visual Features Extraction• Table1. 32 low-level visual features
Features Dimension
Color mean, standard deviation and skew in HSV space 9
Standard deviation of the wavelet coefficients in 10 pyramid de-correlated
sub-bands10
13 statistical elements extracted from the edge map such as max fill time, max fork
count, etc.13
Robotics Lab, CSIE NCKU
System Overview
Wavelet Transformation
Low-LowSubband
Attended View Extraction
Image Database
Ranking by Euclidean Distance
User Feedback
?
Query Image
END
No
Yes
Best Matches
UserRe-clustering
Ranking by GBDA Learning
Offline Module : Attention-Based Image Retrieval
Online Module : Heuristic Pre-Clustering Relevance Feedback
Feature Extraction from Attended View
HeuristicPre-clustering
Got features!
PCA
VQ
Robotics Lab, CSIE NCKU
Pre-Clustering• Principal Component Analysis (PCA)
+• Vector Quantization algorithm (VQ)
Robotics Lab, CSIE NCKU
User Re-clustering
User Re-clustering
System Pre-clustering Result User Re-clustering Result
Robotics Lab, CSIE NCKU
System Overview
Wavelet Transformation
Low-LowSubband
Attended View Extraction
Image Database
Ranking by Euclidean Distance
User Feedback
?
Query Image
END
No
Yes
Best Matches
UserRe-clustering
Ranking by GBDA Learning
Offline Module : Attention-Based Image Retrieval
Online Module : Heuristic Pre-Clustering Relevance Feedback
Feature Extraction from Attended View
HeuristicPre-clustering
PCA
VQ
Robotics Lab, CSIE NCKU
Re-weighting Scheme
• Group-Based Discriminant Analysis (GBDA)• Multiple positive and multiple negative classes• Clustering each positive class• Scattering the negative example away from each
positive class
Single Flower
Bouquets of Flowers
Negative Samples
Positive Samples
Robotics Lab, CSIE NCKU
GBDA
WSWWSWW
WTPN
T
Wmaxarg
c
iNiPN SS
1
Sw : the sum of the within-class scatter matrix of the positive groups
SPN is the sum of between-class scatter matrices of positive-to-negative
iCxT
iii )m)(xm(xSmi : the mean of the ith positive class Ci
c
iiw SS
1 c: the number of positive groups
DyT
iiNi )m)(ym(yS
D : a set of negative examples
Robotics Lab, CSIE NCKU
Experiment Result (1)
• COREL image database• QS2: 1000 images from 10 selected categories• Each of 10 categories contains 100 images and is
used as queries.
1. Sunset 2. Flower 3. Car 4. Ape 5. Mountain
6. Penguin 7. Tiger 8. Bird 9. Horse 10. Building
Table 1. Image Categories in Query Set 2
N
N returns in top retrieved imagesrelevant precision
Robotics Lab, CSIE NCKU
Experiment Result (2)
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
50.00%
55.00%
60.00%
10 20 30 40 50 60 70 80 90 100
Scope
Pre
cisi
on
Attention-Based System Global
Robotics Lab, CSIE NCKU
Experiment Result (3)
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
1 2 3 4 5 6 7 8 9 10
Category ID
Pre
cisi
on
Attention-Based System Global
Robotics Lab, CSIE NCKU
Experimental Results (4)
Precision = 5/10Precision = 7/20
Query Image
First-time retrieval results
Robotics Lab, CSIE NCKU
Experimental Results (5)
Precision = 8/10Precision = 17/20
First-time feedbackresults
Robotics Lab, CSIE NCKU
Experimental Results (6)
Precision = 10/10Precision = 20/20
Second-time feedback results
Robotics Lab, CSIE NCKU
Conclusion• The major work in this study is integrating
attention-based image retrieval with the relevance feedback algorithm using multiple positive and negative groups.
• The system guides the user in clustering positive feedbacks by providing heuristic pre-clustering results. Then the user can revise the clusters manually.
Robotics Lab, CSIE NCKU
Experiment Result - Video Demo
![Application of the Filter approach and the Clustering …Max-relevance, Min-Redundancy (MRMR) [7] III. Materials and Methods To prove the importance of the k-means clustering step,](https://img.dokumen.tips/doc/110x75/5f856a5e1d9ebd47655f5dfd/application-of-the-filter-approach-and-the-clustering-max-relevance-min-redundancy.jpg)
