LAPI @ 2017 Retrieving Diverse Social Images Task:A Pseudo-Relevance Feedback Diversification Perspective
Bogdan Boteanu, Mihai Gabriel Constantin, Bogdan IonescuLAPI - University ”Politehnica” of Bucharest, 061071, RomaniaEmail: {bboteanu,mgconstantin,bionescu}@alpha.imag.pub.ro
UniversityPOLITEHNICAof Bucharest
§ HC pseudo-relevance feedback (HC-RF)
1. selection of positive and negative examples
2. hierarchical clustering & pruning scheme withfeedback determined automatically from initial data
3. diversification is achieved by traversing HC image clusters with respect to the Flickr initial ranking
Proposed approach (1)
MediaEval 2017, Dublin, Ireland 1/10
1. Selection of positive and negative examples
Proposed approach (2)
I 1 I 2 I N
Image Database(Flickr’s rank)
I 3 I N-1 Np+Nn <= N…Nn
un-relevantNp
relevant
MediaEval 2017, Dublin, Ireland 2/10
2. HC clustering and pruning
Proposed approach (3)
Hierarchical Clustering
cut point
I 1
I 2
Class 1I N
I N-1
Class k(un-relevant)
I 3…
MediaEval 2017, Dublin, Ireland 3/10
3. Diversification
Proposed approach (4)
I 1 I 3 … I 4
Class 1 Class 2 Class n
I 9
I 8
I 2
I 7
I 5
I 15
... ... ...1
2
4
9
3
158 7
5Output
MediaEval 2017, Dublin, Ireland 4/10
Parameter tuning
§ positive examples (Np): 100 – 280 with a step of 20
§ negative examples (Nn): 0 – 20 with a step of 10
§ inconsistency coefficient (Nc - no. of classes): 0.5 – 1.3 with a step of 0.2
ü Best combination of Np-Nn-Nc (highest F1@20)
MediaEval 2017, Dublin, Ireland 5/10
Results - devsetRuns P@20 CR@20 F1@20
1 . all visual 0.575 0.3969 0.4473
2. all text 0.575 0.3969 0.4473
3. all vis - all text 0.6136 0.4234 0.4773
4. CNN 0.575 0.3969 0.4473
5. cred. 0.575 0.3969 0.4473
Flickr init. res. 0.5864 0.3646 0.42277
MediaEval 2017, Dublin, Ireland 6/10
Results - testsetRuns P@20 CR@20 F1@20
1 . all visual 0.6333 0.5791 0.5753
2. all text 0.6214 0.5794 0.5733
3. all vis - all text 0.6196 0.5729 0.5741
4. CNN 0.5845 0.5216 0.5253
5. cred. 0.6018 0.6045 0.5777
MediaEval 2017, Dublin, Ireland 7/10
Results - Visual Example (Flickr Initial)
MediaEval 2017, Dublin, Ireland 8/10
Easter Eggs
P@20=0.75 CR@20=0.4 F1@20=0.52
X X XX X
MediaEval 2017, Dublin, Ireland 9/10
Results - Visual Example (best run)Easter Eggs
P@20=0.9 CR@20=0.53 F1@20=0.67 X X
Conclusions
• credibility information was useful in the context of overalldiversification (Run5 - CR@20 = 0.6045), with more than 2% overother types of descriptors
• in terms of F1 metric score, the use of credibility information, (Run5- F1@20 = 0.5777), allows for better performance over visual andtextual descriptors by more than 3% and by more than 5% overCNN descriptors
MediaEval 2017, Dublin, Ireland 10/10