Embed Size (px)
Citation preview
Turning a Blind Eye:Explicit Removal of Biases and
Variation from Deep Neural Network Embeddings
Mohsan Alvi, Andrew Zisserman, and Christoffer NellåkerBias Estimation in Face Analytics
ECCV 2018 Workshop, 14/09/2018
Introduction
Convolutional Neural Networks are the state-of-the-art in image classification
Rely on “good” training data
Not clear what the network has
learned
Networks can cheat by learning
spurious variations
Lack of comprehensively annotated data
Contents
• Removing a bias from a network• Removing multiple spurious variations from a network• LAOFIW – Labeled Ancestral Faces in the Wild
IMDB Faces Dataset [1]
• Dataset consists of celebrity faces from International Movie DataBase• Contains Age, Gender, Identity Labels
• Created two subsets of the dataset• Age• Gender
• Cleaned labels
[1] Rothe, Timofte, Van Gool. (2015)
Training on Extremely Biased DataEvaluated on a gender/age balanced test dataset
Gender classification accuracy: 70%
Turning a Blind Eye
• Primary task is the attribute of interest• Gender Classification
• Secondary task denotes bias to be unlearned• Age Classification
• Objective:learn feature representation that is informative for primary task, and uninformative for secondary task• Repurpose work in field of domain adaptation [2]
[2] Tzeng, Hoffman, Darrell, and Saenko. (2015)
Methods (1/3)
Based on VGG-M Network [2]
[2] Chatfield, Simonyan, Vedaldi, and Zisserman. (2014)
Minimize:
Methods (3/3)
Cross-entropy between classifier output and uniform distribution
Minimize:
Confusion Loss
Secondary Loss
Secondary Classification Secondary
Confusion
Act in opposition to each other
Alternate:
Primary Classification
&Secondary Confusion
Secondary Classification
Primary Classification
Secondary Confusion
Results – Removing a bias
Baseline Gender classification accuracy: 70%Age-Blind Gender classification accuracy: 86%
Removing multiple spurious variations (1/2)
Problem 1: Multiple biases may be present in dataset
AgeGender
Pose
Expression
Ancestry
Removing multiple spurious variations (2/2)
• Problem 2: no single dataset contains labels for all biases
• Each labeled for a single purpose• Leverage information from multiple datasets
Dataset 3:Ancestry
Dataset 1:Gender
Dataset 4:Pose
Dataset 2:Age
14,000 images in four classes:• Sub-Saharan Africa• Western Europe• East Asia• Indian subcontinent
Labeled Ancestral Origin Faces in the
Wild
LAOFIW
Removing multiple spurious variations experiments
AncestryGender* PoseAge
GenderAncestry PoseAge
Experiment(1)
Experiment(2)
* Not extremely biased
Results - Removing multiple spurious variations (1/2)
Primary task: Gender Secondary tasks: Age, Ancestry, Pose
Results - Removing multiple spurious variations (2/2)
Primary task: Ancestry Secondary tasks: Age, Gender, Pose
Conclusions
• Can improve generalizability of models train on biased datasets• Can remove multiple spurious variations from feature representation
of network• LAOFIW – ancestral origin dataset
