Commissariat à l’énergie atomique et aux énergies alternatives Institut List | CEA SACLAY NANO-INNOV | BAT. 861 – PC142 91191 Gif-sur-Yvette Cedex | FRANCE www-list.cea.fr CONTACT | [email protected] FROM CLASSICAL TO GENERALIZED ZERO-SHOT LEARNING Yannick LE CACHEUX 1 , Hervé LE BORGNE 1 and Michel CRUCIANU 2 1 : CEA LIST / 2 : CNAM Harmonic mean of accuracies on seen and unseen classes for state-of-the art models on two standard ZSL datasets, Caltech-UCSD Birds and Animals with Attributes2. Our method enables a significant increase. Generalized Zero-Shot Learning Zero-shot learning (ZSL) aims to recognize samples from unseen classes, i.e. classes for which no training example is available. This is achieved thanks to the use of additional semantic knowledge, for example vectors of attributes. In Generalized Zero-Shot Learning (GZSL), test samples can be from either a seen or an unseen class. In practice, a problem that frequently arises is that samples from unseen classes are mistaken for samples from seen classes. Experimental results A new process to adapt any existing ZSL method to the GZSL setting is proposed. This process can be divided into two steps and relies on a training / validation / testing split specific to the GZSL task. - 1 st step: an explicit penalization of similarity scores of seen classes called calibration is introduced. Its optimal value is determined by cross-validation. - 2 nd step: the value of the regularization parameter λ , used to control the model’s complexity and prevent overfitting, is chosen specifically for the GZSL task – as opposed to reused from the ZSL task. These two ideas can be substantiated by an analysis of the bias-variance trade-off in a GZSL context. Proposed approach References ● Y. Le Cacheux, H. Le Borgne and M. Crucianu. From classical to generalized zero- shot learning: a simple adaptation process . Submitted to BMVC 2018. ● Y. Xian, B. Schiele and Z. Akata. Zero-shot learning - the good, the bad and the ugly . CVPR 2017. ● W. L. Chao, S. Changpinyo, B. Gong and F. Sha. An empirical study and analysis of generalized zero-shot learning for object recognition in the wild . ECCV 2016. ● C. H. Lampert, H. Nickisch and S. Harmeling. Attribute-based classification for zero- shot visual object categorization . PAMI 2014. Stripes: ✔ Orange: ✘ Hooves: ✔ Zebra Visual training samples Seen classes’ attributes Predicted attributes Training phase Visual testing samples Zero-Shot Learning Stripes: ✘ Orange: ✔ Hooves: ✘ Fox Testing phase Unseen classes’ attributes Stripes: 0.8 Orange: 0.9 Hooves: 0.1 Stripes: ✔ Orange: ✔ Hooves: ✘ Tiger Stripes: ✘ Orange: ✘ Hooves: ✔ Horse Stripes: 0.0 Orange: 0.3 Hooves: 0.8 Stripes: 0.6 Orange: 0.8 Hooves: 0.0 Class predictions Learned relations The tiger is the largest cat species, most recognizable for its pattern of dark vertical stripes on reddish-orange fur with a lighter underside. [wikipedia]

FROM CLASSICAL TO GENERALIZED ZERO-SHOT LEARNING2018.ds3-datascience-polytechnique.fr/wp-content/... · Institut List | CEA SACLAY NANO-INNOV | BAT. 861 – PC142 91191 Gif-sur-Yvette

Download PDF Report

Upload
others
View
1
Download
0

Embed Size (px)

Citation preview

Page 1: FROM CLASSICAL TO GENERALIZED ZERO-SHOT LEARNING2018.ds3-datascience-polytechnique.fr/wp-content/... · Institut List | CEA SACLAY NANO-INNOV | BAT. 861 – PC142 91191 Gif-sur-Yvette

Commissariat à l’énergie atomique et aux énergies alternativesInstitut List | CEA SACLAY NANO-INNOV | BAT. 861 – PC14291191 Gif-sur-Yvette Cedex | FRANCEwww-list.cea.fr

CONTACT | [email protected]

FROM CLASSICAL TO GENERALIZEDZERO-SHOT LEARNING

Yannick LE CACHEUX1, Hervé LE BORGNE1 and Michel CRUCIANU2

1 : CEA LIST / 2 : CNAM

Harmonic mean of accuracies on seen and unseen classes for state-of-the art models on two standard ZSL datasets, Caltech-UCSD Birds and Animals with Attributes2. Our method enables a significant increase.

Generalized Zero-Shot LearningZero-shot learning (ZSL) aims to recognize samples from unseen classes, i.e. classes for which no training example is available. This is achieved thanks to the use of additional semantic knowledge, for example vectors of attributes.

In Generalized Zero-Shot Learning (GZSL), test samples can be from either a seen or an unseen class. In practice, a problem that frequently arises is that samples from unseen classes are mistaken for samples from seen classes.

Experimental results

A new process to adapt any existing ZSL method to the GZSL setting is proposed. This process can be divided into two steps and relies on a training / validation / testing split specific to the GZSL task.

- 1st step: an explicit penalization of similarity scores of seen classes called calibration is introduced. Its optimal value is determined by cross-validation.

- 2nd step: the value of the regularization parameter λ, used to control the model’s complexity and prevent overfitting, is chosen specifically for the GZSL task – as opposed to reused from the ZSL task.

These two ideas can be substantiated by an analysis of the bias-variance trade-off in a GZSL context.

Proposed approach

References● Y. Le Cacheux, H. Le Borgne and M. Crucianu. From classical to generalized zero-

shot learning: a simple adaptation process. Submitted to BMVC 2018.● Y. Xian, B. Schiele and Z. Akata. Zero-shot learning - the good, the bad and the ugly.

CVPR 2017.

● W. L. Chao, S. Changpinyo, B. Gong and F. Sha. An empirical study and analysis of generalized zero-shot learning for object recognition in the wild. ECCV 2016.

● C. H. Lampert, H. Nickisch and S. Harmeling. Attribute-based classification for zero-shot visual object categorization. PAMI 2014.

Stripes: ✔Orange: ✘Hooves: ✔

Zebra

Visual training samples Seen classes’ attributes Predicted attributes

Training phaseVisual testing samples

Zero-Shot Learning

Stripes: ✘Orange: ✔Hooves: ✘

Fox

Testing phaseUnseen classes’ attributes

Stripes: 0.8Orange: 0.9Hooves: 0.1

Stripes: ✔Orange: ✔Hooves: ✘

Tiger

Stripes: ✘Orange: ✘Hooves: ✔

Horse

Stripes: 0.0Orange: 0.3Hooves: 0.8

Stripes: 0.6Orange: 0.8Hooves: 0.0

Cla

ss p

redi

ctio

Lear

ned

rel

atio

The tiger is the largest cat species, most recognizable for its pattern of dark vertical stripes on reddish-orange fur with a lighter underside. [wikipedia]