103
HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial optimization and machine learning for graph classification and matching Romain Raveaux Polytech’Tours - Universit´ e de Tours - LIFAT 26 Juin 2019 1/58

HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

  • Upload
    others

  • View
    6

  • Download
    0

Embed Size (px)

Citation preview

Page 1: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

HABILITATION A DIRIGER DES RECHERCHESContributions and perspectives on combinatorial optimization

and machine learning for graph classification and matching

Romain Raveaux

Polytech’Tours - Universite de Tours - LIFAT

26 Juin 2019

1/58

Page 2: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

2/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Content

1 Curriculum vitæ and pedagogical activities

2 Scientific activities

3 Research on graph matching and classification

4 Conclusions and perspectives

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 3: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

3/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Educations, diplomas and qualificationsPedagogical activities

Curriculum vitæ and pedagogical activities

1 Curriculum vitæ and pedagogical activities

2 Scientific activities

3 Research on graph matching and classification

4 Conclusions and perspectives

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 4: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

4/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Educations, diplomas and qualificationsPedagogical activities

Education

Diplomas:Date Degree Discipline Establishment2004-2006 Master Electrical/computer

engineeringUniversite de Rouen

2004-2006 Master Computer science Universite de Rouen

2006-2010 PhD Computer science Universite de La Rochelle

Master thesis: Symbol recognition in electrical diagram images.Supervised by Sebastien Adam and Pierre Heroux.

PhD thesis: Graph mining and graph classification: Application tocadastral map analysis. Supervised by Jean-Marc Ogier andJean-Christophe Burie.

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 5: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

4/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Educations, diplomas and qualificationsPedagogical activities

Education

Diplomas:Date Degree Discipline Establishment2004-2006 Master Electrical/computer

engineeringUniversite de Rouen

2004-2006 Master Computer science Universite de Rouen2006-2010 PhD Computer science Universite de La Rochelle

Master thesis: Symbol recognition in electrical diagram images.Supervised by Sebastien Adam and Pierre Heroux.PhD thesis: Graph mining and graph classification: Application tocadastral map analysis. Supervised by Jean-Marc Ogier andJean-Christophe Burie.

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 6: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

5/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Educations, diplomas and qualificationsPedagogical activities

Qualifications and positions

1 In 2010, qualified to be assistant professor

CNU sections: 27, 6127: Computer science61: Automatic and signal processing

2 In 2012, assistant professor

Universite de ToursTeaching: Engineering school: Polytech’ToursResearch: Computer science laboratory: LIFAT

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 7: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

6/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Educations, diplomas and qualificationsPedagogical activities

Pedagogical activities

1 Teaching: around 240h (EqTD) per year in average.

Licence level:Networking(L3), Python and data sciences(L2)Master level: Mobile systems(M2), Multimedia systems(M2)2012-2015: Lecturer in the International Master onComputer-Aided Decision Support

2 Pedagogical responsibilities: around 25h (EqTD) per year.

In charge of the fifth year of an engineering specialtyIn charge of student projects (collective and final projects)

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 8: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

7/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination

Scientific activities

1 Curriculum vitæ and pedagogical activities

2 Scientific activities

3 Research on graph matching and classification

4 Conclusions and perspectives

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 9: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

8/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination

Co-supervisions of PhD students

Date Name Funding Teaching Co-supervised by2012-2016 Zeina Abu-Aisheh Minister Assistant lecturer J.Y Ramel, P. Martineau∗

2015-2018 Mostafa Darwiche Region Assistant lecturer D. Conte, V. T’Kindt∗

2015-2019 Maxime Martineau Project Assistant lecturer D. Conte, G. Venturini

∗: A collaborative work: RFAI and ROOT teamsQuality of the management:

1 In numbers:3.3 Journal papers in average by PhD4.6 Conference, workshop papers in average by PhD

2 Without numbers:After the PhD: they stay with us (postdoc, lecturer)Then, they find jobs (Shape.ai, CIRELT, ...)

PhD supervision: a method

1 Available, relevant feed-backs, schedule

2 PhDs as young researchers, kindness communication

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 10: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

8/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination

Co-supervisions of PhD students

Date Name Funding Teaching Co-supervised by2012-2016 Zeina Abu-Aisheh Minister Assistant lecturer J.Y Ramel, P. Martineau∗

2015-2018 Mostafa Darwiche Region Assistant lecturer D. Conte, V. T’Kindt∗

2015-2019 Maxime Martineau Project Assistant lecturer D. Conte, G. Venturini

∗: A collaborative work: RFAI and ROOT teamsQuality of the management:

1 In numbers:3.3 Journal papers in average by PhD4.6 Conference, workshop papers in average by PhD

2 Without numbers:After the PhD: they stay with us (postdoc, lecturer)Then, they find jobs (Shape.ai, CIRELT, ...)

PhD supervision: a method

1 Available, relevant feed-backs, schedule

2 PhDs as young researchers, kindness communication

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 11: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

9/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination

Scientific expertise

1 Program Committee: Conferences

International Conference on Document Analysis andRecognition: ICDAR 2019, 2017Graph-Based Representation for pattern recognition: GBR2019Graphics Recognition: GREC 2019, 2015, 2013, 2011International Conference on Hybrid Artificial IntelligenceSystems: HAIS 2010, 2011

2 Reviewer for:

Pattern Recognition Letters (PRL) (1 per year)Pattern Recognition (PR)(1 per year)IEEE Transaction on Image Processing (TIP) (occasionally)International Journal On Document Analysis and Recognition(IJDAR) (occasionally)Expert Systems with Applications (ESWA)(1 per year)

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 12: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

10/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination

Scientific animation

Science administration:

Since 2016, member of the board of VALCONUM.VALCONUM is a European public-private structure aiming toaccelerate the technological transfer in the field ofdematerialization and the valorization of digital contents.

Contest organization:

2011, participation to the organization of a contest on symbolrecognition.

Conference organizations:

Participation to the organization of GBR 2019, DAS 2014 andGREC 2009

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 13: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

10/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination

Scientific animation

Science administration:

Since 2016, member of the board of VALCONUM.VALCONUM is a European public-private structure aiming toaccelerate the technological transfer in the field ofdematerialization and the valorization of digital contents.

Contest organization:

2011, participation to the organization of a contest on symbolrecognition.

Conference organizations:

Participation to the organization of GBR 2019, DAS 2014 andGREC 2009

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 14: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

11/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination

Projects and collaborations

Projects:

Name Date Scope Funding Involvement Statuse-trap2 2019 National 450K 15% Submitted

LOR 2019 National 600K 33% SubmittedVISIT 2019 Regional 150K (LIFAT) 20% Accepted. Running

ADAM-IoT 2019 European 1M – RejectedFibravasc 2018 Regional 100K (LIFAT) 10% Accepted. RunningMalagga 2018 National - - Rejected

ScannerLoire 2018 Regional - - RejectedADT 2016 Regional - - Rejected

Caramba 2016-2019 Regional 200K 33% Accepted. RunningDoD 2014-2015 Industrial 200K 33% Accepted. Done

Collaborations:

Institute Country Topic PublicationGREYC France, Caen Graph edit distance 1 journalLITIS France, Rouen Graph prototype, Graph edit distance 2 journals

Griffith University Australia Image quality 2 journalsCampinas University Brazil Learning graph matching 1 journal

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 15: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

12/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination

Publications

International journal IF 5 years 1 SNIP 2 SRJ 3 #articles

Pattern Recognition 4.3 2.47 1.06 2Expert Systems with Applications 3.7 2.4 1.27 2Computer and Operation Research 3.2 2.094 1.9 1Computer Vision and Image Understanding 2.7 1.8 0.71 1Pattern Recognition Letters 2.3 1.58 0.662 6Journal of Visual Communication and Image Representation 2.02 1.36 0.5 1International Journal on Document Analysis and Recogni-tion

1.26 1.4 0.387 1

Signal Image and Video Processing 1.6 NA NA 115

International Conference Rank4 #articles

International Conference on Document Analysis and Recognition (ICDAR) A 2Structural, Syntactic, and Statistical Pattern Recognition (SSPR) A 1International Workshop on Document Analysis Systems (DAS) B 2European Signal Processing Conference (EUSIPCO) B 1International Conference on Pattern Recognition (ICPR) B 3

8

1Impact factor from editor websites (June 2018).

2Source Normalized Impact per Paper from editor websites (June 2018).

3SCImago Journal Rank from editor websites (June 2018).

4CORE 2018 : http://portal.core.edu.au

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 16: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

13/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination

Data sets, codes and contest

Data sets are available online

http://www.rfai.li.univ-tours.fr/PublicData/

GDR4GED/home.html

Codes are available online

http://www.rfai.lifat.univ-tours.fr/PublicData/

GraphLib/home.html

https://networkx.github.io/networkx/algorithms/

similarity

Participation to a contest ICPR 2016

https://gdc2016.greyc.fr/

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 17: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

14/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Research on graph matching and classification

1 Curriculum vitæ and pedagogical activities

2 Scientific activities

3 Research on graph matching and classification

4 Conclusions and perspectives

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 18: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

15/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Problems and techniques

1 Problems: graph matching and graph classification

2 Techniques: thanks to machine learning and operationsresearch

3 A coherent work: my PhD, PhDs of Zeina Abu-aisheh,Mostafa Darwiche and Maxime Martineau.

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 19: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

15/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Problems and techniques

1 Problems: graph matching and graph classification

2 Techniques: thanks to machine learning and operationsresearch

3 A coherent work: my PhD, PhDs of Zeina Abu-aisheh,Mostafa Darwiche and Maxime Martineau.

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 20: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

15/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Problems and techniques

1 Problems: graph matching and graph classification

2 Techniques: thanks to machine learning and operationsresearch

3 A coherent work: my PhD, PhDs of Zeina Abu-aisheh,Mostafa Darwiche and Maxime Martineau.

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 21: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

16/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Toward Graph matching: A set of nodes

Each node can be a vector, an image or a word, ...

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 22: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

16/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Toward Graph matching: Two sets of nodes

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 23: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

16/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Toward Graph matching: Set matching

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 24: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

16/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Toward Graph matching: Two graphs

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 25: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

16/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Toward Graph matching (f : V1 → V2)

ij kl

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 26: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

17/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph classification

Toxic or not

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 27: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

18/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Operations Research

A problem has to be solved

1 Operations Research models and solves combinatorialproblems

2 Optimization problems need to be formalized and wellstructured

3 The human or (a priori) knowledge about the problems isintroduced through variables, constraints and objectives

=⇒ link with expert systems

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 28: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

19/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Machine Learning

A problem has to be solved

But: lack of formal specifications.

A possible solution:

1 Formulate a (statistical) proxy problem that relies on data

2 then use machine learning.

Machine Learning:

1 discover regularities on a given set of data (from“unstructured” or “not formalized” information)

2 generalizes to unseen data

3 solves the original problem without using explicit instructions

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 29: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

19/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Machine Learning

A problem has to be solved

But: lack of formal specifications.

A possible solution:

1 Formulate a (statistical) proxy problem that relies on data

2 then use machine learning.

Machine Learning:

1 discover regularities on a given set of data (from“unstructured” or “not formalized” information)

2 generalizes to unseen data

3 solves the original problem without using explicit instructions

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 30: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

19/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Machine Learning

A problem has to be solved

But: lack of formal specifications.

A possible solution:

1 Formulate a (statistical) proxy problem that relies on data

2 then use machine learning.

Machine Learning:

1 discover regularities on a given set of data (from“unstructured” or “not formalized” information)

2 generalizes to unseen data

3 solves the original problem without using explicit instructions

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 31: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

20/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Problems and state of the art1 Curriculum vitæ and pedagogical activities

Educations, diplomas and qualificationsPedagogical activities

2 Scientific activitiesCo-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination

3 Research on graph matching and classificationOverview of research topicsProblems and state of the artMy contributions

4 Conclusions and perspectivesConclusionsShort term perspectivesLong term perspectivesRomain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 32: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

21/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph matching: the problem at glance

Input:

1 Two graphs: G1, G2

2 Similarity functions:sV , sE

sV (i , k)

sV (j , k)

sE (ij , kl)

sV (i , l)

sV (j , l)

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 33: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

22/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph matching: the problem at glance

Output:

Node-to-nodematching

Binary variables: YYi,k = 1 if i and kare matched

Yi ,k

Yj ,k

Yi ,l

Yj ,l

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 34: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

23/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph matching: the problem at glance

Objective function: similarity of two graphs: Yi ,k = Yj ,l = 1S(G1,G2,Y ) = sV (i , k).Yi ,k + sV (j , l).Yj ,l + sE (ij , kl).Yi ,k .Yj ,l

sV (i , k) ∗ Yi ,k

sE (ij , kl) ∗ Yi ,k ∗ Yj ,l

sV (j , l) ∗ Yj ,l

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 35: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

24/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph matching: the problem at glance

Graph matching problem: NP-hard [Gold and Rangarajan, 1996]

Find Y such that:

1 the sum of similarities is maximized.

2 a node of G1 is matched to at most one node of G2

3 a node of G2 is matched to at most one node of G1

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 36: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

25/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph matching: state of the art

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 37: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

26/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph matching: state of the art

Bottom lines from the state of the art:

1 Many fast heuristics can be found in the literature

2 How accurate are these heuristics compared to exactmethods?

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 38: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

27/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph matching: pattern recognition applications

Graph matching can be involved in several pattern recognitionapplications.

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 39: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

28/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph matching: pattern recognition applications

1 Graph comparison

2 Graph similarity search

Compare the query withall the graphsSort by similarities

3 Graph classification:K-Nearest Neighbors(KNN)

Compare the query withall the graphsSort by similaritiesRetain the K most similargraphsThe most frequent classlabel

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 40: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

28/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph matching: pattern recognition applications

1 Graph comparison2 Graph similarity search

Compare the query withall the graphsSort by similarities

3 Graph classification:K-Nearest Neighbors(KNN)

Compare the query withall the graphsSort by similaritiesRetain the K most similargraphsThe most frequent classlabel

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 41: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

28/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph matching: pattern recognition applications

1 Graph comparison2 Graph similarity search

Compare the query withall the graphsSort by similarities

3 Graph classification:K-Nearest Neighbors(KNN)

Compare the query withall the graphsSort by similarities

Retain the K most similargraphsThe most frequent classlabel

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 42: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

28/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph matching: pattern recognition applications

1 Graph comparison2 Graph similarity search

Compare the query withall the graphsSort by similarities

3 Graph classification:K-Nearest Neighbors(KNN)

Compare the query withall the graphsSort by similaritiesRetain the K most similargraphsThe most frequent classlabel

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 43: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

28/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph matching: pattern recognition applications

1 Graph comparison2 Graph similarity search

Compare the query withall the graphsSort by similarities

3 Graph classification:K-Nearest Neighbors(KNN)

Compare the query withall the graphsSort by similaritiesRetain the K most similargraphsThe most frequent classlabel

Issue: many calls to the graph matching methodRomain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 44: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

29/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph matching: KNN state of the art

Graph KNN can be sped up thanks to:

1 Fast graph matching methods [K. Riesen, 2015]2 Reducing the number of graphs by graph prototypes [M.

Ferrer, 2011], [R. Raveaux, 2011].1 Keep the most significant graphs

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 45: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

29/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph matching: KNN state of the art

Graph KNN can be sped up thanks to:

1 Fast graph matching methods [K. Riesen, 2015]2 Reducing the number of graphs by graph prototypes [M.

Ferrer, 2011], [R. Raveaux, 2011].1 Keep the most significant graphs

How to reduce the number of comparisons without loss ofinformation?

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 46: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

30/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph matching: pattern recognition applications

We know how to compare graphs thank to graph matching.Graph matching offers an elegant manner to compare graphsdirectly in graph space.

But how to choose the similarity functions between nodes andedges?

Node/Edge similarity functions are crucial

Similarity functions link the graph matching problem to the finalapplication.

BUT:

1 How to compare graphs according to a specific objective?

2 How to reach a goal that is defined by data?

3 Where machine learning can be introduced?

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 47: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

30/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph matching: pattern recognition applications

We know how to compare graphs thank to graph matching.Graph matching offers an elegant manner to compare graphsdirectly in graph space.

But how to choose the similarity functions between nodes andedges?

Node/Edge similarity functions are crucial

Similarity functions link the graph matching problem to the finalapplication.

BUT:

1 How to compare graphs according to a specific objective?

2 How to reach a goal that is defined by data?

3 Where machine learning can be introduced?

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 48: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

30/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Graph matching: pattern recognition applications

We know how to compare graphs thank to graph matching.Graph matching offers an elegant manner to compare graphsdirectly in graph space.

But how to choose the similarity functions between nodes andedges?

Node/Edge similarity functions are crucial

Similarity functions link the graph matching problem to the finalapplication.

BUT:

1 How to compare graphs according to a specific objective?

2 How to reach a goal that is defined by data?

3 Where machine learning can be introduced?

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 49: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

31/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Learning graph matching: the problem at glance

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 50: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

32/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Learning graph matching: the state of the art

[Cho et al., 2013]

[Nowak et al., 2017] [Cortes et al., 2016]

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 51: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

33/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Deadlocks

1 How to design fast and accurate graph matching methods?

2 How to reduce the number of graph comparisons?

3 How to learn graph matching for classification?

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 52: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

34/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

My contributions1 Curriculum vitæ and pedagogical activities

Educations, diplomas and qualificationsPedagogical activities

2 Scientific activitiesCo-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination

3 Research on graph matching and classificationOverview of research topicsProblems and state of the artMy contributions

4 Conclusions and perspectivesConclusionsShort term perspectivesLong term perspectivesRomain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 53: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

35/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Deadlock 1: How to design fast and accurate graphmatching methods?

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 54: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

36/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Branch and bound for graph matching

2010

2011

2013

2015

2017

2018

2019

2015

Search space is a tree

A tree node is a partialmatching

Depth-first search withbacktracking[Abu-Aisheh et al., 2015]

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 55: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

36/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Branch and bound for graph matching

2010

2011

2013

2015

2017

2018

2019

2015

Search space is a tree

A tree node is a partialmatching

Depth-first search withbacktracking[Abu-Aisheh et al., 2015]

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 56: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

36/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Branch and bound for graph matching

2010

2011

2013

2015

2017

2018

2019

2015

Search space is a tree

A tree node is a partialmatching

Depth-first search withbacktracking[Abu-Aisheh et al., 2015]

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 57: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

36/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Branch and bound for graph matching

2010

2011

2013

2015

2017

2018

2019

2015

Search space is a tree

A tree node is a partialmatching

Depth-first search withbacktracking[Abu-Aisheh et al., 2015]

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 58: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

37/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Branch and bound for graph matching

Anytime branch and bound [Abu-Aisheh et al., 2016]:

finds quickly a solution: Depth-first search[Abu-Aisheh et al., 2015]

keeps on searching for improving solutions

can be easily interrupted at each tree node

can be interrupted anytime to provide the best solution foundso far.

Anytime algorithm: suitable when the time given to graphmatching methods is uncertain. Good for pattern recognitionapplications

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 59: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

37/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Branch and bound for graph matching

Anytime branch and bound [Abu-Aisheh et al., 2016]:

finds quickly a solution: Depth-first search[Abu-Aisheh et al., 2015]

keeps on searching for improving solutions

can be easily interrupted at each tree node

can be interrupted anytime to provide the best solution foundso far.

Anytime algorithm: suitable when the time given to graphmatching methods is uncertain. Good for pattern recognitionapplications

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 60: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

38/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Branch and bound for graph matching

2010

2011

2013

2015

2017

2018

2019

2015

Anytime branch and bound[Abu-Aisheh et al., 2016]

Quick first solution:Depth-first search[Abu-Aisheh et al., 2015]

Parallel branch and bound[Abu-Aisheh et al., 2018].Work-stealing strategy tobalance the load.

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 61: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

39/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Mathematical programming for graph matching

2010

2011

2013

2015

2017

2018

2019

2017

ILP for graph matching (F1,F2, F3):[Lerouge et al., 2017]

Linearization of thequadratic problem

Edge matching variables (Z)

sV (i , k) ∗ Yi ,k

sE (ij , kl) ∗ Yi ,k ∗ Yj ,l

sV (j , l) ∗ Yj ,l

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 62: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

39/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Mathematical programming for graph matching

2010

2011

2013

2015

2017

2018

2019

2017

ILP for graph matching (F1,F2, F3):[Lerouge et al., 2017]

Linearization of thequadratic problem

Edge matching variables (Z)

sV (i , k) ∗ Yi ,k

sE (ij , kl) ∗ Zij ,kl

sV (j , l) ∗ Yj ,l

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 63: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

40/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Mathematical programming for graph matching

New constraints should be addedto ensure that if 2 edges arematched (Zij ,kl = 1) then relatednodes are matched too.Topological constraints:

Zij ,kl ≤ Yi ,k ∀(ij , kl) ∈ E1 × E2

Zij ,kl ≤ Yj ,l ∀(ij , kl) ∈ E1 × E2

sV (i , k) ∗ Yi ,k

sE (ij , kl) ∗ Zij ,kl

sV (j , l) ∗ Yj ,l

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 64: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

41/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Mathematical programming for graph matching

2010

2011

2013

2015

2017

2018

2019

2017

1 Local Branching Heuristic: LocBra [Darwiche et al., 2018]2 The goal is to intelligently explore the solution space

By taking advantage of an ILP and a mathematical solver(Cplex)

3 It is an iterative process starting from an initial solution (x0)

4 x is a vector of binary variables grouping Y and Z

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 65: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

42/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Mathematical programming for graph matching

Local search: Search for an improving solution inside a smallregion of the solution space.

Neighborhood: no more than k variables should be changedbetween the new solution and x0.This neighborhood definition is a new constraint in the ILP

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 66: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

43/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Mathematical programming for graph matching

No improved solutions found → Diversification operator for GM:

Goal: Helps skipping a local optimum.

To change the region, change important variables

Important variables=a high impacts on the objective functionvalue.

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 67: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

43/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Mathematical programming for graph matching

No improved solutions found → Diversification operator for GM:

Goal: Helps skipping a local optimum.To change the region, change important variables

Important variables=a high impacts on the objective functionvalue.

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 68: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

44/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Deadlock 2: How to reduce the number of graphcomparisons?

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 69: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

45/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Fused graph matching and KNN problems for classification

2010

2011

2013

2015

2017

2018

2019

2017

Problem: Solve the KNN problemfor graphs

Issue: Many graph comparisonsand the process is slow

Answer: Merge GM and KNN in asingle problem[Abu-Aisheh et al., 2017]

Why: Global reasoning instead ofconsidering each comparisonindependently

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 70: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

46/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Fused graph matching and KNN problems for classification

(GM+KNN):

The search space is organized as a tree

First floor: a tree node is a graph comparison

Each graph comparison is an instance of graph matching

It can be solved by a Branch and Bound method

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 71: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

47/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Fused graph matching and KNN problems for classification

Use case: We are looking for the nearest neighbor1 Compare Gq with G1: S(Gq,G1) = 4

2 Start to compare Gq with G2 and use S(Gq,G1) = 4 as alower bound to cut branches.

3 From the first floor, we estimate that S(Gq,G2) ≤ 24 G1 is a better neighbor, no need to explore fully the sub-tree

(Gq,G2)5 Avoid (full) comparisons of very dissimilar graphs

C is function that evaluates a tree node (over estimate)Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 72: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

47/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Fused graph matching and KNN problems for classification

Use case: We are looking for the nearest neighbor1 Compare Gq with G1: S(Gq,G1) = 42 Start to compare Gq with G2 and use S(Gq,G1) = 4 as a

lower bound to cut branches.

3 From the first floor, we estimate that S(Gq,G2) ≤ 24 G1 is a better neighbor, no need to explore fully the sub-tree

(Gq,G2)5 Avoid (full) comparisons of very dissimilar graphs

C is function that evaluates a tree node (over estimate)Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 73: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

47/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Fused graph matching and KNN problems for classification

Use case: We are looking for the nearest neighbor1 Compare Gq with G1: S(Gq,G1) = 42 Start to compare Gq with G2 and use S(Gq,G1) = 4 as a

lower bound to cut branches.3 From the first floor, we estimate that S(Gq,G2) ≤ 2

4 G1 is a better neighbor, no need to explore fully the sub-tree(Gq,G2)

5 Avoid (full) comparisons of very dissimilar graphs

C is function that evaluates a tree node (over estimate)Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 74: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

47/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Fused graph matching and KNN problems for classification

Use case: We are looking for the nearest neighbor1 Compare Gq with G1: S(Gq,G1) = 42 Start to compare Gq with G2 and use S(Gq,G1) = 4 as a

lower bound to cut branches.3 From the first floor, we estimate that S(Gq,G2) ≤ 24 G1 is a better neighbor, no need to explore fully the sub-tree

(Gq,G2)

5 Avoid (full) comparisons of very dissimilar graphs

C is function that evaluates a tree node (over estimate)Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 75: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

47/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Fused graph matching and KNN problems for classification

Use case: We are looking for the nearest neighbor1 Compare Gq with G1: S(Gq,G1) = 42 Start to compare Gq with G2 and use S(Gq,G1) = 4 as a

lower bound to cut branches.3 From the first floor, we estimate that S(Gq,G2) ≤ 24 G1 is a better neighbor, no need to explore fully the sub-tree

(Gq,G2)5 Avoid (full) comparisons of very dissimilar graphs

C is function that evaluates a tree node (over estimate)Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 76: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

48/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Deadlock 3: How to learn graph matching?

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 77: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

49/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Learn graph matching for classification

2010

2011

2013

2015

2017

2018

2019

2018

Parametrized graphmatching[Raveaux et al., 2017]

Learning graph matching forclassification[Martineau et al., 2018]

sV (i , k)

sE (ij , kl)

sV (j , l)Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 78: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

49/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Learn graph matching for classification

2010

2011

2013

2015

2017

2018

2019

2018

Parametrized graphmatching[Raveaux et al., 2017]

Learning graph matching forclassification[Martineau et al., 2018]

sV (i , k).βk

sE (ij , kl).βkl

sV (j , l).βlRomain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 79: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

50/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Learn graph matching for classification

2010

2011

2013

2015

2017

2018

2019

2018

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 80: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

50/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

Overview of research topicsProblems and state of the artMy contributions

Learn graph matching for classification

2010

2011

2013

2015

2017

2018

2019

2018

Parameters β are learned by gradient descent.Weakly supervised learning of (discriminative) graph matching

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 81: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

51/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Conclusions and perspectives

1 Curriculum vitæ and pedagogical activities

2 Scientific activities

3 Research on graph matching and classification

4 Conclusions and perspectives

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 82: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

52/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Take a step back on graph matchingFrom the operations research side

NP-hard problem

No single method can effectively address all instances.

Computer vision and pattern recognition:

low computational time usually dominates the optimalityguarantees.

Complementary of the methods

There is a need to combine heuristics

Select an heuristic according to the instance at hand

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 83: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

53/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Take a step back on graph matchingFrom the machine learning viewpoint

Similarity functions are crucial

Learning node/edge embedding, learning similarity functions

To reach a specific objective (the user need).

Good similarity functions

allow to easily differentiate between vertices/edges

can make the problem easier to solve.

Node embedding integrating topological information

can help to recast the quadratic problem to linear assignmentproblem

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 84: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

53/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Take a step back on graph matchingFrom the machine learning viewpoint

Similarity functions are crucial

Learning node/edge embedding, learning similarity functions

To reach a specific objective (the user need).

Good similarity functions

allow to easily differentiate between vertices/edges

can make the problem easier to solve.

Node embedding integrating topological information

can help to recast the quadratic problem to linear assignmentproblem

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 85: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

53/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Take a step back on graph matchingFrom the machine learning viewpoint

Similarity functions are crucial

Learning node/edge embedding, learning similarity functions

To reach a specific objective (the user need).

Good similarity functions

allow to easily differentiate between vertices/edges

can make the problem easier to solve.

Node embedding integrating topological information

can help to recast the quadratic problem to linear assignmentproblem

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 86: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

54/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Take a step back on graph matchingGraph matching for graph classification

Question: What is the meaning of graph matching for graphclassification?

GM imposes node assignment constraints

GM brings constraints to the learning problem:

Do constraints act like a regularization term to bettergeneralize on unseen data?

Are constraints useful to reduce the number of training data?

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 87: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

54/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Take a step back on graph matchingGraph matching for graph classification

Question: What is the meaning of graph matching for graphclassification?

GM imposes node assignment constraints

GM brings constraints to the learning problem:

Do constraints act like a regularization term to bettergeneralize on unseen data?

Are constraints useful to reduce the number of training data?

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 88: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

54/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Take a step back on graph matchingGraph matching for graph classification

Question: What is the meaning of graph matching for graphclassification?

GM imposes node assignment constraints

GM brings constraints to the learning problem:

Do constraints act like a regularization term to bettergeneralize on unseen data?

Are constraints useful to reduce the number of training data?

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 89: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

54/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Take a step back on graph matchingGraph matching for graph classification

Question: What is the meaning of graph matching for graphclassification?

GM imposes node assignment constraints

GM brings constraints to the learning problem:

Do constraints act like a regularization term to bettergeneralize on unseen data?

Are constraints useful to reduce the number of training data?

These open questions are important to legitimate graph matchingfor graph classification.

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 90: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

55/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Graph matching what do we need?

From the operations research side:

1 Fast, scalable and accurate heuristics are always welcome forcomputer vision and pattern recognition communities.

From the machine learning viewpoint:

1 Low complexity (near linear time)

2 Graph matching methods suited to execution on GPU

3 Learning problems are often solved by gradient descent sodifferentiable methods are wanted → link with robustoptimization?

4 Avoid the storage of similarity matrices (|V1|.|V2| × |V1|.|V2|)

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 91: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

55/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Graph matching what do we need?

From the operations research side:

1 Fast, scalable and accurate heuristics are always welcome forcomputer vision and pattern recognition communities.

From the machine learning viewpoint:

1 Low complexity (near linear time)

2 Graph matching methods suited to execution on GPU

3 Learning problems are often solved by gradient descent sodifferentiable methods are wanted → link with robustoptimization?

4 Avoid the storage of similarity matrices (|V1|.|V2| × |V1|.|V2|)

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 92: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

55/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Graph matching what do we need?

From the operations research side:

1 Fast, scalable and accurate heuristics are always welcome forcomputer vision and pattern recognition communities.

From the machine learning viewpoint:

1 Low complexity (near linear time)

2 Graph matching methods suited to execution on GPU

3 Learning problems are often solved by gradient descent sodifferentiable methods are wanted → link with robustoptimization?

4 Avoid the storage of similarity matrices (|V1|.|V2| × |V1|.|V2|)

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 93: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

55/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Graph matching what do we need?

From the operations research side:

1 Fast, scalable and accurate heuristics are always welcome forcomputer vision and pattern recognition communities.

From the machine learning viewpoint:

1 Low complexity (near linear time)

2 Graph matching methods suited to execution on GPU

3 Learning problems are often solved by gradient descent sodifferentiable methods are wanted → link with robustoptimization?

4 Avoid the storage of similarity matrices (|V1|.|V2| × |V1|.|V2|)

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 94: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

55/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Graph matching what do we need?

From the operations research side:

1 Fast, scalable and accurate heuristics are always welcome forcomputer vision and pattern recognition communities.

From the machine learning viewpoint:

1 Low complexity (near linear time)

2 Graph matching methods suited to execution on GPU

3 Learning problems are often solved by gradient descent sodifferentiable methods are wanted → link with robustoptimization?

4 Avoid the storage of similarity matrices (|V1|.|V2| × |V1|.|V2|)

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 95: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

56/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Short term perspectives

Future master students (low hanging fruits)

1 (KNN + GM) solved by the local branching heuristic

2 Parametrized GM as an input layer of a MLP

3 ILP for the MCS problem

Future PhD students

1 Learning graph matching: hierarchical feature learning (GraphNeural Network) + a combinatorial layer (graph matchingmethod)

2 Learning graph matching: learn to branch in a branch andbound (reinforcement learning)

3 Structured classification: graph matching as a loss function

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 96: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

57/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Long term perspectives

To be curious:1 Cross fertilization: OR, ML: LOR project2 Inspired by other problems:

From computer vision: CRF MAP-inferenceFrom OR: TSP, scheduling problems

Fundamental:

On the relation between graph matching and OptimalTransport (OT) (Gromov-Wasserstein distance).

Applications:1 Graph matching for multiple object tracking in videos, table

comparisons in documents, ...2 Graph matching for unsupervised domain adaptation3 Benchmarks

Still a lot of interesting work to do ...

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 97: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

57/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Long term perspectives

To be curious:1 Cross fertilization: OR, ML: LOR project2 Inspired by other problems:

From computer vision: CRF MAP-inferenceFrom OR: TSP, scheduling problems

Fundamental:

On the relation between graph matching and OptimalTransport (OT) (Gromov-Wasserstein distance).

Applications:1 Graph matching for multiple object tracking in videos, table

comparisons in documents, ...2 Graph matching for unsupervised domain adaptation3 Benchmarks

Still a lot of interesting work to do ...Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 98: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

58/58

Curriculum vitæ and pedagogical activitiesScientific activities

Research on graph matching and classificationConclusions and perspectives

ConclusionsShort term perspectivesLong term perspectives

Thank you

Any questions?

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 99: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

0/0

Abu-Aisheh, Z., Raveaux, R., and Ramel, J. (2016).Anytime graph matching.Pattern Recognition Letters, 84:215–224.

Abu-Aisheh, Z., Raveaux, R., and Ramel, J. (2017).Fast nearest neighbors search in graph space based on abranch-and-bound strategy.In [Foggia et al., 2017], pages 197–207.

Abu-Aisheh, Z., Raveaux, R., Ramel, J., and Martineau, P.(2015).An exact graph edit distance algorithm for solving patternrecognition problems.In Marsico, M. D., Figueiredo, M. A. T., and Fred, A. L. N.,editors, ICPRAM 2015 - Proceedings of the InternationalConference on Pattern Recognition Applications and Methods,

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 100: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

0/0

Volume 1, Lisbon, Portugal, 10-12 January, 2015., pages271–278. SciTePress.

Abu-Aisheh, Z., Raveaux, R., Ramel, J., and Martineau, P.(2018).A parallel graph edit distance algorithm.Expert Syst. Appl., 94:41–57.

Cho, M., Alahari, K., and Ponce, J. (2013).Learning graphs to match.In IEEE International Conference on Computer Vision, ICCV2013, pages 25–32.

Cortes, X., Serratosa, F., and Serratosa, F. (2016).Learning graph matching substitution weights based on theground truth node correspondence.IJPRAI, 30(2).

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 101: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

0/0

Darwiche, M., Conte, D., Raveaux, R., and T’Kindt, V.(2018).A local branching heuristic for solving a graph edit distanceproblem.Computers and Operations Research.

Foggia, P., Liu, C., and Vento, M., editors (2017).Graph-Based Representations in Pattern Recognition - 11thIAPR-TC-15 International Workshop, GbRPR 2017, Anacapri,Italy, May 16-18, 2017, Proceedings, volume 10310 of LectureNotes in Computer Science.

Gold, S. and Rangarajan, A. (1996).A graduated assignment algorithm for graph matching.IEEE Transactions on Pattern Analysis and MachineIntelligence, 18(4):377–388.

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 102: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

0/0

Lerouge, J., Abu-Aisheh, Z., Raveaux, R., Heroux, P., andAdam, S. (2017).New binary linear programming formulation to compute thegraph edit distance.Pattern Recognition, 72:254–265.

Martineau, M., Raveaux, R., Conte, D., and Venturini, G.(2018).Learning error-correcting graph matching with a multiclassneural network.Pattern Recognition Letters.

Nowak, A., Villar, S., Bandeira, A. S., and Bruna, J. (2017).A note on learning algorithms for quadratic assignment withgraph neural networks.CoRR, abs/1706.07450.

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES

Page 103: HABILITATION A DIRIGER DES RECHERCHESromain.raveaux.free.fr/document/PresentationHDRfinalsans...HABILITATION A DIRIGER DES RECHERCHES Contributions and perspectives on combinatorial

0/0

Raveaux, R., Martineau, M., Conte, D., and Venturini, G.(2017).Learning graph matching with a graph-based perceptron in aclassification context.In [Foggia et al., 2017], pages 49–58.

Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES