Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
HABILITATION A DIRIGER DES RECHERCHESContributions and perspectives on combinatorial optimization
and machine learning for graph classification and matching
Romain Raveaux
Polytech’Tours - Universite de Tours - LIFAT
26 Juin 2019
1/58
2/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Content
1 Curriculum vitæ and pedagogical activities
2 Scientific activities
3 Research on graph matching and classification
4 Conclusions and perspectives
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
3/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Educations, diplomas and qualificationsPedagogical activities
Curriculum vitæ and pedagogical activities
1 Curriculum vitæ and pedagogical activities
2 Scientific activities
3 Research on graph matching and classification
4 Conclusions and perspectives
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
4/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Educations, diplomas and qualificationsPedagogical activities
Education
Diplomas:Date Degree Discipline Establishment2004-2006 Master Electrical/computer
engineeringUniversite de Rouen
2004-2006 Master Computer science Universite de Rouen
2006-2010 PhD Computer science Universite de La Rochelle
Master thesis: Symbol recognition in electrical diagram images.Supervised by Sebastien Adam and Pierre Heroux.
PhD thesis: Graph mining and graph classification: Application tocadastral map analysis. Supervised by Jean-Marc Ogier andJean-Christophe Burie.
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
4/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Educations, diplomas and qualificationsPedagogical activities
Education
Diplomas:Date Degree Discipline Establishment2004-2006 Master Electrical/computer
engineeringUniversite de Rouen
2004-2006 Master Computer science Universite de Rouen2006-2010 PhD Computer science Universite de La Rochelle
Master thesis: Symbol recognition in electrical diagram images.Supervised by Sebastien Adam and Pierre Heroux.PhD thesis: Graph mining and graph classification: Application tocadastral map analysis. Supervised by Jean-Marc Ogier andJean-Christophe Burie.
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
5/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Educations, diplomas and qualificationsPedagogical activities
Qualifications and positions
1 In 2010, qualified to be assistant professor
CNU sections: 27, 6127: Computer science61: Automatic and signal processing
2 In 2012, assistant professor
Universite de ToursTeaching: Engineering school: Polytech’ToursResearch: Computer science laboratory: LIFAT
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
6/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Educations, diplomas and qualificationsPedagogical activities
Pedagogical activities
1 Teaching: around 240h (EqTD) per year in average.
Licence level:Networking(L3), Python and data sciences(L2)Master level: Mobile systems(M2), Multimedia systems(M2)2012-2015: Lecturer in the International Master onComputer-Aided Decision Support
2 Pedagogical responsibilities: around 25h (EqTD) per year.
In charge of the fifth year of an engineering specialtyIn charge of student projects (collective and final projects)
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
7/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination
Scientific activities
1 Curriculum vitæ and pedagogical activities
2 Scientific activities
3 Research on graph matching and classification
4 Conclusions and perspectives
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
8/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination
Co-supervisions of PhD students
Date Name Funding Teaching Co-supervised by2012-2016 Zeina Abu-Aisheh Minister Assistant lecturer J.Y Ramel, P. Martineau∗
2015-2018 Mostafa Darwiche Region Assistant lecturer D. Conte, V. T’Kindt∗
2015-2019 Maxime Martineau Project Assistant lecturer D. Conte, G. Venturini
∗: A collaborative work: RFAI and ROOT teamsQuality of the management:
1 In numbers:3.3 Journal papers in average by PhD4.6 Conference, workshop papers in average by PhD
2 Without numbers:After the PhD: they stay with us (postdoc, lecturer)Then, they find jobs (Shape.ai, CIRELT, ...)
PhD supervision: a method
1 Available, relevant feed-backs, schedule
2 PhDs as young researchers, kindness communication
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
8/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination
Co-supervisions of PhD students
Date Name Funding Teaching Co-supervised by2012-2016 Zeina Abu-Aisheh Minister Assistant lecturer J.Y Ramel, P. Martineau∗
2015-2018 Mostafa Darwiche Region Assistant lecturer D. Conte, V. T’Kindt∗
2015-2019 Maxime Martineau Project Assistant lecturer D. Conte, G. Venturini
∗: A collaborative work: RFAI and ROOT teamsQuality of the management:
1 In numbers:3.3 Journal papers in average by PhD4.6 Conference, workshop papers in average by PhD
2 Without numbers:After the PhD: they stay with us (postdoc, lecturer)Then, they find jobs (Shape.ai, CIRELT, ...)
PhD supervision: a method
1 Available, relevant feed-backs, schedule
2 PhDs as young researchers, kindness communication
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
9/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination
Scientific expertise
1 Program Committee: Conferences
International Conference on Document Analysis andRecognition: ICDAR 2019, 2017Graph-Based Representation for pattern recognition: GBR2019Graphics Recognition: GREC 2019, 2015, 2013, 2011International Conference on Hybrid Artificial IntelligenceSystems: HAIS 2010, 2011
2 Reviewer for:
Pattern Recognition Letters (PRL) (1 per year)Pattern Recognition (PR)(1 per year)IEEE Transaction on Image Processing (TIP) (occasionally)International Journal On Document Analysis and Recognition(IJDAR) (occasionally)Expert Systems with Applications (ESWA)(1 per year)
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
10/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination
Scientific animation
Science administration:
Since 2016, member of the board of VALCONUM.VALCONUM is a European public-private structure aiming toaccelerate the technological transfer in the field ofdematerialization and the valorization of digital contents.
Contest organization:
2011, participation to the organization of a contest on symbolrecognition.
Conference organizations:
Participation to the organization of GBR 2019, DAS 2014 andGREC 2009
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
10/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination
Scientific animation
Science administration:
Since 2016, member of the board of VALCONUM.VALCONUM is a European public-private structure aiming toaccelerate the technological transfer in the field ofdematerialization and the valorization of digital contents.
Contest organization:
2011, participation to the organization of a contest on symbolrecognition.
Conference organizations:
Participation to the organization of GBR 2019, DAS 2014 andGREC 2009
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
11/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination
Projects and collaborations
Projects:
Name Date Scope Funding Involvement Statuse-trap2 2019 National 450K 15% Submitted
LOR 2019 National 600K 33% SubmittedVISIT 2019 Regional 150K (LIFAT) 20% Accepted. Running
ADAM-IoT 2019 European 1M – RejectedFibravasc 2018 Regional 100K (LIFAT) 10% Accepted. RunningMalagga 2018 National - - Rejected
ScannerLoire 2018 Regional - - RejectedADT 2016 Regional - - Rejected
Caramba 2016-2019 Regional 200K 33% Accepted. RunningDoD 2014-2015 Industrial 200K 33% Accepted. Done
Collaborations:
Institute Country Topic PublicationGREYC France, Caen Graph edit distance 1 journalLITIS France, Rouen Graph prototype, Graph edit distance 2 journals
Griffith University Australia Image quality 2 journalsCampinas University Brazil Learning graph matching 1 journal
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
12/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination
Publications
International journal IF 5 years 1 SNIP 2 SRJ 3 #articles
Pattern Recognition 4.3 2.47 1.06 2Expert Systems with Applications 3.7 2.4 1.27 2Computer and Operation Research 3.2 2.094 1.9 1Computer Vision and Image Understanding 2.7 1.8 0.71 1Pattern Recognition Letters 2.3 1.58 0.662 6Journal of Visual Communication and Image Representation 2.02 1.36 0.5 1International Journal on Document Analysis and Recogni-tion
1.26 1.4 0.387 1
Signal Image and Video Processing 1.6 NA NA 115
International Conference Rank4 #articles
International Conference on Document Analysis and Recognition (ICDAR) A 2Structural, Syntactic, and Statistical Pattern Recognition (SSPR) A 1International Workshop on Document Analysis Systems (DAS) B 2European Signal Processing Conference (EUSIPCO) B 1International Conference on Pattern Recognition (ICPR) B 3
8
1Impact factor from editor websites (June 2018).
2Source Normalized Impact per Paper from editor websites (June 2018).
3SCImago Journal Rank from editor websites (June 2018).
4CORE 2018 : http://portal.core.edu.au
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
13/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Co-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination
Data sets, codes and contest
Data sets are available online
http://www.rfai.li.univ-tours.fr/PublicData/
GDR4GED/home.html
Codes are available online
http://www.rfai.lifat.univ-tours.fr/PublicData/
GraphLib/home.html
https://networkx.github.io/networkx/algorithms/
similarity
Participation to a contest ICPR 2016
https://gdc2016.greyc.fr/
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
14/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Research on graph matching and classification
1 Curriculum vitæ and pedagogical activities
2 Scientific activities
3 Research on graph matching and classification
4 Conclusions and perspectives
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
15/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Problems and techniques
1 Problems: graph matching and graph classification
2 Techniques: thanks to machine learning and operationsresearch
3 A coherent work: my PhD, PhDs of Zeina Abu-aisheh,Mostafa Darwiche and Maxime Martineau.
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
15/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Problems and techniques
1 Problems: graph matching and graph classification
2 Techniques: thanks to machine learning and operationsresearch
3 A coherent work: my PhD, PhDs of Zeina Abu-aisheh,Mostafa Darwiche and Maxime Martineau.
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
15/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Problems and techniques
1 Problems: graph matching and graph classification
2 Techniques: thanks to machine learning and operationsresearch
3 A coherent work: my PhD, PhDs of Zeina Abu-aisheh,Mostafa Darwiche and Maxime Martineau.
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
16/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Toward Graph matching: A set of nodes
Each node can be a vector, an image or a word, ...
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
16/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Toward Graph matching: Two sets of nodes
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
16/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Toward Graph matching: Set matching
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
16/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Toward Graph matching: Two graphs
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
16/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Toward Graph matching (f : V1 → V2)
ij kl
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
17/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph classification
Toxic or not
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
18/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Operations Research
A problem has to be solved
1 Operations Research models and solves combinatorialproblems
2 Optimization problems need to be formalized and wellstructured
3 The human or (a priori) knowledge about the problems isintroduced through variables, constraints and objectives
=⇒ link with expert systems
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
19/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Machine Learning
A problem has to be solved
But: lack of formal specifications.
A possible solution:
1 Formulate a (statistical) proxy problem that relies on data
2 then use machine learning.
Machine Learning:
1 discover regularities on a given set of data (from“unstructured” or “not formalized” information)
2 generalizes to unseen data
3 solves the original problem without using explicit instructions
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
19/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Machine Learning
A problem has to be solved
But: lack of formal specifications.
A possible solution:
1 Formulate a (statistical) proxy problem that relies on data
2 then use machine learning.
Machine Learning:
1 discover regularities on a given set of data (from“unstructured” or “not formalized” information)
2 generalizes to unseen data
3 solves the original problem without using explicit instructions
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
19/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Machine Learning
A problem has to be solved
But: lack of formal specifications.
A possible solution:
1 Formulate a (statistical) proxy problem that relies on data
2 then use machine learning.
Machine Learning:
1 discover regularities on a given set of data (from“unstructured” or “not formalized” information)
2 generalizes to unseen data
3 solves the original problem without using explicit instructions
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
20/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Problems and state of the art1 Curriculum vitæ and pedagogical activities
Educations, diplomas and qualificationsPedagogical activities
2 Scientific activitiesCo-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination
3 Research on graph matching and classificationOverview of research topicsProblems and state of the artMy contributions
4 Conclusions and perspectivesConclusionsShort term perspectivesLong term perspectivesRomain Raveaux HABILITATION A DIRIGER DES RECHERCHES
21/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph matching: the problem at glance
Input:
1 Two graphs: G1, G2
2 Similarity functions:sV , sE
sV (i , k)
sV (j , k)
sE (ij , kl)
sV (i , l)
sV (j , l)
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
22/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph matching: the problem at glance
Output:
Node-to-nodematching
Binary variables: YYi,k = 1 if i and kare matched
Yi ,k
Yj ,k
Yi ,l
Yj ,l
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
23/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph matching: the problem at glance
Objective function: similarity of two graphs: Yi ,k = Yj ,l = 1S(G1,G2,Y ) = sV (i , k).Yi ,k + sV (j , l).Yj ,l + sE (ij , kl).Yi ,k .Yj ,l
sV (i , k) ∗ Yi ,k
sE (ij , kl) ∗ Yi ,k ∗ Yj ,l
sV (j , l) ∗ Yj ,l
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
24/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph matching: the problem at glance
Graph matching problem: NP-hard [Gold and Rangarajan, 1996]
Find Y such that:
1 the sum of similarities is maximized.
2 a node of G1 is matched to at most one node of G2
3 a node of G2 is matched to at most one node of G1
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
25/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph matching: state of the art
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
26/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph matching: state of the art
Bottom lines from the state of the art:
1 Many fast heuristics can be found in the literature
2 How accurate are these heuristics compared to exactmethods?
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
27/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph matching: pattern recognition applications
Graph matching can be involved in several pattern recognitionapplications.
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
28/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph matching: pattern recognition applications
1 Graph comparison
2 Graph similarity search
Compare the query withall the graphsSort by similarities
3 Graph classification:K-Nearest Neighbors(KNN)
Compare the query withall the graphsSort by similaritiesRetain the K most similargraphsThe most frequent classlabel
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
28/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph matching: pattern recognition applications
1 Graph comparison2 Graph similarity search
Compare the query withall the graphsSort by similarities
3 Graph classification:K-Nearest Neighbors(KNN)
Compare the query withall the graphsSort by similaritiesRetain the K most similargraphsThe most frequent classlabel
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
28/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph matching: pattern recognition applications
1 Graph comparison2 Graph similarity search
Compare the query withall the graphsSort by similarities
3 Graph classification:K-Nearest Neighbors(KNN)
Compare the query withall the graphsSort by similarities
Retain the K most similargraphsThe most frequent classlabel
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
28/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph matching: pattern recognition applications
1 Graph comparison2 Graph similarity search
Compare the query withall the graphsSort by similarities
3 Graph classification:K-Nearest Neighbors(KNN)
Compare the query withall the graphsSort by similaritiesRetain the K most similargraphsThe most frequent classlabel
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
28/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph matching: pattern recognition applications
1 Graph comparison2 Graph similarity search
Compare the query withall the graphsSort by similarities
3 Graph classification:K-Nearest Neighbors(KNN)
Compare the query withall the graphsSort by similaritiesRetain the K most similargraphsThe most frequent classlabel
Issue: many calls to the graph matching methodRomain Raveaux HABILITATION A DIRIGER DES RECHERCHES
29/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph matching: KNN state of the art
Graph KNN can be sped up thanks to:
1 Fast graph matching methods [K. Riesen, 2015]2 Reducing the number of graphs by graph prototypes [M.
Ferrer, 2011], [R. Raveaux, 2011].1 Keep the most significant graphs
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
29/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph matching: KNN state of the art
Graph KNN can be sped up thanks to:
1 Fast graph matching methods [K. Riesen, 2015]2 Reducing the number of graphs by graph prototypes [M.
Ferrer, 2011], [R. Raveaux, 2011].1 Keep the most significant graphs
How to reduce the number of comparisons without loss ofinformation?
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
30/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph matching: pattern recognition applications
We know how to compare graphs thank to graph matching.Graph matching offers an elegant manner to compare graphsdirectly in graph space.
But how to choose the similarity functions between nodes andedges?
Node/Edge similarity functions are crucial
Similarity functions link the graph matching problem to the finalapplication.
BUT:
1 How to compare graphs according to a specific objective?
2 How to reach a goal that is defined by data?
3 Where machine learning can be introduced?
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
30/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph matching: pattern recognition applications
We know how to compare graphs thank to graph matching.Graph matching offers an elegant manner to compare graphsdirectly in graph space.
But how to choose the similarity functions between nodes andedges?
Node/Edge similarity functions are crucial
Similarity functions link the graph matching problem to the finalapplication.
BUT:
1 How to compare graphs according to a specific objective?
2 How to reach a goal that is defined by data?
3 Where machine learning can be introduced?
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
30/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Graph matching: pattern recognition applications
We know how to compare graphs thank to graph matching.Graph matching offers an elegant manner to compare graphsdirectly in graph space.
But how to choose the similarity functions between nodes andedges?
Node/Edge similarity functions are crucial
Similarity functions link the graph matching problem to the finalapplication.
BUT:
1 How to compare graphs according to a specific objective?
2 How to reach a goal that is defined by data?
3 Where machine learning can be introduced?
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
31/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Learning graph matching: the problem at glance
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
32/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Learning graph matching: the state of the art
[Cho et al., 2013]
[Nowak et al., 2017] [Cortes et al., 2016]
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
33/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Deadlocks
1 How to design fast and accurate graph matching methods?
2 How to reduce the number of graph comparisons?
3 How to learn graph matching for classification?
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
34/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
My contributions1 Curriculum vitæ and pedagogical activities
Educations, diplomas and qualificationsPedagogical activities
2 Scientific activitiesCo-supervisions of studentsAnimations and responsibilitiesProjects and collaborationsDissemination
3 Research on graph matching and classificationOverview of research topicsProblems and state of the artMy contributions
4 Conclusions and perspectivesConclusionsShort term perspectivesLong term perspectivesRomain Raveaux HABILITATION A DIRIGER DES RECHERCHES
35/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Deadlock 1: How to design fast and accurate graphmatching methods?
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
36/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Branch and bound for graph matching
2010
2011
2013
2015
2017
2018
2019
2015
Search space is a tree
A tree node is a partialmatching
Depth-first search withbacktracking[Abu-Aisheh et al., 2015]
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
36/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Branch and bound for graph matching
2010
2011
2013
2015
2017
2018
2019
2015
Search space is a tree
A tree node is a partialmatching
Depth-first search withbacktracking[Abu-Aisheh et al., 2015]
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
36/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Branch and bound for graph matching
2010
2011
2013
2015
2017
2018
2019
2015
Search space is a tree
A tree node is a partialmatching
Depth-first search withbacktracking[Abu-Aisheh et al., 2015]
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
36/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Branch and bound for graph matching
2010
2011
2013
2015
2017
2018
2019
2015
Search space is a tree
A tree node is a partialmatching
Depth-first search withbacktracking[Abu-Aisheh et al., 2015]
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
37/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Branch and bound for graph matching
Anytime branch and bound [Abu-Aisheh et al., 2016]:
finds quickly a solution: Depth-first search[Abu-Aisheh et al., 2015]
keeps on searching for improving solutions
can be easily interrupted at each tree node
can be interrupted anytime to provide the best solution foundso far.
Anytime algorithm: suitable when the time given to graphmatching methods is uncertain. Good for pattern recognitionapplications
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
37/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Branch and bound for graph matching
Anytime branch and bound [Abu-Aisheh et al., 2016]:
finds quickly a solution: Depth-first search[Abu-Aisheh et al., 2015]
keeps on searching for improving solutions
can be easily interrupted at each tree node
can be interrupted anytime to provide the best solution foundso far.
Anytime algorithm: suitable when the time given to graphmatching methods is uncertain. Good for pattern recognitionapplications
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
38/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Branch and bound for graph matching
2010
2011
2013
2015
2017
2018
2019
2015
Anytime branch and bound[Abu-Aisheh et al., 2016]
Quick first solution:Depth-first search[Abu-Aisheh et al., 2015]
Parallel branch and bound[Abu-Aisheh et al., 2018].Work-stealing strategy tobalance the load.
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
39/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Mathematical programming for graph matching
2010
2011
2013
2015
2017
2018
2019
2017
ILP for graph matching (F1,F2, F3):[Lerouge et al., 2017]
Linearization of thequadratic problem
Edge matching variables (Z)
sV (i , k) ∗ Yi ,k
sE (ij , kl) ∗ Yi ,k ∗ Yj ,l
sV (j , l) ∗ Yj ,l
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
39/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Mathematical programming for graph matching
2010
2011
2013
2015
2017
2018
2019
2017
ILP for graph matching (F1,F2, F3):[Lerouge et al., 2017]
Linearization of thequadratic problem
Edge matching variables (Z)
sV (i , k) ∗ Yi ,k
sE (ij , kl) ∗ Zij ,kl
sV (j , l) ∗ Yj ,l
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
40/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Mathematical programming for graph matching
New constraints should be addedto ensure that if 2 edges arematched (Zij ,kl = 1) then relatednodes are matched too.Topological constraints:
Zij ,kl ≤ Yi ,k ∀(ij , kl) ∈ E1 × E2
Zij ,kl ≤ Yj ,l ∀(ij , kl) ∈ E1 × E2
sV (i , k) ∗ Yi ,k
sE (ij , kl) ∗ Zij ,kl
sV (j , l) ∗ Yj ,l
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
41/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Mathematical programming for graph matching
2010
2011
2013
2015
2017
2018
2019
2017
1 Local Branching Heuristic: LocBra [Darwiche et al., 2018]2 The goal is to intelligently explore the solution space
By taking advantage of an ILP and a mathematical solver(Cplex)
3 It is an iterative process starting from an initial solution (x0)
4 x is a vector of binary variables grouping Y and Z
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
42/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Mathematical programming for graph matching
Local search: Search for an improving solution inside a smallregion of the solution space.
Neighborhood: no more than k variables should be changedbetween the new solution and x0.This neighborhood definition is a new constraint in the ILP
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
43/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Mathematical programming for graph matching
No improved solutions found → Diversification operator for GM:
Goal: Helps skipping a local optimum.
To change the region, change important variables
Important variables=a high impacts on the objective functionvalue.
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
43/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Mathematical programming for graph matching
No improved solutions found → Diversification operator for GM:
Goal: Helps skipping a local optimum.To change the region, change important variables
Important variables=a high impacts on the objective functionvalue.
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
44/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Deadlock 2: How to reduce the number of graphcomparisons?
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
45/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Fused graph matching and KNN problems for classification
2010
2011
2013
2015
2017
2018
2019
2017
Problem: Solve the KNN problemfor graphs
Issue: Many graph comparisonsand the process is slow
Answer: Merge GM and KNN in asingle problem[Abu-Aisheh et al., 2017]
Why: Global reasoning instead ofconsidering each comparisonindependently
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
46/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Fused graph matching and KNN problems for classification
(GM+KNN):
The search space is organized as a tree
First floor: a tree node is a graph comparison
Each graph comparison is an instance of graph matching
It can be solved by a Branch and Bound method
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
47/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Fused graph matching and KNN problems for classification
Use case: We are looking for the nearest neighbor1 Compare Gq with G1: S(Gq,G1) = 4
2 Start to compare Gq with G2 and use S(Gq,G1) = 4 as alower bound to cut branches.
3 From the first floor, we estimate that S(Gq,G2) ≤ 24 G1 is a better neighbor, no need to explore fully the sub-tree
(Gq,G2)5 Avoid (full) comparisons of very dissimilar graphs
C is function that evaluates a tree node (over estimate)Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
47/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Fused graph matching and KNN problems for classification
Use case: We are looking for the nearest neighbor1 Compare Gq with G1: S(Gq,G1) = 42 Start to compare Gq with G2 and use S(Gq,G1) = 4 as a
lower bound to cut branches.
3 From the first floor, we estimate that S(Gq,G2) ≤ 24 G1 is a better neighbor, no need to explore fully the sub-tree
(Gq,G2)5 Avoid (full) comparisons of very dissimilar graphs
C is function that evaluates a tree node (over estimate)Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
47/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Fused graph matching and KNN problems for classification
Use case: We are looking for the nearest neighbor1 Compare Gq with G1: S(Gq,G1) = 42 Start to compare Gq with G2 and use S(Gq,G1) = 4 as a
lower bound to cut branches.3 From the first floor, we estimate that S(Gq,G2) ≤ 2
4 G1 is a better neighbor, no need to explore fully the sub-tree(Gq,G2)
5 Avoid (full) comparisons of very dissimilar graphs
C is function that evaluates a tree node (over estimate)Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
47/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Fused graph matching and KNN problems for classification
Use case: We are looking for the nearest neighbor1 Compare Gq with G1: S(Gq,G1) = 42 Start to compare Gq with G2 and use S(Gq,G1) = 4 as a
lower bound to cut branches.3 From the first floor, we estimate that S(Gq,G2) ≤ 24 G1 is a better neighbor, no need to explore fully the sub-tree
(Gq,G2)
5 Avoid (full) comparisons of very dissimilar graphs
C is function that evaluates a tree node (over estimate)Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
47/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Fused graph matching and KNN problems for classification
Use case: We are looking for the nearest neighbor1 Compare Gq with G1: S(Gq,G1) = 42 Start to compare Gq with G2 and use S(Gq,G1) = 4 as a
lower bound to cut branches.3 From the first floor, we estimate that S(Gq,G2) ≤ 24 G1 is a better neighbor, no need to explore fully the sub-tree
(Gq,G2)5 Avoid (full) comparisons of very dissimilar graphs
C is function that evaluates a tree node (over estimate)Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
48/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Deadlock 3: How to learn graph matching?
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
49/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Learn graph matching for classification
2010
2011
2013
2015
2017
2018
2019
2018
Parametrized graphmatching[Raveaux et al., 2017]
Learning graph matching forclassification[Martineau et al., 2018]
sV (i , k)
sE (ij , kl)
sV (j , l)Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
49/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Learn graph matching for classification
2010
2011
2013
2015
2017
2018
2019
2018
Parametrized graphmatching[Raveaux et al., 2017]
Learning graph matching forclassification[Martineau et al., 2018]
sV (i , k).βk
sE (ij , kl).βkl
sV (j , l).βlRomain Raveaux HABILITATION A DIRIGER DES RECHERCHES
50/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Learn graph matching for classification
2010
2011
2013
2015
2017
2018
2019
2018
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
50/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
Overview of research topicsProblems and state of the artMy contributions
Learn graph matching for classification
2010
2011
2013
2015
2017
2018
2019
2018
Parameters β are learned by gradient descent.Weakly supervised learning of (discriminative) graph matching
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
51/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Conclusions and perspectives
1 Curriculum vitæ and pedagogical activities
2 Scientific activities
3 Research on graph matching and classification
4 Conclusions and perspectives
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
52/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Take a step back on graph matchingFrom the operations research side
NP-hard problem
No single method can effectively address all instances.
Computer vision and pattern recognition:
low computational time usually dominates the optimalityguarantees.
Complementary of the methods
There is a need to combine heuristics
Select an heuristic according to the instance at hand
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
53/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Take a step back on graph matchingFrom the machine learning viewpoint
Similarity functions are crucial
Learning node/edge embedding, learning similarity functions
To reach a specific objective (the user need).
Good similarity functions
allow to easily differentiate between vertices/edges
can make the problem easier to solve.
Node embedding integrating topological information
can help to recast the quadratic problem to linear assignmentproblem
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
53/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Take a step back on graph matchingFrom the machine learning viewpoint
Similarity functions are crucial
Learning node/edge embedding, learning similarity functions
To reach a specific objective (the user need).
Good similarity functions
allow to easily differentiate between vertices/edges
can make the problem easier to solve.
Node embedding integrating topological information
can help to recast the quadratic problem to linear assignmentproblem
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
53/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Take a step back on graph matchingFrom the machine learning viewpoint
Similarity functions are crucial
Learning node/edge embedding, learning similarity functions
To reach a specific objective (the user need).
Good similarity functions
allow to easily differentiate between vertices/edges
can make the problem easier to solve.
Node embedding integrating topological information
can help to recast the quadratic problem to linear assignmentproblem
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
54/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Take a step back on graph matchingGraph matching for graph classification
Question: What is the meaning of graph matching for graphclassification?
GM imposes node assignment constraints
GM brings constraints to the learning problem:
Do constraints act like a regularization term to bettergeneralize on unseen data?
Are constraints useful to reduce the number of training data?
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
54/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Take a step back on graph matchingGraph matching for graph classification
Question: What is the meaning of graph matching for graphclassification?
GM imposes node assignment constraints
GM brings constraints to the learning problem:
Do constraints act like a regularization term to bettergeneralize on unseen data?
Are constraints useful to reduce the number of training data?
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
54/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Take a step back on graph matchingGraph matching for graph classification
Question: What is the meaning of graph matching for graphclassification?
GM imposes node assignment constraints
GM brings constraints to the learning problem:
Do constraints act like a regularization term to bettergeneralize on unseen data?
Are constraints useful to reduce the number of training data?
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
54/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Take a step back on graph matchingGraph matching for graph classification
Question: What is the meaning of graph matching for graphclassification?
GM imposes node assignment constraints
GM brings constraints to the learning problem:
Do constraints act like a regularization term to bettergeneralize on unseen data?
Are constraints useful to reduce the number of training data?
These open questions are important to legitimate graph matchingfor graph classification.
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
55/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Graph matching what do we need?
From the operations research side:
1 Fast, scalable and accurate heuristics are always welcome forcomputer vision and pattern recognition communities.
From the machine learning viewpoint:
1 Low complexity (near linear time)
2 Graph matching methods suited to execution on GPU
3 Learning problems are often solved by gradient descent sodifferentiable methods are wanted → link with robustoptimization?
4 Avoid the storage of similarity matrices (|V1|.|V2| × |V1|.|V2|)
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
55/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Graph matching what do we need?
From the operations research side:
1 Fast, scalable and accurate heuristics are always welcome forcomputer vision and pattern recognition communities.
From the machine learning viewpoint:
1 Low complexity (near linear time)
2 Graph matching methods suited to execution on GPU
3 Learning problems are often solved by gradient descent sodifferentiable methods are wanted → link with robustoptimization?
4 Avoid the storage of similarity matrices (|V1|.|V2| × |V1|.|V2|)
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
55/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Graph matching what do we need?
From the operations research side:
1 Fast, scalable and accurate heuristics are always welcome forcomputer vision and pattern recognition communities.
From the machine learning viewpoint:
1 Low complexity (near linear time)
2 Graph matching methods suited to execution on GPU
3 Learning problems are often solved by gradient descent sodifferentiable methods are wanted → link with robustoptimization?
4 Avoid the storage of similarity matrices (|V1|.|V2| × |V1|.|V2|)
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
55/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Graph matching what do we need?
From the operations research side:
1 Fast, scalable and accurate heuristics are always welcome forcomputer vision and pattern recognition communities.
From the machine learning viewpoint:
1 Low complexity (near linear time)
2 Graph matching methods suited to execution on GPU
3 Learning problems are often solved by gradient descent sodifferentiable methods are wanted → link with robustoptimization?
4 Avoid the storage of similarity matrices (|V1|.|V2| × |V1|.|V2|)
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
55/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Graph matching what do we need?
From the operations research side:
1 Fast, scalable and accurate heuristics are always welcome forcomputer vision and pattern recognition communities.
From the machine learning viewpoint:
1 Low complexity (near linear time)
2 Graph matching methods suited to execution on GPU
3 Learning problems are often solved by gradient descent sodifferentiable methods are wanted → link with robustoptimization?
4 Avoid the storage of similarity matrices (|V1|.|V2| × |V1|.|V2|)
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
56/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Short term perspectives
Future master students (low hanging fruits)
1 (KNN + GM) solved by the local branching heuristic
2 Parametrized GM as an input layer of a MLP
3 ILP for the MCS problem
Future PhD students
1 Learning graph matching: hierarchical feature learning (GraphNeural Network) + a combinatorial layer (graph matchingmethod)
2 Learning graph matching: learn to branch in a branch andbound (reinforcement learning)
3 Structured classification: graph matching as a loss function
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
57/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Long term perspectives
To be curious:1 Cross fertilization: OR, ML: LOR project2 Inspired by other problems:
From computer vision: CRF MAP-inferenceFrom OR: TSP, scheduling problems
Fundamental:
On the relation between graph matching and OptimalTransport (OT) (Gromov-Wasserstein distance).
Applications:1 Graph matching for multiple object tracking in videos, table
comparisons in documents, ...2 Graph matching for unsupervised domain adaptation3 Benchmarks
Still a lot of interesting work to do ...
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
57/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Long term perspectives
To be curious:1 Cross fertilization: OR, ML: LOR project2 Inspired by other problems:
From computer vision: CRF MAP-inferenceFrom OR: TSP, scheduling problems
Fundamental:
On the relation between graph matching and OptimalTransport (OT) (Gromov-Wasserstein distance).
Applications:1 Graph matching for multiple object tracking in videos, table
comparisons in documents, ...2 Graph matching for unsupervised domain adaptation3 Benchmarks
Still a lot of interesting work to do ...Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
58/58
Curriculum vitæ and pedagogical activitiesScientific activities
Research on graph matching and classificationConclusions and perspectives
ConclusionsShort term perspectivesLong term perspectives
Thank you
Any questions?
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
0/0
Abu-Aisheh, Z., Raveaux, R., and Ramel, J. (2016).Anytime graph matching.Pattern Recognition Letters, 84:215–224.
Abu-Aisheh, Z., Raveaux, R., and Ramel, J. (2017).Fast nearest neighbors search in graph space based on abranch-and-bound strategy.In [Foggia et al., 2017], pages 197–207.
Abu-Aisheh, Z., Raveaux, R., Ramel, J., and Martineau, P.(2015).An exact graph edit distance algorithm for solving patternrecognition problems.In Marsico, M. D., Figueiredo, M. A. T., and Fred, A. L. N.,editors, ICPRAM 2015 - Proceedings of the InternationalConference on Pattern Recognition Applications and Methods,
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
0/0
Volume 1, Lisbon, Portugal, 10-12 January, 2015., pages271–278. SciTePress.
Abu-Aisheh, Z., Raveaux, R., Ramel, J., and Martineau, P.(2018).A parallel graph edit distance algorithm.Expert Syst. Appl., 94:41–57.
Cho, M., Alahari, K., and Ponce, J. (2013).Learning graphs to match.In IEEE International Conference on Computer Vision, ICCV2013, pages 25–32.
Cortes, X., Serratosa, F., and Serratosa, F. (2016).Learning graph matching substitution weights based on theground truth node correspondence.IJPRAI, 30(2).
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
0/0
Darwiche, M., Conte, D., Raveaux, R., and T’Kindt, V.(2018).A local branching heuristic for solving a graph edit distanceproblem.Computers and Operations Research.
Foggia, P., Liu, C., and Vento, M., editors (2017).Graph-Based Representations in Pattern Recognition - 11thIAPR-TC-15 International Workshop, GbRPR 2017, Anacapri,Italy, May 16-18, 2017, Proceedings, volume 10310 of LectureNotes in Computer Science.
Gold, S. and Rangarajan, A. (1996).A graduated assignment algorithm for graph matching.IEEE Transactions on Pattern Analysis and MachineIntelligence, 18(4):377–388.
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
0/0
Lerouge, J., Abu-Aisheh, Z., Raveaux, R., Heroux, P., andAdam, S. (2017).New binary linear programming formulation to compute thegraph edit distance.Pattern Recognition, 72:254–265.
Martineau, M., Raveaux, R., Conte, D., and Venturini, G.(2018).Learning error-correcting graph matching with a multiclassneural network.Pattern Recognition Letters.
Nowak, A., Villar, S., Bandeira, A. S., and Bruna, J. (2017).A note on learning algorithms for quadratic assignment withgraph neural networks.CoRR, abs/1706.07450.
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES
0/0
Raveaux, R., Martineau, M., Conte, D., and Venturini, G.(2017).Learning graph matching with a graph-based perceptron in aclassification context.In [Foggia et al., 2017], pages 49–58.
Romain Raveaux HABILITATION A DIRIGER DES RECHERCHES