Page 1
IDS London
Auto-Tagging, Search and Image Synthesis
- Mayank Sagar
Page 2
In brief
– Auto-tagging or keywording
– Search with computer vision
– Semantic Labelling
– Object Detection and Analysis
– IDS AutoColourise and Style Transfer
– the future with Image synthesis
© [email protected]
IDS London
Page 3
• A brief history of search
– Build an index
– Add content
• Finding pictures
– Keywording or Tagging
– Too few – images get missed
– Too many words makes keywording meaningless
© [email protected]
IDS London
Page 4
© [email protected]
IDS London
Page 5
• Guessing game between researcher & keyworder
• AI driven keywording
© [email protected]
IDS London
Page 6
© [email protected]
IDS London
Page 7
• Major players
– Clarifai
– AWS Rekognition
– Imagga
• RESTful APIs with JSON formatting
• Translations
• Variables – confidence score
• Custom training
© [email protected]
IDS London
Page 8
Visual Search
• Use an image to find the same or duplicate image – reverse image search
• Use an image to find similar images
© [email protected]
IDS London
Page 9
© [email protected]
IDS London
Page 10
© [email protected]
IDS London
Page 11
© [email protected]
IDS London
Page 12
Enter Machine Learning..
• Train a network to identify presence of features of interest within images
• Object detection and Analysis
© [email protected]
IDS London
Page 13
Object Detection and Analysis
© [email protected]
IDS London
Page 14
Object Detection and Analysis
© [email protected]
IDS London
Page 15
Machine Learning
• Train a network to identify presence of features of interest within videos
• Logos
• Objects
© [email protected]
IDS London
Page 16
• Semantic labelling
• Colour histogram matching
© [email protected]
IDS London
Page 17
• Semantic labelling = What’s in the picture
• Colour histogram matching
© [email protected]
IDS London
Page 18
• Semantic labelling = What’s in the picture
• Colour histogram matching = How it looks
© [email protected]
IDS London
Page 19
© [email protected]
IDS London
Page 20
© [email protected]
IDS London
Page 21
© [email protected]
IDS London
Page 22
© [email protected]
IDS London
Page 23
© [email protected]
IDS London
Page 24
IDS AutoColourise
© [email protected]
IDS London
Page 25
IDS AutoColourise
© [email protected]
IDS London
Page 26
IDS AutoColourise
© [email protected]
IDS London
Page 27
Style transfer
© [email protected]
IDS London
Page 28
Style transfer
© [email protected]
IDS London
Page 29
Style transfer
© [email protected]
IDS London
Page 30
Style transfer
© [email protected]
IDS London
Page 31
Image Synthesis
• GANs – Generative Adversarial Networks
• Making stuff from basic inputs
• Making stuff from nothing
© [email protected]
IDS London
Page 32
© [email protected]
IDS London
Page 33
© [email protected]
IDS London
Page 34
© [email protected]
IDS London
Page 35
© [email protected]
IDS London
Page 36
© [email protected]
IDS London
Page 37
© [email protected]
IDS London
Page 38
Making stuff from basic inputs
© [email protected]
IDS London
Page 39
Making stuff from basic inputs
© [email protected]
IDS London
Page 40
Making stuff from basic inputs
© [email protected]
IDS London
Page 41
Making stuff from nothing
© [email protected]
IDS London
Page 42
© [email protected]
IDS London
Page 43
© [email protected]
IDS London
Page 44
© [email protected]
IDS London
Page 45
© [email protected]
IDS London
Page 46
© [email protected]
IDS London
Page 47
IDS AutoTag
religion, art, furniture,
painting, god,
decoration, gold,
church, interior,
sculpture, spirituality,
temple, holy, prayer,
symbol, people, worship, saint, ancient
© [email protected]
IDS London
Page 48
Thanks to the team at IDS
© [email protected]
IDS London
Page 49
who can’t be with us in person
© [email protected]
IDS London
Page 50
as they don’t actually exist
© [email protected]
IDS London
Page 51
but were synthesised by a neural network
© [email protected]
IDS London
Page 52
IDS London
Mayank Sagar
[email protected]