Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Face Recognition Memory with NLP backend of Social Robot for HRI
8th June 2018, Friday
Nidhi Mishra
Project Officer, IMI
IMI Research Seminar
1
Contents• Background of Robots
• Social Robots
• Motivation
• Proposed Model
• Algorithm
• Conclusion
• Future Work
• References
2
3
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Industrial Service~27.5M 2
~1.7M
- https://assets.kpmg.com/content/dam/kpmg/pdf/2016/06/social-robots.pdf
4
70%
17%
9%2%2%
Robots
Cleaning work Hobby Research Lawn-mowing Others
Hobby
Non-Social Social
Research
Non-Social Social
- https://assets.kpmg.com/content/dam/kpmg/pdf/2016/06/social-robots.pdf
Robot roll-call
5
- https://assets.kpmg.com/content/dam/kpmg/pdf/2016/06/social-robots.pdf
SocialHumanoid Robot
6
Aging population
7
• The world is aging at a rapid rate and by 2030 there will be 34 nations where more than 20% of the population is over 65. This has broad implications for economic growth and immigration trends.
• With an older population that works less, support and dependency ratios get out of whack.
• Robots will perform many elder-care tasks within a decade.
- http://money.cnn.com/interactive/news/aging-countries/
8
Robot as Companion
Companion robots are robots that can make themselves useful, carry out various type of tasks to assist humans in a domestic environment. They should behave socially and interact with a socially acceptable manner with humans [Dautenhahn et al., 2005]
9
Challenges
Several research have suggested characteristics that a social robot should visibly possess [Fong et al., 2003, Dautenhahn, 2007b, Heerink, 2010]:
• Express and perceive emotions
• Communicate with high level dialogue
• Learn and recognise models of other agents
• Establish and maintain social relationships
• Use natural cues of communication
• Exhibit distinctive personality and character
• Learn and develop social competencies
10
Have a MemoryRecognize it’s CompanionUnderstand Human Language Naturally
11
12
Proposed Solution
Real Time face Recognition and Training
13
OpenFace
• Detect faces with a pre-trained models from dlib or OpenCV.
• Transform the face for the neural network. This repository uses dlib's real-time pose estimation with OpenCV's affine transformation to try to make the eyes and bottom lip appear in the same location on each image.
• Use a deep neural network to represent (or embed) the face on a 128-dimensional unit hypersphere.
• Apply classification technique to the features to complete recognition task.
- https://cmusatyalab.github.io/openface/
14
15
16
17
18
Extracting NLP features
• Part of speech (reads a sentence and assigns parts of speech to each word)
• Named entity recognition (labels sequences of words in a text which are the names of things, such as person and company names, or gene and protein names.)
• Find dependencies (representation of grammatical relations between words in a sentence)
• Coreference ( the task of finding all expressions that refer to the same entity in a text.)
-https://stanfordnlp.github.io/CoreNLP/index.html
19
Sentence as input :-
1. Nidhi is working at NTU Singapore.
2. She works on Nadine robot.- http://nlp.stanford.edu:8080/corenlp/process
20
WordNet
• WordNet is lexical database for English Language based conceptual look-up.
• Organizes lexical information in terms of word meanings rather than word form.
How wordNet can help us?One person can say a sentence in different ways using different words whose meaning are same.Example : I am going to downtown this weekend.
I am going to downtown this sunday.I am going to downtown this week.
WordNet can help me find similar words for one word. -http://www.nltk.org/howto/wordnet.html
21
22
23
https://stanfordnlp.github.io/CoreNLP/index.htmlFabrizio Sebastiani. Machine Learning in Automated Text Categorization. ACM Computing Surveys, 34(1):1–47, Mars
Conclusion
• For robots, it is crucial that they possess human-like social interaction skills. It is vital for a robot to know who it is talking with and remember facts about the human companion.
• Aims to explore the possibility of improving human-robot interaction(HRI) by exploiting natural language resources and using natural language processing (NLP) methods.
24
Future Plans
• An illustration through a real case of this model.
• Compare the model using standard database and a new database for HRI applications.
25
Thank you!
Q & A
26