View
353
Download
0
Category
Tags:
Preview:
Citation preview
Tips and Trikes from
Real Case Studies
Matteo Valoriani
mvaloriani@gmail.com
@MatteoValoriani
#RealSense
PhD at Politecnico of Milano
CEO of Fifth Element
Speaker and Consultant
Intel Software Innovator: Perceptual Computing
Microsoft MVP on Kinect
Who I am…
2
mvaloriani@gmail.com
@MatteoValoriani
Intel® RealSense™ Hands-On Lab - Milan
3Intel® RealSense™ Hands-On Lab - Milan
Follow me on
Twitter or the
Kitten gets it:
@MatteoValoriani
4Intel® RealSense™ Hands-On Lab - Milan
You have to be a magician…
or at least a good illusionist
Understands 4 basic types of input
6Intel® RealSense™ Hands-On Lab - Milan
Categories of
Input
Capabilities Features
Hands • Hand and Finger Tracking
• Gesture Recognition
• 22-point Hand and Finger Tracking
• 9 static and dynamic mid-air gestures
Face • Face Detection and
Tracking
• Multiple Face Detection and tracking
• 78-point Landmark Detection (facial features)
• Emotion Recognition (7 emotions, coming post-Beta)
• Pulse Estimation
• Face Recognition (Coming post-beta)
Speech • Speech Recognition • Command and Control
• Dictation
• Text to Speech
Environment • Segmentation
• 3D Scanning
• Augmented Reality
• Background Removal
• 3D Object / Face / Room Scanning (Coming post-beta)
• 2D/3D Object Tracking
• Scene Perception (coming post-beta)
Competitive
technologies focus
on a living-room
experience or a
sub-set of Intel
RealSense
technology
features
Designed for close-range interactions
8Intel® RealSense™ Hands-On Lab - Milan
120 cm
Intel®RealSense™
3D camera
56°(v) x 72° (v)
20 cm
Vertical Rages
10Intel® RealSense™ Hands-On Lab - Milan
60cm
58 c
m
120cm
56°
20cm
17 c
m
70cm 35cm73 c
m
Effective Range
Gestures Range
Effective Range
3D Facial Traking Range
2D Facial Traking Range
115cm
Horizontal Rages
12Intel® RealSense™ Hands-On Lab - Milan
60cm
87cm
120cm
72°
20cm
24cm
Effective Range
Gestures Range
Effective Range
3D Facial Traking Range
2D Facial Traking Range
170cm
70cm 35cm108cm
50cm
Capture Volumes
13Intel® RealSense™ Hands-On Lab - Milan
The user is performing a hand gesture outside of the capture volume.
The camera will not see this gesture
RealSense Camera use IR light and Sunlight can blind the
camera!!!
• Check exposition during all day
• Verify that there isn’t direct light on the camera
Indoor/Outdoor
15Intel® RealSense™ Hands-On Lab - Milan
RealSense isn’t a Rugged device:
• Check temperatures (+3/33°)
• Check humidity
Indoor/Outdoor (2)
16Intel® RealSense™ Hands-On Lab - Milan
Feedback, feedback, feedback,…
20Intel® RealSense™ Hands-On Lab - Milan
View of user:
• User Viewport
• User Overlay
… where actions performed for some other purpose or unconscious signs are interpreted in order to influence/improve/facilitate the actors' future interaction or day-to-day life (from Alan Dix)
• The interaction is not purposeful from the person side, but it is designed “to happen”
• It “happens” in relation to signs which are not done for that (body temperature, unconscious reactions such as blink rate, or unconscious aspects of activities such as typing rate, vocabulary shifts (e.g. modal verbs), actions done for other purposes, …
• It is designed for people acting
Manage Incidental Interaction
22Intel® RealSense™ Hands-On Lab - Milan
Gestures should have a clear cognitive association with the semantic
functions they perform and the effects they achieve. Intuitiveness can be
enforced by appropriate interface and feedbacks.
The semantics of gestural patterns that belong to everyday life or
common task should be as consistent as possible to their “conventional”
meaning, but also take into account that intuitiveness is strongly
associated with users’ cultural background, general knowledge, and
experience.
Semantic intuitiveness
Gestural communication involves more muscles than keyboard interaction
or speech. Gestural commands must therefore be concise and quick, and
minimize user’s effort and physical stress.
Two types of muscular stress are known: static, the effort required
maintaining a posture for a fixed amount of time; dynamic, related to the
effort required to move a portion of the body through a trajectory.
Minimalize Fatigue
It must be easy for the user to learn how to perform and remember
gestures, minimizing the mental load of recalling movement trajectories
and associated actions.
The learning rate depends on tasks, user experience, skills, as well as the
size of the gesture language (more gestures decrease the learnability
rate).
Favor ease of learning (Learnability) 1/2
The gestures that are most natural, easy to learn and are immediately assimilated by the user are those that belong to everyday life, or involve the least physical effort. These gestures should be associated to the most frequent interactions.
Complex gestures can be more expressive and give more control, but have a higher learnability burden.
Hence there is clearly a tension between design requirements, among which a compromise must be made: naturalness of gestures, minimum size of the gesture language, expressiveness and completeness of the gesture language.
Favor ease of learning (Learnability) 2/2
Users can perform unintended gestures, i.e., movements that are not
meant to communicate with the system they are interacting with.
The “immersion syndrome” occurs if every movement is interpreted by
the system, whether or not it was intended, and may determine
interaction effects against the user’s will.
Intentionality (Immersion Syndrome) 1/2
The designer must identify well-defined means to detect the intention of
the gestures, as distinguishing useful movements from unintentional ones
is not easy.
Body tension and non-relaxed posture of users can be used to make
explicit the user intention to start interaction, issue a command, or
confirm a choice.
The tense period should be short to not generate fatigue.
Intentionality (Immersion Syndrome) 2/2
Appropriate feedback indicating the effects and correctness of
the gesture performed is necessary for successful interaction,
and to improve the user's confidence in the system.
Not-self-revealing
https://software.intel.com/sites/campaigns/perceptualshowcase/
winners.htm
http://www.intel.com/content/www/us/en/architecture-and-
technology/realsense-jim-parsons-flight-attendant-to-mars.html
Intel® Perceptual Computing Showcase
33Intel® RealSense™ Hands-On Lab - Milan
34
• The robot is built using Legos* Mindstorm
• A standard Ultrabook is place on top
• It acts as the brains
• As well as the interface
• Intel 3D Cameras are used to capture the
environment & steer the robot around
• It also allows for some facial & gesture recognition
Rover the
Self-Driving Car
Intel® RealSense™ Hands-On Lab - Milan
Space Between
• Used both Gesture & Voice commands in this
Puzzle Oriented Game about teaching a creature to
survive in a virtual world
• Used Unity* 3D as its game engine
35Intel® RealSense™ Hands-On Lab - Milan
3D Head Scanner
• Rapidly (about a minute) does a full scan of
the user’s head for avatar creation or other
uses such as fashion & beauty applications
• In the original app, users could change hair
styles, skin complexion, etc.
• Built as a Unity* project, it could be used by
other applications & games for a wild array of
different usage models
36Intel® RealSense™ Hands-On Lab - Milan
Stargate SG1
Gunship V2
• An evolution of the award
winning Perceptual Computing
game
• Controller-free, immersive UX
• Two-handed, multi-axis camera
and fire controls
• Menu and in-game voice
commands
37Intel® RealSense™ Hands-On Lab - Milan
Redwall: The
Warrior Reborn
• Facial landmark mapping to
character avatars
• Fully controller free UI
• Expression sensitive NPC
interactions.
38Intel® RealSense™ Hands-On Lab - Milan
Recommended