Upload
sirris
View
260
Download
0
Embed Size (px)
DESCRIPTION
Astute symposiume 10/10/2013 - Smart sensors userstate
Citation preview
A PROBABILISTIC MODEL FOR
COGNITIVE-AFFECTIVE USER STATE
AWARENESS
ARTEMIS JOINT UNDERTAKING
GRANT AGREEMENT: 269334
PROPOSAL ACRONYM: ASTUTE
Astute project 2
INTRODUCTION
Higher data flow
Complex services
Embedded Systems are becoming more and more complex
Different applications
Data
Information
Advice
Actions
Support
Confirm
Error
WHY?
Astute project 3
INTRODUCTION
ASTUTE project aims to improve usability of embedded systems by using user state and context situation capture to provide pro-active decision
support via multi-modal interfaces.
Pro-active decision support system based on human centered design able to support user intentions while keeping him in control.
CONTEXT CAPTURE
CONTEXT MODELING
ENGINE
PROACTIVE DECISION ENGINE
MULTI-MODAL HMI
Astute project 4
USER AWARENESS
PHYSICAL ENVIRONMENT
CONDITIONS
PHYSIOLOGIC MEASURES
PROCESSING DATA INPUTS USER
STATE
•Information Priority
•Decision Support
•Data delivery
•Keep in control
Astute project 5
USER STATE
USER STATE
PHYSICAL ENVIRONMENT
CONDITIONS
SENSOR NETWORKS
EXTERNAL DATABASES
VISUAL ODOMETRY
SPECIFIC MEASUREMENTS
PHYSIOLOGIC MEASURES
ECG
EEG
USER FEEDBACK
MOTION
ARTIFICIAL VISION
BIOMETRICS
Astute project 6
USER STATE
EEG:
Electroencephalography is the measurement of electrical activity resulting from ionic currents flows in brain neurons using multiple electrodes placed on the scalp.
It is commonly used in medicine for diagnostic applications, like epilepsy, encephalopathies or sleep disorders, by analyzing its spectral content.
Further applications include EEG average analysis for cognitive sciences by analyzing response to time-locked events and stimulus.
STRESS FATIGUE
FATIGUE
CONCENTRATION
RELAXATION
DISEASES
Astute project 7
USER STATE
ECG:
Electrocardiography is the transthoracic measurement of electrical activity in heart using different electrodes attached to user’s skin.
It is commonly used in medicine to measure heartbeats rate, size and position of the different heart chambers and any effect of external source on heart.
Although ECG information is limited to physiological user status, it is useful to complement EEG data to complement obtained information increasing system performance and reliability.
How this information is merged?
Astute project 8
RATIONALE & MOTIVATION
• Cognitive-affective states are relevant in the realisation of tasks that:
– Manage a large volume of information in the interface/system/process – are a cognitive challenge
– Are critical (urgency, safety, health, sports)
– Involve people (human resources, leadership, coaching, social and personal relationships)
• Availability of information on user state facilitates interface/system/process pro-activeness, which is accomplished via decision support built on top of data-/knowledge-based models, providing:
– Alarms, feedback
– Recommendations for adapting the interface/system/process
Astute project 9
RATIONALE & MOTIVATION
• Present work provides a description of research and development of a user state diagnostic system, within the broader context of the European Artemis ASTUTE project ASTUTE aims to develop advanced and innovative pro-active HMI
supported by reasoning engine system, for improving the way the human being deals with complex and huge information quantities in different operative conditions and contexts.
• A number of previous projects generally used a limited range of sensors network, mainly focused on autonomous psycho-physiological information, and used concrete context scenarios
• More technical effort may be done to incorporate measures of brain activity, and thus to delineate a full picture of brain-body reaction
• A probabilistic model has been developed, given its capability to handle uncertainty in sources information and inference, and sound mathematical framework.
Astute project 10
OVERVIEW
Sensors
User Profile Context
Web services
Cloud App Mobile
User state
Mobile Cloud App
Astute project 11
DATA CAPTURE
EEG raw data Frequency bands
α, β, θ,... Calibration & normalisation
Heart rate
User’s profile
Calibration & normalisation
To user state model
Web services connectivity
Astute project 12
OUR COGNITIVE MODEL
• Three user states that are relevant in working environments and safety-critical tasks: – stress, mental workload and fatigue
• A fourth state, namely inaptitude, is derived as a combination of the three aforementioned user-states (stress, mental workload and fatigue): – this is input to decision support, where recommendation for assistance will be
given
Astute project 13
• Input used for diagnosis of user state comes from – brain activity (EEG) and heart activity sensors
• This is complemented by selected – predictive factors from context (context complexity, task workload) and – user profile (experience, age, fitness), – extracted from each user-case
OUR COGNITIVE MODEL
Astute project 14
OUR COGNITIVE MODEL
p(Symptom2 | Disease1, Disease2) Symptom2
Disease1 Disease2 False True
False False 80 % 20 %
False True 40 % 60 %
True False 30 % 70 %
True True 20 % 80 %
As the number of parents of a node increase, conditional probability tables (CPT) become larger. Limit relations between nodes if there’s conditional independence Use canonical distributions
NoisyOR is used in our model Adapt distributions with statistical analysis
of a large amount of training data
Astute project 15
OUR COGNITIVE MODEL
• This first model can be extended with “contrasts”
– for verifying the need for assistance via the state inaptitude.
– In the end, this strategy improves the decision reliability
Astute project 16
OUR COGNITIVE MODEL
• Assistance recommendation is provided – alarms, voice, warning
messages, etc.
– assistance is based on the expected utility in order to advice the user
Astute project 17
USAGE EXAMPLE
Astute project 18
USAGE EXAMPLE
Astute project 19
SCHEMATIC ARCHITECTURE
Astute project 20
FUTURE WORK
• additional sensors – for example to complement existing information or to cope with the
unavailability of specific sensors in particular settings
• further input parameters extracted from existing sensors • improved pre-processing and fine-tuning of features can enhance
robustness • new user’s cognitive and affective states • further exploitation of user’s state contrasts and assistance • exploring a dynamic version for anticipation or prediction of the
user’s state • adaptability to users via training procedures and to improve the
prediction capacity of the cognitive model • mobility and user profiling and personalization are key to our
system and deserve significant attention
Astute project 21
CONCLUSIONS
• we’ve shown present and future work within the framework of ASTUTE project
• emphasis is on providing an integrated solution to monitor and adapt the user’s state to the task demands in complex contexts – to design and implement a probabilistic cognitive model that is
predicting the user’s state based on complex context use-cases – a set of sensors is diagnosing such user’s states based of brain
and heart rate evidence
• this solution is partially overcome by other projects, however we add value in increasing the set of sensors, range of user’s states in real and intensive scenarios
Astute project 22
THANKS FOR
YOUR ATTENTION!