View
5.492
Download
1
Embed Size (px)
DESCRIPTION
Brain and Body Interfaces (BBI) were discussed in this workshop. Its goal was to provide a platform for creating synergies between two related and emerging HCI disciplines (PC and BCI). Find out more at the workshop website: http://www.eecs.tufts.edu/~agirou01/workshop/
Citation preview
BBB Workshop Overview
Audrey Girouard, Erin Treacy Solovey,Regan Mandryk, Desney Tan,
Lennart Nacke, Robert J.K. Jacob
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
BBB Workshop: Bringing Together BCI& Physiological Computing
3
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Goals
1. Provide a platform for creating synergies between two related and emerging HCI disciplines (PC and BCI).
2. Kick-start BBB research in HCI by identifying key research questions and application areas
3. Enhancing HCI methodologies by adding these new methods and techniques to the toolbox of HCI researchers
4
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
About Us
5
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
About you
• 25 Participants
• 9 Brain papers• 19 Body papers
• X EEG papers• Y fNIRS papers• Z EMG papers• A GSR• B Eye tracking
6
32%
68%
Brain Body
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
7
<on your tables>
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
What terms/acronyms should we use to describe our work?
Psychophysiological computing seems a bit long… BBB?BBI?
Other suggestions?
8
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Today’s Schedule
9
09:00-10:30 Introduction of organizers & participants, CHI madness style
10:30-11:00 Break
11:00-12:30 Group Discussion (assign groups + topic)
12:30-14:00 Lunch
14:00-15:30 Small group discussion (1h of discussion + 30 minutes of reporting)
15:30-16:00 Break
16:00-17:00 Report from small groups, recap, group discussion – barriers to publication in HCI
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Staying connected
10
Website http://www.eecs.tufts.edu/~agirou01/workshop/
Twitter http://twitter.com/bbbCHI2010
Twitter Hashtag
#bbbCHI2010
Facebook http://www.facebook.com/home.php#/pages/Atlanta-GA/Brain-Body-and-Bytes-Psychophysiological-User-Interaction-at-CHI-2010/136581171786
Flickr http://www.flickr.com/groups/bbbchi/
Slide Share http://www.slideshare.net/group/brain-body-and-bytes-psychophysiological-user-interaction
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
And now… 2 Minute Introductions
• About your research:– Application or domain space? (e.g. video games)
– Signal you are measuring? (e.g. fNIRS)
– Device/hardware you are using? (e.g. ISS, Inc)
• What would you like to get out of workshop?– What problems are you facing?– What do you see in common with others here?– What device would you like to know more about?
11
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
List of Presentations1. Adriane Randolph2. Krzysztof Z. Gajos3. Saraswathi Bellur4. Danny Plass-Oude Bos5. Kai Kuikkaniemi6. Mitchel Benovoy7. Yee Chieh Chew8. Martha Crosby9. Jan Kallenbach10. Eva Oliveira11. Lucia Filgueiras12. Marc Grootjen13. Melody Moore Jackson14. Johann Schrammel15. Daniel Sjölie
12
16. Katarzyna Wac17. Feng Tian18. Veronica Zammitto19. Johanna Octavia20. Bob Wray21. Marjolein van der Zwaag
Brain-Computer Interfaces as Child’s Play: Using Mental Training to Improve
Motor Skills in Children
Adriane B. Randolph, Ph.D.Ashley Ingraham
Kennesaw State University
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline• Research Position
– Examine ties between sensorimotor area and physical ability to uncover a training tool to aid children’s motor skill development
• Pilot Study– 3 able-bodied adults (2 Male, 1 Female)– Acquire baseline Wii tennis score– Conduct mental training with BCI2000 mu training protocol for 5 days– Retest Wii tennis score for improvements– Mixed results
• Future Work– Extend study with 20 able-bodied adults (currently in progress)– Analyze link between training and baseline ability– Test with elementary-aged children
14
Participant Session One Session Five
AZ 98 60
CM 70 64
MW 103 118
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Screening for MuAshley prepares participant to use mental imagery of hand and foot movements to determine his best channels for use in mu BCI training before he plays a match of Wii Tennis.
15
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Adriane B. Randolph, Ph.D.Kennesaw State University BrainLab
(770) 423-6083 [email protected]
http://coles.kennesaw.edu/brainlab/
16
Understanding How to Design ComplexBrain-Controlled Applications
Krzysztof Z. Gajos
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
How to Design Brain-Controlled Applications?• Basic operations
18
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
How to Design Brain-Controlled Applications?
19
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
How to Design Brain-Controlled Applications?
20
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
How to Design Brain-Controlled Applications?• Basic operations
21
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
How to Design Brain-Controlled Applications?• Interaction between primary task and BCI
performance
22
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
How to Design Brain-Controlled Applications?• Error prevention, recovery, and actions
with side effects
23
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Krzysztof [email protected]://www.eecs.harvard.edu/~kgajos/
24
Psychophysiological Responsesto Media Interfaces
S. Shyam SundarSaraswathi Bellur
Media Effects Research Labhttp://www.psu.edu/dept/medialab
Pennsylvania State University
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Biopac MP 35 & MP 150 ACQknowledge software
26
Skin Conductance (EDA) Heart Rate (ECG) Facial EMG Brain Wave (EEG) Eye Movement (EOG)
Application AreasInterface presentationInformation Processing Usability Testing & Evaluation User Experience
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 27
Physiological Measures: Affect, Arousal & Attention
1. EEG: Bipolar recordings Occipital Alpha Blocking
2. Skin Conductance (EDA) Tonic arousal levels
3. ECG: Heart Rate recording
OrientingResponse viaBPM change
4. Facial EMG:Corrugator &Zygomaticrecordings
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Theoretical Frameworks• The Media Equation: Mediated Life = Real Life (Reeves & Nass,
1996). • Structural aspects of media technologies: Automatic or
Controlled processing? Voluntary or Involuntary responses?• Specific cues triggered via interactions with four types of
media affordances, MAIN model (Sundar, 2008)
28
M: MODALITY
I: INTERACTIVITY
A: AGENCY
N: NAVIGABILITY
Animation
Customization
Contingency
PerceptualBandwidth
InformationScent
UGC & Sourceness
Heuristics
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Saras Bellur ([email protected])S. Shyam Sundar ([email protected])
29
Media Effects Research Labhttp://www.psu.edu/dept/medialab
Pennsylvania State University
BCI for Games
Danny Plass-Oude BosBoris Reuderink, Bram van de Laar, Hayrettin Gürkök, Christian Mühl,
Mannes Poel, Anton Nijholt, Dirk HeylenHMI, University of Twente
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Target Group: the general population
31
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
BCI: issues
32
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
BCI: new information, new features
33
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Acceptance: Usability & Realistic Settings
34
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Subjects Users
35
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
[email protected]://hmi.ewi.utwente.nl/~oudebos
36
Designing Biofeedback for Games and Playful Applications
Kai KuikkaniemiHelsinki Institute for Information
Technology
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Emoshooter
38
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Emoshigeru - cinema gamesReal-timePublic displayBiosignal adaptationMassive co-locatedShooter / StrategyPart of Shigeru-games
localwlan
Public displayConnected to game
server
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Presemo – presentation engine
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Emolisten – biosignal sonification
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Research statement
• Social Games and Playful Applications, which Utilize Biosignals
• Grey Area between Adaptation and Biofeedback– Implicit and Explicit Biofeedback
• Biosignal Synchrony (Inter-Personal)
42
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Our team and projects
Professor Marko TurpeinenProfessor Timo SaariIlkka Kosunen, Toni Laitinen, Petri Lievonen, Pauli Ojala
Fun of Gaming (EU FP6) (2006-2009)Enactive Social Media and Games (Aalto) (2009)Emokeitai (Tekes – National Innovation Agency) (2009-2011) Thanks:CKIR / Niklas Ravaja
43
Psychophysiological signal analysis and classification
Mitchel BenovoyCentre for Intelligent Machine, McGill University
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline
• Signals
• Hardware
• Application domains
45
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Signals
46
0.5 1 1.5 2 2.5
x 104
-0.5
0
0.5
1
1.5
Sample
∆C
ond.
( µS
)
1000 2000 3000 4000 5000 6000 7000 8000 9000 10000323436384042
Ref
l.
1000 2000 3000 4000 5000 6000 7000 8000 9000 100005
6
7
Am
p.
1000 2000 3000 4000 5000 6000 7000 8000 9000 1000090
95
100
105
HR
(bpm
)
1000 2000 3000 4000 5000 6000 7000 8000 9000 100001.21.41.61.8
22.2
Exp
.
1000 2000 3000 4000 5000 6000 7000 8000 9000 100000.20.40.60.8
1A
mp.
1000 2000 3000 4000 5000 6000 7000 8000 9000 1000015
20
25
30
RR
(bpm
)
Galvanic Skin Response Phalange Temperature
Electromyogram
Blood Volume Pulse – HR & Amp. Respiration – Rate & Amp.
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Hardware• Thought Technology ProCom Infinity
47
Images via www.thoughttechnology.com
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Application Domains• Affective Computing
– Interaction evaluation (UI, devices, latency)– Music & emotion research– Psychoneurology
48
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
49
BCI and Creativity
Yee Chieh (Denise) ChewGeorgia Institute of Technology
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline
• fNIR and Creative Expression• EEG and Music
51
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Character Templates for fNIRCreated with a mouse, these are templates that subjects traced using fNIR
52
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
53
Physiological Measures Used for Identification of Cognitive States and
Continuous AuthenticationCurtis S. Ikehara & Martha E. Crosby
University of Hawaii at Manoa
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline
• Overview of our Research Using Physiological sensors– Assessing Cognitive States– Continuous Authentication
• Possible Applications
55
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 5656
Multi-modal Physiological Sensors System
MTF(Moving Target Fractions) HUMAN
Custom Electronics(Analog to Digital Converter)
Eye Tracker(Gaze Position, Fixation Number, Fixation
Duration, Repeat Fixations, Search Patterns, Pupil Size, Blink Rate, Blink Duration)
Pressure Mouse(Pressures on Mouse Button and Body)
Laptop(Data Collection, Analysis & Display) EDA / TSA Finger Sensors
(Electrodermal and Temperature Sensor Activity)
Oximeter(Pulse, Blood Oxygen)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 20105757
A Composite of Over 20 Mouse Clicks of Three People (Click Signature)
Subject 5
Subject 4
Subject 7
US Patent # 7,245,218
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Possible Applications
• The student’s confidence can be assessed by evaluating the distortions of the mouse click signature.
• High Value Vehicle Control– Pressure and other physiological sensors can be
attached to a steering (e.g. aircraft yoke or steering wheel) .
– Sensors can be used to indicate that:• The authorized pilot or driver is in control.• The cognitive state of the pilot is consistent with the
vehicle’s activity.
58
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
University of Hawaii at Manoa1680 East-West Road - POST 317
Honolulu, Hawaii 96822 USA
59
Martha E. [email protected]
Curtis S. [email protected]
Measuring Interaction Experiences:Integration of Multiple
Psychophysiological Methods
Jan KallenbachAalto UniversityHelsinki, Finland
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
OutlineUser Experience
=Choice & Interaction
+Cost-Benefit Evaluations
Measuring Interaction Experiences?
Software Framework61
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Client Application for UX ResearchSampling frequency: max 1000HzPsychophysiological Measures: Gaze, Pupil Size, EDA, EMG, EEG, …Behavioral Measures: Keyboard, Mouse, Slider, …System Measures: Window Events
62
MultimodalMeasurements
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Jan KallenbachResearch Scholar
Aalto UniversitySchool of Science & TechnologyDepartment of Media TechnologyP.O. Box 15500FIN - 00076 Aalto
63
Towards enhanced video access and recommendation through emotions
Eva OliveiraNuno Ribeiro
Teresa ChambelIPCA | LASIGE
65Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
We Want
• To access to videos based on emotions– user actual emotional (psychophysiological) state– video emotional content– director’s perspective
• To recommend videos based on emotions– create a set of rules from every affective
classification perspective (user and video, director)
66Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Affective video recommendation system requirements ()• Every video must be classified affectively from the user
and from the content perspective.• Every user has an emotional profile constituted by all his
implicit (physiological) classification of every classified movie.
• User physiological signals can change the way we watch the videos
67Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
User Classification Issues
• Automatic Classification– Users emotional state
• emotional model– Classification techniques
• Result output• Accuracy• Lab contexts
68Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Current State
• Capturing users phychophysiological signals (BIOPAC System) while watching movies
• Next step: several classification techniques to convert signal result into emotional categories
This is my exciting picture
69Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
• Eva Oliveira :
Your users can’t talk!How can you tell they liked
your system? Lucia Filgueiras and Claudia Tambascia
Escola PolitecnicaUniversidade de São Paulo
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
What we want to do
• Develop products for cerebral palsy users• Perform participatory design• Understand non-verbal communication
71
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Direct measures and usability evaluation
Variabilidade da frequência cardíaca OP13
2,6
8,3
5,2
2,6 2,63,9
4,6 5,0
11,7
2,3
9,1
3,82,9
6,6
4,0 4,4
2,4
14,1
5,3 5,64,1
7,3
3,3
8,5
5,9
0,0
2,0
4,0
6,0
8,0
10,0
12,0
14,0
16,0
18,0
09:35
09:40
09:45
09:50
09:55
10:00
10:05
10:10
10:15
10:20
10:25
10:30
10:35
10:40
10:45
10:50
10:55
11:00
11:05
11:10
11:15
11:20
11:25
11:30
11:35
HORA
Índi
ce d
o B
alan
ço S
impá
tico?
Par
assi
mpá
tico
LF nu / HF nu LSC Média
Alarme de temperarura alta:
10:15
Operador analisa sistema após vários alarmes: temperatura alta (10:29);
nível alto (10:32) e segurança (10:32; 10:37; 10:43)
Ação do operador para correção
Ação do operador para correção
Eye tracking in iTV apps evaluation
NASA TLX, SWAT, Heart Rate Variability
Heart Rate Variability of an operator in a unstable unit situation
Comparison of eye tracker findings to conventional usability tests results
HRV and workload measures in control room operability evaluation
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Our questions:• How personalized are body expressions for cerebral palsy patients?
• Are there general body expressions that can mean the same for everyone?
• Is it possible to determine a generic way of measuring these expressions?
• Is there any other liaison besides gestures and facial expressions , like an unconscious connection of limbic systems that can be responsible for the mutual sensitive communication?
• Is there a reference body signal that can be used as a reliable parameter to evaluate body expressions?
• How can we establish a correlation between body expressions and patient´s opinion of approval or disapproval of a design solution?
73
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you!Contact us:
Lucia V. L. FilgueirasEscola Politecnica, Universidade de São [email protected]
Claudia A.TambasciaFundação [email protected]
http://lts-i.pcs.usp.br
74
Human Computer Symbiosis by Mutual Understanding
Marc GrootjenEagleScience-TNO
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline
• Symbiosis & mutual understanding• Information models• Interfaces
76
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 77
Tasks (goals)
Disturbance
computerMutual
understanding
Symbiotic entity
Maximum task performance
human
Human-computer symbiosis with mutual understanding as key issue for maximum task performance
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 78
Domesticating Brain-Computer Interfaces: Systems
for the HomeMelody Moore Jackson
BrainLabSchool of Interactive Computing
Georgia Tech
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Extended Home-based BCI study
• fNIR-based BCI for communication• 32 People with ALS (so far) over 1 year• 78% (25) could operate the device on
initial visit with accuracy > 70%• Accuracy to 100%, average 76.5%• Issues:
– Motivation– Data Collection– Remote Support
80
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
The Kokoro GatariBrain Computer Interface based on Functional Near Infrared imaging
81
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Melody Moore [email protected]
82
Get up, move on! Using Electromyography to Explore
the Relationship of Experience and Motion
Johann SchrammelCentre for Usability Research and
Engineering
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline
• Compare experiences and emotions in conditions with and without movements with Electromyography(EMG) and other methods
84
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline
• Can EMG successfully be used to estimate a user‘s emotional experience in motion contexts (artefacts, feasibility, etc.)?
85
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline
• Is a more embodied way of controlling interaction actually amplifying experience?
86
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Results
• EMG is suitable for motion contexts.• Both EMG and self-report data indicate
that natural body movement intensifies emotions.
• It’s crucial that body movements are in accordance with activity.
87
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
88
Reality-Based Brain-Computer Interaction
Daniel SjölieUmeå University, Umeå, Sweden
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Reality-Based Brain-Computer Interaction• Complex, reality-based, interaction with the
ability to adapt to the user in real-time– Using brain measurements to evaluate
• Direct brain-computer interaction– The user can be unaware
• We’re using a combination of virtual reality (VR) and functional MRI (fMRI)
90
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
HCI and cognition
• An improved understanding of the brain in this interactive and complex context is needed
• Build on theories of cognition as a foundation for the development of advanced HCI– As interaction moves towards being reality-based
we move closer to “interacting with reality”, and thus to general cognition
• Cognitive neuroscience and computational models of the brain are important tools
91
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Working with fMRI and Virtual RealityWorking with fMRI requires specialist competence and forces us to deal with powerful magnetic fields and restricted mobility, but the ability to measure the whole brain is an invaluable research tool.
92
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Daniel Sjölie, [email protected]
93
emoBAN: Improving Quality of Life
via Psychophysiological Mobile Computing
Katarzyna Wac, Anind K. DeyCarnegie Mellon University, HCII
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
emoBAN idea
• “what always speaks silently is the body” (Brown, 1990)
– measure QoL via psychophysiology
95
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
emoBAN: Usable Useful Functional
• Body Area Network– sensors
• vital signs, e.g., ECG, temp, resp• context, e.g., location, time, activity
– actuators• feedback via, e.g., AV, tactile, light
96
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank youDr Katarzyna WacCarnegie Mellon UniversityHuman-Computer Interaction [email protected]/~wac
97
Leveraging Psychophysical Data in Monitoring and Analyzing the States of Badminton Players
Feng Tian, Wencan Luo, Jing Dai, Reza Naima, Yongmin Cheng, HonganWang, Guozhong Dai
Institute of Software, Chinese Academy of SciencesPhysical Education Collage, Beijing Sport University
Berkeley Institute of Design, UC. BerkeleyChina Badminton Team, General Administration of Sport of China
4/20/2010
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Traces and physical actions - Camera, accelerometers
Psychophysical data- Berkeley TricorderECG, EMG, Pulse Oximetry, BioImpedance Respiration
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Exploring Quantitative Methods for Evaluating Sports Games
Veronica ZammittoSimon Fraser University
Magy Seif El-NasrSimon Fraser University
Paul NewtonElectronic Arts Canada
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Current Study: NBA Live 10
• Eye tracking– Eye Tech TM3
• Psycho-physiological data– ProComp5 Infiniti– EMG (corrugator supercilii and zygomatic major),
EKG (heart rate), GSR and BVP• Gameplay Telemetry Data
– event-based RPC telemetry collection system• Triangulation
102
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Data Triangulation - NBA Live 10Game user experience
103
What was the user looking at?
Where user’s attention was?
What was the emotional response?
Was positive? Was negative?
Event, when
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Data Triangulation - NBA Live 10Game user experience
104
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Veronica [email protected]
Magy Seif [email protected]
Paul [email protected]
105
Exploring Psycho-physiological Measures for the Design and Behavior of Socially-Aware
Avatars in Ubicomp Environments
Johanna Renny OctaviaHasselt University, Belgium
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Avatars in Ubicomp Environment User’s representation Mediator between system and user (companion) To design a socially-aware avatar that adapts its behavior according to user’s emotion Any incorrect actions of the avatar can lead to loss of identification of the user
107
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Our informal experiment ProComp Infiniti device, measured GSR, EMG and BVP Higher physiological values when playing against a friend How different social relationships between players influence those values?
108
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Expertise Centre for Digital Media (EDM)Web: www.edm.uhasselt.beEmail: [email protected]
109
Tailoring Practice for Individualized Learning via
Cognitive StateBob Wray
Soar Technology, Inc.
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Why I need Psychophysiological Computing• Practice environments are becoming
common-place• Adaptive tailoring of practice is important for
individualized experience/learning• Need more than traditional adaptations (e.g.,
scaffolding)– Adapting to engage learner / maintain interest – Adaptation to reducing “gaming” in environment– Adaptation to elicit learner affect/arousal (“real”)
111
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Research TestbedNarrative, conversational, inter-cultural training application
112
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Bob WraySoar [email protected], 919.967.5079
Brian MagerkoGeorgia Institute of [email protected]
113
Guidelines to biosignaldriven HCI
Marjolein van der Zwaag
Philips Research Europe, University of Groningen
Guidelines to biosignaldriven HCI
Marjolein van der ZwaagEgon L. van den Broek
Joris H. Janssen
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline
• Signal(s) measured: psycho-physiological • Application domain: Affective computing• Device used: Nexus 10, mind media
• Guidelines for processing biosignals– Triangulation– Identification of users– Biosignal characteristics – Temporal construction
116
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Unobtrusive affective computingMusic direct your mood concept
117
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
118
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
What is brain/body computing good for?
How can we think systematically about this space?
120
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
What are the major themes of HCI that can be explored using brain/body interaction?
121
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
What specific brain & body sensing modalities are interesting and how can we use them?
What are most innovative technologies in this space?
How do these compare with existing practices (NASA-TLX, Likert-style questionnaires, etc)?
122
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
How can BCI and PC be defined within an HCI context?
How is a HCI context different from neuropsychology, biomedicine or biomedical
engineering?
How does HCI contribute to psychophysiological computing (or more
specifically neural engineering, bio-engineering, etc)?
123
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
If we could measure anything on brain and body, what would we want to measure and
why?
What cognitive states and emotions have been investigated with sensors?
What methodologies are best for what signals?
124
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
How can accurate physiological, psychological and neurophysiological
measurements improve HCI?
125
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
With what software techniques and hardware technologies could these signals
be optimally processed?
How can software account for the combination of user actions taken to accomplish a task versus actions that
would surprise the user, yet optimize their experience?
126
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
What are the issues that must be considered when using brain/body computing for
direct control, passive sensing, adaptation, etc?
127
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Groups
• How to go from raw data to something else– Emotions: TABLE 1– Data Fusion: TABLE 4
• How to use these things in interactive interfaces: TABLE 3
128
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
BBB Workshop: Bringing Together BCI& Physiological Computing
129
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Staying connected
130
Website http://www.eecs.tufts.edu/~agirou01/workshop/
Twitter http://twitter.com/bbbCHI2010
Twitter Hashtag
#bbbCHI2010
Facebook http://www.facebook.com/home.php#/pages/Atlanta-GA/Brain-Body-and-Bytes-Psychophysiological-User-Interaction-at-CHI-2010/136581171786
Flickr http://www.flickr.com/groups/bbbchi/
Grouplist http://groups.google.com/group/brain-body-interfaces/
Slide Share http://www.slideshare.net/group/brain-body-and-bytes-psychophysiological-user-interaction
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
BBB at CHI2010Brains and Brawn session: Tuesday 11:30-1pm
Making Muscle-Computer Interfaces More PracticalT. Scott Saponas, Desney S. Tan, Dan Morris, Jim Turner, James A. LandayUniversity of Washington, USAMicrosoft Corporation, USA
A Novel Brain-Computer Interface Using a Multi-Touch SurfaceBeste F. Yuksel, Michael Donnerer, James Tompkin, Anthony Steed.University College London, UK
The Influence of Implicit and Explicit Biofeedback in First-Person Shooter GamesKai Kuikkaniemi, Toni Laitinen, Marko Turpeinen, Timo Saari, Ilkka Kosunen, Niklas Ravaja, Helsinki Institute for Information Technology, FinlandCenter for Knowledge and Innovation Research, Finland
Effects of Interactivity and 3D-motion on Mental Rotation Brain Activity in an Immersive Virtual Environment
Daniel Sjölie, Kenneth Bodin, Eva Elgh, Johan Eriksson, Lars-Erik Janlert, Lars NybergUmeå University, Sweden
131
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
THANK YOU
• Keep in touch!
132