Using Neuroscience theories and tools to inform information system research

PIERRE-MAJORIQUE LÉGER, Ph.D.Co-director, Tech3LabDepartment of Information Technologies, HEC Montréal

November 2015

Using neuroscience theories & tools to inform system research

© Copyright Tech3Lab 2015

Prof. Pierre-Majorique Léger, Ph.D.

Full ProfessorDirector ERPsim LabCo-director Tech3Lab pml@hec.ca

GRAD stUDIEs

POst-DOctORAtE

LAbs VIsItInG PROfEssOR

Lab Serious games to learn enterprise systems and business analytics

https://www.researchgate.net/profile/Pierre-Majorique_Leger

EASE OF USE?

© Copyright Tech3Lab 2015© Copyright Tech3Lab 2015

EASE OF USE?

EASE OF USE?

EASE OF USE?

© Copyright Tech3Lab 2015

UXManaging Business User Experience

BUSINeSS INteLLIGeNCe

MArketING AND eLeCtrONIC COMMerCe

INFOrMAtION teChNOLOGIeS

INNOVAtION 2013

Management comittee

SYLVAIN SÉNÉCAL, Ph.D.Co-directorFull Professor

FrANÇOIS COUrteMANChe, Ph.D.Research professional

MArC FreDette, Ph.D.ResearcherAssociate Professor

PIerre-MAJOrIQUe LÉGer, Ph.D.Co-directorFull Professorn

eLISe LABONtÉ-LeMOYNe, Ph.D.Post-Doctoral researcher

© Copyright Tech3Lab 2015

Financing Agencies

© Copyright Tech3Lab 2015

Tech3Lab 2015 : The most important UX Lab in North America

© Copyright Tech3Lab 2015

sEctIOn 1 Emotion and Attention in HcIsEctIOn 2

Mobility and tactile interfaces in HcI sEctIOn 3

Learning and adaption in HcI

sEctIOn 1 Emotion and Attention in HcI

© Copyright Tech3Lab 2015

What is eye-tracking?

© Copyright Tech3Lab 2015

tobii Eyetracker X60

© Copyright Tech3Lab 2015

© Copyright Tech3Lab 2015

tobii Eyetracker X60

© Copyright Tech3Lab 2015

© Copyright Tech3Lab 2015

Source: http://eyetracking.me/?page_id=9

What is Eye Tracking

© Copyright Tech3Lab 2015

Processing order: Gaze path

© Copyright Tech3Lab 2015

Visual attention: Heatmap

© Copyright Tech3Lab 2015

!!

!

83(4'1'9+!120:0-!&'()*+!,"#$!'//)01*21+0!83(4'1'9+!120:0!"+207<!

Figure 1. Illustration of pupillary responses in cognitive taskexecution, (A) Mental multiplication task and pupil size change, (B) Difficulty effect on pupil dilation evoked by mental multi-plication of two numbers displayed together for eight seconds (adopted from Klingner's Dissertation (Figure 5.5) [1]).

(B)

(A)

Source: Hossein & Yeasin, 2014

© Copyright Tech3Lab 2015

© Copyright Tech3Lab 2015

Emotional attention: HeatmaptEcHnOLOGy DEVELOPED At tech3LabCourtemanche, F., et al., Method and Product forVisualizing the Emotions of a User, in Provisional patent application 2015. p. p. 14.

sEctIOn 2 Mobility and tactile interfaces in HcI

The effect of touching on advertising effectiveness

• Needfortouch+• tact iledevice=

•upto2XmorechaNcestorecogNizebraNds

Senecal, S., Leger, P. M., Fredette, M., Courtemanche, F., Cameron, A. F., Mirhoseini, S., & Riedl, R. (2013). Touch Screen as Input Device: Does it Influence Memory Retrieval?.

The touch effectvirtually touching the product has significant effect on brand and product recognition

The impact of treadmill desk on cognition

Seated condition

Walking condition

Treamillhabituation

Reading taskwhile walking

Reading taskwhile seated

Adjustmentof equipment

5 min. 40 min. 10 min. 10 min. 10 min.EEG

Recall taskPost-

experimentalsurvey

▪ Proportion of EEG frequency band power for electrodes with significant differences between groups

Labonté-LeMoyne, É., Santhanam, R., Léger, P. M., Courtemanche, F., Fredette, M., & Sénécal, S. (2015). The delayed effect of treadmill desk usage on recall and attention. Computers in Human Behavior, 46, 1-5.

Avec la permission de ICI Radio-Canada

Texting while walking:an expensive switch cost

Courtemanche, F., Léger, P.-M., Cameron, A.-F., Faubert, J., Labonté-LeMoyne, É., Sénécal, S., Fredette, M. et Bellavance, F. (2014). «Texting While Walking: Measuring the Impact on Pedestrian Visual Attention», Gmunden Retreat on NeuroIS 2014, Gmunden, Autriche, 5-7

sEctIOn 3 Learning & adaption in HcI

Measuringbrain activity

© Copyright Tech3Lab 2015

1011

© Copyright Tech3Lab 2015

© Copyright Tech3Lab 2015

© Copyright Tech3Lab 2015

ÜííéWLLìéäç~ÇKïáâáãÉÇá~KçêÖLïáâáéÉÇá~LÉåLÉLÉQLqÉíêáë|alp|NV=ÜííéWLLìéäç~ÇKïáâáãÉÇá~KçêÖLïáâáéÉÇá~LÉåLÉLÉQLqÉíêáë|alp|NVUSKéåÖ=USKéåÖ

© Copyright Tech3Lab 2015

Learning in the brain

Source: Hill and Schneider, 2006

Controlled processing

Automatic processing

Learning in the brain

Source: Hill and Schneider, 2006

Rapid acquisition Easy alteration of memory

Slow and high effort Deteriorates under stress

Slow acquisition Difficult to alter

Fast and low effort Robust under stress

Reduction of control network activation

which scaffold learning

http://upload.wikimedia.org/wikipedia/en/e/e4/Tetris_DOS_19 http://upload.wikimedia.org/wikipedia/en/e/e4/Tetris_DOS_1986.png 86.png

Source: Haier, 1992

Amateur Expert

ALPHA relaxed/reflecting

alert/working

drowsy/ideating

sleep/dreaming

deep dreamless

BETA

THETA

DELTA

© Copyright Tech3Lab 2015

Astronaut/aquanaut Nicole P. Stott and Ross Hein, a professional aquanaut from the University of North Carolina at Wilmington (UNCW), assist University of Cincinnati physician Tim Broderick apply a net with electrodes for an electroencephalogram (EEG) study

© Copyright Tech3Lab 2015

ALPHA relaxed/reflecting

alert/working

drowsy/ideating

sleep/dreaming

deep dreamless

BETA

THETA

DELTA

Pope, Alan T., Edward H. Bogart, and Debbie S. Bartolome. “Biocybernetic system evaluates indices of operator engagement in automated task.” Biological psychology 40.1 (1995): 187-195.

© Copyright Tech3Lab 2015

ERPsim on SAP HANA

now available on sAP HAnA

Automated administrative tasks

Simulated market

Simulated passing of time

Business analytics

Business decisions

VIRtUAL systEM PARtIcIPAntsERP systEM

1 12 23

floW=d iff icultyaNd compet eNcy+

psychophys iological stat eoft helearNer

Predicting the flow state

16%

18%

© Copyright Tech3Lab 2015

Léger, Pierre-Majorique, et al. “Neurophysiological correlates of cognitive absorption in an enactive training context.” Computers in Human Behavior 34 (2014): 273-283.

© Copyright Tech3Lab 2015

Attentional heatmap in during software interaction

© Copyright Tech3Lab 2015

Mobile Eye-tracking AssEssInG AttEntIOn AnD bEHAVIOUR In sOftwARE AnD MEcHAnIcAL DEVELOPMEnt

© Copyright Tech3Lab 2015

Understanding problem solvingAssEssInG sELEctIVE AttEntIOn In A MULtIPLAn EnVIROnMEnt

© Copyright Tech3Lab 2015

Monitoring learner’s engagement in real-time

© Copyright Tech3Lab 2015

HyPOtHèsEGénérer des émotions compatibles avec l’engagement dès la prise en main du jeu

MEsUREs Activité électrodermale, mouvement oculaire et rythme cardiaque

L’EffEt DEs tyPEs DE tUtORIEL DE JEU sUR L’IntEntIOn D’UtILIsAtIOn D’APPLIcAtIOns MObILEs

Chercheur : Pierre-Majorique Léger, Ph.D. (HEC Montréal) Partenaire industriel : Hibernum Créations inc Financement : CRSNG EGP 484281 – 15

cARtE DE cHALEUR OcULOMÉtRIQUE

PARcOURs OcULAIRE

© Copyright Tech3Lab 2015

EffEt DE LA RÉDUctIOn DE L’ÉMOtIOn nÉGAtIVE LIÉE AU DEsIGn sUR L’EffIcAcItÉ DEs APPLIcA-tIOns ÉDUcAtIVEs Et DE fORMAtIOn

Chercheur : Pierre-Majorique Léger, Ph.D. (HEC Montréal) Partenaire industriel : HMH Tribal Nova Financement : CRSNG EGP 485446 - 15

cARtE DE cHALEUR OcULOMÉtRIQUE

PARcOURs OcULAIRE

HyPOtHèsEMaintenir l’apprenant dans un état compatible avec la volonté de persévérer

MEsUREs Mesure de la charge mentale avec le diamètre de la pupille et l’analyse automatique des émotions faciales

floWofplayer(a)=d iff iculty+compet eNcy+

player(a)+player(b)

Predicting flow in multiplayer context

66%

Bastarache, Léger, Courtemanche, Sénécal and Fredette (2015) Measuring Flow Using Psychophysiological Data in a Multiplayer Gaming Context, Information Systems and Neuroscience, Lecture Notes in Information Systems and Organisation.

conclusion

© Copyright Tech3Lab 2015

Upcoming book at Springer: R. RIEDL, P.-M. LÉGER

fundamentals of neuroisInformation Systems and the Brain

Series: Studies in Neuroscience, Psychology and Behavioral Economics

▪ Provides the fundamentals of NeuroIS

▪ Guides the reader to theories and tools which are relevant for this field

▪ Contains tutorial-like illustrations which enhance understanding

▪ Written by experts in the field

© Copyright Tech3Lab 2015

sEctIOn 1 Emotion and Attention in HcIsEctIOn 2

Mobility and tactile interfaces in HcI sEctIOn 3

Learning and adaption in HcI

Thank you!