Sleepers’ Lag: Study on Motion and AttentionKeywords: Video analysis, computer vision, tracking, body motion, classroom, interpersonal synchronization, orchestration 1 INTRODUCTION

(2016).Sleeper’slag:Studyonmotionandattention.JournalofLearningAnalytics,3(2),239–260.http://dx.doi.org/10.18608/jla.2016.32.12

ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 239

Sleepers’ Lag: Study on Motion and Attention

MirkoRaca

CHILILaboratoryÉcolepolytechniquefédéraledeLausanne,Switzerland

[email protected]

RolandTormey

CAPEÉcolepolytechniquefédéraledeLausanne,Switzerland

PierreDillenbourg

CHILILaboratoryÉcolepolytechniquefédéraledeLausanne,Switzerland

ABSTRACT:Bodylanguageisanessentialsourceofinformationineverydaycommunication.Lowsignal-to-noiseratiopreventsusfromusingitintheautomaticprocessingofstudentbehaviour,anobstacle thatwe are slowly overcomingwith advanced statisticalmethods. Insteadof profilingindividualbehaviourofstudentsintheclassroom,theideaistocomparestudentsandconnecttheobservedtraitstodifferentlevelsofattention.Withtheusageofnoveltechniquesfromthefieldof computer vision,we focuson features that canbeautomatically extractedwitha systemofcameras,bymeansofpassiveobservationoftheclassroompopulation.Weshowparallelsbetweenourworkandprevioustheoriesandformulateanewconceptformeasuringthelevelofattentionbasedonsynchronizationofstudentbodymovement.Weobservedthatstudentswithlowerlevelsofattentionareslowertoreactthanfocusedstudents,aphenomenonwenamed“sleepers’lag.”Thisrealizationmaygiverisetonovelmeasurementsthatcanactasatechnologicalsupportforteachermetacognition.Thegoalistoimprovetheteacher–studentconversationandtoproposetechniquesthatcanenableashorterfeedbackloopoftheteacher’sperformancecomparedtothecurrent-daymethods.

Keywords: Video analysis, computer vision, tracking, body motion, classroom, interpersonalsynchronization,orchestration

1 INTRODUCTION

Attentionisthe“gateway”throughwhichstudentslearn(Shelletal.,2010),butthisessentialtraitiseasytoloseandhardtoassess.So,howcanthelecturer“measure”theattentionofstudentsduringtheclass?Typically,classroominteractions(Q-A,interactions,demonstrations)areusedasproxies,butinstandardlecturesettings,studentparticipationisverylow.Teacherobservationstendtobebasedonasmallsampleof high-interaction individuals, while fewer than 40% of students actively engage in the conversation(Howard&Henney,1998).



Ourapproachisbasedonanattempttoformalizeobservationofteachereffectonstudents.Coe,Aloisi,Higgins, and Major (2014) confirmed the validity of classroom observation as a method of teacherassessment.Butwhentheoperationiscarriedoutbyindividualsassessingtheteacher,Bernstein(2008)noted that the quality of the process is largely dependent on the training of the observers. AnotherconstraintthatRange,Duncan,andHvidston(2013)notedwasthetimelimitforthepost-observationalconference,whichshouldbewithinfivedaysoftheintervention.

Thedirectproblem—evaluatingtheattentionthattheteacher’sactionwillattract,ishighlysubjectiveand impossible to automatize. Modern approaches attempt to classify appealing body language andpresentationstyles,butinordertoassesstheeffectoftheapproach,weneedtoturntotheaudience.

Weconsiderteacherperformanceandstudentunderstandingastwosidesofthesamecoin.AsHattie’s(2013)meta-analysisnoted,presentingtheeffectoftheirinterventionbacktotheteachersisoneofthestrongesteffectsamongeducationalinterventions.Inordertore-connectthetwosidesoftheclassroomintoamutuallybeneficialconversation,weaimtopresentatechnologythatcanprovideteacherswithseamlessfeedback.Timperley,Wilson,Barrar,andFung(2008)describeabroadersetofprinciplesasa“knowledge-buildingcycle”—asetofeffortsneededtocontinueteachers’professional improvement.Themethodsinourapproacharealreadywellestablishedinthehumanconversationasthegroundingprinciple(Clark&Brennan,1991)andthebackchannel(Vinciarellietal.,2012).

Ourtechnological interventionisaimedtowardsamplifyingthebackchannel,andfocusingtheteachertowardsclassroominteractions.Itisimportanttonoteourintentiontoaugment(expand)thefeedbackloop,andnottoreplacethepoweroftheteacher’sownobservation.Eventhoughcurrentchallengesinthetechnologicaldomainlie inachievinghumanperformance,excludingtheteacherfromthelearningloopwouldbeamistake;astheorchestratorofthelearningprocess(Dillenbourg&Jermann,2010),theteacherisresponsibleforintegratingtheinformationintotheoveralllearningexperience.

Withoutoverloadingthestudentswithgadgetsandformallystructuredproceduresthatdictatetheformatof the learningexperience,weaim to implementour systemwith a set of cameras. Thebase for ourobservationsishumanactivityinitsmostbasicform—movement.

Inthispaper,wepresentthemethodformeasuringmovementinaclassroomandtheprocedureusedtorelate the gathered information to students’ subjective perceptions of their own attention. Themaincontributionistheconceptofmeasuringthespeedofstudentreactionsinclasstodetectstudentswithlowerattention.Theconceptisbasedontheideathatstudentsfocusedonthelecturewouldreactinthemomenttotheimportantinformationbeingpresented,whiledistractedstudentswouldbeslowertonoteit.This is theconceptwecall “sleepers’ lag.”Thehigher thevariance in reaction time to thecommonstimuli(inourcasetotheteacher’spresentation)—thelowertheattentionoftheclassroomaudience.

Our other conclusions go further into exploring how the geometry of the classroom and immediatesurroundingsaffecttheindividualstudent.Thissetsthegroundfor“student-centred”observationoftheclassroom, as opposed to the dominant trend of exploration that considers the teacher as the onlystimuluspresent.



2 RELATED WORK

Traditional classrooms (in both talk format and seating configuration) remain the dominant format oflecturingonall levelsofformaleducationtoday(Moore,1989).Therehavebeenmanycritiquesoftheformat,notingthattheclassroom’sgeographicalconfigurationmakesitdifficulttodeveloptheteacher–student relationshipandunderstandingbeyondstereotypes (Hargreaves,2000).Andwhilesomeclaimthatthecurrentorganizationalsetupevolvedforpracticalreasons(Koneya,1976)wecannotignorethedifficultiesthatteacherfaceinkeepingstudentattentionovertime(Middendorf&Kalish,1996;Wilson&Korn,2007)andon-task(Rosengrantetal.,2012).

Thesetoftheorieswegroupunderthename“teacher-centric”focusontheteacherandtheteacher’simpactontheclassroom.Astheprimaryorchestratorofthelearningprocess(Dillenbourgetal.,2011;Dillenbourg & Jermann, 2010), teachers take on the responsibility that begins with educationalpresentation, follows throughpedagogical guidance (Corcoran&Tormey, 2012), andhopes to achievestudents’personaltransformation(Whitcomb,Borko,&Liston,2008).Theteacher’sroleintheclassroomhas been characterized as emotional labour (Hargreaves, 2000) and cognitively demanding (Emmer&Stough,2001).Inmanyinstances,agoodteacherischaracterizedbytheabilitytopresenttheteachingmaterial in a way that engages students, this being the major difference between a novice and anexperiencedteacher(Borko&Livingston,1989),confirmingtheneedfortheteachertobeareflectivepractitioner(Schön,1983).

Thegeometryoftheclassroomcanalsobeanemotionalbarrierformorenaturalinteraction(Hargreaves,2000).Studentsinthefrontrowsareperceivedasbeing“moreinterested”(Daly&Suite,1981).ThebulkofcommunicationisorientedinaT-shapedregionwiththehighestconcentrationofinteractionfocusedonthefrontandcentreoftheclassroom(Adams,1969).Thisnotonlyaffectstheteacher’sperception,butstudentsalsoadjusttothegeometryoftheclassroom,withthoseseekinginteractiontendingtositinthe high-interaction zone (Altman & Lett, 1970). The seating arrangement also amplifies studentinteractions—makinghigh-verbalizersmoreactiveinthehigh-interactionzone,andlow-verbalizersevenless active in the low-interaction zone (the edges of the classroom) (Koneya, 1976). The classroomenvironmentgreatlyaffects theperceptionsof teacherandstudents,but thisdoesnotalwayswork infavourofthelearningprocess.

Beingfarawayfromtheteachergoesbeyondjustteacherperception.Onthe“student-centric”sideofresearch,Daum(1972)foundthatdistancefromtheteacheralsohasasignificanteffectonthesuccessofstudents.Finn,Pannozzo,andAchilles(2003)foundthatsmallerclasssizes(fewerthan15students)affectthequalityofthelectureintwoways—theteacherstakelesstimetomanagethelearningprocess,butmoreimportantlystudent-to-studentinteractionalsoimproved.Asstudentsgrowupintheschoolsystem,therelationshipbetweenteacherandstudentbecomeslessemotionallyinvolved(Hargreaves,2000)andtheirparticipationinclassroomactivitydecreases(Marks,2000).Thisseemscloselyrelatedtostudentsbecomingmoreaccustomedtostudyinginalargegroups,whereindividualvisibilityisuncertain(Finnetal.,2003)andcircumstancesallowforeasydiffusionofresponsibilityandsocialloafing(Forsyth,2009).It



is common for students tohavemorepractical goals (i.e., good grades) thanpurely academic growth(Allen,1986).

Irrespective of position or grades, students have difficultymaintaining their attention throughout theduration of a lecture (Rosengrant et al., 2012). Attention “can be partially defined as the selection,activation,andmaintenanceofmentalfocusonsomestimuli(externalorinternal)accompaniedbytheblockingofotherstimuli”(Rapp,2006).RodaandThomas(2006)noteditasourbiologicaldefenseagainstinformational overload coming primarily from the external environment. Even if it is not clearlyquantifiable how long it takes students to “zone out” during a lecture, proposed measurements ofbetween10minutes(Wilson&Korn,2007)and20minutes(Middendorf&Kalish,1996)arefarlessthantheaveragedurationofa lecture.Moore (1989) recognized that studentattention isdividedbetweenthreetypesofinteractions:i)learner–content,ii)learner–instructor,andiii)learner–learner,inwhichthesecondtypehaspriorityovertheothertwoinclass,duetoitslimitedavailability.RodaandThomas(2006)producedadetailedspecificationofhowattentionshouldbehandledinthedomainofhuman–computerinteraction, but outside of this strictly technical domain, the rules become less defined. Variousapproachestodetermineuserattentionwereformulatedwitheye-trackingresearchbeingtheprevalentmethodfor itsmeasurability (Nüssli,2011).Head-posewasalso foundtobeagood indicatorofvisualattentionwith88%accuracy(Stiefelhagen&Zhu,2002).Withthegoalofraisingtheaccuracyofprediction,othermethods introducedvarious complementarymeasurements, suchasEEGdevicesandheart-ratemonitors(Chen&Vertegaal,2004)andothercontextualinformation(Arroyoetal.,2009;ElKaliouby&Robinson,2004;Horvitz,Kadie,Paek,&Hovel,2003)withtheconstanttrade-offbetweenthecomplexityofthemeasuringapparatusandtheconfidenceoftheprediction.Intheareaofmeasuring“expertise,”focusingsolelyontheactivityasthecue,successfulattemptsatobservingdifferentbehaviouralpatternshavebeenobservedinbothexpertandnovicecategories(Worsley&Blikstein,2013).

Inabroaderscope,SocialSignalProcessing(SSP)researchfield(Vinciarelli,Pantic,&Bourlard,2009)posestheideathatmachineinterpretationofsimplehumanactionshasreacheditslimit,andinordertoimproveautomaticanalysis,weneedtoencodesocialcontext(Vinciarellietal.,2012).Withscopewellbeyondtheclassroom,attemptshavealreadybeenmadeininterpretingthebehavioursoflargegroupsatsportingevents(Conigliaro,Setti,Bassetti,Ferrario,&Cristani,2013)andinpublicspacesingeneral(Bazzanietal.,2013). The first results showed promise, but with the crudeness of the initial findings, we are againremindedofthecomplexityofhumaninteraction.Gatica-Perez(2009)showedtheneedforidentifyingthisnewbranchofresearch,aspapersonthetopicarecurrentlydistributedoverseveralscientificdomainsbasedonthemethods,applications,etc.

Our researchaims toscaffold teacher’sperceptionof thestudentsandraiseawarenessaboutstudentreception of the lecture. Some of the current methods of doing so are focused on the web-domaininteractions (Dyckhoff, Lukarov,Muslim, Chatti, & Schroeder, 2013), feedback devices such as clickers(Caldwell,2007),ormobilephoneapps(Rivera-Pelayo,Munk,Zacharias,&Braun,2013).Welookedtotheresearchonunobtrusivemeasurements (Webb,Campbell,Schwartz,&Sechrest,1999) inordernot todisturbtheclassroomecosystem.Thetopicissensitivebecause,asHeylighen(2002)hasalreadynoted,informationoverload leadstosuchdangerouspitfallsasanxiety,stress,andalienation. Inthemidstof



suchamentallydemandingtaskasteaching,wemustbecarefulwhenintroducingnewelementssincethemain bottleneckmay still remain in the teacher’s head.We took important cues fromubiquitouscomputing principles (Weiser, 1991), and interventions in which the information was available whenneeded,butwasnotthefocusoftheactivity(Bachour,2010;Alavi,2012).

3 THEORETICAL BACKGROUND

Ineverydaycommunication,groundingoccursseamlesslythroughouttheconversation.TheworkofClarkandBrennan(1991)definesgroundingasthecollectiveprocessbywhichparticipantstrytoestablishthemutualbeliefthatallsidesunderstandeachotherinordertocontinuetheconversationsuccessfully.Inone-on-onecommunication,groundingisessentialandcompletelyinterwovenwithotheractivities;intheclassroom, however, the feedback component is much weaker. Lecturing is inherently imbalancedbetweenthetwogroundingphases—i)presentation,andii)acceptanceofinformation—largelyinfavouroftheteacher.

The“acceptancephase”iswelldevelopedintheeducationaldomain,outoftheneedtoformalizetheprocess.Theevaluationofstudentknowledgetakesmanyforms,andinordertoshowhowteachersusedifferenttypesofevaluation,weemphasizethefollowingproperties:

• Social scope: Differentiating between the evaluation of a single person, work-group, class,generation,etc.

• Delay: Time between the presented information and proof of its assimilation. Whileconversationalgroundinghappensinstantaneously,moreformaltechniqueshavelongerdelays,eitherwithinonework-unit(questionandanswerpairduringclass),orseveraldays(quizresults,finalexam,etc.).

• Confidence:Wecanneverbecertainifthepresentedknowledgedisplaysactualcomprehension,butdifferentmethodsofferresultsofhigherorlowerreliability.Whilestudentsnoddingcanbenomorethanaminimal-effortconversationalcontinuer,afullyansweredopen-endedquestioninthefinalquizwillbeamorereliableindicatorofactualunderstanding.

• Material scope:Dependingon the formulationof thequestion, the answermight require thestudenttodemonstrateknowledgeofasingledefinition,explainmaterialpresentedwithinthelesson(topic),orconnectseveralscientificareas.

While the formal education process requires widematerial and social scope, there is little space forinterventionandcorrectionofstudentknowledge.Inordertodopreventiveevaluation,teachersoftenuseasmallermaterialscopeandshorterdelay(e.g.,continuoustesting).Indoingso,lowperformancecanbeexplainedby“didnotstudyhardenough”insteadof“materialwasnotappropriatelypresented.”Duetothelargenumberoffactorsinfluencinglearning,thelongerthedelay—thehigherthedistributionofresponsibility.



Thevalueoffeedbacktoteachershasbeenprovenhighlyeffective.Inanexhaustivemeta-studyontheeffects of different factors on learning, Hattie (2013) placed feedback to teachers as the tenth mostinfluential factoranalyzed intermsofstudentsuccess.Butcurrentsystemsforteachingevaluationaretypicallycarriedoutattheendofterm,whicheffectivelydissociatesthestudentgradefromanysingleactiononthepartoftheteacher.Intermsofdeliberatepractice,Ericsson(2008)suggeststhatthe“besttraining situations focus on activities of short duration with opportunities for immediate feedback,reflection,andcorrection.”Butwhatdoesthismeanforthefeedbacklooptotheteacherastheperformerofteachingactivities?

Toperform spontaneous self-evaluation, teachers are reduced to the conversational check-inwith theclass,which offers short delay and lowmaterial scope, but also low social scope and confidence.Weaddresseachpointseparately.

Lowmaterial scopemeans frequent requests for feedback fromstudents,which canbeautomaticallycarriedoutbymaintainingeyecontact.This“focusedattention”ontheindividualstudentisusedbothasafeedbackdeviceandasamethodofreconnectingtheabsent-mindedstudenttotheclassroommaterial.

Lowsocialscopecomespurelyfromourmentalconstraints.Confrontedwithagroupofpeople,ahumanobserverissequentiallyanalyzingeachindividual.Again,towidenthescopeoftheanalysis,theteacherwouldneedtospendmoretimeevaluating.Apotentialwayaroundthisbottleneck is togeneralizeorextrapolateinformationaboutthestudentstate,whichwewilladdressshortly.

Wecanassumethatlowconfidenceiscausedinpartbyconversationalconformityandpeerpressure.Inthebriefinteractionwiththeteacher,astudentengageddirectlyintheconversationisoftentrickedintosimulatingpositivegroundingevidencebyprovidingaminimal-effort“continuer”—suchasaheadnod(Clark & Brennan, 1991) — motivated primarily by the need to continue the lecture (effectively a“conversation”betweenteacherandstudent).Asecondaryobstacleforreportingactualunderstandingofthelessonispeer-pressureandconformity,whichimplicatethatthestudentneedstostepawayfromtheanonymityoftheclassroom(Forsyth,2009)andadmitalackofunderstandingpublicly.Thesourceofbothproblems is that the feedback requires direct and intentional interaction with the teacher. The“intentionality”offeedbackiscommoninmostotherapproaches,andthemainissueweovercomewiththeobservationalapproach.

Inordertokeepupwiththeteachingschedule,teachershaveseveralgeneralizationtoolsthattheycanusetoinferattentionandcomprehension:

• Teacherexperience:Developingintuitionaboutstudentreactionsistheslowestmethodtotrain.This automation of thinking andmental shortcuts (Kahneman, 2011) is usually found inmoreexperiencedteachers.Unfortunately,duetotheslowfeedback loop,thiscanbealsothemosterroneousmethod(Ericsson,2008).

• Familiaritywiththestudentinquestion:Builtthroughthelensofexperience,butshorterinthe



time-scope,familiaritywithindividualstudentscanprovideusefulfeedback.Themaindifferenceliesinthelocusofinformation—whiletheexperienceisprimarilyassociatedwiththeactionsoftheteacher,mainsourceofinformationremainstheindividualstudent.

• Howthelectureisgoingsofar:Thisisashort-termtemporalmethod,inwhichnoindividualisbeingconsidered,butrathertheoverallreceptionofthelessonbytheclass.

• Attitudeintheclassroom:Thesocialdimension.Evenifsomestudentsarenotvisible,theteachercaninfer“generalacceptance”ofthematerialby“readingtheaudience”asactorsdoonstage.

Dominantdimensionsthatoverlapinthenotedmethodsincludei)experience,ii)timeandiii)thesocialdimension.Giventhateachmethodinterpolatesthesethreecomponentstodifferentdegrees,webaseourapproachprimarilyonthesocio-temporaldimensions,inserviceofscaffoldingthethirdcomponent,whichremainsconnectedtotheteachersthemselves.Thisnaturallyassignstheapproachwithattributessuch as wide social scope and independence of the material scope — given that the automatedmeasurementscanbeappliedatanytime.Theapproachattemptstoaccessthesociallyvisibleinformationinto which we have limited access due to our biological limitations, amplifying the back-channelcommunication. Previous work stated that body language, while rich in semantics, is low on syntax(Vinciarellietal.,2012)—whichmakesitimplicitlyunreliable.Buttheavailabilityofdataemittedfromthe students as informative (carries meaning) if not communicative (not purposefully used forcommunication)signalsprovidesfertilegroundforanalysis.

3.1 Theoretical Assumptions

Our initial hypothesis for the experiment was that we could detect consistent groups of students bycommonbehaviourpatterns.Anexampleofconsistencywouldbeagroupofstudents listeningto thelecture versus students looking out the window. Second hypothesis was that people in the visiblesurroundingsofanindividualaffectthatperson(student)bytheirnon-verbalcues.Weconsideredbodylanguageinitsmostbasicformandcomparedtheco-occurrencesofmotion(co-movement)betweenpairsofstudents.Wealsorelatedourobservationstostudents’levelsofattention.

Fromthedualeye-trackingtheory,weknowthatthequalityofcollaboration(Richardson,Dale,&Kirkham,2007)andunderstanding(Jermann&Nüssli,2012)betweentwopersonscanbeassessedbyanalyzingtheconsistencyoftheirgazepatterns.Wedrawananalogywiththeseconclusionsinthedomainofmotionintheclassroom,withthehypothesisthatstudentswholistentotheteacherwillbemorelikelytomoveina synchronized manner, while an absent-minded student will act on his/her own internal rhythm.Synchronizedmotionisnotlimitedtoanyspecificaction,butcanbeexplainedusingtheexampleoftakingnotes—attentivestudentswouldturnthepagesonthehandoutsandnoteimportantfactsastheyarepresentedinclass.Morethanareactiontothelecture’saudio/visualstimulus,motioncanbeseenasanagreementoftheaudience.Ifthestudentsagreethataneventoutsidetheclassroom(e.g.,loudnoise,truck)ismoreimportantthanthelecture,theywouldstillhaveasynchronizedmotion(everybodylookingoutthewindow)butcausedbyadifferentstimulusthantheteacher.



Synchronization in the class was studied in a dyadic fashion, by comparing each pair of students.Dependingontherelativelocationbetweenthetwostudentsconsideredinthepair,wedividedthedyadsintothreeconditionsbasedontheirmutualvisibility(asdescribedinSection247.2).

Giventhat learning isnotastrictly formalizedactivity, reactionsofstudentscanvaryorbecompletelyblank.Indualeyetracking,adelayof2secondsbetweenthespeaker’sandthelistener’sgazeduringthemoments of referencing has been identified (Richardson et al., 2007), with the conclusion that thecomprehensionbetweenparticipantsisinverselyproportionaltothetimelag.Basedonthis,wedefinetwomovementsasco-movementifithappenswithin±4secondsfromeachother(depictedinFigure1a).Wedifferentiatebetweeni)perfectsynchronization(<2secapart),ii)synchronization(2–4secondsapart),andiii)weaksynchronization(4–6secondsapart).ThesethreeperiodsaredisplayedinFigure1bastheverticalaxis.

a) b)

Figure1:Synchronizedmovement.a)Co-movementmatrixofPersonAandPersonBoveraperiodof12seconds(6timesteps).Perfectsynchronizationisrepresentedbythediagonalofthematrix,markedwithredsquares.<±4secondsynchronizationisrepresentedwithbluecellsandweak

synchronization(<±6seconds)ismarkedwithgreencells.Periodstoofaraparttobeconsideredaregrayed-out.b)Co-movementtimeline,consideredfromtheperspectiveofPersonB.Thefigureshowsthesamevaluesastheco-movementmatrix,alignedonthediagonalcellsofthematrix(redsquares).

Transparentsectionsarenotpresentintheexamplematrix.

The additional third periodwas introduced to take into account indirect synchronization—when thepersonisnotreactingtotheteacher’sstimulusbutisfollowingthereactionsofothers,forwhichweadded2secondsforthepersontoobservethereactionofothersandthenreproduceit.Thisiswhatwecallthe“sleepers’lag”—theideathatthosemimickingattentioninsteadofactuallypayingattentionwillhaveadelay(a“lag”)intheiractions.

Algorithmically,motion synchronization between two personswas calculated asmatrixmultiplication.Eachpersonisrepresentedwithatimeseriesofmotionintensityvalues,sampledin2-secondsteps.The



co-movementmatrixiscreatedbymultiplyingthetwotimeseriesastheNx1and1xNmatrix(visualizedinFigure1a).Nrepresentsthenumberofsamplescollectedforeachpersonduringthelecture.

Withinthetwotimeseries,valueswiththesameindexrepresentthesametime-periodinthelecture.Thismeansthatperfectsynchronizationmomentswillbefoundonthediagonaloftheco-movementmatrix,coordinates(t,t).Toanalyzesynchronizationinstances(2–4secondsapart),PersonA,whomovedbefore,will occur 1 time-step before, and the co-movement with Person B is located at coordinates (t-1, t).Similarly,“weaksynchronization”withPersonAmoving4secondsbeforePersonBisshownatcoordinates(t-2,t).Incasesofmutualvisibility,reversedirectionofinfluence(PersonBmovingbeforePersonA)isalsopossibleandshownatcoordinates(t+1,t)and(t+2,t).

Themajorityoftheco-movementmatrixrepresentssynchronizedmovementinstancestoofaraparttoberelevant(biggerdifferencebetweencoordinatesrepresentsbiggertimedelaysbetweenactions).Forthatreason,we focuson thediagonaland the twobandsaround it:±2sec,±4sec.FromtheperspectiveofPersonB,wecandensely represent synchronizationmomentswithPersonAas the timelineshown inFigure1b.

Becausethevaluesintheco-movementmatrixrepresentmultiplicationofmotionintensitiesintherange(0.0–1.0),thevalueproducedwillbehighonlyifbothmovementswereofhighintensity.

4 METHOD

Oursetupandmethodforgatheringdataisnovelintheclassroomenvironment.Wewilldescribethemaintechnologicalpoints,coverthedata-gatheringmethodology,andprovideourcurrentworkingsample.

a) b) c)

Figure2:Motiondetectionandgrouping.a)Individualmotionvectorsshownaspurplearrows,b)motionvectorsgroupedovertimeintomotiontracksthatcanbeassignedtoanindividual,andc)markedstudentareasandcentresofGaussianprobabilities,whichmodeltheprobabilityofmotion

belongingtoeachstudent.



4.1 Motion Analysis

Analysisofmotionisbasedontrackingfeaturepointsinthevideo(Bouguet,1999).Oursetupconsistsofthreecamerasusedforcoverageofthestudents(showninFigure3)andoneobservingtheteacher.Initialstepsofanalysis—synchronizationofvideostreamsfromallsourcesandannotatingvisibleregions inwhichstudentsresideduringthelecture—aredescribedinRacaandDillenbourg(2013).

Ourmainchallenges intheprocessofextractingameasurementofmotionforfurtheranalysiswere i)interpersonal occlusions, ii) perspective distortion, and iii) normalization of the amount ofmovementrecordedfromasinglepersonintoacomparablemeasurementbetweenseveralpersons.

i)Interpersonalocclusionsarehandledbytakingseveralpre-processingstepsbeforeassigningthemotiontoaperson.Themainideaisthatbygroupingthemotionvectorsintomotiontracks,wecanmorereliablyassign the whole track to a single person, instead of taking each motion vector as an isolatedmeasurement.

StepsoftheprocessareillustratedinFigure2.RawmotionvectorsareshowninFigure2aaspurplearrowswhoseintensitiesaddtotheamountofmotionofonepersonatonetimeinstance.Motionvectors(v)arenextgroupedintotracks(T)whichconsistof“cloud”ofmotionvectorsoverseveralframes.Thecriteriumfor grouping is based on proximity, direction similarity, and intensity of the vectors. For visualizationpurposes,asetofcloudcentresfromseveralframesareconnectedintoatrack,showninFigure2b.Finallytheentiretrackisassignedtothestudentofhighestprobability(gf),definedbytheformulabelow.Eachstudent(g)hasaGaussiandistributioncentredonthepositionofhishead(depictedinFigure2c).Theentiretrackisassessedovereverycentre(i.e.,everystudent)andmotionisassignedtothestudentwiththehighestprobability.

𝑔" = 𝑎𝑟𝑔𝑚𝑎𝑥( ∀*∈,

𝑝(𝑣 ∣ 𝑔)

In cases where a student was occluded on more than 80% of tracked area, the movements wereindistinguishablefromthepersoninfrontofhim/her.Dependingonthequalityofthemeasurementsforthepersoninfront,eitheroneorbothstudentswereremovedfromfurtherprocessingiftheywerebelowasetthreshold.

Takingintoaccountthatourprimaryinterestwasmotionbetweenstudents,itisimportanttonoticethatthismethodwasdesignedsothat

• Amotionoccurringbetweentwostudentswouldnotbeassignedtobothstudents,

• Largemotionsspanningseveraltrackedareaswouldbeassignedtoasingleperson,andnottoagroupofpeople.



Figure3:Arrangementofcamerasforrecordingstudentactions.

ii)Perspectivedistortion:Tocompensatefortheperspectiveeffect,thenumberoftrackingpointsremainsconstantoverallannotatedtrackedareas.Oursecondprecautionwastonormalizetheintensityofthemotionvectorbythediagonalofthestudentregion.Thisensuresthatthehand-motionofthestudentinthebackrowwillberegisteredwiththesameintensityasthehand-motionofthestudentinthefrontrow.

iii)Normalizingtheamountofmotionofapersonhasproventobedifficult.Webasedournormalizationontwopremises:i)thestudentis,onaverage,sittingstillduringtheclass;ii)thestudenthasatleastonefull-bodymovementintherecordedfootage(e.g.,poseshift).Toscalethistoarangeof0–100%motion,we take the median value of movement intensity as the 5% motion (which corresponds to a smallmotion/sittingstillbeingregisteredas5%motion),andweverifythatgiventhisbasicmotionintensitythestudentreaches100%motionatleastonceduringtheclass.Motionthatregistersabovethethresholdof100%isclippedtothemaximumvalue.ThefinalmotionintensityovertimecanbevisualizedasshowninFigure4b.

4.2 Experimental Procedure

Weobservedeach lecture for thedurationof30minutes.Afterarandominterval (averageduration7minutes)atonesignalwasgiventhatinterruptedthelecture.Atthattime,studentswereaskedtofillouta questionnaire sampling their activities and self-reportedperceptionof the classroom. In addition tostudentsamples,wehand-annotatedclasseventsthatwereproductsofteacheractionorteacher–studentinteraction. Events were annotated into following categories: i) slide change, ii) slide animation, iii)questionbegin/endperiod, iv)answerbegin/endperiod,andv)otherevents.Ourquestionnaire fillingperiods(typicallylastingaround1minute)weredesignatedas“questionanswering”periods.Sincetheydo not represent a normal part of a lecture, student activity in those periods was not taken intoconsiderationinfurtherdataanalysis.TheeventsareshownasannotationsinthetoppartofthetimelinevisualizationinFigure4b.



a) b) Figure4:Motionintensitygraphs.Horizontalaxisrepresentsthetimeandverticalaxis0–100%ofrelativemotionoftheperson.a)Exampleofco-movementfortwopersons.Person2shiftedher

seatingposition(blueline),2secondslater,neighbouringPerson1(markedingreen)alsostartedre-adjustingherself.b)Motionofasingleperson(darkgreentrace)overlaidontheaveragemotionofthe

wholeclassroom(graytrace).Thehorizontalredlinemarksthe30%thresholdthatweusedformovementanalysis.Colour-codedlabelsontopindicatedifferenteventsduringtheclass,asdescribed

inSection247.2.Annotationspresenthereare:Bluerectangles—slidechange;Redperiods—questionansweringperiodsorquestionnairefillingperiods;Greenverticallines—slideanimations.

Questionnaires

Byusinga10-pointLikertscale,participantsregisteredthefollowing:

• theirattentionlevel• theirperceptionoftheteacher(energetic/boring)• theirperceptionoftheclassroomattention(high/low)• theimportanceofthematerialpresented(important/irrelevant)

Inadditiontothis,thequestionnaireenumeratedactivitiesthatthestudentsdidduringtheprevioustime-period:

• listening• takingnotes• repeatingkeyideas• thinkingaboutotherthings• interactingwithpeoplearoundyou(ifnotscheduledbytheclassroomactivity)• usingyourlaptop/phone

Studentscouldcheckmorethanoneactivity.

4.3 Student sample

Webaseour resultsonanalysisof twoclasses,described inTable1.Both studentgroupswere in thebachelorprogramofÉcolepolytechniquefédéraledeLausanne(EPFL).Theteachersweretwoexperienced



lecturers teaching social science (Class 1) and technical science (Class 2). The lectures were given atdifferenttimesoftheday—oneinthemorning,theotherinlateafternoon—andindifferentrooms.

Table1:Basicinformationaboutanalyzedclasses.Class Size Analyzed Femaleratio Rows Columns

1 38 29 36.8% 6 7

2 18 14 22.2% 4 5

Eventhoughweinitiallyconsiderbothclassescomparable,thesmallnumberofstudentsinthesecondclassrenderedconclusionsfromthatobservationstatisticallyinvalid.WeshowtheresultsfoundinClass2heretodemonstratetheconsistenttrendinbothcases.

4.4 Location and Surroundings

Oneofourmainconsiderationswhenthinkingabouthowthestudentperceivesthelecturecamefromproxemiczones(Hall&Hall,1969).Sincetheperceptionoftheteacherchangessignificantlydependingonhowfarthestudentisfromthefront,wedecidedagainstnormalizingthespaceinthewayitwasdoneinAdams(1969),whichwouldallowustocreateonebigsamplebymakingthetwoclassescomparable.

Emulatingtheproxemicsconceptintheclassroomenvironment,wedefinedthethreezonesdepicted:

• Immediateneighbourmodels“personalspace.”Thepersontotheimmediateleftorrightofthestudent, with whom the student shares desk- and leg-space. This is partially dictated by thedimensionsofthedesks,whichinthiscasearemadefortwopersonsperdesk.

• Visibleneighbourhoodrepresentsthezoneoftworowsinfrontofthestudent2personswide.This represents the “social zone” of proximal theory (which spans from 1.2m–3m). The zonemodelspeoplewhowouldbeintentionallyorunintentionallyobservedbythestudentfollowingthematerialontheslidesorlookingtowardstheteacher.

• Non-visible students are those either too far to the side or behind the individual to be seenwithoutintentionalaction.

5 OBSERVATIONS

5.1 Questionnaire Data

Thecollectedquestionnairedatawasusedprimarilyasthebasisforfurtheranalysisofthecollectedvideomaterial.Nevertheless,wereportthecondensedfindingstodepictthegeneralsituationintheclassrooms.A general noteon the findings is that becauseof the small numberof samples,weare reportingourfindingswithKendall’scorrelation.



Reportedlevelsofattentioninbothcaseswerehigh,withthemeanlocatedaroundseven.InthecaseofClass1,μ=6.822,σ=2.344,and incaseofClass2 thenormaldistributionhas theparametersμ=7.444,σ=1.100(showninFigure7).Thistrendwasalsoconfirmedbyourfurtherstudiesusingalargersampleofparticipants(194participants),showninFigure7c,ofμ=6.71,σ=1.456.

Itisalsointerestingtoobservethattheattentionreportedwassignificantlycorrelatedwiththedistancefromtheteacher(representedastherowinwhichthestudentwassitting).ResponsesshowninFigure6showthedownwardtrendofcorrelationr(192)=-0.29(p<0.05).ThisfurtherconfirmstheobservationsofDaum(1972).

There is a significant correlation between the personal level of attention and the perceived level ofattentionoftheentireclass(Class1:τ(38)=0.477(p<0.05);Class2:τ(18)=0.413(p<0.05)).Weconsideredthisan interestingwayofexpressingdissatisfactionwithpersonalorclassperformanceas thestudentwould mark a bigger difference between personal and classroom attention if there were a biggerdissatisfactionwiththelearningconditions.Classesweregenerallyperceivedbyparticipantsasexhibitingbothhighteacher-energyandhighstudentattention.

Table2:Parametersofperceivedclassquality.Class Classattention(meanvariance) Teacherenergy(mean,variance)1 6.776(3.711) 7.783(1.866)2 7.125(3.266) 8.347(1.920)

Figure5:Attentionovertimeatfor4differentmomentsduringtheclass.Datacapturedinourextendedstudy,samplesize194participants.



Wealsostudied thevariationofattention levelsover time inhopesof capturing the reporteddrop inconcentrationafter10minutes(Wilson&Korn,2007),butfoundnocleartrend(seeFigure5).

Figure6:Meanattentionreportedoverrows.Linearfitdisplaysaslope-0.29,p<0.05.

a)

b) c)

Figure7:Averageattentionofstudentsinbothclasseswassubjectivelyperceivedashigh.a)Class1(μ=6.822,σ=2.344)andb)Class2(μ=7.444,σ=1.100)c)additionalfindingsfromoursecondstudy(194participants)showsacleanerGaussiandistribution(withtheright-sidetailcut-off)(μ=6.71,σ=1.456)



a) b)

Figure8:Percentageofactivitiesperattentionlevelina)Class1andb)Class2.Numberofreportedinstanceswasnormalizedbythetotalnumberofinstancesonthatattentionleveltoproducethe

percentages.

Activitiesstudentsreported(showninFigure8)showanexpectedtendencytoreportmaterial-relatedactivities (listening to lecture, taking notes, and repeating ideas) in higher attention levels. Off-taskactivities(“thinkingaboutotherthings,”“talkingtoothers”)werereportedonalllevelsuptothemaximumlevelofattention.NotethatthestudentsinClass2wereusingtabletsaspartoftheirregularstudiestoviewtheclassmaterial,whichwasnotrequiredforClass1.

5.2 Motion Data

Synchronizedmovementisdefinedasbodymovementwithmorethan30%intensityfromeachofthetwopersonsbeingcompared(shownasthehorizontalredlineinFigure4b).The30%thresholdwastakentoseparateminorbodymovementsfrommotionlikelytobenoticedbyothers.Wetookintoconsiderationthevisibilityofthetwopersons,meaningthatinorderforthemovementofPerson1tobeconsideredasastimulus,itmustbevisibletoPerson2.Visibilityreasoningwasdonebasedontheseatinglocationofthetwopersons.

Wecomparedtheaveragenumberofsyncedmovementsbetweenpairssittingimmediatelynexttoeachother andother pairs.We found that immediate neighbours had a higher probability of synchronizedmovementthananon-neighbouringpair(usingat-test(p≤0.05)),showninTable3.

Table3:Averagenumberofsynchronizedmomentsbetweenimmediateneighboursandotherpairs.

Class Neighbouringpairsmean(variance) Otherpairsmean(variance)

1 76.54(32.47) 54.43(15.64)

2 63.33(24.33) 44.88(18.42)



Figure9:CorrelationbetweendistancefromteacherandmotionintensityinClass1;Kendallcorrelationt(38)=-0.284(p=0.03)

Weanalyzedbutfoundnosignificantdifferenceinthenumberofsynchronizedmovementsbetweenthepairfromavisibleneighbourhoodandthenon-visiblestudents.

TocomparethemotionmetricswiththepreviousfindingsofAdams(1969)onstudentactivity,wealsotestedtheinfluenceofteacherproximitytothemovementofthestudents.Thefurtherawaystudentsarefromthe front-centreof theclassroom(thepointclosest to the teacher inbothcases, representedasdistancedinFigure5)thelessactivetheyare(Kendallcorrelationisτ(38)=-0.284(p=0.03)forClass1;andτ(18)=-0.172(p=0.45)forClass2).Analyzingthesamples,wehaveseenthesametrendinbothcases,eventhough thecorrelationwas insignificant for the secondclassroom.Figure10shows thecorrelation forClass1.

Figure10:AveragemotionlagcomparedwiththeaveragelevelofattentioninClass1.Kendallcorrelationτ(29)=-0.259(p=0.06).



Ourthirdtestwastofindthecorrelationoftheaveragereportedlevelofattentiontothereactionspeed.The questionwaswhether studentswith lower attention levelsweremore likely to lag behind otherstudents in their visible field. The correlation foundhad theexpected trend in theKendall correlation(τ(29)=-0.259 (p=0.06)) but was marginally insignificant. The result is shown in Figure 10. Class 2’scorrelationhadasimilartrendbutwasnotstatisticallysignificant(τ(18)=-0.222(p=0.32)).Thedatathussuggests a phenomenon of “sleeper’s lag,” but the current sample is not conclusive. In addition, thedifferenceinaveragespeedofreactionisinsub-secondintervals,whichleadsustoquestionifthiswouldbenoticeabletotheteacher’seyewithoutthetechnologicalenhancementoftheclassroom.

6 CONCLUSION

Inthispaper,wedemonstratedourconceptofmeasuringspeedofreactioninthestudentpopulationofthe classroom. We gathered insight about the subjective perception of classroom attention with aquestionnaire, which shows that students will project their level of attention onto others. Our firstconclusionaboutsynchronizationofmotionbetweenimmediateneighboursshowsthattwopersonscanaffecteachotherjustbysittingtogetherwithoutactualdirectinteraction.

Wefoundasimilaritywithpreviousstudiesontheeffectofteacherproximityonstudents(Adams,1969;Daum,1972)andfoundthatstudentswhoarefurtherawaynotonlyparticipateless,butalsomovelessandreportlowerattention.

Finally,weproposedanewwayofevaluatingtheoverallattentionoftheclassroombycomparingpairsofstudents and analyzing how synchronously they move. By comparing the motion results to the datagatheredinthequestionnaire,weshowedacorrelationbetweenslowerreactiontimeandlowerlevelsofreportedattention—the“sleepers’lag,”butourdatawasnotconclusive.

Wehavenotyettouchedonthesubjectofpresentingtheinformationtotheteachersduringthelecture,andweareplanningtostartadialoguewiththeparticipatingteacherstofindthebestrepresentationfordisplaying the informationduring the lecture.Ournext stepsare toconfirmthe findingsonabroadersampleofstudentsandcontinuetorefinethetechnologicalmethods.Inadditiontothe“sleepers’lag”wewouldalsoliketoexplorefurtherthephenomenonwecall“distractionripples”—assumingthetransitivityofmotionsyncing,wewould liketocapturethespreadof influencefromoneclass-membertopeoplearoundhim/her.Wearealsointerestedincorrelatinghowwellthese“ripples”spreadinhigh-attentionandlow-attentiongroupsofstudentsinordertoformulateanewmetricofclassattention.

Inadditiontomotion,weaimtointroduceadditionalcuesintoourreasoningaboutstudentattentionandperceptionoftheclass,specificallygazedirection.Thegoalistoprovideaholisticimageofclassroomlifeinordertofindthemostsalientcuesthatcanbeunobtrusivelycollected.Ourintentionisthat,intheend,theentiresystemwouldactasatrainingexperiencefornoviceteacherswhilealsoprovidingfeedbacktoexperiencedteachersforcontinuedprofessionaldevelopment.

Stepping back from the trend of individual learning with massive online open courses (MOOCs),classrooms remain the dominant site of learning at all educational levels. Introducing technological



solutions to the classroom can potentially have a huge impact on the way students learn. Bysupplementing teacher observationswith advancedmeasures,we hope to create a blend superior tocurrentmethodsthatexcludeteachers,onethatwillbebeneficialforstudentsandteachersboth.

ACKNOWLEDGEMENTS

ThisworkhasbeensponsoredbytheProDocSNFGrant,projectPDFMP1135108.Wewouldalsoliketothankalloftheexperimentparticipants(studentsandteachers)fortheirhelpingatheringthedata.

REFERENCES

Adams,R.S.(1969).Locationasafeatureofinstructionalinteraction.Merrill-PalmerQuarterlyofBehaviorandDevelopment,15(4),309–321.

Alavi,H.S. (2012).Ambientawareness for theorchestrationofcollaborativeproblemsolving (Doctoraldissertation, École polytechnique fédérale de Lausanne, Lausanne).http://dx.doi.org/10.5075/epfl-thesis-5275

Allen, J. D. (1986). Classroom management: Students’ perspectives, goals, and strategies. AmericanEducationalResearchJournal,23(3),437–459.http://dx.doi.org/10.3102/00028312023003437

Altman, I.,& Lett, E. E. (1970). The ecology of interpersonal relationships: A classification systemandconceptual model. In J. McGrath (Ed.), Social and psychological factors in stress (pp. 177–201).NewYork:Holt,Rinehart&Winston.

Arroyo, I.,Cooper,D.G.,Burleson,W.,Woolf,B.P.,Muldner,K.,&Christopherson,R. (2009).Emotionsensorsgotoschool.Proceedingsofthe14thInternationalConferenceonArtificialIntelligenceinEducation: Building Learning Systems that Care: From Knowledge Representation to AffectiveModelling (AIED ’09), 17–24. Retrieved fromhttp://centerforknowledgecommunication.com/newPubs/AIEDSENSORSCameraReady.pdf

Bachour, K. (2010).Augmenting face-to-face collaborationwith low-resolution semi-ambient feedback(Doctoral dissertation, École Polytechnique Fédérale de Lausanne, Lausanne).http://dx.doi.org/10.5075/epfl-thesis-4895

Bazzani,L.,Cristani,M.,Tosato,D.,Farenzena,M.,Paggetti,G.,Menegaz,G.,&Murino,V.(2013).Socialinteractions by visual focus of attention in a three-dimensional environment. Expert Systems,30(2),115–127.http://dx.doi.org/10.1111/j.1468-0394.2012.00622.x

Bernstein, D. J. (2008). Peer review and evaluation of the intellectual work of teaching.Change: TheMagazineofHigherLearning,40(2),48–51.

Borko,H.,&Livingston,C.(1989).Cognitionandimprovisation:Differencesinmathematicsinstructionbyexpert and novice teachers. American Educational Research Journal, 26(4), 473–498.http://dx.doi.org/10.3102/00028312026004473

Bouguet, J.-Y. (1999). Pyramidal implementation of the Affine Lucas Kanade feature tracker. IntelCorporation.Retrievedfromhttp://robots.stanford.edu/cs223b04/algo_tracking.pdf

Caldwell, J. E. (2007). Clickers in the large classroom:Current researchandbest-practice tips.CBE-LifeSciencesEducation,6(1),9–20.http://dx.doi.org/10.1187/cbe.06-12-0205

Chen,D.,&Vertegaal,R.(2004).Usingmentalloadformanaginginterruptionsinphysiologicallyattentive



userinterfaces.InProceedingsofCHIEA’04,ExtendedAbstractsonHumanFactorsinComputingSystems,1513–1516.http://dx.doi.org/10.1145/985921.986103

Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. Perspectives on Socially SharedCognition,13(1991),127–149.http://dx.doi.org/10.1037/10096-006

Coe, R., Aloisi, C., Higgins, S., & Major, L. E. (2014). What Makes Great Teaching? Review of theUnderpinning Research (Research Report). Retrieved from the Sutton Trust websitehttp://www.suttontrust.com/wp-content/uploads/2014/10/What-Makes-Great-Teaching-REPORT.pdf

Conigliaro,D.,Setti,F.,Bassetti,C.,Ferrario,R.,&Cristani,M.(2013).ATTENTO:ATTENTionObservedforautomated spectator crowd analysis. In A. Salah, H. Hung, O. Aran, H. Gunes (Eds.), HumanBehavior Understanding: proceedings of the 4th International Workshop (pp. 102–111).Berlin:Springer.http://dx.doi.org/10.1007/978-3-319-02714-2_9

Corcoran,R.P.,&Tormey,R.(2012).Developingemotionallycompetentteachers:Emotionalintelligenceandpre-serviceteachereducation.Oxford:PeterLang/ERIC.

Daly,J.A.,&Suite,A.(1981).Classroomseatingchoiceandteacherperceptionsofstudents.TheJournalofExperimentalEducational,50(2),64–69.http://dx.doi.org/10.1080/00220973.1981.11011803

Daum, J. (1972). Proxemics in the classroom: Speaker–subject distance and educational performance.PaperpresentedattheannualmeetingofSoutheasternPsychologicalAssociation.

Dillenbourg,P.,&Jermann,P.(2010).Technologyforclassroomorchestration.InKhine,M.S,I.M.Saleh(Eds.),Newscienceof learning:Cognition,ComputersandCollaboration inEducation (pp.525–552).Berlin:Springer.http://dx.doi.org/10.1007/978-1-4419-5716-0_26

Dillenbourg,P.,Zufferey,G.,Alavi,H.,Jermann,P.,Do-Lenh,S.,Bonnard,Q.,…Kaplan,F.(2011).Classroomorchestration: The third circle of usability. In H. Spada, G. Stahl, N. Miyake, N. Law (Eds.),Proceedingsof the9th InternationalConferenceonComputer-SupportedCollaborative Learning(CSCL2011),(Vol.1,pp.510–517).Lulu:ISLS.

Dyckhoff,A.L.,Lukarov,V.,Muslim,A.,Chatti,M.A.,&Schroeder,U.(2013).Supportingactionresearchwithlearninganalytics.Proceedingsofthe3rdInternationalConferenceonLearningAnalyticsandKnowledge(LAK’13),220–229.http://dx.doi.org/10.1145/2460296.2460340

ElKaliouby,R.,&Robinson,P.(2004).Mindreadingmachines:Automatedinferenceofcognitivementalstatesfromvideo.Proceedingsofthe2004IEEEInternationalConferenceonSystems,ManandCybernetics(SMC2004),(Vol.1,pp.682–688).http://dx.doi.org/10.1109/ICSMC.2004.1398380

Emmer,E.T.,&Stough,L.M.(2001).Classroommanagement:Acriticalpartofeducationalpsychology,with implications for teacher education. Educational Psychologist, 36(2), 103–112.http://dx.doi.org/10.1207/S15326985EP3602_5

Ericsson,A.,K. (2008).Deliberatepracticeandacquisitionofexpertperformance:Ageneraloverview.Academic Emergency Medicine, 15(11), 988–994. http://dx.doi.org/10.1111/j.1553-2712.2008.00227.x

Finn,J.D.,Pannozzo,G.M.,&Achilles,C.M.(2003).The“why’s”ofclasssize:Studentbehaviorinsmallclasses. Review of Educational Research, 73(3), 321–368.http://dx.doi.org/10.3102/00346543073003321

Forsyth,D.(2009).Groupdynamics(5thedition).Boston,MA:CengageLearning.



Gatica-Perez,D.(2009).Automaticnonverbalanalysisofsocialinteractioninsmallgroups:Areview.ImageandVisionComputing,27(12),1775–1787.http://dx.doi.org/10.1016/j.imavis.2009.01.004

Hall,E.T.,&Hall,E.T.(1969).Thehiddendimension(reprinted1990).NewYork:AnchorBooks.Hargreaves,A.(2000).Mixedemotions:Teachers’perceptionsoftheirinteractionswithstudents.Teaching

andTeacherEducation,16(8),811–826.http://dx.doi.org/10.1016/S0742-051X(00)00028-7Hattie,J.(2013).Visiblelearning:Asynthesisofover800meta-analysesrelatingtoachievement.London:

Routledge.Heylighen,F.(2002).Complexityandinformationoverloadinsociety:Whyincreasingefficiencyleadsto

decreasingcontrol.TheInformationSociety,4,1–44.Horvitz,E.,Kadie,C.,Paek,T.,&Hovel,D.(2003).Modelsofattentionincomputingandcommunication:

From principles to applications. Communications of the ACM, 46(3), 52–59.http://dx.doi.org/10.1145/636772.636798

Howard,J.R.,&Henney,A.L.(1998).Studentparticipationandinstructorgenderinthemixed-agecollegeclassroom.JournalofHigherEducation,69,384–405.

Jermann, P., & Nüssli, M.-A. (2012). Effects of sharing text selections on gaze cross-recurrence andinteractionqualityinapairprogrammingtask.Proceedingsofthe2012ConferenceonComputerSupported Cooperative Work (CSCW ʼ12), 1125–1134.http://dx.doi.org/10.1145/2145204.2145371

Kahneman,D.(2011).Thinking,fastandslow.NewYork:Farrar,StrausandGiroux.Koneya,M.(1976).Locationandinteractioninrow-and-columnseatingarrangements.Environmentand

Behavior,8(2),265–282.http://dx.doi.org/10.1177/001391657682005Marks,H.M.(2000).Studentengagementininstructionalactivity:Patternsintheelementary,middle,and

high school years. American Educational Research Journal, 37(1), 153–184.http://dx.doi.org/10.3102/00028312037001153

Middendorf,J.,&Kalish,A.(1996,February).The“change-up”inlectures.NationalTeaching&LearningForum,5(3),1–5.http://dx.doi.org/10.1002/ntlf.10026

Moore,M.G.(1989).Editorial:Threetypesofinteraction.TheAmericanJournalofDistanceEducation,3(2).http://dx.doi.org/10.1080/08923648909526659

Nüssli,M.-A. (2011).Dual eye-trackingmethods for the studyof remote collaborativeproblem solving(Doctoral dissertation, École polytechnique fédérale de Lausanne, Lausanne).http://dx.doi.org/10.5075/epfl-thesis-5232

Raca, M., & Dillenbourg, P. (2013). System for assessing classroom attention. Proceedings of the 3rdInternational Conference on Learning Analytics and Knowledge (LAK ’13), 265–269.http://dx.doi.org/10.1145/2460296.2460351

Range,B.,Duncan,H.,&Hvidston,D. (2013).How faculty supervise andmentorpre-service teachers:Implicationsforprincipalsupervisionofnoviceteachers.LeadershipPreparation,8(2)43–58.

Rapp,D.N.(2006).Thevalueofattentionawaresystemsineducationalsettings.Computers inHumanBehavior,22(4),603–614.http://dx.doi.org/10.1016/j.chb.2005.12.004

Richardson,D. C.,Dale, R.,&Kirkham,N. Z. (2007). The art of conversation is coordination: Commongroundandthecouplingofeyemovementsduringdialogue.PsychologicalScience,18(5),407–413.http://dx.doi.org/10.1111/j.1467-9280.2007.01914.x



Rivera-Pelayo,V.,Munk,J.,Zacharias,V.,&Braun,S.(2013).Liveinterestmeter:Learningfromquantifiedfeedbackinmasslectures.Proceedingsofthe3rdInternationalConferenceonLearningAnalyticsandKnowledge(LAK’13),23–27.http://dx.doi.org/10.1145/2460296.2460302

Roda, C., & Thomas, J. (2006). Attention aware systems: Theories, applications, and research agenda.ComputersinHumanBehavior,22(4),557–587.http://dx.doi.org/10.1016/j.chb.2005.12.005

Rosengrant,D.,Hearrington,D.,Alvarado,K.,Keeble,D.,Rebello,N.S.,Engelhardt,P.V.,&Singh,C.(2012).Followingstudentgazepatternsinphysicalsciencelectures.AIPConferenceProceedingsof2011PhysicsEducationResearchConference(Vol.1413,p.323).http://dx.doi.org/10.1063/1.3680060

Schön,D.A.(1983).Thereflectivepractitioner:Howprofessionalsthinkinaction.NewYork:BasicBooks.Shell,D. F.,Brooks,D.W., Trainin,G.,Wilson,K.M.,Kauffman,D. F.,&Herr, L.M. (2010).Theunified

learningmodel.Berlin:Springer.Stiefelhagen,R.,&Zhu,J.(2002).Headorientationandgazedirectioninmeetings.InProceedingsofCHI’02

Extended Abstracts on Human Factors in Computing Systems (pp. 858–859).http://dx.doi.org/10.1145/506443.506634

Timperley,H.,Wilson,A.,Barrar,H.,&Fung,I.(2008).TeacherProfessionalLearningandDevelopment:BestEvidenceSynthesisIteration[BES].IterativeNewZealand:BestEvidenceSynthesisProgram.Retrievedfromhttp://www.oecd.org/edu/school/48727127.pdf

Vinciarelli,A.,Pantic,M.,&Bourlard,H.(2009).Socialsignalprocessing:Surveyofanemergingdomain.ImageandVisionComputing,27(12),1743–1759.http://dx.doi.org/10.1016/j.imavis.2008.11.007

Vinciarelli,A.,Pantic,M.,Heylen,D.,Pelachaud,C.,Poggi,I.,D’Errico,F.,&Schröder,M.(2012).Bridgingthegapbetweensocialanimalandunsocialmachine:Asurveyofsocialsignalprocessing. IEEETransactionsonAffectiveComputing,3(1),69–87.http://dx.doi.org/10.1109/T-AFFC.2011.27

Webb,E.J.,Campbell,D.T.,Schwartz,R.D.,&Sechrest,L. (1999).Unobtrusivemeasures (Vol.2).NewYork:SagePublications.

Weiser, M. (1991). The computer for the 21st century. Scientific American, 265(3), 94–104.http://dx.doi.org/10.1145/329124.329126

Whitcomb,J.,Borko,H.,&Liston,D.(2008).Whyteach?PartII.JournalofTeacherEducation,59(4),267–272.http://dx.doi.org/10.1177/0022487108322199

Wilson,K.,&Korn,J.H.(2007).Attentionduringlectures:Beyondtenminutes.TeachingofPsychology,34(2),85–89.http://dx.doi.org/10.1177/009862830703400202

Worsley,M.,&Blikstein,P. (2013). Towards thedevelopmentofmultimodal actionbasedassessment.Proceedingsofthe3rdInternationalConferenceonLearningAnalyticsandKnowledge(LAK’13),94–101.http://dx.doi.org/10.1145/2460296.2460315

Documents

Sleepers’ Lag: Study on Motion and AttentionKeywords: Video analysis, computer vision, tracking, body motion, classroom, interpersonal synchronization, orchestration 1 INTRODUCTION