1
1 D’Mello, S. K. & Graesser, A. C. (2014). Confusion. In R. Pekrun & L. LinnenbrinkGarcia (Eds.). International handbook of emotions in education. New York, NY: Routledge. 2 Hammer, D. (1996). Misconceptions or PPrims: How may alternative perspectives of cognitive structure influence instructional perceptions and intentions. Journal of the Learning Sciences, 5(2), 97–127. doi:10.1207/s15327809jls0502_1 3 Christensen, S. M., & Turner, D. R. (Eds.). (1993). Folk psychology and the philosophy of mind. New York: Psychology Press. 4 Lee, G., & Byun, T. (2011). An explanation for the difficulty of leading conceptual change using a counterintuitive demonstration: The relationship between cognitive conflict and responses. Research in Science Education, 42(5), 943–965. doi:10.1007/s1116501192345 5 Jacobson, M. J., & Wilensky, U. (2006). Complex systems in education: Scientific and educational importance and implications for the learning sciences. Journal of the Learning Sciences, 15(1), 11–34. doi:10.1207/s15327809jls1501_4 6 Jacobson, M. J., Kapur, M., So, H.J., & Lee, J. (2010). The ontologies of complexity and learning about complex systems. Instructional Science, 39(5), 763–783. doi:10.1007/s1125101091470 7 Meyer, J. H., & Land, R. (2005). Threshold concepts and troublesome knowledge (2): Epistemological considerations and a conceptual framework for teaching and learning. Higher Education, 49(3), 373388. 8 diSessa, A. A. (2014). The Construction of Causal Schemes: Learning Mechanisms at the Knowledge Level. Cognitive Science, 38(5), 795–850. doi:10.1111/cogs.12131 9 Alexander, P. A., Kulikowich, J. M., & Schulze, S. K. (1994). The influence of topic knowledge, domain knowledge, and interest on the comprehension of scientific exposition. Learning and Individual Differences, 6(4), 379397. 10 Kapur, M. (2008). Productive Failure. Cognition and Instruction, 26(3), 379–424. doi:10.1080/07370000802212669 11 VanLehn, K., Siler, S., Murray, C., & Yamauchi, T. (2003). Why do only some events cause learning during human tutoring? Cognition and Instruction, 21(3), 209–249. doi:10.1207/S1532690XCI2103_01 12 Storm, C., & Storm, T. (1987). A taxonomic study of the vocabulary of emotions. Journal of Personality and Social Psychology, 53, 805–816. 13 D’Mello, S., & Graesser, A. (2014). Confusion and its dynamics during device comprehension with breakdown scenarios. Acta Psychologica, 151(C), 106116. doi:10.1016/j.actpsy.2014.06.005 14 Hess, U. (2003). Now you see it, now you don'tthe confusing case of confusion as an emotion: Commentary on Rozin and Cohen (2003). Emotion, 3(1), 76–80. doi:10.1037/15283542.3.1.76 15 Silvia, P. J. (2010). Confusion and interest: The role of knowledge emotions in aesthetic experience. Psychology of Aesthetics, Creativity, and the Arts, 4(2), 75. 16 D’Mello, S., & Graesser, A. (in press) Inducing and tracking confusion and cognitive disequilibrium with breakdown scenarios. Memory and Cognition. 17 Hays, M. J., Kornell, N., & Bjork, R. A. (2010). The costs and benefits of providing feedback during learning. Psychonomic Bulletin & Review, 17(6), 797–801. doi:10.3758/PBR.17.6.797 18 Lehman, B., D'Mello, S., & Graesser, A. (2013). Who benefits from confusion induction during learning? An individual differences cluster analysis. In K. Yacef et al. (Eds.) Artificial Intelligence in Education (51–60), Berlin: SpingerVerlag. 19 D’Mello, S., & Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2), 145–157. doi:10.1016/j.learninstruc.2011.10.001 20 D’Mello, S., Lehman, B., Pekrun, R., & Graesser, A. (2014). Confusion can be beneficial for learning. Learning and Instruction, 29, 153–170. doi:10.1016/j.learninstruc.2012.05.003 21 Pekrun, R., & Stephens, E. J. (2011). Academic emotions. In APA educational psychology handbook, Vol 2: Individual differences and cultural and contextual factors. (pp. 3–31). Washington: American Psychological Association. doi: 10.1037/13274001 22 Acee, T. W., Kim, H., Kim, H. J., Kim, J.I., Chu, H.N. R., Kim, M., et al. (2010). Academic boredom in under and overchallenging situations. Contemporary Educational Psychology, 35(1), 17–27. doi:10.1016/j.cedpsych.2009.08.002 23 Graesser, A. C., Conley, M. W., & Olney, A. (2011). Intelligent tutoring systems. In APA educational psychology handbook, Vol 3: Application to learning and teaching. (pp. 451–473). Washington: American Psychological Association. doi: 10.1037/13275018 24 EstebanMillat, I., & MartínezLópez, F. J. (2014). Modelling students' flow experiences in an online learning environment. Computers & Education. doi:10.1016/j.compedu.2013.09.012 25 Chi, M. T. H. (2005). Commonsense Conceptions of Emergent Processes: Why Some Misconceptions Are Robust. Journal of the Learning Sciences, 14(2), 161–199. doi:10.1207/s15327809jls1402_1 26 Dweck, C. S. (2002). Messages that motivate: How praise molds students’ beliefs, motivation, and performance (in surprising ways). In J. Aronson (Ed.), Improving academic achievement: Impact of psychological factors on education (pp. 37–60). Orlando, FL: Academic Press. Predicting confusion 1. Is there a robust method for inducing learner confusion? 2. What are the similarities/differences in functional activation across epistemological domains? 3. What are the neural correlates of constructive confusion? 4. Can cognitive neuroscience research on error correction and feedback inform learning design and interventions for nonconstructive confusion? Predicting learner confusion for enhanced feedback and self-regulation: A conceptual framework Jason Lodge 1 , Mariya Pachman 2 , Amael Arguel 2 , Rachel Buckley 3 , Gregor Kennedy 1 , Lori Lockyer 2 , Ottmar Lipp 4 , Rob Hester 3 , & Mike Timms 5 1 Centre for the Study of Higher Education & Melbourne Graduate School of Education, University of Melbourne, 2 School of Education, Macquarie University, 3 School of Psychological Sciences, University of Melbourne, 4 School of Psychology & Speech Pathology, Curtin University, 5 Australian Council for Educational Research A framework for learner confusion Research Questions Understanding confusion 1. Does confusion contribute to conceptual change if so, how? 2. What are the behavioural and information processing factors underpinning constructive confusion? 3. Does confusion differ between novices and experts? If so, can the differences be used to inform learning design? 4. What is the relationship between confusion, interest and motivation? Designing for constructive confusion 1. What types of learning designs lead to constructive confusion? 2. How can feedback be used to ensure that confusion leads to constructive outcomes? 3. What is the role of selfregulation in constructive confusion? 4. How can students be assisted to recover from the effects of nonconstructive confusion? Methods engagement/flow 24 (equilibrium) confusion 1 (disequilibrium) 16 frustration 21 boredom 22 (equilibrium) impasse detected impasse resolved failure/ goals blocked additional impasse persistent failure/ hopelessness lack of control/ forced effort Adapted from D’Mello & Graesser, 2014 1 misconception 2 folk knowledge 3 counterintuitive 4 complex knowledge 5 systemic 6 troublesome 7 novel information 8 unfamiliar 6 knowledge domain 9 Discovery Translation Application neural and physiological correlates of confusion fMRI facial electromyography skin conductance neuroscience of productive confusion error correction feedback processing and confusion recognition, recall, transfer judgments of learning disfluency/cognitive load/ disequilibrium design and confusion performance irt memory prototype testing analytics/data mining experience of confusion confidence self-regulation frustration & boredom interest & motivation learning designs learner performance confidence metacognition conceptual change constructive confusion 18 non- constructive confusion 19 reinforcement of misconception 25 cognitive 12 processing difficulty 13 emotional 14 epistemic emotion 15 combine epistemological antecedents underlying processes phenomenological experience intervention/ feedback 17 conceptual change process conceptual change 20 disengagement 26 (learned helplessness) intervention/ feedback 17 academic recovery 23 primary outcome secondary outcome meta-level outcome learning design 10,11 References

Predicting learner confusion for enhanced feedback and

  • Upload
    others

  • View
    4

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Predicting learner confusion for enhanced feedback and

1D’Mello,  S.  K.  &  Graesser,  A.  C.  (2014).  Confusion.  In  R.  Pekrun  &  L.  Linnenbrink-­‐Garcia  (Eds.).  International  handbook  of  emotions  in  education.  New  York,  NY:  Routledge.  2Hammer,  D.  (1996).  Misconceptions  or  P-­‐Prims:  How  may  alternative  perspectives  of  cognitive  structure  influence  instructional  perceptions  and  intentions.  Journal  of  the  Learning  Sciences,  5(2),  97–127.  doi:10.1207/s15327809jls0502_1  3Christensen,  S.  M.,  &  Turner,  D.  R.  (Eds.).  (1993).  Folk  psychology  and  the  philosophy  of  mind.  New  York:  Psychology  Press.  4Lee,  G.,  &  Byun,  T.  (2011).  An  explanation  for  the  difficulty  of  leading  conceptual  change  using  a  counterintuitive  demonstration:  The  relationship  between  cognitive  conflict  and  responses.  Research  in  Science  Education,  42(5),  943–965.                                doi:10.1007/s11165-­‐011-­‐9234-­‐5  5Jacobson,  M.  J.,  &  Wilensky,  U.  (2006).  Complex  systems  in  education:  Scientific  and  educational  importance  and  implications  for  the  learning  sciences.  Journal  of  the  Learning  Sciences,  15(1),  11–34.  doi:10.1207/s15327809jls1501_4  6Jacobson,  M.  J.,  Kapur,  M.,  So,  H.-­‐J.,  &  Lee,  J.  (2010).  The  ontologies  of  complexity  and  learning  about  complex  systems.  Instructional  Science,  39(5),  763–783.  doi:10.1007/s11251-­‐010-­‐9147-­‐0  7Meyer,  J.  H.,  &  Land,  R.  (2005).  Threshold  concepts  and  troublesome  knowledge  (2):  Epistemological  considerations  and  a  conceptual  framework  for  teaching  and  learning.  Higher  Education,  49(3),  373-­‐388.  8diSessa,  A.  A.  (2014).  The  Construction  of  Causal  Schemes:  Learning  Mechanisms  at  the  Knowledge  Level.  Cognitive  Science,  38(5),  795–850.  doi:10.1111/cogs.12131  9Alexander,  P.  A.,  Kulikowich,  J.  M.,  &  Schulze,  S.  K.  (1994).  The  influence  of  topic  knowledge,  domain  knowledge,  and  interest  on  the  comprehension  of  scientific  exposition.  Learning  and  Individual  Differences,  6(4),  379-­‐397.  10Kapur,  M.  (2008).  Productive  Failure.  Cognition  and  Instruction,  26(3),  379–424.  doi:10.1080/07370000802212669  11VanLehn,  K.,  Siler,  S.,  Murray,  C.,  &  Yamauchi,  T.  (2003).  Why  do  only  some  events  cause  learning  during  human  tutoring?  Cognition  and  Instruction,  21(3),  209–249.  doi:10.1207/S1532690XCI2103_01  12Storm,  C.,  &  Storm,  T.  (1987).  A  taxonomic  study  of  the  vocabulary  of  emotions.  Journal  of  Personality  and  Social  Psychology,  53,  805–816.  13D’Mello,  S.,  &  Graesser,  A.  (2014).  Confusion  and  its  dynamics  during  device  comprehension  with  breakdown  scenarios.  Acta  Psychologica,  151(C),  106-­‐116.  doi:10.1016/j.actpsy.2014.06.005  14Hess,  U.  (2003).  Now  you  see  it,  now  you  don't-­‐-­‐the  confusing  case  of  confusion  as  an  emotion:  Commentary  on  Rozin  and  Cohen  (2003).  Emotion,  3(1),  76–80.  doi:10.1037/1528-­‐3542.3.1.76  !

15Silvia,  P.  J.  (2010).  Confusion  and  interest:  The  role  of  knowledge  emotions  in  aesthetic  experience.  Psychology  of  Aesthetics,  Creativity,  and  the  Arts,  4(2),  75.  16D’Mello,  S.,  &  Graesser,  A.  (in  press)  Inducing  and  tracking  confusion  and  cognitive  disequilibrium  with  breakdown  scenarios.  Memory  and  Cognition.  17Hays,  M.  J.,  Kornell,  N.,  &  Bjork,  R.  A.  (2010).  The  costs  and  benefits  of  providing  feedback  during  learning.  Psychonomic  Bulletin  &  Review,  17(6),  797–801.  doi:10.3758/PBR.17.6.797  18Lehman,  B.,  D'Mello,  S.,  &  Graesser,  A.  (2013).  Who  benefits  from  confusion  induction  during  learning?  An  individual  differences  cluster  analysis.  In  K.  Yacef  et  al.  (Eds.)  Artificial  Intelligence  in  Education  (51–60),  Berlin:  Spinger-­‐Verlag.  19D’Mello,  S.,  &  Graesser,  A.  (2012).  Dynamics  of  affective  states  during  complex  learning.  Learning  and  Instruction,  22(2),  145–157.  doi:10.1016/j.learninstruc.2011.10.001  20D’Mello,  S.,  Lehman,  B.,  Pekrun,  R.,  &  Graesser,  A.  (2014).  Confusion  can  be  beneficial  for  learning.  Learning  and  Instruction,  29,  153–170.  doi:10.1016/j.learninstruc.2012.05.003  21Pekrun,  R.,  &  Stephens,  E.  J.  (2011).  Academic  emotions.  In  APA  educational  psychology  handbook,  Vol  2:  Individual  differences  and  cultural  and  contextual  factors.  (pp.  3–31).  Washington:  American  Psychological  Association.  doi:10.1037/13274-­‐001  22Acee,  T.  W.,  Kim,  H.,  Kim,  H.  J.,  Kim,  J.-­‐I.,  Chu,  H.-­‐N.  R.,  Kim,  M.,  et  al.  (2010).  Academic  boredom  in  under-­‐  and  over-­‐challenging  situations.  Contemporary  Educational  Psychology,  35(1),  17–27.  doi:10.1016/j.cedpsych.2009.08.002  23Graesser,  A.  C.,  Conley,  M.  W.,  &  Olney,  A.  (2011).  Intelligent  tutoring  systems.  In  APA  educational  psychology  handbook,  Vol  3:  Application  to  learning  and  teaching.  (pp.  451–473).  Washington:  American  Psychological  Association.  doi:10.1037/13275-­‐018  24Esteban-­‐Millat,  I.,  &  Martínez-­‐López,  F.  J.  (2014).  Modelling  students'  flow  experiences  in  an  online  learning  environment.  Computers  &  Education.  doi:10.1016/j.compedu.2013.09.012  25Chi,  M.  T.  H.  (2005).  Commonsense  Conceptions  of  Emergent  Processes:  Why  Some  Misconceptions  Are  Robust.  Journal  of  the  Learning  Sciences,  14(2),  161–199.  doi:10.1207/s15327809jls1402_1  26Dweck,  C.  S.  (2002).  Messages  that  motivate:  How  praise  molds  students’  beliefs,  motivation,  and  performance  (in  surprising  ways).  In  J.  Aronson  (Ed.),  Improving  academic  achievement:  Impact  of  psychological  factors  on  education  (pp.  37–60).  Orlando,  FL:  Academic  Press.  

Predicting  confusion  !

1. Is  there  a  robust  method  for  inducing  learner  confusion?  

!2. What  are  the  similarities/differences  in  functional  

activation  across  epistemological  domains?  !3. What  are  the  neural  correlates  of  constructive  

confusion?  !4. Can  cognitive  neuroscience  research  on  error  correction  

and  feedback  inform  learning  design  and  interventions  for  non-­‐constructive  confusion?

Predicting learner confusion for enhanced feedback and self-regulation: A conceptual frameworkJason Lodge1, Mariya Pachman2, Amael Arguel2, Rachel Buckley3, Gregor Kennedy1, Lori Lockyer2, Ottmar Lipp4, Rob Hester3, & Mike Timms5 1Centre for the Study of Higher Education & Melbourne Graduate School of Education, University of Melbourne, 2School of Education, Macquarie University, 3School of Psychological Sciences, University of Melbourne, 4School of Psychology & Speech Pathology, Curtin University, 5Australian Council for Educational Research

A framework for learner confusion

Research QuestionsUnderstanding  confusion  

!1. Does  confusion  contribute  to  conceptual  change  -­‐  if  so,  

how?  !2. What  are  the  behavioural  and  information  processing  

factors  underpinning  constructive  confusion?  !3. Does  confusion  differ  between  novices  and  experts?  If  

so,  can  the  differences  be  used  to  inform  learning  design?  

!4. What  is  the  relationship  between  confusion,  interest  

and  motivation?

Designing  for  constructive  confusion  !

1. What  types  of  learning  designs  lead  to  constructive  confusion?  

!2. How  can  feedback  be  used  to  ensure  that  confusion  

leads  to  constructive  outcomes?  !3. What  is  the  role  of  self-­‐regulation  in  constructive  

confusion?  !4. How  can  students  be  assisted  to  recover  from  the  

effects  of  non-­‐constructive  confusion?

Methods

engagement/flow24!(equilibrium)

confusion1!(disequilibrium)16 frustration21

boredom22!(equilibrium)

impasse detected

impasse resolved

failure/

goals blocked

additional impasse

persistent failure/

hopelessness

lack of control/

forced effort

Adapted from D’Mello & Graesser, 20141

misconception2!folk knowledge3 counterintuitive4

complex knowledge5!systemic6

troublesome7

novel information8!unfamiliar6

knowledge domain9

Discovery Translation Application

neural and physiological !correlates of confusion!

fMRI facial electromyography

skin conductance

neuroscience of productive !confusion!

error correction feedback

processing and confusion!recognition, recall, transfer

judgments of learning disfluency/cognitive load/

disequilibrium

design and confusion!performance irt memory

prototype testing analytics/data mining

experience of confusion!confidence

self-regulation frustration & boredom interest & motivation

learning designs!learner performance

confidence metacognition

conceptual change

constructive!confusion18

non-constructive!confusion19

reinforcement of!misconception25

cognitive12 !processing difficulty13

emotional14 !epistemic emotion15

combine

epistemological antecedents underlying processes phenomenological experience

intervention/ feedback17

conceptual change

process

conceptual !change20

disengagement26!(learned

helplessness)

intervention/ feedback17

academic recovery23

primary outcome secondary outcome meta-level outcome

learning design10,11

References