View
44
Download
0
Category
Preview:
DESCRIPTION
Studying and achieving robust learning with PSLC resources. Prepared by: Ken Koedinger HCII & Psychology, CMU CMU Director of PSLC Presented by: Vincent Aleven HCII, CMU Member PSLC Executive Committee. 7th Annual Pittsburgh Science of Learning Center Summer School. 11th overall - PowerPoint PPT Presentation
Citation preview
1
Studying and achieving robust learning with PSLC resourcesPrepared by: Ken Koedinger
HCII & Psychology, CMUCMU Director of PSLC
Presented by: Vincent AlevenHCII, CMUMember PSLC Executive Committee
2
7th Annual Pittsburgh Science of Learning Center Summer School
• 11th overall– ITS was focus in
2001 to 2004
• Goals:– Learning science &
technology concepts & tools
– Hands-on project => poster on Fri
3
Vision for PSLC
• Why? Chasm between science & practiceIndicators: Ed achievement gaps persist,Low success rate of randomized controlled trials
• Underlying problem: Many ideas, too little sound scientific foundation
• Need: Basic research studies in the field
=> PSLC Purpose: Identify the conditions that cause robust student learning– Field-based rigorous science – Leverage cognitive & computational theory,
educational technologies
“rigorous, sustained scientific research in education” (NRC, 2002)
4
Builds off past success: Intelligent Tutors Bringing Learning Science to Schools!
• Intelligent tutoring systems– Automated 1:1 tutor– Artificial Intelligence– Cognitive Psychology
• Andes: College Physics Tutor– Replaces homework
• Algebra Cognitive Tutor – Part of complete course
Students: model problems with Students: model problems with diagrams, graphs, equationsdiagrams, graphs, equations
Tutor: feedback, help, Tutor: feedback, help, reflective dialogreflective dialog
5
Tutors make a significant difference in improving student learning!
• Andes: College Physics Tutor– Field studies: Significant
improvements in student learning
• Algebra Cognitive Tutor – 10+ full year field
studies: improvements on problem solving, concepts, basic skills
– Regularly used in 1000s of schools by 100,000s of students!
0
10
20
30
40
50
60
Iowa SAT subset ProblemSolving
Represent-ations
Traditional Algebra Course
Cognitive Tutor Algebra
6
President Obama on Intelligent Tutoring Systems!“we will devote more than three percent of our GDP to research and development. …. Just think what this will allow us to accomplish: solar cells as cheap as paint, and green buildings that produce all of the energy they consume; learning software as effective as a personal tutor; prosthetics so advanced that you could play the piano again; an expansion of the frontiers of human knowledge about ourselves and world the around us. We can do this.”
How close to this vision are we now?What else do we need to do?
7
Overview
• PSLC Background– Intelligent Tutoring Systems– Cognitive Task Analysis
• PSLC Methods & Tech Resources– In vivo experimentation– LearnLab courses, CTAT, TagHelper,
DataShop
• PSLC Theoretical Framework
Next
b
CognitiveTutorTechnology
Algebra I GeometryEquationSolver
Algebra II
Cognitive Tutors
ArtificialIntelligence
CognitivePsychology
Research base
Math InstructorsMath EducatorsNCTM Standards
Curriculum Content
Cognitive Tutor Approach
9
• Cognitive Model: A system that can solve problems in the various ways students can
Strategy 1: IF the goal is to solve a(bx+c) = d THEN rewrite this as abx + ac = d
Strategy 2: IF the goal is to solve a(bx+c) = d THEN rewrite this as bx + c =
d/a
Misconception: IF the goal is to solve a(bx+c) = d THEN rewrite this as abx + c = d
Cognitive Tutor Technology
10
3(2x - 5) = 9
6x - 15 = 9 2x - 5 = 3 6x - 5 = 9
Cognitive Tutor Technology
• Cognitive Model: A system that can solve problems in the various ways students can
If goal is solve a(bx+c) = dThen rewrite as abx + ac = d
If goal is solve a(bx+c) = dThen rewrite as abx + c = d
If goal is solve a(bx+c) = dThen rewrite as bx+c = d/a
• Model Tracing: Follows student through their individual approach to a problem -> context-sensitive instruction
11
3(2x - 5) = 9
6x - 15 = 9 2x - 5 = 3 6x - 5 = 9
Cognitive Tutor Technology
• Cognitive Model: A system that can solve problems in the various ways students can
If goal is solve a(bx+c) = dThen rewrite as abx + ac = d
If goal is solve a(bx+c) = dThen rewrite as abx + c = d
• Model Tracing: Follows student through their individual approach to a problem -> context-sensitive instruction
Hint message: “Distribute a across the parentheses.”
Bug message: “You need tomultiply c by a also.”
• Knowledge Tracing: Assesses student's knowledge growth -> individualized activity selection and pacing
Known? = 85% chance Known? = 45%
b
CognitiveTutorTechnology
Algebra I GeometryEquationSolver
Algebra II
Cognitive Tutors
ArtificialIntelligence
CognitivePsychology
Research base
Math InstructorsMath EducatorsNCTM Standards
Curriculum Content
Cognitive Tutor Approach
14
Difficulty Factors Assessment:Discovering What is Hard for Students to Learn
Which problem type is most difficult for Algebra students?
Story Problem
As a waiter, Ted gets $6 per hour. One night he made $66 in tips and earned a total of $81.90. How many hours did Ted work?
Word Problem
Starting with some number, if I multiply it by 6 and then add 66, I get 81.90. What number did I start with?
Equation
x * 6 + 66 = 81.90
15
Algebra Student Results:Story Problems are Easier!
70%61%
42%
0%
20%
40%
60%
80%
Story Word Equation
Problem Representation
Percent Correct
Koedinger, & Nathan, (2004). The real story behind story problems: Effects of representations on quantitative reasoning. The Journal of the Learning Sciences.
Koedinger, Alibali, & Nathan (2008). Trade-offs between grounded and abstract representations: Evidence from algebra problem solving. Cognitive Science.
17
“The Student Is Not Like Me”
• To avoid your expert blind spot, remember the mantra:
“The Student Is Not Like Me”
• Perform Cognitive Task Analysis to find out what students are like
Use Data!
19
Prior achievement:Intelligent Tutoring Systems bring learning science to schools
A key PSLC inspiration:Educational technology as research platform to generate new learning science
20
Overview
• PSLC Background– Intelligent Tutoring Systems– Cognitive Task Analysis
• PSLC Methods & Tech Resources– In vivo experimentation– LearnLab courses, CTAT, TagHelper,
DataShop
• PSLC Theoretical Framework
Next
21
PSLC Statement of Purpose
Leverage cognitive and computational theory to identify the instructional conditions that cause robust student learning.
Leverage cognitive and computational theory to identify the instructional conditions that cause robust student learning.
22
What is Robust Learning?
• Achieved through:– Conceptual understanding & sense-making
skills– Refinement of initial understanding– Development of procedural fluency with basic
skills
• Measured by:– Transfer to novel tasks– Retention over the long term, and/or – Acceleration of future learning
23
PSLC Statement of Purpose
Leverage cognitive and computational theory to identify the instructional conditions that cause robust student learning.
24
In Vivo Experiments: Laboratory-quality principle testing in real classrooms
25
In Vivo Experimentation MethodologyImportant features of different research
methodologies:• What is tested?
– Instructional solution vs. causal principle
• Where & who?– Lab vs. classroom
• How?– Treatment only vs. Treatment + control
• Generalizing conclusions:– Ecological validity: What instructional activities work in real classrooms?– Internal validity: What causal mechanisms explain & predict?
33
In Vivo Experimentation MethodologyLab
ExperimentsDesign
ResearchRandomized Field Trials
In vivo Experiments
What?
Instructional solution
√ √
Causal principle √ × × √
Where & who?
Lab √
Classroom × √ √ √
How?
Treatment only √
Treatment + control
√ × √ √
Generalizes how?
Internal validity √ × +/– √
Ecological validity
× √ √ √
34
LearnLabA Facility for Principle-Testing
Experiments in Classrooms
35
LearnLab courses at K12 & College Sites
• 6+ cyber-enabled courses: Chemistry, Physics, Algebra, Geometry, Chinese, English
• Data collection– Students do home/lab work
on tutors, vlab, OLI, …– Log data, questionnaires,
tests DataShop
Researchers Schools
Learn Lab
Chemistry virtual labChemistry virtual lab
Physics intelligent tutorPhysics intelligent tutor
REAP vocabolary tutorREAP vocabolary tutor
36
PSLC Technology Resources
• Tools for developing instruction & experiments– CTAT (cognitive tutoring systems)
• SimStudent (generalizing an example-tracing tutor)
– OLI (learning management)– TuTalk (natural language dialogue)– REAP (authentic texts)
• Tools for data analysis – DataShop– TagHelper
37
PSLC Statement of Purpose
Leverage cognitive and computational theory to identify the instructional conditions that cause robust student learning.
38
Overview
• PSLC Background– Intelligent Tutoring Systems– Cognitive Task Analysis
• PSLC Methods & Tech Resources– In vivo experimentation– LearnLab courses, CTAT, TagHelper,
DataShop
• PSLC Theoretical Framework Next
39
KLI Framework: Designing Instruction for Robust Learning
The conditions that yield robust learning can be decomposed at three levels:Knowledge componentsLearning eventsInstructional events
Get framework report at learnlab.org
40
Instructional events
Explanation, practice, text, rule, example, teacher-student discussion
Assessment eventsQuestion, feedback, step in ITS
Learning events
KC accessible from long-term memory
Immediate performance
Robust performance
KEYOvals – observableRectangle - inferredSolid line – causeDashed line – inferencesKC = Knowledge Component
State test, belief survey
KLI Event Decomposition
• Decompose temporal progress of learning
• Observe/control instructional & assessment events
• Infer learning events & changes in knowledge
41
KLI Framework: Designing Instruction for Robust Learning
The conditions that yield robust learning can be decomposed at three levels:Knowledge componentsLearning eventsInstructional events
42
What’s the best form of instruction? Two choices?• More assistance vs. more challenge
– Basics vs. understanding– Education wars in reading, math, science…
• Researchers like binary oppositions too.We just produce a lot more of them!– Massed vs. distributed (Pashler)– Study vs. test (Roediger)– Examples vs. problem solving (Sweller,Renkl)– Direct instruction vs. discovery learning (Klahr)– Re-explain vs. ask for explanation (Chi, Renkl)– Immediate vs. delayed (Anderson vs. Bjork)– Concrete vs. abstract (Pavio vs. Kaminski)– …
Koedinger & Aleven (2007). Exploring the assistance dilemma in experiments with Cognitive Tutors. Ed Psych Review.
43
How many options are there really? And what works best when?
What’s best?
Focused practice
Distributed practice
Study examples
Test onproblems
50/50Mix
Concrete AbstractMix
Delayed No feedback
Immediate
Block topics in chapters
Interleave topics
Fade
Explain Ask for explanations
Mix
Gradually widen
Study Test50/50 Study Test50/50Study
Concrete Mix Abstract
ImmediateNo
feedbackDelayed
Block topics in chapters
Fade Interleave topics
Explain Ask for explanations
Mix
More helpBasics
More challenge Understanding
44
205,891,132,094,649“Big Science” effort needed to tackle this complexity
Cumulative theory development Field-based basic research with microgenetic data collection
Derivation:15 instructional dimensions 3 options per dimension 2 stages of learning=> 315*2 options
45
An example of some PSLC studies in this space
• Researchers like binary oppositions too.We just produce a lot more of them!– Massed vs. distributed (Pashler)– Study vs. test (Roediger)– Examples vs. problem solving (Sweller,Renkl)– Direct instruction vs. discovery learning (Klahr)– Re-explain vs. ask for explanation (Chi, Renkl)– Immediate vs. delayed (Anderson vs. Bjork)– Concrete vs. abstract (Pavio vs. Kaminski)– …
46
Learn by doing or by studying?• Testing effect (e.g., Roediger & Karpicke, 06)
– “Tests enhance later retention more than additional study of the material”
Kirschner, Sweller, & Clark (2006). Why minimal guidance during instruction does not work:…failure of… problem-based…teaching. • Worked example effect
– “a worked example constitutes the epitome of strongly guided instruction [aka optimal instruction]”
– Paas and van Merrienboer (1994), Sweller et al. (1998), van Gerven et al. (2002), van Gog et al. (2006), Kalyuga et al. (2001a, 2001b), Sweller et al. (1998), Ayres (2006), Trafton & Reiser (1993), Renkl and Atkinson (2003), … the list goes on …
• Theoretical goal: Address debate between desirable difficulties, like “testing effect”, and direct instruction, like “worked examples”
• Limitation of past worked example studies have weaker control, untutored practice– PSLC studies compare to tutored practice
47
Worked Example Experiments within Geometry Cognitive Tutor (Alexander Renkl, Vincent Aleven, Ron Salden, et al.)
• 8 studies in US & Germany– Random assignment, vary single principle– Over 500 students– 3 in vivo studies run in Pittsburgh area
schools
• Cognitive Science ’08 Conference IES Best Paper Award
48
Ecological Control = Standard Cognitive Tutor Students solve problems step-by-step & explain
49
Worked out steps with calculation shown by Tutor
Treatment condition: Half of steps are given as examples
Student still has to self explain worked out step
50
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Pretest Transfer:Procedural
Transfer:Declarative
Percentage correct
Example
Problem
d = .73 *
Worked examples improve efficiency & understanding
Lab results• 20% less time
on instruction• Conceptual
transfer in study 2
In Vivo• Adaptively fading examples to problems yields
better long-term retention & transfer
51
Worked example effect generalizes across domains, settings, researchers• Geometry tutor studies• Chemistry tutor studies in vivo at High School & College
(McLaren et al.)
– Same outcomes in 20% less time• Algebra Tutor study in vivo (Anthony et al.)
– Better long term retention in less time • Theory: SimStudent model (Matsuda et al.)
– Problems provide learning process with negative examples to prune misconceptions
• Research to practice– Influencing Carnegie Learning development– New applied projects with SERP, WestEd
52
Processes of Learning within the KLI Framework
The conditions that yield robust learning can be decomposed at three levels:Knowledge
componentsLearning eventsInstructional events
Learning eventsFluency building, refinement, sense making
53
Learning Events in the Brain & reflected in Dialogue
• Fluency buildingMemory, speed, automaticity
• Refinement processesClassification, co-training, discrimination, analogy, non-verbal explanation-based learning
• Sense-making processesReasoning, experimentation, explanation, argument, dialogue
Some PSLC Examples• ACT-R models of spacing,
testing effects & instructional efficiency (Pavlik)
• SimStudent models of learning by example & by tutoring– Inductive logic prgrming,
probabilistic grammars (Matsuda, Cohen , Li, Koedinger)
• Transactivity+ analysis of peer & classroom learning dialogues (Rose, Asterhan, Resnick)
54
Knowledge components carry the results of learning• Knowledge component =
an acquired unit of cognitive function or structure that can be inferred from performance on a set of related tasks.
• Used in broad sense of a knowledge base– From facts to mental
models, metacognitive skills
55
Example KCs with different features• Chinese vocabulary KCs:
const->const, explicit, no rationale– If the Chinese pinyin is “lao3shi1”, then the English word is “teacher”– If the Chinese radical is “ 日” , the English word is “sun”
• English Article KCs: var->const, implicit, no rationale
– If the referent of the target noun was previously mentioned, then use “the”
• Geometry Area KCs: var->var, implicit & explicit, rationale
– If the goal is to find the area of a triangle,and the base <B> and the height is <H>,then compute 1/2 * <B> * <H>
– If the goal is to find the area of irregular shape made up of regular shapes <S1> and <S2>, then find area <S1> and <S2> and add
Integrated KCs for mental models, central conceptual structures, strategies & complex planning
56
Kinds of Knowledge Components
• Other kinds of KCs– Integrative, probabilistic, metacognitive, misconceptions or “buggy”
knowledge
57
Knowledge components are not just about domain knowledge• Examples of possible domain-general KCs• Metacognitive strategy
– Novice KC: If I’m studying an example, try to remember each step– Desired KC: If I’m studying an example, try to explain how each
step follows from the previous• Motivational belief
– Novice: I am no good at math– Desired: I can get better at math by studying and practicing
• Social communicative strategy– Novice: When an authority figure speaks, remember what they
say. – Desired: Repeat another's claim in your own words and ask
whether you got it right
Can these be assessed, learned, taught? Broad transfer?
58
Learning curves: Measuring behavior on tasks over time• Data from flash-card tutor• Tasks present a Chinese word & request the English translation• Learning curve shows average student performance (e.g., error rate, time on
correct responses) after each opportunity to practice
6 secs
3 secs
59
Empirical comparison of KC complexity
Example KC typesChinese vocabulary KCs const->const, explicit, no rationale
English Article KCs var->const, implicit, no rationale
Geometry Area KCs var->var, implicit/explicit, rationale
Time 6 -> 3 secs
10 -> 6 secs
14 -> 10 secs
6 secs
3 secs
10 secs
6 secs
14 secs
10 secs
60
Which instructional principles are effective for which kinds of knowledge components?
Do complex instructional events aid simple knowledge acquisition?
Do simple instructional events aid complex knowledge acquisition?
61
Prompted self-explanation studies across domains• Physics Course - field principles
– Better transfer than providing explanations
• Geometry Course - properties of angles– Better transfer than just practice despite
solving 50% fewer problems in same time
• English Course - article use– Pure practice appears more efficient;
self-explanation may help for long-term retention & for novices
• Cross-domain hypoth: Type of KC determines when self-explanation will be effective.
Var->Var, explicit
Var->Const, implicit
62
Summary• Obama: “learning software as effective as a
personal tutor”• How close to this vision are we now?
– Many fielded Intelligent Tutors– Students learn as much or more
• What else do we need to do?– Expand to more areas => CTAT– More sophisticated interaction => CSCL– Use tutors to advance science & improve
educational practice => In Vivo & EDM
In other words …Take the PSLC Summer School!
63
64
To do • Other possibilities
– Reduce in vivo slide by deleting redundancy between text and table
– Define ITS as about the intelligence in the design – Add an opener?
• Not techy enough– TEDx talk => KDD Cup => learning curves
• Lots of issues -- gaming vs. not (Shih)• Tie to WE and SE
– Get from HCI AB talk (CSCL example) Too much listing at the end
65
LearnLab Products
Infrastructure created and highly used• LearnLab courses have supported over
150 in vivo experiments
• Established DataShop: A vast open data repository & associated tools– 110,000 student hours of data
• 21 million transactions at ~15 second intervals – New data analysis & modeling algorithms– 67 papers, >35 are secondary data analysis not
possible without DataShop
66
Novice Expert
Learning
Instruction
Pre-test Post-test
Typical Instructional Study• Compare effects of 2 instructional conditions in lab • Pre- & post-test similar to tasks in instruction
67
Post-test
Novice Expert(desired)
Learning
Instruction
Pre-test Post-test: Long-term retention, transfer, accelerated future learning, or desire for future learning
• Macro: Measures of robust learning
PSLC Instructional Experiments
68
Post-test
Novice Expert(desired)
Learning
Instruction
Pre-test Post-test: Long-term retention, transfer, accelerated future learning, or desire for future learning
• Macro: Measures of robust learning
PSLC Instructional Experiments
Assessment Events
Knowledge:Shallow percepts & concepts
Knowledge: Deep percepts & concepts, fluent
• Micro analysis: knowledge, learning, interactions
Instructional Events
69
Post-test
Coursegoals &cultural context
• Studies run in vivo as part of existing courses
Novice Expert(desired)
Learning
Instruction
Pre-test Post-test: Long-term retention, transfer, accelerated future learning, or desire for future learning
• Macro: Measures of robust learning
PSLC Instructional Experiments
Assessment Events
Knowledge:Shallow percepts & concepts
Knowledge: Deep percepts & concepts, fluent
• Micro analysis: knowledge, learning, interactions
Instructional Events
70
Develop a research-based, but practical framework• Theoretical framework key goals
– Support reliable generalization from empirical studies to guide design of effective ed practices
Two levels of theorizing: • Macro level
– What instructional principles explain how changes in the instructional environment cause changes in robust learning?
• Micro level– Can learning be explained in terms of what knowledge
components are acquired at individual learning events?
71
Example study at macro level: Hausmann & VanLehn 2007
• Research question– Should instruction provide explanations and/or elicit
“self-explanations” from students?
• Study design– All students see 3 examples & 3 problems
• Examples: Watch video of expert solving problem• Problems: Solve in the Andes intelligent tutor
– Treatment variables: • Videos include justifications for steps or do not• Students are prompted to “self-explain” or paraphrase
72
Para-phrase
Self-explain
Explan XNo
explan
73
Para-phrase
Self-explain
Explan
No explan X
74
Self-explanations => greater robust learning
0.670.670.90 0.980
0.2
0.4
0.6
0.8
1
1.2
Paraphrase,With just.
Paraphrase,Without just.
Self-explain,With just.
Self-explain,Without just.
(hints+errors) / steps
• Justifications: no effect!
• Immediate test on electricity problems:
• Transfer to new electricity homework problems
0.450.370.69 1.040
0.2
0.4
0.6
0.8
1
1.2
1.4
Paraphrase,With just.
Paraphrase,Without just.
Self-explain,With just.
Self-explain,Without just.
(hints+errors) / steps
• Instruction on electricity unit => accelerated future learning of magnetism!
0.911.17 0.75 0.680
0.2
0.4
0.6
0.8
1
1.2
1.4
Paraphrase,With just.
Paraphrase,Without just.
Self-explain,With just.
Self-explain,Without just.
(hints+errors) / steps
75
Key features of H&V study
• In vivo experiment– Ran live in 4 physics sections at US Naval
Academy – Principle-focused: 2x2 single treatment
variations– Tight control manipulated through
technology
• Use of Andes tutor => repeated embedded assessment without
disrupting course
• Data in DataShop (more later)
76
Develop a research-based, but practical framework• Theoretical framework key goals
– Support reliable generalization from empirical studies to guide design of effective ed practices
Two levels of theorizing: • Macro level
– What instructional principles explain how changes in the instructional environment cause changes in robust learning?
• Micro level– Can learning be explained in terms of what knowledge
components are acquired at individual learning events?
77
Knowledge Components
• Knowledge Component– A mental structure or process that a learner uses,
alone or in combination with other knowledge components, to accomplish steps in a task or a problem-- PSLC Wiki
• Evidence that the Knowledge Component level functions in learning …
79
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
Problem1 Problem2 Problem3
(hints+errors)/steps
Instructionalexplanation
Self-explanation
Example1 Example2 Example3
Back to H&V study: Micro-analysisLearning curve for main KCSelf-explanation effect tapers but not to zero
80
PSLC wiki: Principles & studies that support them
Points to Hausmann’s study page (and other studies too)
Instructional Principle pages unify across studies
81
PSLC wiki: Principles & studies that support them
With links to concepts in glossary Hausmann’s study description:
82
PSLC wiki: Principles & studies that support them
~200 concepts in glossarySelf-explanation glossary entry
83
Research Highlights• Synthesizing worked examples &
self-explanation research – 10+ studies in multiple 4 math & science domains– New theory: It’s not just cognitive load!
• Examples for deep feature construction, problems & feedback for shallow feature elimination
This work inspired new question: Does self-explanation enhance language learning? Experiments in progress …
• Computational modeling of student Learning – Simulated learning benefits of examples/demonstrations vs.
problem solving (Masuda et al., 2008)• Theory outcome: problem solving practice is an important source of
negative examples • Engineering: “programming by tutoring” is more cost-effective than
“programming by demonstration”– Shallow vs. deep prior knowledge changes learning rate (Matsuda et
al., in press)
84
Research Highlights (cont) Computational modeling of instructional assistance
Assistance formula: Optimal learning (L) depends on right level of assistance
Relevant to multiple experimental paradigms & dimensions of instructional assistance Direct instruction (worked examples) vs. constructivism (testing effect)
Concrete manipulatives vs. simple abstractions
Formula provides path to resolve hot debates
P*Sb+(1-P)FbP*Sc+(1-P)Fc
L =
Kirschner, Sweller, & Clark (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist
Kaminski, Sloutsky, & Heckler (2008). The advantage of learning abstract examples in learning math. Science.
L
PAssistance
85
Research Highlights (cont) Synthesis paper on computer tutoring of metacognition
Generalizes results across 7 studies, 3 domains, 4 populations Posed new questions about role of motivation
Lasting effects of metacognitive support Computer-based tutoring of self-regulatory learning
Technologically possible & can have a lasting effect Students who used help-seeking tutor demonstrated better learning
skills in later units after support was faded Spent 50% more time reading help messages
Data mining for factors that affect student motivation Machine learning to analyze vast student interaction data
from full year math courses (Baker et al., in press a & b) Students more engaged on “rich” story problems than standard Surprise: Also more engaged on abstract equation exercises!
Koedinger, Aleven, Roll, & Baker. (in press). In vivo experiments on whether supporting metacognition in intelligent tutoring systems yields robust learning. In Handbook of Metacognition in Education.
86
Social Communication
Social context of classroom Teacher
Interaction
Thrusts investigate overlapping factors
Novice Expert
Learning
Instruction
Knowledge:Shallow, perceptual
Knowledge: Deep, conceptual, fluent
THRUSTSCognitive Factors
MetacognitionMotivation
MetacognitionMotivation
Metacognition & Motivation
Motivation Metacognition
Comp Modeling & Data Mining
Learning
Instruction
Knowledge Knowledge
87
Thrust Research Questions• Cognitive Factors. How do instructional events affect
learning activities and thus the outcomes of learning?• Metacognition & Motivation. How do activities
initiated by the learner affect engagement with targeted content?
• Social Communication. How do interactions between learners and teachers and computer tutors affect learning?
• Computational Modeling & Data Mining. Which models are valid across which content domains, student populations, and learning settings?
88
4th Measure of Robust Learning • Existing robust learning measures
– Transfer– Long-term retention– Acceleration of future learning
• New measure:– Desire for future learning
• Is student engaged in subject? • Do they chose to pursue further math, science, or
language?
89
END
Recommended