1

Problem Order Implications for Learning TransferNan Li, William Cohen, and Kenneth Koedinger

School of Computer ScienceCarnegie Mellon University

2

Order of ProblemsOne of the most important variables that

affects learning effectivenessBlocked order vs. interleaved order

Interleaved is better! Why?

Type I Type II Type III

Type I Type II Type III Type I Type II Type III

Most existing textbooks

Numerous previous studies

3

Need for Better Theory Studies

Contextual interference (CI) effect (Shea and Morgan, 1979)

Mixed results on complex tasks or novices

… Hypothesis

Elaboration hypothesis (Shea and Morgan, 1979)

Forgetting or reconstruction hypothesis (Lee and Magill, 1983)

… Proposed Approach

A controlled simulation study Using a machine-learning agent,

SimStudent Given problems of blocked orders or

interleaved orders

A precise implementation.

Easier to inspect SimStudent’s learning processes and

outcomes.

Lacks the precision of a computational theory.

4

A Brief Review of SimStudent

• A learning agent that• Acquires production

rules• From examples and

problem-solving experience

• Given a perceptual representation, a set of feature predicates and operator functions

Matsuda et al., CogSci-09

5

SimStudent Learns Production Rules

Skill divide (e.g. -3x = 6)

Retrieval path: Left side (-3x) Right side (6)

Precondition: Left side (-3x) does not

have a constant term

=> Function sequence:

Get-coefficient (-3) of left side (-3x)

Divide both sides by the coefficient

7

Retrieval Path Learner A perceptual learner

Finding paths to identify useful information (percepts) in GUI E.g. <-3x, 6> <Cell 11, Cell 21> <4x, 12> <Cell 12, Cell 22>

Specific general E.g. Cell 21 Cell 2? Cell ??

The most specific path that covers all of the training percepts

Retrieval path:Left side (-3x)Right side (6)

8

Precondition Learner A feature test learner

Acquiring the precondition of the production rule Given a set of feature predicates

A boolean function that describes relations among objects E.g. (has-coefficient -3x), (has-constant 2x+5)

Utilize FOIL (Quinlan, 1990) Input:

Positive and negative examples based on the percepts <percept1, percept2> E.g. positive: <-3x, 6>, negative: <2x+4, 8>

Output: A set of feature tests that

describe the desired situation to fire the production rule E.g. (not (has-constant ?percept1))

Different problem orders Different intermediate production rules Incorrect rule applications Different negative feedback

Precondition:Left side (-3x) does not have constant term

9

Function Sequence Learner

An operator function sequence learner Acquires a sequence of operator functions to apply in

producing the next step Given a set of operator functions

E.g. (coefficient -3x), (add-term 5x-5 5)

Input: A set of records, Ri = <perceptsi, stepi>

E.g. <<-3x, 6>, (divide -3)>

Output: A sequence of operator functions, op = (op1, op2, … opk), that

explains all recordsE.g.

(bind ?coef (coefficient ?percepts1)), (bind ?step (divide ?coef))

Function sequence:Get-coefficient (-3) of left side (-3x)Divide both sides with the coefficient

(coefficient -3x)

(divide -3)<-3x,

6>

-3(divide -3)

22

Problem Order StudyBlocked order vs. Interleaved orderThree domains

Fraction addition Equation solving Stoichiometry

Training and testing problems Solved by human students in classroom studies

SimStudent Tutored by automatic tutors that simulate the

automatic tutors used by human students

23

Fraction AdditionProblem

TypesType Feature ExampleType I denominator1 =

denominator2

1/4 + 3/4

Type II One denominator is a multiple of the

other denominator

1/2 + 3/4

Type III No denominator is a multiple of the other

denominator

1/3 + 3/4

24

Equation SolvingProblem

S1 + S2V = S3

TypesType Form ExampleType I S1 + S2V = S3 -2 + 5x = 7Type II V/S1 + S2 = S3 x/3 + 1 = 4Type III S1/V = S2 -6/x = 3

25

Stoichiometry Problem

How many moles of atomic oxygen (O) are in 250 grams of P4O10? (Hint: the molecular weight of P4O10 is 283.88 g P4O10 / mol P4O10.)

Skills Unit conversion: 0.6 kg H2O = 600 g H2O Molecular weight: There are 2 moles of P4O10 in 283.88 * 2 g

P4O10

Composition stoichiometry: There are 10 moles of O in each mole of P4O10

Types Type Skills NeededType I Unit conversionType II Unit conversion + Molecular weight

Type III Unit conversion + Molecular weight + Composition stoichiometry

26

Problem OrdersBlocked-Ordering

CurriculaInterleaved-Ordering

Curricula

Type I Type I Type II Type II Type III Type III Type I Type II Type III Type I Type II Type III

Type I Type I Type III Type III Type II Type II Type I Type III Type II Type I Type III Type II

Type II Type II Type I Type I Type III Type III Type II Type I Type III Type II Type I Type III

Type II Type II Type III Type III Type I Type I Type II Type III Type I Type II Type III Type I

Type III Type III Type I Type I Type II Type II Type III Type I Type II Type III Type I Type II

Type III Type III Type II Type II Type I Type I Type III Type II Type I Type III Type II Type I

27

Speed of Learning

Fraction Addition Equation Solving Stoichiometry

28

Cause of the EffectSimStudent vs. Human Student

More controllableMore observable

Conjecture: Interleaved order Receive feedback from all three

types

Blocked order Receive feedback from some types

Interleaved order More explicit negative feedback More effective learning

Type I Type II Type III

Type I Type II Type III Type I Type II Type III

29

Explicit Negative Feedback

More negative feedback More effective precondition learning

Opportunities to expose to over-general preconditions

30

ExampleE.g., S1+S2V=S3 (Type I)

Subtract both sides by S1

Subtract both sides by Si, if Si is a signed number &

there is a “+”

Negative feedback: Subtract both sides of S1/V=S2 by V or

S1 (Type III)

Subtract both sides by S1Subtract both sides by Si, if Si is a signed number

Negative feedback: Subtract both sides of S1V+S2=S3 by

S1V (Type I)

31

SummaryInterleaved order More negative feedback

Better precondition learningSimStudent with limited memory

Blocked order More training examples Better function sequence learning

Future studiesGenerality across problem setsSimStudent with limited memoryA study on human students

32