Putting the ‘x’ back Putting the ‘x’ back into goal-free into goal-free problems: a problems: a

brainstorming brainstorming approachapproach

Carina SchubertCarina Schubert11, Paul Ayres, Paul Ayres22, , Katharina ScheiterKatharina Scheiter11 and John Sweller and John Sweller22

1 1 University of TuebingenUniversity of Tuebingen22 University of New South Wales University of New South Wales

HypothesesHypotheses(Theory follows from John Sweller talk)(Theory follows from John Sweller talk)

o No group differences for higher ability No group differences for higher ability students (impact of prior knowledge)students (impact of prior knowledge)

o For students with lower ability:For students with lower ability:o Goal Free with X will be more effective Goal Free with X will be more effective

than the Goal Free and the Conventional than the Goal Free and the Conventional groupsgroups

o Goal Free will be more effective than the Goal Free will be more effective than the Conventional groupConventional group

Planned contrasts will be conducted Planned contrasts will be conducted accordinglyaccordingly

MethodMethod

MethodMethod

ParticipantsParticipantso 82 year 9 students from two Sydney Girls high 82 year 9 students from two Sydney Girls high

schoolsschoolso 44 students were of higher mathematical ability, 44 students were of higher mathematical ability,

38 students of lower mathematical ability 38 students of lower mathematical ability (according to school grading in general (according to school grading in general mathematical ability - top 20% excluded)mathematical ability - top 20% excluded)

o Randomly assigned to one of three treatment Randomly assigned to one of three treatment groups: groups: o ConventionalConventionalo Goal- free Goal- free o Goal-free with XGoal-free with X

MethodMethod

MaterialsMaterials

Learning materialsLearning materials• 16 geometry problems16 geometry problems• 8 different geometry theorems 8 different geometry theorems

employedemployed• 14 two-step problems, 2 three-step 14 two-step problems, 2 three-step

problemsproblems• For Conventional and Goal-free with X For Conventional and Goal-free with X

group, the problems had a goal angle Xgroup, the problems had a goal angle X

MethodMethod

MethodMethodMaterialsMaterials Testing materials ( with X for all groups)Testing materials ( with X for all groups)

8 Test questions:8 Test questions: o similar to acquisition problems similar to acquisition problems o same combination of theorems in same order same combination of theorems in same order

but with different numbersbut with different numberso 7 two-step, 1 three-step7 two-step, 1 three-step

8 Transfer questions:8 Transfer questions:o 4 with same combinations of theorems as in 4 with same combinations of theorems as in

acquisition problems in inverted order acquisition problems in inverted order o 4 with different combinations of theorems – 4 with different combinations of theorems –

different, unusual configurations ( see Ayres & different, unusual configurations ( see Ayres & Sweller, 1990)Sweller, 1990)

o 3 two-step, 5 three-step3 two-step, 5 three-step

MethodMethodMaterialsMaterials Self-rating measures (after acquisition Self-rating measures (after acquisition

& after testing): 7-point Likert scale& after testing): 7-point Likert scale• How How difficultdifficult did you find it to answer the did you find it to answer the

questions? (Sweller et al.)questions? (Sweller et al.)• How much How much mental effortmental effort did you use to did you use to

answer the questions? (Paas et al.)answer the questions? (Paas et al.)• How much did you How much did you concentrateconcentrate when you when you

answered the questions? (Cierniak, answered the questions? (Cierniak, Scheiter & Gerjets, 2008)Scheiter & Gerjets, 2008)

• How How motivatedmotivated were you to answer the were you to answer the questions? (Ayres & Youssef, 2008)questions? (Ayres & Youssef, 2008)

MethodMethod

Rationale for CL measuresRationale for CL measures

o Explore potential differences Explore potential differences between difficulty and mental effort between difficulty and mental effort (Van Gog & Paas, in press)(Van Gog & Paas, in press)

o Explore relation between CL Explore relation between CL measures and test performancemeasures and test performance

MethodMethod

Instruction (acquisition):Instruction (acquisition):o Conventional: “For each question find the Conventional: “For each question find the

value of angle x”value of angle x”o Goal-free: “For each question find as many Goal-free: “For each question find as many

angles as you can”angles as you can”o Goal-free with X: “For each question, find as Goal-free with X: “For each question, find as

many angles as you can in any order you many angles as you can in any order you like”like”

Instruction (test + transfer): Instruction (test + transfer): ““For each question, find the value of angle x”For each question, find the value of angle x”

MethodMethod

ProcedureProcedure

SELF-RATING

ACQUISITION

SELF-RATING

TEST

TRANSFER

15

mins

1 min

10 mins

10 mins

1 min

ResultsResults

Results: Scores in Results: Scores in acquisitionacquisition

0

5

10

15

20

25

30

Conven-tional Goal-free Goal-Free X

Higher Abilty

Lower Ability

o max. score 32

o all planned contrasts were n.s.

o Higher ability students > lower ability students (F (1, 80) = 95.81; p = .00)

Results: TestscoreResults: Testscore

0

2

4

6

8

10

12

14

16

Conventional Goal-free Goal-Free X

Higher Abilty

Lower Ability

o max. score 16

o all planned contrasts were n.s.

o Higher ability students > lower ability students (F (1, 80) = 23.14; p= .00)

Results: Transfer ScoreResults: Transfer Score

0

2

4

6

8

10

12

14

16

Conventional Goal-free Goal-Free X

Higher Abilty

Lower Ability

o max. score 16

o Higher ability > lower ability (F (1, 80) = 62.17; p = .00)

o for higher ability students all planned contrasts were n.s.

Lower ability students:

o Goal-free X > Conventional (t (22) = 2.06; p = .05)

o Goal-free X > Goal-free

(t (24) = 2.20; p < .05)

o Goal-free not better than Conventional

Hypotheses summaryHypotheses summary

o For the higher ability group there For the higher ability group there were no differences between were no differences between treatment groups (supported)treatment groups (supported)

o For the lower ability group there For the lower ability group there were significant group differences on were significant group differences on the transfer problems onlythe transfer problems onlyo Goal-free with X was superior to the Goal-free with X was superior to the

other two groups (supported)other two groups (supported)o Goal-free = Conventional (not supported)Goal-free = Conventional (not supported)

Further analysisFurther analysis

o Two potential explanations of the results:Two potential explanations of the results:o Schema acquisition (learnt the geometry Schema acquisition (learnt the geometry

problems)problems)o Strategy acquisition (learnt the problem Strategy acquisition (learnt the problem

solving strategy)solving strategy)o We examine the number of angles We examine the number of angles

calculated calculated o We examine the use of extra constructions We examine the use of extra constructions

(many students used a strategy of drawing (many students used a strategy of drawing extra lines to find angles – a strategy extra lines to find angles – a strategy evidently learnt at school)evidently learnt at school)

Results: Number of Angles Results: Number of Angles in acquisitionin acquisition

o 3 x 2 ANOVA3 x 2 ANOVA

o significant group significant group differencesdifferences (F(2,76) = 7.38; (F(2,76) = 7.38; p<.01): p<.01):

Goal Free, Goal Goal Free, Goal Free X > Free X > ConventionalConventional

o Higher ability > Higher ability > lower abilitylower ability (F(1,76) = 57.38; (F(1,76) = 57.38; p=.00)p=.00)

0

5

10

15

20

25

30

35

40

45

Conventional Goal-free Goal-Free X

Higher Abilty

Lower Ability

Results: Number of Results: Number of angles in testangles in test

o No group No group differencesdifferences

o Higher ability Higher ability > lower ability > lower ability (F(1,76) = 7.63; (F(1,76) = 7.63; p<.01)p<.01)

10

11

12

13

14

15

16

17

18

19

20

Conventional Goal-free Goal-Free X

Higher Abilty

Lower Ability

Results: Number of angles Results: Number of angles in Transferin Transfer

o Higher ability > Higher ability > lower ability lower ability (F(1,76)=27.01; (F(1,76)=27.01; p<.01)p<.01)

o group x ability group x ability interaction interaction (F(2,76)=3.78; p<.05)(F(2,76)=3.78; p<.05)

o Simple effects + Simple effects + post-hoc on lower post-hoc on lower ability students: ability students: Goal-free with X > Goal-free with X > Goal-Free and Goal-Free and ConventionalConventional 10

12

14

16

18

20

22

24

26

28

30

Conventional Goal-free Goal-Free X

Higher Abilty

Lower Ability

Strategy acquisitionStrategy acquisition

o Evidence suggests that for the lower Evidence suggests that for the lower ability students the Goal-free with X ability students the Goal-free with X group continued with the strategy of group continued with the strategy of finding additional angles for the finding additional angles for the transfer problems. In contrast, the transfer problems. In contrast, the Goal-free group seemed to abandon Goal-free group seemed to abandon the strategy.the strategy.

Results: Constructions Results: Constructions AcquisitionAcquisition

o Higher ability > Higher ability > lower ability lower ability (F(1,76)=6.78;p=.01)(F(1,76)=6.78;p=.01)

o Group x ability Group x ability interaction interaction (F(2,76)=5.60;p<.01)(F(2,76)=5.60;p<.01)

o Simple effects Simple effects and post-hoc: and post-hoc: o no group effect no group effect

for lower for lower ability studentsability students

o For higher For higher ability ability students, students, Conventional > Conventional > Goal Free Goal Free

0

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

0,9

1

Conventional Goal-free Goal-Free X

Higher Abilty

Lower Ability

Results: Constructions Results: Constructions TestTest

o No group effectNo group effecto No ability effectNo ability effecto No interactionNo interaction

Results: Constructions Results: Constructions TransferTransfer

o Group effect Group effect (F(2,76)=5.27; (F(2,76)=5.27;

p<.01) ;p<.01) ; ConventionaConventional > Goal l > Goal FreeFree

0

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

Conven-tional

GoalFree

GoalFree

with X

Higher AbilityLower Ability

Results: CL measuresResults: CL measures

3 x 2 x (2) ANOVA Repeated 3 x 2 x (2) ANOVA Repeated measuresmeasuresBetweeBetwee

n n phasesphases

GroupGroup AbilityAbility Group x Group x AbilityAbility

MotivationMotivation p=.06 p=.06 ↓↓

n.s.n.s. p=.00 p=.00 ↑↑

n.s.n.s.

ConcentratiConcentrationon

n.s.n.s. n.s.n.s. p=.00 p=.00 ↑↑

n.s.n.s.

DifficultyDifficulty p=.00 p=.00 ↑↑

n.s.n.s. P=.00 P=.00 ↓↓

n.s.n.s.

Mental Mental EffortEffort

p=.00 p=.00 ↑↑

n.s.n.s. n.s.n.s. p=.03^p=.03^

^ Simple effects and post-hoc: high ability students invested less mental effort in Goal-free with X group than in Goal-free and Conventional group

Results: CL measuresResults: CL measures

Correlations Correlations

DifficultDifficulty 1y 1

DifficultDifficulty 2y 2

Effort 1Effort 1 Effort 2Effort 2

Difficulty Difficulty 11

.66**.66** .52**.52** .27*.27*

Difficulty Difficulty 22

.32**.32** .39**.39**

Effort 1Effort 1 .61**.61**

Results: CL measuresResults: CL measures

Correlations of CL measures and Correlations of CL measures and Transfer ScoreTransfer ScoreMoMo

t 1t 1CoCon 1n 1

DifDiff 1f 1

Eff Eff 11

MoMot 2t 2

CoCon 2n 2

DifDiff 2f 2

Eff Eff 22

TransfTransferer

.55.55

****.57.57

****-.58-.58

****-.05-.05 .48.48

****.62.62

****-.58-.58

****.12.12

ConclusionsConclusions

ConclusionsConclusions

o On transfer problems, Goal-free with X was On transfer problems, Goal-free with X was superior to Goal-free and Conventionalsuperior to Goal-free and Conventional

o The data on the number of angles The data on the number of angles calculated suggest that the Goal-free with calculated suggest that the Goal-free with X group have learnt the problem solving X group have learnt the problem solving strategy of generating anglesstrategy of generating angles

o Surprisingly the Goal-free was not superior Surprisingly the Goal-free was not superior to the Conventional group. The Goal-free to the Conventional group. The Goal-free group rejected the problem solving group rejected the problem solving strategy of generating angles.strategy of generating angles.

ConclusionsConclusions

o How has adding an X to the Goal-free group How has adding an X to the Goal-free group influenced the results so much?influenced the results so much?

o It is notable that in some previous research It is notable that in some previous research into goal-free problems the learning domain into goal-free problems the learning domain has been quite restricted. In studies by has been quite restricted. In studies by Owen and Sweller (1985) in trigonometry Owen and Sweller (1985) in trigonometry and Ayres (1990) in geometry, few sides and Ayres (1990) in geometry, few sides and angles could be calculated. However, in and angles could be calculated. However, in this study many more angles could be this study many more angles could be calculated – it may be that the X gives the calculated – it may be that the X gives the group more focus.group more focus.