Strategy Flexibility Matters for Student Mathematics Achievement: A Meta-Analysis Kelley Durkin Bethany Rittle-Johnson Vanderbilt University, United States

Strategy Flexibility Matters for Student Mathematics

Achievement: A Meta-Analysis

Kelley DurkinBethany Rittle-Johnson

Vanderbilt University, United States

Jon R. StarHarvard University, United States

Defining Strategy Flexibility• Simplest definition:

Knowing more than one strategy for solving a particular type of problem (e.g., Heirdsfield & Cooper, 2002)

• Most complex definition: Being able to use a variety of strategies and information from the problem context, the learner’s environment, and the sociocultural context to select the most appropriate problem solving procedure (e.g., Verschaffel, Luwel, Torbeyns, & Van Dooren, 2007)

2

Recent Focus on Strategy Flexibility• Previously, flexibility rarely measured as an

instructional outcome (Star, 2005).

• Standardized tests in the U.S. include sections on:– Concepts– Procedures– Problem solving– But not flexibility

• Recently, flexibility examined as a separate outcome (Star, 2007; Verschaffel et al., 2007).

3

Importance of Strategy Flexibility• Helps adapt existing procedures to unfamiliar

problems (e.g., Blöte, Van der Burg, & Klein, 2001)

• Greater understanding of domain concepts (e.g., Hiebert & Wearne, 1996)

• Crucial component of expertise in problem solving (Dowker, 1992; Dowker, Flood, Griffiths, Harris, & Hook, 1996; Star & Newton, 2009)

4

Current Study• Is strategy flexibility related to other

mathematical constructs?– Conceptual Knowledge• Success recognizing and explaining key domain

concepts (Carpenter et al., 1998; Hiebert & Wearne, 1996)

– Procedural Knowledge• Success executing action sequences to solve problems

(Hiebert & Wearne, 1996; Rittle-Johnson, Siegler, & Alibali, 2001)

– General Mathematics Achievement

• Meta-analysis of our past work

5

Our Definition of Strategy Flexibility

• Knowing multiple strategies and their relative efficiencies (Flexibility Knowledge)

AND• Adapting strategy choice to specific problem

features (Flexible Use)

(e.g., Blöte et al., 2001; National Research Council, 2001; Rittle-Johnson & Star, 2007)

6

Method Overview

• Selected Studies• Measures• Analysis Strategies

7

Included StudiesStudy Authors Year Topic N GradeRittle-Johnson & Star 2007 Equation Solving 70 7

Star & Rittle-Johnson 2008 Equation Solving 155 6

Rittle-Johnson & Star 2009 Equation Solving 162 7 & 8

Rittle-Johnson, Star, & Durkin

2009 Equation Solving 236 7 & 8

Star et al. 2009 Estimation 65 5

Star & Rittle-Johnson 2009 Estimation 157 5 & 6

Rittle-Johnson, Star, & Durkin

2011 Equation Solving 198 8

Schneider, Rittle-Johnson & Star

2011 Equation Solving 293 7 & 8

8

Measures

• Flexibility Knowledge• Flexible Use• Conceptual Knowledge• Procedural Knowledge• Standardized Tests

9

Measures• Flexibility Knowledge– Knowing multiple procedures and the relative

efficiency of the procedures5(x + 3) + 6 = 5(x + 3) + 2x

6 = 2xa. What step did the student use to get from the first line to the second line?b. Do you think that this is a good way to start this problem? Circle One:

(a) a very good way(b) OK to do, but not a very good way(c) Not OK to do

c. Explain your reasoning. 10

Measures• Flexible Use– Students using the most appropriate strategy

depending on problem features3(h + 2) + 4(h + 2) = 357(h + 2) = 35

• Sometimes know a more appropriate strategy for solving a problem before actually use it (Blöte et al., 2001; Siegler & Crowley, 1994)

11

Measures• Conceptual Knowledge– Ability to recognize and explain key domain concepts

Which of the following is a like term to (could be combined with) 7(j + 4)?

(a) 7(j + 10) (b) 7(p + 4) (c) j (d) 2(j + 4) (e) a and d

• Procedural Knowledge– Ability to execute action sequences to solve problems

3(h + 2) + 4(h + 2) = 35

12

Measures• Standardized Tests

National Tests• Comprehensive Testing Program (CTP)• Measures of Academic Progress (MAP)

State Tests• Massachusetts Comprehensive Assessment System (MCAS)• Tennessee Comprehensive Assessment Program (TCAP)

• Collected scores from school records

13

Coding and Analysis Strategies• Calculated correlation between each pair of

outcomes for each study• Fischer’s z to transform correlations to get effect

sizes, ESr, for each study (Lipsey & Wilson, 2001).

• The mean correlation effect size was calculated using a random effects model.

14

Results• Mean correlations between outcomes

FlexibilityKnowledge

Flexible Use

ConceptualKnowledge

ProceduralKnowledge

Standardized Test

FlexibilityKnowledge 1 .635 .563 .610 .535

Flexible Use 1 .541 .627 .404

ConceptualKnowledge 1 .544 .520

ProceduralKnowledge 1 .475

Note: All correlations were significant (p < .001) 15



Flexible Use

ConceptualKnowledge

ProceduralKnowledge

Standardized Test


Flexible Use 1 .541 .627 .404






Flexible Use

ConceptualKnowledge

ProceduralKnowledge

Standardized Test


Flexible Use 1 .541 .627 .404




Flexibility knowledge and flexible use strongly related



Flexible Use

ConceptualKnowledge

ProceduralKnowledge

Standardized Test


Flexible Use 1 .541 .627 .404






Flexible Use

ConceptualKnowledge

ProceduralKnowledge

Standardized Test


Flexible Use 1 .541 .627 .404




Conceptual knowledge had moderately strong relationships to flexibility



Flexible Use

ConceptualKnowledge

ProceduralKnowledge

Standardized Test


Flexible Use 1 .541 .627 .404




Conceptual knowledge had moderately strong relationships to flexibility

Procedural knowledge had moderately strong relationships to flexibility



Flexible Use

ConceptualKnowledge

ProceduralKnowledge

Standardized Test


Flexible Use 1 .541 .627 .404




Similar to correlation between conceptual and procedural knowledge



Flexible Use

ConceptualKnowledge

ProceduralKnowledge

Standardized Test


Flexible Use 1 .541 .627 .404






Flexible Use

ConceptualKnowledge

ProceduralKnowledge

Standardized Test


Flexible Use 1 .541 .627 .404




Standardized test measures significantly correlated with flexibility



Flexible Use

ConceptualKnowledge

ProceduralKnowledge

Standardized Test


Flexible Use 1 .541 .627 .404





Standardized test measures significantly correlated with other outcomes



Flexible Use

ConceptualKnowledge

ProceduralKnowledge

Standardized Test


Flexible Use 1 .541 .627 .404





Standardized test measures significantly correlated with other outcomes

Correlations between flexibility and standardized tests similar to other correlations

3 Main Findings

• Flexibility knowledge and flexible use are separate constructs

• Flexibility is related to other constructs• Standardized tests relate to flexibility as well

as they relate to other constructs

26

The Construct of Strategy Flexibility

• May be important to measure flexible use and flexibility knowledge separately.– Appears measures of knowledge and use are

tapping different aspects of flexibility.

• Conceptual and procedural knowledge are related to flexibility (Schneider et al., 2011).

27

Relation to Standardized Tests• Standardized test scores relate to flexibility just

as well as they relate to conceptual and procedural knowledge.

• Teachers can feel pressured to teach to the test, and the lack of flexibility items on assessments could lead to less time on flexibility in the classroom.

• Push for standardized tests to include items that assess flexibility.

• Flexibility a valued outcome when evaluating interventions.

28

Conclusion

• Strategy flexibility is important for developing expertise and efficient problem solving

• Need to measure and encourage students’ strategy flexibility in the future

29

30

Acknowledgements

• E-mail: [email protected]• Visit our Contrasting Cases Website at

http://gseacademic.harvard.edu/contrastingcases/index.html for more information

• Thanks to the Children’s Learning Lab at Vanderbilt University

• Funded by a grant from the Institute for Education Sciences, U.S. Department of Education– The opinions expressed are those of the authors and do not

represent views of the U.S. Department of Education.

30

http://gseacademic.harvard.edu/contrastingcases/index.html

http://gseacademic.harvard.edu/contrastingcases/index.html

ReferencesBlöte, A. W., Van der Burg, E., & Klein, A. S. (2001). Students' flexibility in

solving two-digit addition and subtraction problems: Instruction effects. Journal of Educational Psychology, 93(3), 627-638.

Carpenter, T. P., Franke, M. L., Jacobs, V. R., Fennema, E., & Empson, S. B. (1998). A longitudinal study of invention and understanding in children's multidigit addition and subtraction. Journal for Research in Mathematics Education, 29(1), 3-20.

Dowker, A. (1992). Computational estimation strategies of professional mathematicians. Journal for Research in Mathematics Education, 23(1), 45-55.

Dowker, A., Flood, A., Griffiths, H., Harris, L., & Hook, L. (1996). Estimation strategies of four groups. Mathematical Cognition, 2(2), 113-135.

Heirdsfield, A. M., & Cooper, T. J. (2002). Flexibility and inflexibility in accurate mental addition and subtraction: Two case studies. The Journal of Mathematical Behavior, 21, 57-74.

31

ReferencesHiebert, J., & Wearne, D. (1996). Instruction, understanding and skill in

multidigit addition and subtraction. Cognition and Instruction, 14, 251-283.

Lipsey, M. W., & Wilson, D. B. (2001). Practical Meta-Analysis (Vol. 49). Thousand Oaks, CA: Sage Publications.

National Research Council. (2001). Adding it up: Helping children learn mathematics. Washington, DC: National Academy Press.

Rittle-Johnson, B., Siegler, R. S., & Alibali, M. W. (2001). Developing conceptual understanding and procedural skill in mathematics: An iterative process. Journal of Educational Psychology, 93, 346-362.

Rittle-Johnson, B., & Star, J. R. (2007). Does comparing solution methods facilitate conceptual and procedural knowledge? An experimental study on learning to solve equations. Journal of Educational Psychology, 99(3), 561-574.

32

ReferencesRittle-Johnson, B., & Star, J. R. (2009). Compared with what? The effects of

different comparisons on conceptual knowledge and procedural flexibility for equation solving. Journal of Educational Psychology, 101, 529-544.

Rittle-Johnson, B., Star, J. R., & Durkin, K. (2009). The importance of prior knowledge when comparing examples: Influences on conceptual and procedural knowledge of equation solving. Journal of Educational Psychology, 101(4), 836-852.

Rittle-Johnson, B., Star, J. R., & Durkin, K. (2011, June 28). Developing procedural flexibility: Are novices prepared to learn from comparing procedures? British Journal of Educational Psychology. Advance online publication.

Schneider, M., Rittle-Johnson, B., & Star, J. R. (2011, August 8). Relations among conceptual knowledge, procedural knowledge, and procedural flexibility in two samples differing in prior knowledge. Developmental Psychology. Advance online publication.

33

ReferencesSiegler, R. S., & Crowley, K. (1994). Constraints on learning in nonprivileged

domains. Cognitive Psychology, 27(2), 194-226.Star, J. R. (2005). Reconceptualizing procedural knowledge. Journal for

Research in Mathematics Education, 36, 404-411.Star, J. R. (2007). Foregrounding Procedural Knowledge. [Peer Reviewed].

Journal for Research in Mathematics Education, 38(2), 132-135.Star, J. R., & Newton, K. J. (2009). The nature and development of experts’

strategy flexibility for solving equations. ZDM-International Journal on Mathematics Education, 41, 557-567.

Star, J. R., & Rittle-Johnson, B. (2008). Flexibility in problem solving: The case of equation solving. Learning and Instruction, 18, 565 - 579.

Star, J. R., & Rittle-Johnson, B. (2009). It pays to compare: An experimental study on computational estimation. Journal of Experimental Child Psychology, 102, 408 - 426.

34

ReferencesStar, J. R., Rittle-Johnson, B., Lynch, K., & Perova, N. (2009). The role of prior

knowledge in the development of strategy flexibility: The case of computational estimation. ZDM, 41(5), 569-579.

Verschaffel, L., Luwel, K., Torbeyns, J., & Van Dooren, W. (2007). Developing adaptive expertise: A feasible and valuable goal for (elementary) mathematics education? Ciencias Psicologicas, 2007(1), 27-35.

35

Documents

Strategy Flexibility Matters for Student Mathematics Achievement: A Meta-Analysis Kelley Durkin Bethany Rittle-Johnson Vanderbilt University, United States