Iowa Content Network ProjectMathematics

Reviewing Research on

Instructional Strategies and Programs

Background & Overview

1. Scholarly work within the field that provides a context for the Iowa Mathematics Network (IMN) research reviews, and on which the IMN builds

2. Overview of the IMN project thus far

Background and ContextOther

Research Reviews

National Tests

Internatl.Tests

Some Cognitive Science

Program Ratings

National Review Panels Iowa Math

Network Project

Published Research Reviews• Adding It Up (National Research Council)http://www.nap.edu/books/0309069955/html/

• NCTM Standards Research Companionhttp://my.nctm.org/store/ECat/product.asp?id=12341

• Improving Student Achievement in Mathematics (International Academy of Education)

http://www.ibe.unesco.org/International/Publications/EducationalPractices/prachome.htm

• Standards-Based School Mathematics Curriculahttps://www.erlbaum.com/shop/tek9.asp?pg=products&specific=0-8058-4337-X

National Tests• National Assessment of Educational

Progress (NAEP)http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2000469

http://nces.ed.gov/nationsreportcard/mathematics/trendsnational.asp• SAThttp://www.collegeboard.com/press/article/0,3183,26858,00.html

International Tests

• Third International Mathematics and Science Study (TIMSS) - 1995 12th Grade

• TIMSS - 1999 8th Grade Video Studyhttp://nces.ed.gov/pubsearch

/pubsinfo.asp?pubid=2003013• TIMSS-R - 1999 Repeat with US Consortiahttp://isc.bc.edu/timss1999b/mathbench

_report/t99b_math_report.html

Some Cognitive Science

• How We Learn: Ask the Cognitive Scientist. American Educator, Winter, 2002.

http://www.aft.org/american_educator/winter2002/CogSci.html

Program Ratings

• American Association for the Advancement of Science (AAAS) - rating of algebra texts

http://www.project2061.org/research/textbook/hsalg/charts.htm

• US Department of Education - Exemplary and Promising Programs in Mathematics

http://www.enc.org/professional/federalresources/exemplary/promising/document.shtm?input=CDS-000496-496_toc,00.shtm

National Research Review Panels

• What Works Clearinghousehttp://w-w-c.org/

• Mathematical Sciences Education Board (MSEB)

http://www4.nas.edu/cp.nsf/57b01c7b1b6493c485256555005853cf/5cf09421beb746d185256b7c00568d05?

OpenDocument

Background Details

1. Some findings from Cognitive Science

2. International and National Tests• TIMSS

• NAEP

• SAT

3. Research on NCTM-Standards based approach• NSF curricula: US DE, AAAS, NAEP, new book

• Research companion to Principles and Standards

4. Existing Reviews & Summaries of Research• Improving Student Achievement in Math• Adding It Up• NCTM Research Companion

5. In-Progress Research Review Projects• What Works Clearinghouse (US DE)• National Research Council• Iowa Math Network Project

1. Cognitive Science

“Findings from the field [Cognitive Science] that are strong and clear enough to merit classroom application.”

Willingham, Daniel T. How We Learn: Ask the Cognitive Scientist. American Educator, Winter, 2002.

Finding 1

“The mind much prefers that new ideas be framed in concrete rather than abstract terms.”

Finding 2

Rote Knowledge, Inflexible knowledge, and Deep Structure

Rote:

Q: What is the equator?

A: A managerie lion running around the Earth through Africa.

“We rightly want students to understand;

we seek to train creative problem solvers, not parrots. Insofar as we can prevent students from absorbing knowledge in a rote form, we should do so. ”

Inflexible Knowledge:

• Deeper than rote knowledge, but at the same time, clearly the student has not completely mastered the concept.

• Understanding is somehow tied to the surface features.

• Meaningful, yet narrow.

• The student does not yet have flexibility . (Knowledge is flexible when it can be accessed out of the context in which it was learned and applied in new contexts.)

Deep Structure Knowledge:

• Deeper than inflexible knowledge• Transcends specific examples• Knowledge is flexible -- it can be accessed

out of the context in which it was learned and applied in new contexts

• Knowledge is no longer organized around surface forms, but rather is organized around deep structure

Finding 3Develop deep structure knowledge:

• Solve more problems

• Multiple contexts

• Focus on meaning

• Don’t despair of inflexible knowledge, and don’t confuse it with rote knowledge

2. International and National Tests

• TIMSS

• NAEP

• SAT

TIMSS - 12th grade

US performs at the bottom of the list of nations

A key difference between US high schools and all others in the world:

Integrated Curriculum

(not Alg 1, Geom, Alg 2, Pre-Calc)

TIMSS 8th Grade Video Study

• US and 6 countries that outperformed the US in 1999, plus Japan from ‘95 study

• March 2003• Teaching Mathematics in Seven Countries:

Results from the TIMSS 1999 Video Study. NCES, 2003

• US performance is below average (19th out of 38 in 1999)

• No single method of mathematics instruction was observed in all of the high-performing countries examined

• Hong Kong: most emphasis on procedures

• Netherlands: most calculator use

• All countries: little use of computers

• Japan: most connections and relationships

• Japan: do more than repeat procedures during private work time

• Japan: how to use procedures, not just execute

• Netherlands: most use of real-life applications

• US reduces complexity of problems

Lessons taught by US and Australia teachers most often translated connections problems into procedure problems

Tendency in U.S. classrooms for teachers to transform intellectually demanding tasks in ways that reduce the cognitive challenge for students

• US does most review and least new content

Review of previously taught lessons plays a larger role in mathematics lessons in the Czech Republic and the United States than in the five other countries where more time is devoted to introducing new content.

• US least likely to emphasize connectionsWhen the researchers examined the ways in

which the mathematical problems in the lesson were actually discussed and worked out during the lessons, they found that eighth-grade mathematics lessons in Australia and the United States were the least likely to emphasize mathematical connections or relationships (8 and less than 1 percent, respectively; other countries ranged from 37 to 52 percent)

Moral so far:

US can learn from other countries that are more successful in mathematics education

TIMSS and Singapore

• Singapore is top ranked• Singapore curriculum is becoming popular in US• Spring 2003 - Ngee Ann Polytechnic in Singapore

announced an agreement to collaborate with the Curriculum Research & Development Group (CRDG) of the University of Hawaii to develop an introductory engineering mathematics course for Ngee Ann students based on a Standards-based curriculum developed at CRDG, Algebra I: A Process Approach.

• Ngee Ann selected the program because it uses problem-solving and communication strategies--reading, writing, speaking, critical listening, and multiple representations--features that they believe lead to students' deeper understanding of mathematics.

• Though she was initially surprised by the request, Barb Dougherty of CRDG said, "On thinking about it, our Algebra I program is a natural fit with the way math is taught in Singapore. Like us, they use increasingly complex word problems to teach students problem-solving skills, encouraging students to find different ways to solve and express problems, not simply memorize formulas taught by a teacher."

• Singapore is also seeking to collaborate with “reform” mathematics education curriculum developers at Cambridge University in England.

Moral:

Learn from other countries, but don’t try to emulate their curricula

TIMSS 1999:Michigan Invitational Group

• U.S. groups participating in this international comparative study include states, large school districts, and consortia of schools.

• The top four U.S. groups are: the Naperville school district in Illinois, the First in the World consortium on the North Shore in the Chicago area, Montgomery County in Maryland, and a 21-school consortium called the Michigan Invitational Group.

• The top-scoring US groups are using "hands-on learning” and "progressive curriculum strategies.”

• Michigan Invitational Group (MIG) also uses "National Science Foundation materials” and has strong implementation.

• MIG is significant since it is the only top-scoring U.S. group with a diverse population. A diverse group of students can perform at the top level, right along with the top countries in the world and the top affluent suburban US school districts.

Moral:

Use NCTM-Standards Based Approach, implemented well, for high achievement with diverse students.

“The Nation’s Report Card”National Assessment of

Educational Progress (NAEP) Trends in Computation

• 17-year-olds. After declining between 1973 and 1982, average scores increased during the 1980s, and more modestly in the 1990s. The average score in 1999 was higher than that in 1973.

• 13-year-olds. An increase in scores between 1978 and 1982, followed by additional increases in the 1990s, resulted in an average score in 1999 that was higher than that in 1973.

• 9-year-olds. After a period of stable performance in the 1970s,average scores increased in the 1980s. Additional modest gains were evident in the 1990s, and the 1999 average score was higher than that in 1973.

MORAL:

Changing curricula to emphasize computation skills is not warranted.

“Reports today saying the curriculum must change to emphasize computational skills are no more valid today than in 1973.”

Johnny Lott, NCTM President, NCTM News Bulletin, November 2002

SAT Scores

• SAT scores highest in 35 years• Continues trend of rising scores• “NCTM has done a tremendous job in its

reform efforts. This has really begun to pay off.” -- Wayne Camara, College Board VP of

Research and Development(NCTM News Bulletin, October 2003, p. 1)

3. Research on the NCTM Standards-Based Approach

• Research on NCTM’s Principles and Standards for School Mathematics

• Research on the “NSF Curricula”

Research on NCTM’s Principles and Standards

A Research Companion to Principles and Standards for School Mathematics

Edited by Jeremy Kilpatrick, Gary Martin, and Deborah Schifter

NCTM, 2003

Does Research Support the NCTM Standards?

• Yes

• “The Standards are consistent with the best and most recent evidence on teaching and learning mathematics.”

• “However, research does not shine equally brightly on all aspects of the Standards.”

What Research Cannot Do

• Research cannot make value judgments.

• Research cannot prove what works or what is best.– Too many variables, conditions, situations– Implementation is essential

• Traditional scientific experiments cannot be applied to all educational questions.

What Research Can Do

• Probe beneath the surface

• Extend our knowledge of teaching and learning

• Inform our decisions

• Show what is possible and what is promising

What Do We Know?

• Students learn what they have the opportunity to learn.

What do we know about traditional programs?

• “Presuming that traditional approaches have proven to be successful is ignoring the largest database we have.”

• With traditional curricula and pedagogy:– Students knowledge is limited to what the

traditional approach emphasizes.– Students knowledge is not robust nor

extendable.

What do know about alternative “reform” programs?

• Emphasizing conceptual development and understanding can promote significant learning without sacrificing skill proficiency.

• Solving problems can be used effectively as a context for learning new concepts and skills.

• Students in alternative programs implemented with fidelity for reasonable lengths of time have learned more and learned more deeply than in traditional programs.

Research on the “NSF Curricula”

• US Department of Education Exemplary designation, 1999

• AAAS Ratings of Algebra texts

• New book of research

• NAEP (MIG)

• Individual research studies

High School NSF Curricula

• Core-Plus Mathematics Project

• Interactive Mathematics Program

• MATH Connections

• Mathematics Modeling Our World (ARISE)

• SIMMS Integrated Mathematics

• UCSMP Secondary School Curriculum

Middle Grades NSF Curricula

• Connected Mathematics Project

• Mathematics in Context

• MathScape

• MATH Thematics: The STEM Project

Elementary Grades NSF Curricula

• Math Trailblazers

• Everyday Mathematics

• Investigations

• Number Power

US DE Exemplary Programs ‘99

• Cognitive Tutor Algebra

• College Preparatory Mathematics

• Connected Mathematics Project

• Core-Plus Mathematics Project

• Interactive Mathematics Program

American Association for the Advancement of Science (AAAS)

Algebra Text Ratings

• Reviewed traditional and reform texts

• Two categories: With Potential, Little Potential

Algebra Programs With Potential

• Concepts in Algebra

• Core-Plus Mathematics

• Focus on Algebra

• Interactive Mathematics Program

• MATH Connections

• Mathematics Modeling Our World (ARISE)

• UCSMP Algebra

Book of Research on the NSF Curricula

• Standards-Based School Mathematics Curricula: What Are They? What Do Students Learn?

• Editors: Sharon Senk, Denisse Thompson

• Erlbaum, 2003

• Book editors, grade band reviewers, K-12 reviewer

Summary

• “[There is] considerable evidence that the promises of reform mathematics are real and the fears of the anti-reformers unjustified.” Swafford, p 458

• “The studies in this book provide much needed evidence that the new programs work.” Kilpatrick, p 472

4. Existing Reviews and Summaries of Research

• Improving Achievement in Mathematics

• Adding It Up

• NCTM Research Companion (above)

Adding It Up: Helping Children Learn Mathematics

National Research Council

Jeremy Kilpatrick, Jane Swafford, Bradford Findell, editors

2001

Purpose

• Focus on number and operations• Grades preK-8• Synthesize research• Provide research-based

recommendations• Give advice and guidance

Main Recommendation

All students can and should be mathematically proficient.

Mathematical Proficiency

• Conceptual Understanding – comprehension of mathematical concepts, operations and relations

• Procedural fluency – skill in carrying out procedures flexibly, accurately, efficiently, and appropriately

• Strategic competence – ability to formulate, represent, and solve mathematical problems

• Adaptive reasoning - capacity for logical thought, reflection, explanation, and justification

• Productive disposition - habitual inclination to see mathematics as sensible, useful, and worthwhile, coupled with a belief in diligence and one’s own efficacy

Some Findings Related to Need for Improvement

• “On the 23 problem-solving tasks given as part of the 1996 NAEP in which students had to construct an extended response, the incidence of satisfactory or better response was less than 10% on about half of the tasks” (p. 138).

• Performance “on word problems declines dramatically when additional features are included, such as more than one step or extraneous information” (p.139).

• 8th graders experience much difficulty with problems that ask them to “justify and explain their solutions” (p. 139).

Some Key Recommendations

• Integrated and balanced development of all five strands of mathematical proficiency

• On non-routine problems, students need to slow down and ask themselves guiding questions, and not prematurely apply operations to numbers in the problems.

• Students need to develop conceptual understanding of operations, as well as learn standard algorithms.

More Recommendations

• Integrated and balanced development of all five strands of mathematical proficiency

• Instruction should not be based on extreme positions that students learn, on one hand, solely by internalizing what a teacher or book says or, on the other hand, solely by inventing mathematics on their own.

• Efforts to improve students’ mathematics learning should be informed by scientific evidence

• Teachers’ professional development should be high quality, sustained, and systematically designed and deployed

• Assessment should enable, not just gauge

• Time and resources needed

Improving Achievement in Mathematics

• International Academy of Education, 2000• Chapter in the Handbook of Research on

Improving Student Achievement • Douglas Grouws and Kristin Cebulla• Review research on effective teaching in

mathematics• Research-based teaching practices

Research-Based Teaching Strategies and Methods

• Opportunity to learn• Focus on meaning• Learning new concepts and skills while

solving problems• Opportunities for both invention and

practice• Openness to student solution methods and

student interaction

• Small-group learning

• Whole-class discussion

• Number sense

• Concrete materials

• Students’ use of calculators

Note: Implementation is essential.

5. In-Progress Research Review Projects

• What Works Clearinghouse (all)

• National Research Council (NSF + others)

• Iowa Math Network Project (all, based on past work)

What Works Clearinghouse

• Established by the US Department of Education• Review research through contracts to American

Institutes of Research and the Campbell Collaboration

• Identify Scientifically Research Based programs and strategies in reading, math, etc.

• For math, 1st is MS, then Elem, then HS• Alan Schoenfeld is the head of the math section

Mathematical Sciences Education Board (MSEB) Panel

• A Review of the Evaluation Data on the Effectiveness of NSF-Supported and Commercially Generated Mathematics Curriculum Materials

• 13 NSF curricula• 6 others• 1st draft of report done, out for review, final report

in several months.• Jere Confrey, Chair

Iowa Math Network Project Overview

• Established by the Iowa Department of Education

• Review research studies on instructional strategies and programs for improving student achievement in mathematics

• Ongoing, not comprehensive, rate research design not instructional strategies

Process

• Group of 9 reviewers

• Teams of two, 4 grade bands, Chair

• Initial screening (criteria below)

• Review using standard form

• Rate quality of research design based on standard criteria (e.g., comparative groups)

Initial Screening Criteria

• Peer reviewed

• Student achievement

• Quantitative

• Instructional strategy or program

Reviewed So Far

• Instructional strategies (e.g., problem-centered, conceptually oriented)

• Programs (e.g., NSF curricula, CGI, RNP)

• About 35 studies in 1st round -- done

• Coded by content strand (perhaps deeper later)

• 2nd round underway

Key Themes So Far

• Focus on meaning and understanding

• Multiple representations• Problem-centered• NCTM-Standards based

approach

Background and ContextOther

Research Reviews

National Tests

Internatl.Tests

Some Cognitive Science

Program Ratings

National Review Panels Iowa Math

Network Project