The Nation's Report Card: Mathematics 2003 · 2005. 7. 25. · Chapter 1 The Nation’s Report Card™ Mathematics 2003 U.S. Department of Education Institute of Education Sciences

U.S. Department of EducationInstitute of Education SciencesNCES 2005–451

The Nation’s Report Card

The National Assessment of Educational Progress

Mathematics 2003™

What is The Nation’s Report CardTM?THE NATION’S REPORT CARDTM, the National Assessment of Educational Progress (NAEP), is anationally representative and continuing assessment of what America’s students know and can do invarious subject areas. Since 1969, assessments have been conducted periodically in reading, mathematics,science, writing, history, geography, and other fields. By making objective information on studentperformance available to policymakers at the national, state, and local levels, NAEP is an integral part ofour nation’s evaluation of the condition and progress of education. Only information related toacademic achievement is collected under this program. NAEP guarantees the privacy of individualstudents and their families.

NAEP is a congressionally mandated project of the National Center for Education Statistics withinthe Institute of Education Sciences of the U.S. Department of Education. The Commissioner ofEducation Statistics is responsible, by law, for carrying out the NAEP project through competitive awardsto qualified organizations.

In 1988, Congress established the National Assessment Governing Board (NAGB) to oversee and setpolicy for NAEP. The Board is responsible for: selecting the subject areas to be assessed; settingappropriate student achievement levels; developing assessment objectives and test specifications;developing a process for the review of the assessment; designing the assessment methodology; developingguidelines for reporting and disseminating NAEP results; developing standards and procedures forinterstate, regional, and national comparisons; determining the appropriateness of all assessment itemsand ensuring the assessment items are free from bias and are secular, neutral, and non-ideological;taking actions to improve the form, content, use, and reporting of results of the National Assessment;and planning and executing the initial public release of NAEP reports.

The National Assessment Governing Board

Darvin M. Winick, ChairPresidentWinick & AssociatesDickinson, Texas

Sheila M. Ford, Vice ChairPrincipalHorace Mann Elementary

SchoolWashington, D.C.

Francie AlexanderChief Academic Officer,

Scholastic, Inc.Senior Vice President,

Scholastic EducationNew York, New York

David J. AlukonisChairmanHudson School BoardHudson, New Hampshire

Amanda P. AvalloneAssistant Principal &

Eighth-Grade TeacherSummit Middle SchoolBoulder, Colorado

Honorable Jeb BushGovernor of FloridaTallahassee, Florida

Barbara Byrd-BennettChief Executive OfficerCleveland Municipal

School DistrictCleveland, Ohio

Carl A. CohnClinical ProfessorRossier School of

EducationUniversity of

Southern CaliforniaLos Angeles, California

Shirley V. DicksonEducational ConsultantLaguna Niguel, California

John Q. EastonExecutive DirectorConsortium on Chicago

School ReformChicago, Illinois

Honorable Dwight EvansMemberPennsylvania House of

RepresentativesPhiladelphia, Pennsylvania

David W. GordonSacramento CountySuperintendent of SchoolsSacramento County Office

of EducationSacramento, California

Henry L. JohnsonSuperintendent of

EducationMississippi Department

of EducationJackson, Mississippi

Kathi M. KingTwelfth-Grade TeacherMessalonskee High

SchoolOakland, Maine

Honorable Keith KingMemberColorado House of

RepresentativesColorado Springs, Colorado

Kim Kozbial-HessFourth-Grade TeacherFall-Meyer Elementary

SchoolToledo, Ohio

Andrew C. PorterProfessorLeadership Policy

and OrganizationsVanderbilt UniversityNashville, Tennessee

Luis A. RamosCommunity Relations

ManagerPPL SusquehannaBerwick, Pennsylvania

Mark D. ReckaseProfessorMeasurement and

Quantitative MethodsMichigan State UniversityEast Lansing, Michigan

John H. StevensExecutive DirectorTexas Business and

Education CoalitionAustin, Texas

Mary Frances Taymans, SNDExecutive DirectorNational Catholic

Educational AssociationWashington, D.C.

Oscar A. TroncosoPrincipalSocorro High SchoolSocorro Independent

School DistrictEl Paso, Texas

Honorable Thomas J. VilsackGovernor of IowaDes Moines, Iowa

Michael E. WardFormer State Superintendent

of Public InstructionNorth Carolina Public

SchoolsJackson, Mississippi

Eileen L. WeiserMember, State Board

of EducationMichigan Department

of EducationLansing, Michigan

Grover J. Whitehurst (Ex officio)DirectorInstitute of Education

SciencesU.S. Department of

EducationWashington, D.C.

Charles E. SmithExecutive DirectorNAGBWashington, D.C.

Chapter 1The Nation’s Report Card™

Mathematics2003

U.S. Department of EducationInstitute of Education SciencesNCES 2005-451

James S. BraswellGloria S. DionMary C. DaaneYing Jin

In collaboration withScott DavisAmy DresherFrank JenkinsEdward KulickBrenda Tay-LimMei Jang LinYuxin TangEducational Testing Service

Arnold A. GoldsteinProject OfficerNational Center forEducation Statistics

U.S. Department of EducationMargaret SpellingsSecretary

Institute of Education SciencesGrover J. WhitehurstDirector

National Center for Education StatisticsGrover J. WhitehurstActing Commissioner

July 2005

SUGGESTED CITATIONBraswell, J.S., Dion, G.S., Daane, M.C., and Jin, Y. (2005). The Nation’s Report Card: Mathematics 2003. (NCES 2005–451).U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. Washington, DC:U.S. Government Printing Office.

FOR MORE INFORMATIONContent contact:Arnold [email protected]

To obtain single copies of this report, or for ordering information on other U.S. Department of Education products, call tollfree 1-877-4ED-PUBS (877-433-7827), or write:

Education Publications Center (ED Pubs)U.S. Department of EducationP.O. Box 1398Jessup, MD 20794-1398

TTY/TDD 1-877-576-7734FAX 1-301-470-1244

Online ordering via the Internet: http://www.edpubs.orgCopies also are available in alternate formats upon request.This report also is available on the World Wide Web: http://nces.ed.gov/pubsearch

The work upon which this publication is based was performed forthe National Center for Education Statistics by Educational Testing Service,

Pearson Educational Measurement, and Westat.

http://nces.ed.gov/transfer.asp?location=http://www.edpubs.org

http://nces.ed.gov/pubsearch

TA B L E O F C O N T E N T S • N A E P 2 0 0 3 M AT H E M AT I C S R E P O R T C A R D iii

Table of Contents

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xv

Chapter 1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Overview of the 2003 National Assessment of Educational Progress in Mathematics . . . . . . . . . . . . . . . . . 2

Framework for the 2003 Mathematics Assessment Instrument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

The NAEP 2003 Mathematics Assessment Instrument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Description of School and Student Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Reporting the Assessment Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

The Setting of Achievement Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Mathematics Achievement-Level Descriptions for Each Grade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Trial Status of Achievement Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Interpreting NAEP Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Overview of the Remaining Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Chapter 2

Average Mathematics Scale Score and Achievement-Level Results for the Nation and States . . . . . . 15

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

National Mathematics Scale Score Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

National Mathematics Scale Scores by Percentile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

National Mathematics Achievement-Level Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Mathematics Results by Region of the Country . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Mathematics Results for States and Other Jurisdictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Mathematics Scale Score Results by State/Jurisdiction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Cross-State/Jurisdiction Mathematics Scale Score Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Mathematics Achievement-Level Results by State/Jurisdiction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Cross-State/Jurisdiction Mathematics Achievement-Level Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . 35

iv TA B L E O F C O N T E N T S • N A E P 2 0 0 3 M AT H E M A T I C S R E P O R T C A R D

Chapter 3

Subgroup Results for the Nation and States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Performance of Selected Subgroups for the Nation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Gender . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Race/Ethnicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Student Eligibility for Free/Reduced-Price School Lunch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

Parents’ Highest Level of Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

Type of School . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

Type of Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

Performance of Selected Subgroups by State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Gender . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Race/Ethnicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

Student Eligibility for Free/Reduced-Price School Lunch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

Chapter 4

Average Mathematics Scale Scores and Achievement-Level Results for Districts Participating

in the Trial Urban District Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Scale Score Results for Urban Districts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

Scale Scores by Percentiles for Urban Districts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

Achievement-Level Results for Urban Districts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Performance of Selected Subgroups for Urban Districts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Gender . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Race/Ethnicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

Students’ Eligibility for Free/Reduced-Price School Lunch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

Student-Reported Highest Level of Parents’ Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

Chapter 5

Sample Assessment Questions and Student Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

Grade 4 Sample Assessment Questions and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

Grade 8 Sample Assessment Questions and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

Maps of Selected Item Descriptions on the NAEP Mathematics Scale—Grades 4 and 8 . . . . . . . . . . . . 136

Appendix A

Overview of Procedures Used for the NAEP 2003 Mathematics Assessment . . . . . . . . . . . . . . . . . . . 139

Appendix B

Subgroup Percentage Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

Appendix C

State and Urban District Subgroup Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

Appendix D

State- and District-Level Contextual Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249

Appendix E

Members of the NAEP Mathematics Standing Committee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256

TA B L E O F C O N T E N T S • N A E P 2 0 0 3 M AT H E M AT I C S R E P O R T C A R D v

Chapter 1: Figures

Figure 1.1Structure of the NAEP 2003 mathematics assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Figure 1.2Policy definitions of the three NAEP achievement levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Figure 1.3Descriptions of NAEP mathematics achievement levels, grade 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Figure 1.4Descriptions of NAEP mathematics achievement levels, grade 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Chapter 2: Tables and Figures

Figure 2.1Average mathematics scale scores, grades 4 and 8: 1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Figure 2.2Mathematics scale score percentiles, grades 4 and 8: 1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Figure 2.3Percentages of students at or above Basic and Proficient in mathematics,grades 4 and 8: 1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Table 2.1Percentages of students, by mathematics achievement level, grades 4 and 8: 1990–2003 . . . . . . . . . 19

Figure 2.4Map of regions of the country according to U.S. Census . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Table 2.2Average mathematics scale scores, by region of the country, grades 4 and 8: 2003 . . . . . . . . . . . . . . . 20

Table 2.3Percentages of students, by mathematics achievement level and region of the country,grades 4 and 8: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Table 2.4Average mathematics scale scores, grade 4 public schools: By state, 1992–2003 . . . . . . . . . . . . . . . . 23

Table 2.5Average mathematics scale scores, grade 8 public schools: By state, 1990–2003 . . . . . . . . . . . . . . . . 24

Figure 2.5Comparison of state and national public school average mathematics scale scores,grade 4: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Figure 2.6Comparison of state and national public school average mathematics scale scores,grade 8: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Figure 2.7Cross-state comparison of average mathematics scale scores, grade 4 public schools: 2003 . . . . . . . . 27

Figure 2.8Cross-state comparison of average mathematics scale scores, grade 8 public schools: 2003 . . . . . . . . 28

Figure 2.9Percentage of students within each mathematics achievement level,grade 4 public schools: By state, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Figure 2.10Percentage of students within each mathematics achievement level,grade 8 public schools: By state, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

vi TA B L E O F C O N T E N T S • N A E P 2 0 0 3 M AT H E M A T I C S R E P O R T C A R D

Table 2.6Percentage of students at or above Proficient in mathematics,grade 4 public schools: By state, 1992–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Table 2.7Percentage of students at or above Proficient in mathematics,grade 8 public schools: By state, 1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Figure 2.11Cross-state comparison of percentage of students at or above Proficientin mathematics, grade 4 public schools: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Figure 2.12Cross-state comparison of percentage of students at or above Proficientin mathematics, grade 8 public schools: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37


Figure 3.1Average mathematics scale scores, by gender, grades 4 and 8: 1990–2003 . . . . . . . . . . . . . . . . . . . . 41

Figure 3.2Gaps in average mathematics scale scores, by gender, grades 4 and 8: 1990–2003 . . . . . . . . . . . . . . 42

Table 3.1Percentages of students, by mathematics achievement level and gender,grades 4 and 8: 1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Figure 3.3Average mathematics scale scores, by race/ethnicity, grades 4 and 8: 1990–2003 . . . . . . . . . . . . . . . 45

Figure 3.4Gaps in average mathematics scale scores, by race/ethnicity, grades 4 and 8: 1990–2003 . . . . . . . . . 46

Table 3.2Percentages of students, by mathematics achievement level and race/ethnicity,grades 4 and 8: 1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

Figure 3.5Average mathematics scale scores, by student eligibilityfor free/reduced-price school lunch, grades 4 and 8: 1996–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

Figure 3.6Gaps in average mathematics scale scores, by student eligibilityfor free/reduced-price school lunch, grades 4 and 8: 1996–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Table 3.3Percentages of students, by mathematics achievement level and eligibilityfor free/reduced-price school lunch, grades 4 and 8: 1996–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

Table 3.4Average mathematics scale scores, by student eligibilityfor free/reduced-price school lunch and race/ethnicity, grades 4 and 8: 2003 . . . . . . . . . . . . . . . . . . 53

Figure 3.7Average mathematics scale scores, by student-reported parents’ highest levelof education, grade 8: 1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Table 3.5Percentages of students, by mathematics achievement level andstudent-reported parents’ highest level of education, grade 8: 1990–2003 . . . . . . . . . . . . . . . . . . . . . 57

Figure 3.8Average mathematics scale scores, by type of school, grades 4 and 8: 1990–2003 . . . . . . . . . . . . . . . 59

TA B L E O F C O N T E N T S • N A E P 2 0 0 3 M AT H E M AT I C S R E P O R T C A R D vii

Table 3.6Percentages of students, by mathematics achievement level and type of school,grades 4 and 8: 1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Table 3.7Average mathematics scale scores, by student-reported parents’ highest levelof education and type of school, grade 8: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Figure 3.9Average mathematics scale scores, by type of location, grades 4 and 8: 2000 and 2003 . . . . . . . . . . 64

Table 3.8Percentages of students, by mathematics achievement level and type of location,grades 4 and 8: 2000 and 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

Table 3.9Average mathematics scale scores, by gender, grade 4 public schools: By state, 1992–2003 . . . . . . . . 67

Table 3.10Average mathematics scale scores, by gender, grade 8 public schools: By state, 1990–2003 . . . . . . . . 68

Table 3.11Percentage of students at or above Proficient in mathematics, by gender,grade 4 public schools: By state, 1992–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

Table 3.12Percentage of students at or above Proficient in mathematics, by gender,grade 8 public schools: By state, 1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Table 3.13Average mathematics scale scores, by race/ethnicity,grade 4 public schools: By state, 1992–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

Table 3.14Average mathematics scale scores, by race/ethnicity,grade 8 public schools: By state, 1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

Table 3.15Percentage of students at or above Proficient in mathematics, by race/ethnicity,grade 4 public schools: By state, 1992–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Table 3.16Percentage of students at or above Proficient in mathematics, by race/ethnicity,grade 8 public schools: By state, 1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Table 3.17Average mathematics scale scores, by eligibility for free/reduced-price school lunch,grade 4 public schools: By state, 1996–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Table 3.18Average mathematics scale scores, by eligibility for free/reduced-price school lunch,grade 8 public schools: By state, 1996–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

Table 3.19Percentage of students at or above Proficient in mathematics,by eligibility for free/reduced-price school lunch, grade 4 public schools: By state, 1996–2003 . . . . . . 90

Table 3.20Percentage of students at or above Proficient in mathematics,by eligibility for free/reduced-price school lunch, grade 8 public schools: By state, 1996–2003 . . . . . . 91

viii T A B L E O F C O N T E N T S • N A E P 2 0 0 3 M AT H E M A T I C S R E P O R T C A R D


Figure 4.1Average mathematics scale scores, grade 4 public schools: By urban district, 2003 . . . . . . . . . . . . . . 95

Figure 4.2Average mathematics scale scores, grade 8 public schools: By urban district, 2003 . . . . . . . . . . . . . . 95

Table 4.1Mathematics scale score percentiles, grades 4 and 8 public schools: By urban district, 2003 . . . . . . . 97

Table 4.2Percentages of students, by mathematics achievement level,grades 4 and 8 public schools: By urban district, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Table 4.3Average mathematics scale scores, by gender, grades 4 and 8 public schools:By urban district, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Figure 4.3Gaps in average mathematics scores, by gender, grades 4 and 8 public schools:By urban district, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

Table 4.4Percentages of students, by mathematics achievement level and gender,grades 4 and 8 public schools: By urban district, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

Table 4.5Average mathematics scale scores, by race/ethnicity, grades 4 and 8 public schools:By urban district, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

Figure 4.4Gaps in average mathematics scores, by race/ethnicity, grades 4 and 8 public schools:By urban district, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

Table 4.6Percentages of students, by mathematics achievement level and race/ethnicity,grades 4 and 8 public schools: By urban district, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

Table 4.7Average mathematics scale scores, by eligibility for free/reduced-price school lunch,grades 4 and 8 public schools: By urban district, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

Figure 4.5Gaps in average mathematics scores, by eligibility for free/reduced-price school lunch,grades 4 and 8 public schools: By urban district, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

Table 4.8Percentages of students, by mathematics achievement level and eligibilityfor free/reduced-price school lunch, grades 4 and 8 public schools: By urban district, 2003 . . . . . . . 109

Table 4.9Average mathematics scale scores, by student-reported parents’ highest level of education,grade 8 public schools: By urban district, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

Table 4.10Percentages of students, by mathematics achievement level and student-reportedparents’ highest level of education, grade 8 public schools: By urban district, 2003 . . . . . . . . . . . . . . 113


Table 5.1Percentage scored correct for multiple-choice sample question 1,by achievement-level range, grade 4: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

TA B L E O F C O N T E N T S • N A E P 2 0 0 3 M AT H E M AT I C S R E P O R T C A R D ix

Table 5.2aPercentage scored “Partial” or better for short constructed-response sample question 2,by achievement-level range, grade 4: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

Table 5.2bPercentage scored “Correct” for short constructed-response sample question 2,by achievement-level range, grade 4: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Table 5.3Percentage scored correct for multiple-choice sample question 3, by achievement-level range,grade 4: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120



Table 5.6aPercentage scored as “Satisfactory” or better for extended constructed-response samplequestion 6, by achievement-level range, grade 4: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

Table 5.6bPercentage scored as “Extended” for extended constructed-response sample question 6,by achievement-level range, grade 4: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

Table 5.7Percentage scored “Correct” for short constructed-response sample question 7,by achievement-level range, grade 8: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127





Table 5.12aPercentage scored as “Satisfactory” or better for extended constructed-responsesample question 12, by achievement-level range, grade 8: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

Table 5.12bPercentage scored as “Extended” for extended constructed-responsesample question 12, by achievement-level range, grade 8: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

Figure 5.1Map of selected item descriptions on the NAEP mathematics scale, grade 4: 2003 . . . . . . . . . . . . . . 137

Figure 5.2Map of selected item descriptions on the NAEP mathematics scale, grade 8: 2003 . . . . . . . . . . . . . . 138

x TA B L E O F C O N T E N T S • N A E P 2 0 0 3 M AT H E M A T I C S R E P O R T C A R D

Appendix A: Tables and Figures

Figure A.1Descriptions of the five NAEP mathematics content areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

Table A.1Target percentage distribution of items, by content area and grade: 1990–2003 . . . . . . . . . . . . . . . . 142

Table A.2Distribution of questions administered, by question type and grade: 1990–2003 . . . . . . . . . . . . . . . 143

Table A.3Number of students assessed, by sample type, special needs status, and accommodation option,grades 4 and 8: 1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

Table A.4National and state sample sizes and target populations, grades 4 and 8: 2003 . . . . . . . . . . . . . . . . 146

Table A.5District sample sizes and target populations, grades 4 and 8: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . 147

Table A.6National school and student participation rates, by type of school, grades 4 and 8: 2003 . . . . . . . . . 149

Table A.7School and student participation rates, grade 4 public schools: By state, 2003 . . . . . . . . . . . . . . . . . 150

Table A.8School and student participation rates, grade 8 public schools: By state, 2003 . . . . . . . . . . . . . . . . . 151

Table A.9Weighted school and student participation rates, grades 4 and 8 public schools:By urban district, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

Table A.10Students with disabilities and/or limited-English-proficient students identified,excluded, and assessed, when accommodations were not permitted, grades 4 and 8 publicand nonpublic schools: 1992–2000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

Table A.11Percentage of students with disabilities and/or limited-English-proficient students identified,excluded, and assessed, when accommodations were not permitted, grade 4 public schools:By state, 1992–2000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

Table A.12Percentage of students with disabilities and/or limited-English-proficient students identified,excluded, and assessed, when accommodations were not permitted, grade 8 public schools:By state, 1990–2000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

Table A.13Students with disabilities and/or limited-English-proficient students identified, excluded, andassessed, when accommodations were permitted, grades 4 and 8 public and nonpublic schools:1996–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

Table A.14Percentage of students with disabilities and/or limited-English-proficient studentsidentified, excluded, and assessed, when accommodations were permitted, grade 4 public schools:By state, 2000 and 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

Table A.15Percentage of students with disabilities identified, excluded, and assessed, whenaccommodations were permitted, grade 4 public schools: By state, 2000 and 2003 . . . . . . . . . . . . . 162

Table A.16Percentage of limited-English-proficient students identified, excluded, and assessed,when accommodations were permitted, grade 4 public schools: By state, 2000 and 2003 . . . . . . . . 164

TA B L E O F C O N T E N T S • N A E P 2 0 0 3 M AT H E M AT I C S R E P O R T C A R D xi

Table A.17Percentage of students with disabilities and/or limited-English-proficient students identified,excluded, and assessed, when accommodations were permitted, grade 8 public schools:By state, 2000 and 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

Table A.18Percentage of students with disabilities identified, excluded, and assessed, when accommodationswere permitted, grade 8 public schools: By state, 2000 and 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . 168

Table A.19Percentage of limited-English-proficient students identified, excluded, and assessed,when accommodations were permitted, grade 8 public schools: By state, 2000 and 2003 . . . . . . . . 170

Table A.20Percentage of students with disabilities and limited-English-proficient studentsidentified, excluded, and assessed, grade 4 public schools: By urban district, 2003 . . . . . . . . . . . . . 172

Table A.21Percentage of students with disabilities and limited-English-proficient studentsidentified, excluded, and assessed, grade 8 public schools: By urban district, 2003 . . . . . . . . . . . . . 173

Table A.22Students with disabilities and/or limited-English-proficient students assessed withaccommodations, by type of primary accommodation, grades 4 and 8 public and nonpublic schools:1996–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

Table A.23Average mathematics scale scores and standard errors, grades 4 and 8: 1990–2003 . . . . . . . . . . . . 183

Table A.24Percentages of students and standard errors, by mathematics achievement level, grades 4 and 8:1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

Table A.25Average mathematics scale scores and standard errors, by student eligibility forfree/reduced-price school lunch and race/ethnicity, grades 4 and 8: 2003 . . . . . . . . . . . . . . . . . . . . 184

Table A.26Average mathematics scale scores and standard errors, grade 8 public schools: By state,1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

Table A.27Percentage of students at or above Proficient and standard errors, by race/ethnicity, grade 8public schools: By state, 1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

Table A.28Example of False Discovery Rate comparisons of average scale scores for different groups ofstudents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

Figure A.2States within regions of the country defined by the U.S. Census Bureau . . . . . . . . . . . . . . . . . . . . . . . 194

Appendix B: Tables

Table B.1Weighted percentage of students, by region of the country, grades 4 and 8: 2003 . . . . . . . . . . . . . . . 198

Table B.2Weighted percentage of students, by gender, grades 4 and 8: 1990–2003 . . . . . . . . . . . . . . . . . . . . 198

Table B.3Weighted percentage of students, by race/ethnicity, grades 4 and 8: 1990–2003 . . . . . . . . . . . . . . . 199

xii T A B L E O F C O N T E N T S • N A E P 2 0 0 3 M AT H E M A T I C S R E P O R T C A R D

Table B.4Weighted percentage of students, by eligibility for free/reduced-price school lunch, grades 4 and 8:1996–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

Table B.5Weighted percentages of students, by eligibility for free/reduced-price school lunch andrace/ethnicity, grades 4 and 8: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

Table B.6Weighted percentage of students, by student-reported parents’ highest level of education, grade 8:1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

Table B.7Weighted percentage of students, by type of school, grades 4 and 8: 1990–2003 . . . . . . . . . . . . . . . 201

Table B.8Weighted percentages of students, by parents’ highest level of education and type ofschool, grade 8: 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

Table B.9Weighted percentage of students, by type of location, grades 4 and 8: 2000–2003 . . . . . . . . . . . . . . 201

Table B.10Weighted percentage of students, by gender, grade 4 public schools: By state, 1992–2003 . . . . . . . . 202

Table B.11Weighted percentage of students, by gender, grade 8 public schools: By state, 1990–2003 . . . . . . . . 203

Table B.12Weighted percentage of students, by race/ethnicity, grade 4 public schools: By state,1992–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

Table B.13Weighted percentage of students, by race/ethnicity, grade 8 public schools: By state,1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

Table B.14Weighted percentage of students, by eligibility for free/reduced-price school lunch, grade 4public schools: By state, 1996–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210

Table B.15Weighted percentage of students, by eligibility for free/reduced-price school lunch,grade 8 public schools: By state, 1996–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

Table B.16Weighted percentage of students, by gender, grades 4 and 8 public schools:By urban district, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

Table B.17Weighted percentage of students, by race/ethnicity, grades 4 and 8 public schools:By urban district, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

Table B.18Weighted percentage of students, by eligibility for free/reduced-price school lunch,grades 4 and 8 public schools: By urban district, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

Table B.19Weighted percentage of students, by student-reported parents’ highest level ofeducation, grade 8 public schools: By urban district, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

TA B L E O F C O N T E N T S • N A E P 2 0 0 3 M AT H E M AT I C S R E P O R T C A R D xiii

Appendix C: Tables

Table C.1Gaps in average mathematics scale scores, by gender, grade 4 public schools: By state,1992–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

Table C.2Gaps in average mathematics scale scores, by gender, grade 8 public schools: By state,1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

Table C.3Percentages of students, by gender and mathematics achievement level, grade 4 public schools:By state, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

Table C.4Percentages of students, by gender and mathematics achievement level, grade 8 public schools:By state, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

Table C.5Percentage of students at or above Basic in mathematics, by gender, grade 4 public schools:By state, 1992–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

Table C.6Percentage of students at or above Basic in mathematics, by gender, grade 8 public schools:By state, 1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

Table C.7Gaps in average mathematics scale scores, by race/ethnicity, grade 4 public schools: By state,1992–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

Table C.8Gaps in average mathematics scale scores, by race/ethnicity, grade 8 public schools: By state,1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225

Table C.9Percentages of students, by race/ethnicity and mathematics achievement level, grade 4 public schools:By state, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

Table C.10Percentages of students, by race/ethnicity and mathematics achievement level, grade 8public schools: By state, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

Table C.11Percentage of students at or above Basic in mathematics, by race/ethnicity, grade 4public schools: By state, 1992–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

Table C.12Percentage of students at or above Basic in mathematics, by race/ethnicity, grade 8public schools: By state, 1990–2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235

Table C.13Percentages of students, by eligibility for free/reduced-price school lunch andmathematics achievement level, grade 4 public schools: By state, 2003 . . . . . . . . . . . . . . . . . . . . . . 238

Table C.14Percentages of students, by eligibility for free/reduced-price school lunch andmathematics achievement level, grade 8 public schools: By state, 2003 . . . . . . . . . . . . . . . . . . . . . . 239

Table C.15Percentage of students at or above Basic in mathematics, by student eligibility forfree/reduced-price school lunch, grade 4 public schools: By state, 1996–2003 . . . . . . . . . . . . . . . . 240

Table C.16Percentage of students at or above Basic in mathematics, by student eligibility forfree/reduced-price school lunch, grade 8 public schools: By state, 1996–2003 . . . . . . . . . . . . . . . . 241

xiv TA B L E O F C O N T E N T S • N A E P 2 0 0 3 M AT H E M A T I C S R E P O R T C A R D

Table C.17Average mathematics scale scores and achievement-level results, by students withand without disabilities and limited English proficiency, grade 4 public schools: By state, 2003 . . . . . 242

Table C.18Average mathematics scale scores and achievement-level results, by students withand without disabilities and limited English proficiency, grade 8 public schools: By state, 2003 . . . . . 244

Table C.19Average mathematics scale score and achievement-level results, by students withdisabilities and limited-English-proficient students, grade 4 public schools: By urban district, 2003 . . 246

Table C.20Average mathematics scale score and achievement-level results, by students withdisabilities and limited-English-proficient students, grade 8 public schools: By urban district, 2003 . . 247

Appendix D: Tables

Table D.1Population and public-school enrollment, from non-NAEP sources: By state,April 2000 and Fall 2000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

Table D.2Poverty status of school-age children and children served under Individuals with DisabilitiesEducation Act and Chapter 1, from non-NAEP sources: By state, 2001 and school years1990–1991 through 2000–2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

Table D.3Expenditure per pupil, average teacher salary, and pupil/teacher ratio, in public schools,from non-NAEP sources: By state, school years 1999–2000, 2001–2002, and fall 2000 . . . . . . . . . . 252

Table D.4Enrollment, expenditure per pupil, and pupil/teacher ratio in public schools, from non-NAEP sources:By urban district, fall 2000 and school year 1999–2000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

E X E C U T I V E S U M M A R Y • N A E P 2 0 0 3 M A T H E M A T I C S R E P O R T C A R D xv

Executive Summary

The National Assessment of Educational Progress(NAEP) is an ongoing nationally representativesample survey of student achievement in core subjectareas. Authorized by Congress and administered bythe National Center for Education Statistics (NCES),within the Institute of Education Sciences of the U.S.Department of Education, NAEP regularly reports tothe public on the educational progress of fourth-,eighth-, and twelfth-grade students.

This report presents results of the NAEP 2003fourth- and eighth-grade mathematics assessments forthe nation, for regions of the country, forparticipating states and other jurisdictions, and forparticipating urban districts. Assessment results aredescribed in terms of students’ average mathematicsscore on a 0–500 scale and in terms of thepercentage of students attaining each of threeachievement levels: Basic, Proficient, and Advanced.

The achievement levels are performance standardsadopted by the National Assessment GoverningBoard (NAGB) as part of its statutory responsibilities.The achievement levels are a collective judgment ofwhat students should know and be able to do for eachgrade tested. The law requires that the achievementlevels are to be used on a trial basis until theCommissioner of Education Statistics determines“that such levels are reasonable, valid, and

xvi E X E C U T I V E S U M M A R Y • N A E P 2 0 0 3 M A T H E M A T I C S R E P O R T C A R D

informative to the public.”1 Until thatdetermination is made, the law requiresthe Commissioner and the Board to stateclearly the trial status of the achievementlevels in all NAEP reports. However, bothNCES and NAGB believe these perfor-mance standards are useful for under-standing trends in student achievement.They have been widely used by nationaland state officials and others as a com-mon yardstick of academic performance.

Approximately 190,000 fourth-gradersfrom 7,500 schools and 153,000 eighth-graders from 6,100 schools were assessedin 2003. The national results reflect theperformance of students attending bothpublic and nonpublic schools, while theresults for participating states and juris-dictions, and for participating urbandistricts, reflect the performance ofstudents attending public schools. Inaddition to providing average scores andachievement-level percentages in math-ematics for the nation, states and otherjurisdictions, and selected urban districts,this report provides results for subgroupsof students defined by various back-ground characteristics.

A summary of major findings from theNAEP 2003 Mathematics Assessment ispresented on the following pages. Com-parisons are made to results from previ-ous years in which the assessment wasadministered. In addition to the 2003results, national results are reportedfrom the 1990, 1992, 1996, and 2000

assessments. Results for states and otherjurisdictions are also reported from the1990 (eighth grade only), 1992, 1996,2000, and 2003 assessments. Results forparticipating urban districts are reportedfor 2003.

The more recent results, from 2000and 2003, are based on more inclusivesamples using administration proceduresin which testing accommodations werepermitted for students with disabilitiesand limited-English-proficient students.Accommodations were not permitted inearlier assessments. Comparisons betweenresults from 2003 and those from 2000,in which both types of administrationprocedures were used, are discussed inthis executive summary based on theresults when accommodations werepermitted.

Changes in student performanceacross years or differences betweengroups of students in 2003 are discussedonly if they have been determined to bestatistically significant at the .05 levelbased on t-tests adjusted using the FalseDiscovery Rate (FDR) multiple compari-son procedure. Beginning with thereading sample in 2002, the NAEPnational samples were obtained by aggre-gating the samples from each state,rather than obtaining an independentlyselected national sample. As a result, thesize of the national sample increased andsmaller differences between years orbetween subgroups of students werefound to be statistically significant thanwould have been detected in previousassessment years.

1 No Child Left Behind Act of 2001, P. L. 107-110, 115 Stat. 1425 (2002).

E X E C U T I V E S U M M A R Y • N A E P 2 0 0 3 M A T H E M A T I C S R E P O R T C A R D xvii

Overall Mathematics Results for theNation, Regions of the Country, andStates and Other JurisdictionsMathematics Results for the Nation

At grade 4

� The average fourth-grade mathematicsscore was higher in 2003 than in all theprevious assessment years.

� Scores at the 10th, 25th, 50th, 75th,and 90th percentiles were higher in2003 than in any of the previousassessment years, indicating improve-ment for lower-, middle-, and higher-performing students. Gains detectedbetween 2000 and 2003 ranged fromapproximately 5 scale score points forstudents performing at the 90thpercentile to 13 points for students atthe 10th percentile.

� In 2003, 32 percent of fourth-gradersperformed at or above the Proficientlevel. The percentages of fourth-graders performing at or above Basic,at or above Proficient, and at Advancedincreased between 2000 and 2003, andwere higher in 2003 than in 1990. Thepercentage at or above Proficient in-creased by approximately 19 pointsbetween 1990 and 2003.

At grade 8

� The average eighth-grade mathematicsscore was higher in 2003 than in allprevious assessment years.

� Scores at the 10th, 25th, 50th, 75th,and 90th percentiles were higher in2003 than in any of the previousassessment years, indicating improve-ment for lower-, middle-, and higher-performing students. Increases de-tected between 2000 and 2003 rangedfrom approximately 3 scale scorepoints at the 90th percentile to 7points at the 10th percentile.

� In 2003, 29 percent of eighth-gradersperformed at or above the Proficientlevel. The percentages of eighth-graders performing at or above Basicand at or above Proficient increasedbetween 2000 and 2003, and werehigher in 2003 than in 1990. Thepercentage at or above Proficient in-creased by approximately 14 pointsbetween 1990 and 2003.

Mathematics Results for Regionsof the CountryPrior to 2003, NAEP results were re-ported for four NAEP-defined regions ofthe nation: Northeast, Southeast, Central,and West. As of 2003, to align NAEP withother federal data collections, NAEPanalysis and reports have used the U.S.Census Bureau’s definition of “region.”The four regions defined by the U.S.Census Bureau are Northeast, South,Midwest, and West.

At grade 4

� The average fourth-grade mathematicsscore was higher for students in theNortheast and Midwest than for stu-dents in the South and West. Theaverage score for students in the Southwas higher than for students in theWest.

� The percentages of fourth-gradersperforming at or above the Basic andProficient levels were higher in theNortheast and Midwest than in theSouth and West. Higher percentagesof students performed at or aboveBasic and Proficient in the South than inthe West.

xviii E X E C U T I V E S U M M A R Y • N A E P 2 0 0 3 M A T H E M A T I C S R E P O R T C A R D

At grade 8

� The average eighth-grade mathematicsscore was higher for students in theNortheast and Midwest than for stu-dents in the South and West. Theaverage score was higher for studentsin the South than for students in theWest.

� Higher percentages of eighth-gradestudents performed at or above Basicand Proficient in the Northeast andMidwest than in the South and West. Ahigher percentage of eighth-gradersperformed at or above Basic in theSouth than in the West.

Mathematics Results for the States andOther JurisdictionsResults from the 2003 assessment arereported for fourth- and eighth-gradestudents attending public schools only in50 states and 3 other jurisdictions thatparticipated in the assessment. (Through-out this report, the term “jurisdiction” isused to refer to the states, the District ofColumbia, and the two Department ofDefense system schools that participatedin the NAEP mathematics assessment.)

At grade 4

� All 43 jurisdictions that participated inboth the 2000 and 2003 fourth-gradeassessments showed increases in aver-age scores. Similarly, each of the 42jurisdictions that participated in the1992 and 2003 assessments had ahigher average score in 2003.

� Kansas, Massachusetts, Minnesota, NewHampshire, North Carolina, Vermont,and Wyoming were among the jurisdic-

tions with the highest average scores.Average fourth-grade scores in Con-necticut and Virginia were lower onlyin comparison with New Hampshire.

� The percentage of fourth-gradersperforming at or above Proficient washigher in 2003 than in 2000 for all 43jurisdictions that participated in bothyears. The percentage of fourth-graders at or above Proficient was higherin 2003 than in 1992 for all 42 jurisdic-tions that participated in both years.

At grade 8

� Of the 42 jurisdictions that partici-pated in both the 2000 and 2003eighth-grade mathematics assessments,28 had a higher average score in 2003.Each of the 38 jurisdictions that partici-pated in both the 1990 and 2003assessments had a higher average scorein 2003.

� In 2003, Minnesota had the highestaverage mathematics score at grade 8.Eighth-graders in Department ofDefense Overseas schools, Kansas,Massachusetts, Montana, New Hamp-shire, North Dakota, South Dakota,and Vermont all had higher averagescores than the remaining jurisdictionsexcept Minnesota.

� Among the 42 jurisdictions that partici-pated in both the 2000 and 2003eighth-grade assessments, 18 showedan increase in the percentage ofstudents performing at or aboveProficient. The percentage of eighth-graders at or above Proficient was higherin 2003 than in 1990 for all 38 jurisdic-tions that participated in both years.

E X E C U T I V E S U M M A R Y • N A E P 2 0 0 3 M A T H E M A T I C S R E P O R T C A R D xix

Mathematics Results for StudentSubgroups in the Nation and in theStates and Other JurisdictionsIn addition to overall results, NAEPreports on the performance of varioussubgroups of students. In interpretingthese data, readers are reminded thatthe relationship between contextualvariables and student performance is notnecessarily causal. There are many factorsthat play a role in student achievement ina particular subject area.

National Results

Gender

� At both grades 4 and 8, the averagescores for both male students andfemale students were higher in 2003than in any of the previous assessmentyears.

� In 2003, male students outperformedfemale students by 3 points on averageat grade 4 and by 2 points on averageat grade 8. The male-female gap in2003 was not measurably differentfrom the gap in any of the previousassessment years since 1990 for eithergrade.

� At both grades 4 and 8, the percent-ages of male students and femalestudents performing at or aboveProficient were higher in 2003 than inany previous assessment year.

Race/EthnicityBased on information obtained fromschool records, students who took theNAEP mathematics assessment wereidentified as belonging to one of fivemutually exclusive racial/ethnic sub-groups: White, Black, Hispanic, Asian/Pacific Islander, American Indian/AlaskaNative, or Other.

� At both grades 4 and 8, Asian/PacificIslander students scored higher onaverage than White students in 2003.Both White students and Asian/PacificIslander students had higher averagescores than Black, Hispanic, andAmerican Indian/Alaska Native stu-dents at both grades. Hispanic studentsand American Indian/Alaska Nativestudents also scored higher on averagethan Black students at both grades.

� At grade 4, White, Black, and Hispanicstudents all had higher average scoresin 2003 than in any of the previousassessment years. American Indian/Alaska Native fourth-graders had ahigher average score in 2003 than in2000. The average score for Asian/Pacific Islander fourth-graders washigher in 2003 than in 1990.

� At grade 8, White, Black, and Hispanicstudents all showed increases in aver-age scores between 2000 and 2003.The average score for Asian/PacificIslander eighth-graders was higher in2003 than in 1990.

xx E X E C U T I V E S U M M A R Y • N A E P 2 0 0 3 M A T H E M A T I C S R E P O R T C A R D

� At grade 4, the score gap betweenWhite students and Black studentsdecreased between 2000 and 2003,and was smaller in 2003 than in 1990.The gap between White fourth-gradersand Hispanic fourth-graders alsonarrowed between 2000 and 2003, butthere was no measurable differencebetween the gap in 2003 and the gapin 1990.

� At grade 8, the score gap betweenWhite students and Black students wasnarrower in 2003 than in 2000, but thegap in 2003 was not measurably differ-ent from that in 1990.

� At both grades 4 and 8, the percentageof students performing at or above theProficient level was higher in 2003 thanin any of the previous assessment yearsfor White, Black, and Hispanic stu-dents. The percentage of Asian/PacificIslander students performing at orabove Proficient was higher in 2003 thanin 1990.

Eligibility for Free/Reduced-Price SchoolLunchNAEP collects data on students’ eligibilityfor free/reduced-price lunch as anindicator of family economic status.Eligibility for free/reduced-price lunch isdetermined by students’ family income inrelation to the federally establishedpoverty level. The mathematics resultssince 1996 are reported for studentsclassified by their eligibility.

� In 2003, the average mathematicsscores for fourth- and eighth-graderswho were eligible for free/reduced-price lunch were lower than that forstudents who were not eligible.

� For students who were eligible andthose who were not eligible, the aver-age mathematics scores for fourth-grade and eighth-grade studentsincreased between 2000 and 2003 andwere higher in 2003 than in 1996.

� At both grades 4 and 8, the percentageof students at or above Proficient washigher in 2003 than in 2000 and 1996for both students who were eligibleand those who were not eligible.

Parents’ Level of EducationEighth-grade students who participatedin the NAEP mathematics assessmentwere asked to indicate the highest levelof education completed by each parent.Information about parental educationwas not collected at grade 4. Results arereported based on the highest level ofeducation reported for either parent.

� Overall, in 2003, there was a positiverelationship between student-reportedparental education and studentachievement: the higher the parentaleducation level, the higher the averagemathematics score.

� Average scores for eighth-grade stu-dents increased from 2000 to 2003 andwere higher in 2003 than in 1990 foreach level of parental educationreported.

� The percentage of eighth-gradersperforming at or above Proficient washigher in 2003 than in 1990 regardlessof the level of parental educationstudents reported.

E X E C U T I V E S U M M A R Y • N A E P 2 0 0 3 M A T H E M A T I C S R E P O R T C A R D xxi

Type of SchoolThe schools that participate in the NAEPassessment are classified as either publicor nonpublic. A further distinction isthen made between nonpublic schoolsthat are Catholic schools and those thatare some other type of nonpublic school.

� In 2003, fourth- and eighth-gradestudents in nonpublic schools hadhigher average scores than students inpublic schools. Eighth-grade studentsin Catholic schools had lower averagescores than eighth-graders in othernonpublic schools.

� At both grades 4 and 8, the averagemathematics scores for students inpublic and nonpublic schools (includ-ing Catholic and other nonpublicschools) increased from 2000 to 2003and were higher in 2003 than in 1990.

� The percentages of fourth- and eighth-graders performing at or above Profi-cient were higher in 2003 than in 1990for students in public schools, Catholicschools, and other nonpublic schools.

Type of LocationThe schools from which NAEP draws itssamples of students are classified accord-ing to their type of location (central city,rural/small town, or urban fringe/largetown). The methods used to identify thetype of school location in 2000 and 2003were different from those used for priorassessment years; therefore, only the datafrom the 2000 and 2003 assessments arereported.

� In 2003, fourth- and eighth-gradestudents in schools located in urbanfringe/large town and rural/smalltown locations had higher averagemathematics scores than those incentral city locations, and students inurban fringe/large town locationsscored higher on average than stu-dents in rural/small town locations.

� The average mathematics scores in allthree location types were higher in2003 than in 2000 for both grades 4and 8.

� The percentage of students at or aboveProficient increased between 2000 and2003 in all three types of locations atgrade 4.

State and Other Jurisdiction Results

Gender� In 2003, male fourth-graders scoredhigher on average than female fourth-graders in 24 jurisdictions. At grade 8,the average score for male students washigher than for female students in Massa-chusetts, South Carolina, and Depart-ment of Defense Overseas schools.

� The average scores increased between1992 and 2003 for both male and femalefourth-graders in all 42 of the jurisdic-tions that participated in both assess-ments. For the 38 jurisdictions thatparticipated in both the 1990 and 2003eighth-grade assessments, 36 showedincreases for both male and femalestudents and Montana and North Dakotashowed increases only for female students.

xxii E X E C U T I V E S U M M A R Y • N A E P 2 0 0 3 M A T H E M A T I C S R E P O R T C A R D

Race/Ethnicity

� At grade 4, average scores were higherin 2003 than in 1992 for White stu-dents in 42 jurisdictions, for Blackstudents in 35 jurisdictions, for His-panic students in 20 jurisdictions, forAsian/Pacific Islander students in 11jurisdictions, and for American In-dian/Alaska Native students in 3jurisdictions.

� At grade 8, average scores were higherin 2003 than in 1990 for White stu-dents in 37 jurisdictions, for Blackstudents in 25 jurisdictions, for His-panic students in 12 jurisdictions, forAsian/Pacific Islander students in 7jurisdictions, and for American In-dian/Alaska Native students in 5jurisdictions.

Eligibility for Free/Reduced-Price SchoolLunch

� In 2003, students who were eligible forfree/reduced-price lunch scoredlower on average than students whowere not eligible in all 52 jurisdictionsfor which data are available at grade 4and in 51 of the 52 jurisdictions forwhich data are available at grade 8.

� The average fourth-grade mathematicsscore increased between 1996 and2003 both for students who wereeligible and students who were noteligible for free/reduced-price lunchin 44 jurisdictions and for students whowere not eligible in North Dakota. Theaverage eighth-grade mathematicsscores increased between 1996 and2003 for both students who wereeligible and students who were noteligible in 22 jurisdictions, for eligiblestudents in Montana, and for studentswho were not eligible in 10 jurisdic-tions.

Urban District ResultsThe 2003 Trial Urban District Assessment(TUDA) included nine urban public-school districts (Atlanta City SchoolDistrict, Boston Public School District,Charlotte-Mecklenburg Schools, City ofChicago School District 299, ClevelandMunicipal School District, HoustonIndependent School District, Los AngelesUnified School District, New York CityPublic Schools, and San Diego CityUnified School District) plus the Districtof Columbia. Results for the urban dis-tricts are compared with results for publicschools in the nation and public schoolsin large central cities.

Overall Mathematics Resultsfor the Urban Districts

At grade 4

� Fourth-graders in all the participatingdistricts except Charlotte scored loweron average than fourth-graders in thenation. Fourth-graders in Charlottehad a higher average score than publicschool students in the nation, largecentral cities, and the other participat-ing districts.

� With the exception of Charlotte,fourth-grade scores at the 25th, 50th,75th, and 90th percentiles were lowerin each of the districts than in thenation. Scores at the 10th, 25th, 50th,75th, and 90th percentiles were higherin Charlotte than in the nation and inlarge central cities.

� The percentage of fourth-graders inCharlotte performing at or aboveProficient was higher than the corre-sponding percentages in both largecentral cities and the nation.

E X E C U T I V E S U M M A R Y • N A E P 2 0 0 3 M A T H E M A T I C S R E P O R T C A R D xxiii

At grade 8

� Eighth-graders in all the participatingdistricts except Charlotte scored loweron average than eighth-graders in thenation. Eighth-graders in Charlottehad a higher average score than publicschool students in the nation, largecentral cities, and the other participat-ing districts.

� Scores at the 25th, 50th, 75th, and90th percentiles in all the districtsexcept Charlotte were lower than inthe nation. In Charlotte, eighth-gradescores at the 10th, 25th, 50th, 75th,and 90th percentiles were higher thanthe scores in large central cities, andthe scores at the 75th and 90th per-centiles were higher than the corre-sponding national scores.

� The percentage of eighth-graders inCharlotte at or above Proficient washigher than the corresponding per-centages in both large central citiesand in the nation.

Results for Student Subgroups in UrbanDistricts

Gender

� At grade 4, the average scores for bothmale and female students in Charlottewere higher than those for theircounterparts in the nation and in largecentral cities. Male and female fourth-graders in Atlanta, Boston, Chicago,Cleveland, the District of Columbia,and Los Angeles had lower averagescores than their counterparts in largecentral cities and in the nation.

� At grade 8, the average scores for bothmale and female students in Charlottewere higher than the correspondingaverage scores for male and femalestudents in large central cities. Bothmale and female eighth-graders inAtlanta, Chicago, Cleveland, theDistrict of Columbia, and Los Angeleshad lower average scores than theircounterparts in large central cities andin the nation.

Race/Ethnicity

� At grade 4, the average scores forWhite students in Charlotte, theDistrict of Columbia, and Houston;Black students in Boston, Charlotte,Houston, and New York City; andHispanic students in Charlotte andHouston were higher than the corre-sponding scores in large central cities.The average scores for fourth-gradeWhite students in Boston, Chicago,and Cleveland; Black students inChicago and the District of Columbia;and Hispanic students in Boston, theDistrict of Columbia, Los Angeles, andSan Diego were lower than the corre-sponding scores in large central cities.

� At grade 8, the average scores forWhite students in Atlanta, Charlotte,and Houston; Black students in Char-lotte, Houston, and New York City; andHispanic students in Houston werehigher than the corresponding scoresin large central cities. The averagescores for eighth-grade White studentsin Cleveland; Black students in Atlanta,the District of Columbia, and LosAngeles; and Hispanic students in theDistrict of Columbia, Los Angeles, andSan Diego were lower than the corre-sponding scores in large central cities.

xxiv E X E C U T I V E S U M M A R Y • N A E P 2 0 0 3 M A T H E M A T I C S R E P O R T C A R D

Eligibility for Free/Reduced-Price Lunch

� At grade 4, the average scores forstudents eligible for free/reduced-price lunch in Charlotte, Houston, andNew York City were higher than theaverage score in large central cities.The average scores for eligible studentsin Atlanta, Chicago, the District ofColumbia, and Los Angeles were lowerthan the average score for eligiblestudents in large central cities.

� At grade 8, the average scores forstudents eligible for free/reduced-price lunch in Boston, Houston, andNew York City were higher than theaverage score in large central cities.The average scores for eligible studentsin Atlanta, the District of Columbia,and Los Angeles were lower than theaverage score in large central cities.

Parents’ Level of Education

� In 2003, the average score for eighth-graders who indicated that at least oneparent graduated from college waslower in Atlanta, Chicago, Cleveland,the District of Columbia, and LosAngeles than the average score forstudents in the same parental educa-tion category in public schools in largecentral cities and in the nation. Theaverage score for eighth-graders whoreported at least one parent graduatedfrom college was higher in Charlotteand San Diego than for students inlarge central cities.

C H A P T E R 1 • N A E P 2 0 0 3 M A T H E M AT I C S R E P O R T C A R D 1

1Introduction

Mathematics provides the basic processes forquantifying information. Using quantities is essentialeverywhere in our society, in every aspect of our dailylives—at home and in school, for commerce, travel,communications, entertainment, and medicine. Evenif mathematics were not important as a key tounderstanding the structure of our world anduniverse, it would still be one of the key competenciesfor personal, civic, and economic engagement.Students need to understand and be able to applymathematical skills and concepts in order to functioneffectively in daily activities such as understandingfinancial information and evaluating product pricing.

Great importance has long been placed on ensuringthat students acquire mathematical skills and conceptsand these skills have increasingly come to be expectedof all students. This report presents major results fromthe National Assessment of Educational Progress(NAEP) 2003 mathematics assessment of the nation’sfourth- and eighth-grade students. In addition, thereport provides results for fourth- and eighth-gradestudents in 53 states and other jurisdictions and for thenine urban school districts that participated in theTrial Urban District Assessment. This report isintended to inform educators, policymakers, parents,and the general public about students’ achievement inmathematics.

2 C H A P T E R 1 • N A E P 2 0 0 3 M AT H E M A T I C S R E P O R T C A R D

Overview of the 2003 NationalAssessment of Educational Progressin MathematicsFor more than 30 years, NAEP has regu-larly collected, analyzed, and reportedvalid and reliable information about whatstudents know and can do in a variety ofsubject areas. As authorized by the U.S.Congress, NAEP assesses representativenational samples of fourth-, eighth-, andtwelfth-grade students. Since 1990, NAEPhas also assessed representative samples offourth- and eighth-grade students in statesand other jurisdictions that participate inthe NAEP state-by-state assessments.NAEP is administered and overseen bythe National Center for Education Statis-tics (NCES), within the U.S. Departmentof Education’s Institute of EducationSciences.

The content of all NAEP assessments isdetermined by subject-area frameworksthat are developed by the National Assess-ment Governing Board (NAGB) in acomprehensive process involving a broadspectrum of interested parties, includingteachers, curriculum specialists, subject-matter specialists, school administrators,parents, and members of the generalpublic. The framework for the NAEP 2003mathematics assessment, which was up-dated in 1996, is essentially the sameframework that has guided developmentof the NAEP mathematics assessmentssince 1990.

This report describes the results of theNAEP 2003 mathematics assessment atgrades 4 and 8. National results for 2003are compared to those from 1990, 1992,1996, and 2000. Using the same test asthat used nationally, state-level assessmentswere conducted at grade 4 in 1992, 1996,2000, and 2003. At grade 8, state-levelassessments were conducted in 1990, 1992,1996, 2000, and 2003. Results for the nine

districts that participated in the TrialUrban District Assessment (TUDA) arereported for 2003 only. Comparisonsacross assessment years are possiblebecause the assessments were developedunder the same basic framework andshare a common set of mathematicsquestions.

Prior to 1996, administration proce-dures for the NAEP mathematics assess-ments did not permit the use of accommo-dations (e.g., extra time; individual ratherthan group administration) for studentswith special needs who could not partici-pate without them. For the 1996 nationalassessment, however, administrativeprocedures were introduced that allowedexpanding participation in NAEP throughthe use of accommodations by studentswith disabilities (SD) and limited-English-proficient (LEP) students (see appendixA). A split-sample design was used at thenational level in 1996 and 2000 and at thestate level in 2000, so that both administra-tion procedures could be used during thesame assessment, but with differentsamples of students. This made it possibleto report trends in students’ mathematicsachievement across all the assessmentyears and, at the same time, examine theeffects of including students assessed withaccommodations on overall assessmentresults. Based on an examination of howpermitting accommodations affectedoverall population results, it was decidedthat, beginning with the 2003 assessment,NAEP would use only one set of proce-dures—permitting the use of accommoda-tions.

During the period in which accommo-dations were not permitted, students withspecial needs could only be included inthe assessment if it was determined byschool staff that they could be assessedmeaningfully without accommodations.The change in administration procedures


makes it possible for more students to beincluded in the assessments; however, italso represents an important altering ofprocedures from previous assessments.(See the section on Students with Disabili-ties and/or Limited-English-ProficientStudents in appendix A for a more de-tailed discussion.) The reader is encour-aged to consider the difference in accom-modation procedures when interpretingcomparisons between the two sets ofresults.

The charts and tables throughout thisreport distinguish between results fromassessment years in which accommoda-tions were not permitted and results fromassessment years in which accommoda-tions were permitted. In the tables andcharts that display results across assess-ment years, all previous assessment resultsthat were found to be significantly differ-ent (at the .05 level based on t-tests ad-justed using the False Discovery Rate(FDR) multiple comparison procedures)from 2003 results are marked with anasterisk (*). Two sets of results are pre-sented for assessment years in which bothadministration procedures were used(accommodations not permitted andaccommodations permitted). Both sets ofresults may also be notated, if found to besignificantly different from 2003. The textthat accompanies these tables and chartsindicates which previous assessmentresults were significantly different from2003. Comparisons between the 2003results, when accommodations werepermitted, and the 1990 and 1992 results,when they were not permitted, are dis-cussed in the text. However, for previousassessment years with both accommoda-tions-not-permitted results and accommo-dations-permitted results, the text de-scribes comparisons only between theaccommodations-permitted results and 2003.

Framework for the 2003 MathematicsAssessment InstrumentThe NAEP Mathematics Framework is theblueprint that has specified the contentand guided the development of eachNAEP mathematics assessment since 1990.The framework resulted from a nationalprocess involving many organizations andindividuals concerned with mathematicseducation. This cooperative effort wasdirected by the National AssessmentGoverning Board (NAGB) and managedby the Council of Chief State SchoolOfficers (CCSSO). In 1996, the frameworkwas refined so that the 1996, 2000, and2003 assessments could better reflectrecent curricular emphases in mathemat-ics, while maintaining the connection tothe 1990 and 1992 assessments in order tomeasure trends in student performance.1

The framework calls for questions basedon five mathematics content areas: 1)number sense, properties, and operations;2) measurement; 3) geometry and spatialsense; 4) data analysis, statistics, andprobability; and 5) algebra and functions.Questions were categorized according totwo additional domains: mathematicalabilities and mathematical power. The firstdomain, mathematical abilities, describesthree types of knowledge or processesrequired for a student to successfullyrespond to a question: conceptual under-standing; procedural knowledge; andproblem solving, the ability to synthesizeseveral processes when confronting amathematical situation. The seconddomain, mathematical power, reflects thethree processes stressed as major goals ofthe mathematics curriculum: the ability toreason, to communicate, and to makeconnections between concepts and skillseither across the mathematics contentareas, or from mathematics to othercurricular areas. Figure 1.1 summarizesthe structure of the 2003 assessment.

1 National Assessment Governing Board. (2002). Mathematics Framework for the 2003 National Assessment ofEducational Progress. Washington, DC: Author.


Figure 1.1 Structure of the NAEP 2003 mathematics assessment

SOURCE: National Assessment Governing Board. (2002). Mathematics Framework for the 2003 National Assessment of Educational Progress. Washington, DC: Author.

Conceptual Understanding

Procedural Knowledge

Problem Solving

Num

ber S

ense

, Pro

perti

es, a

nd O

pera

tions

Mea

sure

men

t

Geom

etry

and

Spa

tial S

ense

Data

Ana

lysi

s, S

tatis

tics,

and

Pro

babi

lity

Alge

bra

and

Func

tions

Reasoning Connections Communications

Mathematical Power

Content Areas

Mat

hem

atic

s Ab

ilitie

s

A breakdown of the percentage ofquestions in each content area prescribedby the framework for the 1990, 1992,1996, 2000, and 2003 assessments isprovided in appendix A (see table A.1).The framework also incorporates the useof calculators (four-function at grade 4and scientific at grade 8), rulers (at grades4 and 8), protractors (at grade 8), andmanipulatives such as spinners and geo-metric shapes (at grades 4 and 8). The useof these ancillary materials and the use ofcalculators were incorporated into someparts of the assessment, but not all. Calcu-lator use was permitted on approximatelyone-third of the test questions.

The NAEP 2003 Mathematics AssessmentInstrumentThe NAEP mathematics assessment is theonly federally authorized, ongoing, na-tionwide assessment of student mathemat-ics achievement. As such, it is necessary forthe assessment to reflect the framework

and expert perspectives on the measure-ment of mathematics performance.During the development process, theassessment undergoes stringent review byteachers and other educators, as well as bystate officials and measurement specialists.All components of the assessment areevaluated for curricular relevance, devel-opmental appropriateness, and fairnessconcerns.

The assessment comprised 50 bookletsat each grade. Each booklet contained twoseparately timed 25-minute sections ofmathematics questions. The total numbersof test questions used in the 2003 math-ematics assessment at grades 4 and 8 were181 and 197, respectively. Typically, asection, or block, contained approxi-mately 16–20 questions, but there wasconsiderable variation depending on thebalance between multiple-choice andconstructed-response questions.


The mathematics blocks include bothmultiple-choice and constructed-responsequestions designed to assess the frame-work objectives. Approximately 50 percentof student assessment time is devoted toconstructed-response questions. Two typesof constructed-response questions areused: 1) short constructed-responsequestions that require students to provideanswers to computation problems or todescribe solutions in one or two sentences,and 2) extended constructed-responsequestions that require students to givemore detailed responses or explanations.Additional information about the designof the 2003 mathematics assessment ispresented in appendix A.

In order to ensure reliable and validscoring of constructed-response questions,a unique scoring guide describing thespecific criteria for assigning a score levelto each student’s response is developed foreach question. Expert scorers go throughextensive training to understand how toapply these scoring criteria fairly andconsistently. During the scoring process,scorers are consistently monitored toensure that scoring standards are beingapplied appropriately and to ensure ahigh degree of scorer agreement (i.e.,interrater reliability). In addition, forthose constructed-response questions thatwere used in previous assessments, moni-toring of scorers includes checking tomake sure that scoring standards remainconsistent from year to year.

In order to minimize the burden on anyindividual student, NAEP uses a procedurereferred to as matrix sampling in which anindividual student is administered only asmall portion of the entire assessment atany grade. For example, at grades 4 and 8,each student is given only one of the 50different grade-specific test booklets, eachcontaining only two 25-minute blocks.Because each block is administered to a

representative sample at each grade, theresults can then be combined to produceaverage group and subgroup results basedon the entire assessment. In addition tocompleting the two 25-minute blocks ineach student’s test booklet, students areasked to complete two sections of back-ground questions that ask about theirhome or school experiences related tomathematics achievement. The timerequired for each student to participate inthe NAEP mathematics assessment isapproximately one hour.

Description of School and StudentSamplesThe NAEP 2003 mathematics assessmentwas administered to fourth- and eighth-graders at the national and state levels. Atthe national level, results are reported forboth public and nonpublic schoolsamples. At the state or jurisdiction level,results are reported only for public schoolstudents. All 50 states and jurisdictionsthat participated in the 2003 assessmentmet the minimum guidelines for reportingtheir results.

In order to obtain a representativesample of students for reporting nationaland state or jurisdiction results, approxi-mately 190,000 fourth-graders from 7,500schools and 153,000 eighth-graders from6,100 schools were sampled and assessedin 2003. Each selected school that partici-pated in the assessment and each studentassessed represents a portion of thepopulation of interest. The nationalsamples for mathematics were larger in2003 than in previous assessment yearsbecause they were based on the combinedsample of students assessed in each partici-pating state, plus an additional samplefrom nonpublic schools. In the 1990–2000assessments, the national samples weredrawn separately from the state samplesand were smaller than the samples result-ing from aggregating the state samples.


For information on sample sizes andparticipation rates for the nation and bystate or jurisdiction, see tables A.6–A.9 inappendix A.

Results from the 2003 Trial UrbanDistrict Assessment (TUDA) are reportedfor the participating districts for public-school students at grades 4 and 8. TheTUDA employed a larger-than-usualsampling rate within the districts, makingreliable district-level data possible. Thesamples were also large enough to providereliable estimates on subgroups within thedistricts, such as female students or His-panic students.

Reporting the Assessment ResultsResults from the NAEP mathematicsassessment are presented in two ways: asscale scores and as percentages of studentsattaining the various achievement levels.The scale scores, indicating how muchstudents know and can do in mathematics,are presented as average scale scores andas scale scores at selected percentiles. Theachievement-level results indicate thedegree to which student performancemeets the standards set for what theyshould know and be able to do. Results arereported only for groups or subgroups ofstudents; individual student performancecannot be reported based on the NAEPassessment.

Average scale score results are based onthe NAEP mathematics scale, which rangesfrom 0 to 500. To calculate students’average scores on the NAEP mathematicsassessment, the first step is to determinethe percentage of students respondingcorrectly to each multiple-choice questionand the percentage of students respond-ing at each score level for both the shortand extended constructed-response

questions. The determination of averagescale scores entails summarizing theresults on separate subscales for each ofthe five content areas in mathematics andthen combining the separate scales toform a single composite scale. (See appen-dix A for more information on scalingprocedures.) Results by separate subscalesare accessible through the NAEP DataTool on the NAEP web site (http://nces.ed.gov/nationsreportcard/naepdata/).

Achievement-level results are presentedin terms of mathematics achievementlevels as authorized by the NAEP legisla-tion and adopted by NAGB. For eachgrade assessed, NAGB has adopted threeachievement levels: Basic, Proficient, andAdvanced. For reporting purposes, achieve-ment-level cut scores are placed on themathematics scale, resulting in fourranges: below Basic, Basic, Proficient, andAdvanced. The achievement-level resultsare then reported as percentages ofstudents scoring within each range, as wellas the percentage of students at or aboveBasic and at or above Proficient.

The Setting of Achievement LevelsThe 1988 NAEP legislation that createdthe National Assessment Governing Boarddirected that the Board establish achieve-ment-level goals for all the subjects as-sessed by NAEP.2 The NAEP 2001 reau-thorization reaffirmed many of theBoard’s statutory responsibilities, includ-ing “developing appropriate studentachievement levels for each grade or agein each subject area to be tested. . . .” 3 Inorder to follow this directive and toachieve the mandate of the original NAEPlegislation, NAGB undertook the develop-ment of student performance standards(called “achievement levels”). Since 1990,

2 National Assessment of Educational Progress Improvement Act, P. L. 100–297, 20 U.S.C. § 1221 et seq.(1988).

3 No Child Left Behind Act of 2001, P. L. 107-110, 115 Stat. 1425 (2002).

http://nces.ed.gov/nationsreportcard/naepdata/


the Board has adopted achievement levelsin mathematics, reading, U.S. history, worldgeography, science, writing, and civics.

The Board defined three levels for eachgrade: Basic, Proficient, and Advanced. TheBasic level denotes partial mastery of theknowledge and skills that are fundamentalfor proficient work at a given grade. TheProficient level represents solid academicperformance. Students reaching this leveldemonstrate competency over challengingsubject matter. The Advanced level pre-sumes mastery of both the Basic andProficient levels and represents superior

performance. Figure 1.2 presents thepolicy definitions of the achievementlevels that apply across grades and subjectareas. The policy definitions guided thedevelopment of the achievement levelsestablished in all subject areas. Adoptingthree levels of achievement for each gradesignals the importance of looking at morethan one standard of performance. In theBoard’s view, the overall achievement goalfor students is performance at the Profi-cient level or higher as measured by NAEP.The Basic level is not the desired goal, butrepresents partial mastery that is a steptoward Proficient.

Basic

Proficient

Advanced

Figure 1.2 Policy definitions of the three NAEP achievement levels

This level denotes partial mastery of prerequisite knowledge and skills that arefundamental for proficient work at each grade.

This level represents solid academic performance for each grade assessed. Studentsreaching this level have demonstrated competency over challenging subject matter,including subject-matter knowledge, application of such knowledge to real-worldsituations, and analytical skills appropriate to the subject matter.

This level signifies superior performance.

SOURCE: National Assessment Governing Board. (2002). Mathematics Framework for the 2003 National Assessment of Educational Progress. Washington, DC: Author.

Achievement Levels

The achievement levels in this reportwere adopted by the Board based on astandard-setting process designed andconducted under a contract with ACT. Todevelop these levels, ACT convened across-section of educators and interestedcitizens from across the nation and askedthem to judge what students should knowand be able to do relative to a body ofcontent reflected in the mathematicsframework. This process of settingachievement levels was reviewed by anarray of individuals includingpolicymakers, representatives of profes-sional organizations, teachers, parents,

and other members of the general public.Prior to adopting these levels of studentachievement, NAGB engaged a largenumber of people to comment on therecommended levels and to review theresults.

The results of the achievement-level-setting process, after NAGB’s approval,became a set of achievement-level descrip-tions and a set of achievement-level cutscores on the 0–500 NAEP mathematicsscale. The cut scores are the scores thatdefine the boundaries between belowBasic, Basic, Proficient, and Advanced perfor-mance levels at each grade.


Mathematics Achievement-LevelDescriptions for Each GradeSpecific definitions of the Basic, Proficient,and Advanced mathematics achievementlevels for grades 4 and 8 are presented infigures 1.3 and 1.4. As noted previously,the achievement levels are cumulative;therefore, students performing at theProficient level also display the competen-cies associated with the Basic level, andstudents at the Advanced level also demon-

strate the competencies associated withboth the Basic and Proficient levels. Foreach achievement level listed in figures 1.3and 1.4, the scale score that correspondsto the lowest cut score within that level onthe NAEP mathematics scale is shown inparentheses. For example, in figure 1.3,the scale score of 249 corresponds to thelowest score in the range defining thegrade 4 Proficient level of achievement inmathematics.

Figure 1.3 Descriptions of NAEP mathematics achievement levels, grade 4Grade 4

Achievement Levels

Basic Fourth-grade students performing at the Basic level should show some evidence(214) of understanding the mathematical concepts and procedures in the five NAEP

content strands.

Fourth graders performing at the Basic level should be able to estimate and use basicfacts to perform simple computations with whole numbers, show some understanding offractions and decimals, and solve some simple real-world problems in all NAEP contentstrands. Students at this level should be able to use — though not always accurately —four-function calculators, rulers, and geometric shapes. Their written responses are oftenminimal and presented without supporting information.

Proficient Fourth-grade students performing at the Proficient level should consistently apply(249) integrated procedural knowledge and conceptual understanding to problem solving

in the five NAEP content strands.

Fourth graders performing at the Proficient level should be able to use whole numbers toestimate, compute, and determine whether results are reasonable. They should have aconceptual understanding of fractions and decimals; be able to solve real-world problemsin all NAEP content strands; and use four-function calculators, rulers, and geometricshapes appropriately. Students performing at the Proficient level should employ problem-solving strategies such as identifying and using appropriate information. Their writtensolutions should be organized and presented both with supporting information andexplanations of how they were achieved.

Advanced Fourth-grade students performing at the Advanced level should apply integrated(282) procedural knowledge and conceptual understanding to complex and nonroutine

real-world problem solving in the five NAEP content strands.

Fourth graders performing at the Advanced level should be able to solve complexnonroutine real-world problems in all NAEP content strands. They should display masteryin the use of four-function calculators, rulers, and geometric shapes. These students areexpected to draw logical conclusions and justify answers and solution processes byexplaining why, as well as how, they were achieved. They should go beyond the obvious intheir interpretations and be able to communicate their thoughts clearly and concisely.

NOTE: The scores in parentheses indicate the cut point on the scale at which the achievement-level range begins.SOURCE: National Assessment Governing Board. (2002). Mathematics Framework for the 2003 National Assessment of Educational Progress. Washington, DC: Author.


Figure 1.4 Descriptions of NAEP mathematics achievement levels, grade 8

NOTE: The scores in parentheses indicate the cut point on the scale at which the achievement-level range begins.SOURCE: National Assessment Governing Board. (2002). Mathematics Framework for the 2003 National Assessment of Educational Progress. Washington, DC: Author.

Grade 8Achievement Levels

Basic Eighth-grade students performing at the Basic level should exhibit evidence of(262) conceptual and procedural understanding in the five NAEP content strands. This

level of performance signifies an understanding of arithmetic operations—including estimation—on whole numbers, decimals, fractions, and percents.

Eighth graders performing at the Basic level should complete problems correctly withthe help of structural prompts such as diagrams, charts, and graphs. They should beable to solve problems in all NAEP content strands through the appropriate selectionand use of strategies and technological tools—including calculators, computers, andgeometric shapes. Students at this level also should be able to use fundamentalalgebraic and informal geometric concepts in problem solving.

As they approach the Proficient level, students at the Basic level should be able todetermine which of the available data are necessary and sufficient for correctsolutions and use them in problem solving. However, these eighth graders showlimited skill in communicating mathematically.

Proficient Eighth-grade students performing at the Proficient level should apply mathematical(299) concepts and procedures consistently to complex problems in the five NAEP

content strands.

Eighth graders performing at the Proficient level should be able to conjecture, defendtheir ideas, and give supporting examples. They should understand the connectionsamong fractions, percents, decimals, and other mathematical topics such as algebraand functions. Students at this level are expected to have a thorough understandingof basic-level arithmetic operations—an understanding sufficient for problem solvingin practical situations.

Quantity and spatial relationships in problem solving and reasoning should befamiliar to them, and they should be able to convey underlying reasoning skillsbeyond the level of arithmetic. They should be able to compare and contrast math-ematical ideas and generate their own examples. These students should makeinferences from data and graphs, apply properties of informal geometry, and accu-rately use the tools of technology. Students at this level should understand theprocess of gathering and organizing data and be able to calculate, evaluate, andcommunicate results within the domain of statistics and probability.

Advanced Eighth-grade students performing at the Advanced level should be able to reach(333) beyond the recognition, identification, and application of mathematical rules in

order to generalize and synthesize concepts and principles in the five NAEPcontent strands.

Eighth graders performing at the Advanced level should be able to probe examplesand counterexamples in order to shape generalizations from which they can developmodels. Eighth graders performing at the Advanced level should use number senseand geometric awareness to consider the reasonableness of an answer. They areexpected to use abstract thinking to create unique problem-solving techniques andexplain the reasoning processes underlying their conclusions.


Trial Status of Achievement LevelsThe law requires that the achievementlevels are to be used on a trial basis untilthe Commissioner of Education Statisticsdetermines “that such levels are reason-able, valid, and informative to the pub-lic.”4 Until that determination is made, thelaw requires the Commissioner and theBoard to state clearly the trial status of theachievement levels in all NAEP reports. In1993, the first of several congressionallymandated evaluations of the achievement-level-setting process concluded that theprocedures used to set the achievementlevels were flawed and that the percentageof students at or above any particularachievement-level cut point may be under-estimated. 5 Others have critiqued theseevaluations, asserting that the weight ofthe empirical evidence does not supportsuch conclusions.6

In response to the evaluations andcritiques, NAGB sponsored an additionalstudy of the 1992 reading achievementlevels before deciding to use them forreporting NAEP 1994 results.7 Whenreviewing the findings of this study, theNational Academy of Education (NAE)panel expressed concern about what it saw

4 No Child Left Behind Act of 2001, P. L. 107-110, 115 Stat. 1425 (2002).5 United States General Accounting Office. (1993). Education Achievement Standards: NAGB’s Approach

Yields Misleading Interpretations. U.S. General Accounting Office Report to Congressional Requestors.Washington, DC: Author.National Academy of Education. (1993). Setting Performance Standards for Achievement: A Report of theNational Academy of Education Panel on the Evaluations of the NAEP Trial State Assessment: An Evaluation ofthe 1992 Achievement Levels. Stanford, CA: Author.

6 Cizek, G. (1993). Reactions to National Academy of Education Report. Washington, DC: National Assess-ment Governing Board.Kane, M. (1993). Comments on the NAE Evaluation of the NAGB Achievement Levels. Washington, DC:National Assessment Governing Board.

7 American College Testing. (1995). NAEP Reading Revisited: An Evaluation of the 1992 Achievement LevelDescriptions. Washington, DC: National Assessment Governing Board.

8 National Academy of Education. (1996). Reading Achievement Levels. In Quality and Utility: The 1994Trial State Assessment in Reading. The Fourth Report of the National Academy of Education Panel on theEvaluation of the NAEP Trial State Assessment. Stanford, CA: Author.

9 National Academy of Education. (1997). Assessment in Transition: Monitoring the Nation’s EducationalProgress, p. 99. Mountain View, CA: Author.

10 Reckase, M. D. (2000). The Evolution of the NAEP Achievement Levels Setting Process: A Summary of theResearch and Development Efforts Conducted by ACT. Iowa City, IA: ACT, Inc.

as a “confirmatory bias” in the study andabout the inability of this study to “addressthe panel’s perception that the levels hadbeen set too high.”8 In 1997, the NAEpanel summarized its concerns withinterpreting NAEP results based on theachievement levels as follows:

First, the potential instability of thelevels may interfere with the accurateportrayal of trends. Second, the per-ception that few American studentsare attaining the higher standards wehave set for them may deflect atten-tion to the wrong aspects of educa-tion reform. The public has indicatedits interest in benchmarking againstinternational standards, yet it is note-worthy that when American studentsperformed very well on a 1991 inter-national reading assessment, theseresults were discounted because theywere contradicted by poor perfor-mance against the possibly flawedNAEP reading achievement levels inthe following year.9

NCES and NAGB have sought andcontinue to seek new and better ways toset performance standards for NAEP.10


11 National Assessment Governing Board and National Center for Education Statistics. (1995). Proceedingsof the Joint Conference on Standard Setting for Large-Scale Assessments of the National Assessment GoverningBoard (NAGB) and the National Center for Education Statistics (NCES). Washington, DC: U.S. GovernmentPrinting Office.

12 Pellegrino, J. W., Jones, L. R., and Mitchell, K. J. (Eds.). (1998). Grading the Nation’s Report Card:Evaluating NAEP and Transforming the Assessment of Educational Progress. Committee on the Evaluation ofNational Assessments of Educational Progress, Board on Testing and Assessment, Commission onBehavioral and Social Sciences and Education, National Research Council. Washington, DC: NationalAcademy Press.

13 Ibid., 176.14 Forsyth, R. A. (2000). A Description of the Standard-Setting Procedures Used by Three Standardized

Test Publishers. In Student Performance Standards on the National Assessment of Educational Progress:Affirmations and Improvements. Washington, DC: National Assessment Governing Board.Nellhaus, J. M. (2000). States with NAEP-Like Performance Standards. In Student Performance Standardson the National Assessment of Educational Progress: Affirmation and Improvement. Washington, DC: NationalAssessment Governing Board.

For example, NCES and NAGB jointlysponsored a national conference thatexplored many issues related to standardsetting in large-scale assessments.11 Al-though new directions were presented anddiscussed, a proven alternative to thecurrent process has not yet been identi-fied. NCES and NAGB continue to call onthe research community to assist in find-ing ways to improve standard setting forreporting NAEP results.

The most recent congressionally man-dated evaluation conducted by the Na-tional Academy of Sciences (NAS) reliedon prior studies of achievement levels,rather than carrying out new evaluations,on the grounds that the process has notchanged substantially since the initialproblems were identified. Instead, theNAS panel studied the development of the1996 science achievement levels. The NASpanel basically concurred with earliercongressionally mandated studies. Thepanel concluded that “NAEP’s currentachievement-level-setting proceduresremain fundamentally flawed. The judg-ment tasks are difficult and confusing;raters’ judgments of different item typesare internally inconsistent; appropriatevalidity evidence for the cut scores islacking; and the process has producedunreasonable results.”12

The NAS panel accepted the continuinguse of achievement levels in reportingNAEP results on a trial basis, until suchtime as better procedures can be devel-oped. Specifically, the NAS panel con-cluded that “. . . tracking changes in thepercentages of students performing at orabove those cut scores (or in fact, anyselected cut scores) can be of use indescribing changes in student perfor-mance over time.”13

NAGB urges all who are concernedabout student performance levels torecognize that the use of these achieve-ment levels is a developing process and issubject to various interpretations. NAGBand NCES believe that the achievementlevels are useful for reporting trends inthe educational achievement of students.14

In fact, achievement-level results havebeen used in reports by the President ofthe United States, the Secretary of Educa-tion, state governors, legislators, andmembers of Congress. Governmentleaders in the nation and in more than 40states use these results in their annualreports. However, based on the congres-sionally mandated evaluations so far,NCES agrees with the NAS panel’s recom-mendation that caution needs to beexercised in the use of the currentachievement levels. NCES has concluded


that these achievement levels shouldcontinue to be used on a trial basis and beinterpreted with caution.

Interpreting NAEP ResultsThe average scores and percentagespresented in this report are estimatesbased on samples of students rather thanon entire populations. Moreover, thecollection of questions used at each gradelevel is but a sample of the many questionsthat could have been asked to assess theskills and abilities described in the NAEPmathematics framework. As such, theresults are subject to a measure of uncer-tainty, reflected in the standard error ofthe estimates—a range of a few pointsabove or below the score—which accountsfor potential score or percentage fluctua-tion due to sampling and measurementerror. The estimated standard errors forthe estimated scale scores and percentagesin this report are accessible through theNAEP Data Tool on the NAEP web site(http://nces.ed.gov/nationsreportcard/naepdata/). Examples of these estimatedstandard errors are also provided inappendix A, tables A.23 to A.27, of thisreport.

The differences between scale scoresand between percentages discussed in thefollowing chapters take into account thestandard errors associated with the esti-mates. Comparisons are based on statisti-cal tests that consider both the magnitudeof the difference between the groupaverage scores or percentages and thestandard errors of those statistics.

Estimates based on subgroups with smallersample sizes are likely to have relativelylarge standard errors. As a consequence,some seemingly large differences may notbe statistically significant. That is, it cannotbe determined whether these differencesare due to the particular makeup of thesamples of students who were selected, orto true differences in the population ofinterest. When this is the case, the term“apparent difference” or “no measurabledifference” is used in this report. Differ-ences between scores or between percent-ages are discussed in this report only whenthey are significant from a statisticalperspective.

Beginning with the reading sample in2002, the NAEP national samples wereobtained by aggregating the samples fromeach state, rather than obtaining an inde-pendently selected national sample.Consquently, the national sample size in-creased and smaller differences betweenyears or between subgroups of studentswere found to be statistically significantthan would have been detected in previ-ous assessment years. In keeping with pastpractice, all statistically significant differ-ences are indicated in this report. All dif-ferences reported are significant at the .05level with appropriate adjustments formultiple comparisons. The term “signifi-cant” is not intended to imply a judgmentabout the absolute magnitude or the edu-cational relevance of the differences. It isintended to identify statistically depend-able differences in average scores or per-


centages to help inform dialogue amongpolicymakers, educators, and the public.

While the score ranges at each grade inmathematics are identical, the scale wasderived independently at each grade.Therefore, average scale scores acrossgrades cannot be compared. For example,equal scale scores on the grade 4 andgrade 8 scales do not imply equal levels ofmathematics achievement.

Comparisons of performance resultsmay be affected by changes in exclusionrates for students with disabilities andlimited-English-proficient students inNAEP samples. Percentages of studentsexcluded from NAEP may vary consider-ably across states or districts, as well asacross years. Comparisons of achievementresults should be interpreted with cautionif the exclusion rates vary widely. Thepercentages of students who were identi-fied and assessed or excluded based ontheir disability or limited-English-profi-cient status are presented in appendix A.

The results presented are meant todescribe some aspects of the condition ofeducation. They are best viewed as sug-gesting various ideas to be further exam-ined in light of other data, including stateand local data, and in the context of thelarge research literature elaborating onthe many factors contributing to educa-tional achievement.

However, some readers are tempted tomake unwarranted causal inferences fromsimple cross tabulations. At the risk ofsounding dogmatic, it is almost never thecase that a simple cross tabulation of anyvariable with a measure of educationalachievement is conclusive proof thatdifferences in that variable are a cause ofdifferential educational achievement. Theold adage that “correlation is not causa-tion” is a wise precaution to be kept inmind when viewing the results presented

here. Experienced researchers routinelyformulate multiple hypotheses to takethese possibilities into account and read-ers of this volume are encouraged to dolikewise.

Additional NAEP data are available inthe NAEP data tool and in restricted-access research databases. Researchers andpolicy analysts are free to make use of thedata (subject to various confidentialityrestrictions) as they wish. However,as part of the Institute for EducationSciences, NCES has a responsibility to tryto discourage misleading inferences fromthe data presented and to educate thepublic on the difficulty of making validcausal inferences in a field as complex aseducation.

Overview of the Remaining ReportThis report describes the mathematicsperformance of fourth- and eighth-gradersin the nation, participating states andother jurisdictions, large central cityschool districts, and selected urban schooldistricts. Chapter 2 presents overall math-ematics scale scores and achievement-levelresults across years for both the nationand participating states and other jurisdic-tions. Chapter 3 discusses national resultsfor subgroups of students by gender, race/ethnicity, students’ eligibility for free/reduced-price school lunch, parents’highest level of education (for grade 8only), type of school (public andnonpublic), and school’s type of location(central city, urban fringe/large town,rural/small town). State and jurisdictionresults are reported by gender, race/ethnicity, and eligibility for free/reduced-price lunch. Overall and subgroup resultsfor selected urban districts that were partof the TUDA are presented in chapter 4.

Chapter 5 presents sample assessmentquestions and student responses at eachgrade level, including samples of multiple-


choice and constructed-response ques-tions. A table showing the percentage ofstudents at each achievement level whoanswered the questions successfully ac-companies each sample question. Inaddition, item maps for each grade leveldescribe the skill or ability needed toanswer particular mathematics questionsand show the score points at which indi-vidual students had a high probability ofsuccessfully answering particular ques-tions, thereby indicating the relativedifficulty of each question.

The appendices of this report containinformation to expand the results pre-sented in chapters 2–5. Appendix Acontains an overview of assessment devel-opment, sampling, administration, andanalysis procedures. Appendix B presentsthe percentages of students in each of thesubgroups reported for the nation, statesand other jurisdictions, and other selectedurban districts. Appendix C includestables with additional state-level anddistrict-level subgroup results. Finally,appendix D shows state-level and district-level contextual data from sources otherthan NAEP.

Documents

The Nation's Report Card: Mathematics 2003 · 2005. 7. 25. · Chapter 1 The Nation’s Report Card™ Mathematics 2003 U.S. Department of Education Institute of Education Sciences