Tracking Test Scores to Address Educational Accountability Standards in a Public School System

Faculty Sponsor: George Tremblay,

Ph.D. Antioch University New

England

Michael Duffin Program

Evaluation and Educational

Research Associates, Inc.

Student Author:Megan Phillips

Antioch University New

England

Poster presented at the New England Psychological Association Annual Meeting

October 21, 2006

Acknowledgements

This project was undertaken with the support of the Place-Based Education Evaluation Collaborative (PEEC). More information about PEEC can be found at http://www.peecworks.org/

The data presented here were collected as part of an evaluation conducted by Program Evaluation and Educational Research Assocaites, Inc., under the supervision of Michael Duffin.

INTRODUCTION

Place-based education is an innovative approach to teaching that uses interdisciplinary curricula in local natural and built environments. Antioch New England Institute’s Community-based School Environmental Education (CO-SEED) Project strives to:

• expose students to hands-on, real-world learning experiences

• enhance academic engagement• strengthen community ties• promote appreciation for the natural world• increase citizenship among students

INTRODUCTION

Given current pressure on schools to meet standards for educational accountability, educators are understandably eager to demonstrate that innovative programs, whatever other benefits they may offer, do not detract from student performance on standardized achievement tests.

This study describes one aspect of an evaluation of the

CO-SEED curriculum in a New Hampshire public school, comparing longitudinal trends in standardized test performance for the District versus the State.

The CO-SEED program was introduced in Gorham Elementary School in the spring of 1998 on a whole school scale.

METHOD: Research Design

To compare student academic achievement before and after the introduction of CO-SEED, students were grouped into eight sequential cohorts. Cohorts 1-4 completed their Grade 3 testing prior to the introduction of the CO-SEED program; only those in Cohort 1 completed their Grade 6 testing prior to CO-SEED.

Cohort Grade 3 Grade 6

Cohort 1

1993-1994

1996-1997

Cohort 2

1994-1995

1997-1998

Cohort 3

1995-1996

1998-1999

Cohort 4

1996-1997

1999-2000

Cohort 5

1997-1998

2000-2001

Cohort 6

1998-1999

2001-2002

Cohort 7

1999-2000

2002-2003

Cohort 8

2000-2001

2003-2004

METHOD: Measurement Test scores from the New Hampshire Educational Improvement

and Assessment Program (NHEIAP) were obtained through the New Hampshire Department of Education’s website and from published reports available through the New Hampshire Department of Education.

The most consistent measure of student achievement across all academic years was the proportion of students scoring at each of four proficiency levels: Novice, Basic, Proficient, and Advanced. Because adequate yearly progress is measured according to the proportion of students who fall at Basic or above proficiency levels, for purposes of this analysis the Basic, Proficient, and Advanced levels were combined into one category labeled “Passing.” Of particular interest to us was the migration of students into categories representing greater or lesser proficiency, between the standardized assessments in Grade 3 and Grade 6. For 1997 and later, we also had access to Gorham’s test score rank in the State.

RESULTS

• At the State level, we see a downward migration of students from the Passing category into the Novice category as they progress from Grades 3 to 6.

• Gorham shows the opposite pattern. This pattern was present from Cohort 1, before the introduction of CO-SEED, and continued through Cohort 8.

Cohort 1 (1994/1997): Language ArtsPercent of students at each proficiency level

Gor

ham

Gr

3

Gor

ham

Gr

6

Gor

ham

Gr

6

Sta

te G

r 3

Sta

te G

r 3

Sta

te G

r 6

Sta

te G

r 6

Gor

ham

Gr

30

20

40

60

80

100

Novice Passing

% s

tud

ents

Local Versus State Trends in Proficiency Categories

• A noticeable pattern of improvement in state rank between Grades 3 and 6 was seen in cohorts for which state rank data were available (Cohorts 4-7; academic years 1997-98 through 2003-04). Each cohort made a large leap in state rank in both Language Arts and Math.

Change in Gorham State Rank from Grade 3 to 6: Language Arts

0

20

40

60

80

100

Grade 3 Grade 6

Stat

e ra

nk

Cohort 4

Cohort 5

Cohort 6

Cohort 7

RESULTS (cont.)

Change in Gorham State Rank from Grade 3 to 6: Math

0

20

40

60

80

100

Grade 3 Grade 6

Stat

e ra

nk

Cohort 4

Cohort 5

Cohort 6

Cohort 7

Changes in Gorham State Rank from Grade 3 to Grade 6

RESULTS (cont.)

• However, data from the pre-CO-SEED academic year (1997-98) showed that the pattern of dramatic increase in state rank from Grade 3 to Grade 6 may have existed prior to the introduction of CO-SEED, suggesting that the trend may not be CO-SEED specific.

DISCUSSION

The data demonstrate remarkably consistent and impressive trends in Gorham students’ achievement, using State norms as a benchmark, between Grades 3 and 6. While the typical pattern across the State is for students to migrate from Passing proficiency categories into the Novice category, Gorham students are moving in the opposite direction.

Cohort 1 Language Arts: Between Grades 3 and 6, Gorham shows a decrease in Novice level and

an increase in Passing levels; State shows opposite pattern

-20

25

7

-7

-40-30-20-10

010203040

Novice Passing

chan

ge

fro

m G

r 3

to

Gr

6Gorham Gr 3 to Gr 6

State Gr 3 to Gr 6

Although interviews with administrators, educators, and students attest to an important role for CO-SEED in Gorham’s achievement, the data presented here reveal that this trend pre-dated the introduction of CO-SEED, and thus must be at least in part due to some pre-existing attributes of this school system.

While these results do not support CO-SEED as a sole factor responsible for the observed increase in student achievement in Gorham, the curriculum does not appear to be detrimental to student performance on state assessments. Because CO-SEED is a highly experiential and non-traditional curriculum, this result is important in light of the current political climate’s emphasis on accountability for educational programming.

DISCUSSION (cont.)