Assessment and accountability: Assessment and accountability:

From teachable moments to From teachable moments to teststests

Peter AfflerbachPeter Afflerbach

University of MarylandUniversity of Maryland

afflo@umd.eduafflo@umd.edu

Reading Research ConferenceReading Research Conference

International Reading AssociationInternational Reading Association

Atlanta, GAAtlanta, GA

May 3, 2008May 3, 2008

Some past history…and assessment

Public School 33 Queens

Junior High School 109 Queens

Jamaica High School, Jamaica, Queens

1974 Betty Crocker “Leader of Tomorrow” Award

(Afflerbach, 2007; Black & Wiliam, 1998; Calfee & Hiebert, 1991; Crooks, 1988; Davis, 1998; Pellegrino, Chudowsky & Glaser, 2001)

The state of research on assessment

Assessment as the dependent variable in much reading research

Assessment itself is much less likely to be the focus of research

Psychometric inquiry related to validity and reliability

Questions about most useful, most effective,

most efficient assessment need our attention

Assessment in the nexus

Politics

Economics

Agendas

READING ASSESSMENT

Student growth and achievement

Teacher professional development

High quality teaching

Zone of proximal development

Curriculum materials

Framing questions and an observation

Where is our assessment focus?

Do we attend to one type of assessmentat the expense of others?

Is our assessment focus one that may actually

impede student achievement and limit teacher effectiveness?

Is our assessment one that supports students and teachers in their critical, daily work?

How might a balance of assessment, focused on both teachable moments and tests,

help us reach our goals?

Accountability is measured by a single test

on a single day, butaccountability is accomplishedwith daily, useful assessments

that inform our best instruction.

Accountability for teaching and learning Accountability for teaching and learning will be realized when there is balance will be realized when there is balance

in literacy assessment, in literacy assessment, including balance in:including balance in:

• • meeting the needs of different meeting the needs of different audiences and purposes of assessmentaudiences and purposes of assessment

• • formative assessment and summative formative assessment and summative assessmentassessment

• • the assessment that is donethe assessment that is done to to or or forfor students and assessment that is done students and assessment that is done withwith and and byby students students

• • the assessment of what students learn the assessment of what students learn and how students use this knowledgeand how students use this knowledge

• • the assessment of cognitive and the assessment of cognitive and affective factors related to literacyaffective factors related to literacy

……the need for balance in the need for balance in literacy assessment, literacy assessment, including balance in:including balance in:

• • the demands for teacher and school the demands for teacher and school accountability and professional development accountability and professional development opportunities that help teachers develop opportunities that help teachers develop expertise in assessmentexpertise in assessment

The The contextscontexts in which we conduct in which we conduct reading assessment are influenced by (and reading assessment are influenced by (and reflect) political, economic and social reflect) political, economic and social

factorsfactors

• The prevalence of high stakes testing, NCLB or no NCLB

In 2008…In 2008…

• Traditions and habits of assessment

• The tests that our children take today look much like those that we took when we were in school

• How does our current understanding of literacy influence our assessments?

• How does our current understanding of assessment influence our assessments?

‘‘The Report Card’The Report Card’

What do you think Johnny did next?What do you think Johnny did next? 1. hid his report card1. hid his report card 2. sneaked away2. sneaked away 3. scolded his mother3. scolded his mother 4. showed his report card to his parents4. showed his report card to his parents

(Nelson Denny Reading Test, 1966)(Nelson Denny Reading Test, 1966)

‘‘Saved by a Fly’Saved by a Fly’

What is the main idea of the story?What is the main idea of the story? 1. A moose drinks all the water from a river1. A moose drinks all the water from a river 2. A group of animals is afraid of a big moose2. A group of animals is afraid of a big moose 3. A fly gets rid of a troublesome moose3. A fly gets rid of a troublesome moose 4. A group of animals is always quarreling4. A group of animals is always quarreling

(Maryland State Assessment, 2006)(Maryland State Assessment, 2006)

Necessary Necessary alignmentsalignments for for useful reading useful reading assessments:assessments:

The construct of readingThe construct of reading

Reading standards and Reading standards and

benchmarksbenchmarks

Reading curriculum and instructionReading curriculum and instruction

Reading assessmentReading assessment

1. Balance in meeting the needs of different 1. Balance in meeting the needs of different

audiences and purposes of assessmentaudiences and purposes of assessmentAudience Purpose

Students To report on learning and communicate progress To motivate and encourage

To teach children about assessment and how to assess their own work and

progress To build student independence

Teachers To determine nature of student learning To inform instruction

To evaluate students and construct grades

To diagnose student strengths and weaknesses

Afflerbach, 2007

Taxpayers To demonstrate that tax dollars are well spent

Afflerbach, 2007

Politicians To establish accountability of schools

To inform the public of school progress

School To determine instructional program administrators effectiveness

To prove school and teacher accountability

Audience Purpose

Parents To inform of their children’s achievement

To help connect home and school efforts to support student

1. Balance in meeting the needs of different 1. Balance in meeting the needs of different

audiences and purposes of assessment (cont.)audiences and purposes of assessment (cont.)

2. Balance in formative assessment and 2. Balance in formative assessment and summative assessmentsummative assessment

Formative assessment provides information for:

• Understanding individual students and their immediate needs

• The teachable moment

• Creating scaffolded approaches to instruction

(Black & Wiliam, 1998; Crooks, 1988)

Summative assessment provides information for:

• Comparing students’ literacy products

• Judging students’ achievement in relation to benchmarks, standards and each other

(National Reading Panel, 2000; Pellegrino et al, 2001)

In a thoughtful assessment system, formative and summative assessment work

together…

Formative

-ball control

-kicking

-passing

-vision

-resilience

-position

-speed

-team play

-creativity

Summative

-score at end of game

-team standing at end of season

-personalimprovementfrom season to season

Zone of proximal developmentZone of proximal development, , assessmentassessment and and teachable momentsteachable moments

Student's next level of competency and achievement

Assessment and teaching

Zone of proximal development

Teaching and assessment

Student's current level of competency and achievement

(Vygotsky, 1978; Palincsar & Brown, 1984; Rosenshine & Meister, 1994; Palincsar &

Herrenkohl, 2002)

Student's next level of competency and achievement

Where our best teaching happens

Assessment and teaching

Zone of proximal development

Teaching and assessment

Where accountability is created

Student's current level of competency and achievement

A bit more on this…

Formative assessment is used for:

Understanding individual studentsand their immediate needs

Informing the teachable moment

Creating scaffolded approaches to reading instruction

Formative assessment done well leads to good news on summative assessments

Is the daily assessment that identifies individual student needs and allows us to

address them high stakes assessment ?

Important balance questions:

Who is best served by the current array of assessments?

What percentage of assessment across the school year

is formative? Summative?

Is this a good thing?

A test score tells us little or nothing of the means by which it was achieved.

3. Balance the assessment that is done3. Balance the assessment that is done toto or or forfor students students

and assessment that is done and assessment that is done withwith and and byby students students What does assessment mean to our

students?

How do students conceptualize assessment?

If a universal goal of reading instruction is to help

students become independent and successful, are we doing enough

to foster independence?

How can assessment help foster this independence?

“The black box”

My work in my work out = The grade

The need to make transparent and tangible

what assessment “is”…

Independent and successful readers continually monitor and evaluate their learning and

performance

Metacognition and comprehension monitoring are learned…

Who teaches them? How are they taught?

(Black & Wiliam, 1998; Flavell, 1978; Markman, 1979;

Pressley & Afflerbach, 1995)

Do classroom assessments help students become

independent in assessment?

Are students “insiders” or “outsiders” to the culture of assessment?

Is assessment done to or for students?Is assessment done with and by

students?

Without the ability to self-assess reading

(or any other learning), how will students be independent?

It is legitimate to further characterize the broadpoint appearance as a major archeological horizon marker for the eastern seaboard. In the terms of Willey and Phillips, a horizon is “a primarily spatial continuity represented by cultural traits and assemblages whose nature and mode of occurrence permit the assumption of a broad and rapid spread.” That a quick expansion of the broadpoint-using peoples took place is indicated by the narrow range of available radiocarbon dates, along with a correspondingly wide areal distribution of components. Once established, the broadpoint horizon developed as a “whole cultural pattern or tradition” in its own right by persisting and evolving over an expansive region for 500 to 1000 years.

Broadpoint is a type of arrowhead

How are we metacognitive?

Self-assessment happens when students have metacognitive strategies to use, the mindset that they should use them,

and motivation to do so.

Self-assessment is strategic, so our successes with strategy instruction can inform our teaching of self-assessment strategies.

Explain, model, think-aloud and practice.

With reading…

(Pressley, 2005; Duffy, 1993)

4. Balance the assessment of 4. Balance the assessment of cognitivecognitive and and

affectiveaffective factors related to learning factors related to learning Representative State Level Intended Learning Outcomes

Beginning in kindergarten and by the end of second grade students will be able to:

1. Demonstrate a positive learning attitude.

2. Develop social skills and ethical responsibility.

3. Demonstrate responsible emotional and cognitive behaviors.

4. Understand and use basic concepts and skills.

The assessment of cognitive and affective factors

that contribute to and influence achievement

Most state and local standards address affective goals and outcomes in their

curriculum goals and standards and learning benchmark descriptions

Self-concept as a reader

Are these assessed? To what end?

(Chapman & Tunmer, 1995; Guthrie & Wigfield, 1997; CORI Projects; McKenna & Kear, 1990)

Self-esteem as a reader

Motivation to read

When we think of our teaching successes, do we think of students who

scored well on tests?

Or do we think of students who went from reluctant readers to enthusiastic

readers?

Do we think of students who evolved from easily discouraged readers to readers whose motivation helped them

persevere through challenges?

Do we remember students who avoided reading at all costs evolving into

students who learned to love reading?

5. Balance the assessment of 5. Balance the assessment of what students learn from readingwhat students learn from reading and and how students use this knowledgehow students use this knowledge

The 2009 Framework of the National Assessment of Educational Progress derives from expert consensus and conceptualizes reading as:

Reading is an active and complex process that involves• Understanding written text• Developing and interpreting meaning • Using meaning as appropriate to type of text, purpose and situation

(National Center for Education Statistics, 2007)

“Using meaning as appropriate to type of

text, purpose and situation…”

In the 21st century, can our students

•Identify claims and evidence in texts they read?

•Conduct critical evaluations of texts?

•Make judgments of texts and authors?

•Apply what they learn from reading to identify and solve problems?

•Synthesize information from text and non-text sources?

(Coiro & Dobler, 2007; Gee, 2000)

• Meaning construction andcritical evaluation is assessed with …

- Multiple choice items

- Brief constructed responses

- Extended constructed responses

- Performances and judgments

Increasing

complex

performance

and

assessment

(Pellegrino, Chudowsky & Glaser, 2001)

The application of what is learned from reading

Dolgeville, NY

This Factory's Bats Are Going, Going, Gone; As Home of McGwire's 'Big Stick,' Struggling Upstate Town Gets a Lift

And that is the best economic news in a long, long time for Dolgeville, tucked in the hills north of the Mohawk River and 23 miles east of Utica. The century-old clapboard houses and the flower beds here are well tended, but behind the trim exterior, the village is fading, its population and job base declining steadily, like so many places upstate that have lost the manufacturing that once fed them.

Jamestown lost its furniture makers and the Crescent wrench works, Troy lost the Arrow shirt collar factory, and Schenectady lost the American Locomotive Company and most of General Electric. Dolgeville's soon-to-be relic is the Daniel Green shoe factory, a great stone castle along East Canada Creek that once employed nearly 1,000 people, but has seen a series of layoffs. Most of the jobs have moved overseas, and the factory will cease production in June.

RICHARD PEREZ-PENA NY Times April 25, 1999

6. Balance the demands for accountability 6. Balance the demands for accountability with with

professional development opportunities professional development opportunities

that help teachers develop that help teachers develop expertise in assessmentexpertise in assessment

How do teachers become expert at assessment?

How can teachers attain reliability in their

classroom-based assessments?

Opportunity costs of different approaches

to reading assessment

(Darling-Hammond, 1997; Stiggins & Conklin, 1992)

How are teachers supported

in the extremely challenging work of becoming expert

at classroom assessment?

Professional development in classroom assessment is central to school success. For example, a teacher’s checklist for asking appropriate questions can be used to sharpen daily assessment routines:

_____ I ask questions that are appropriately phrased and understood by students._____ I ask questions that are at an appropriate level for the materials being covered. _____ I ask questions that require students to think at various intellectual levels._____ My questions follow a logical sequence._____ Student responses are used to guide my next questions._____ My questions are consistent with the intended goals or objectives of the lesson._____ I ask questions that assess student understanding. _____ I ask processing questions if a student’s answer is incomplete or superficial. _____ I encourage students to answer difficult questions by providing cues or rephrasing._____ I avoid closed-ended questions that restrict students’ demonstration of learning.

Slack, J. Questioning strategies to improve student thinking and comprehension. Southeast Educational Development Laboratory. Reprinted with permission.

Towards expertise in classroom assessment

Vertical and horizontal reading assessment

In for the long term

Avoiding faddism

Opportunity costs

Connect teachable momentswith tests

(Afflerbach, 2007; Johnston, 1991)

ConclusionsConclusions

Effective reading assessment is conceptualized and practiced with clear understanding of

how teachable moments connect with tests

The extreme focus on high stakes tests, AYP and means to meet it skews school assessment towards summary statements, and away from

valuable formative assessment

ConclusionsConclusions

Such formative assessment is that which informs our teachable moments—

Providing information that describesstudents’ zones of proximal development--and where and what and how our instruction

should address students’ needs

Reading assessment is influenced by powers great and small--and this often creates imbalances for teachers and students

ConclusionsConclusions

Our addressing imbalances should be informed by our goals

for (and with) reading assessment

To help us understand student learning and

to help us guide student learning

ConclusionsConclusions

This duality of reading assessment allows us to promote:

-Strategic, skillful reading

-Lifelong, enthusiastic readers

-The Matthew effect

Cognitive and affective and social development

Independence and self-assessment

Lastly…

Achieving balance in our reading assessment programs, materials and procedures promotes accountability

from teachable moment to test