ASSESSMENT LITERACY:ASSESSMENT LITERACY:A BONA FIDE “MAGICA BONA FIDE “MAGIC

BULLET” FOR EDUCATIONBULLET” FOR EDUCATION

W. James PophamW. James PophamUniversity of California, Los AngelesUniversity of California, Los Angeles

California Educational Research California Educational Research Association 88Association 88thth Annual Conference Annual Conference

San FranciscoSan Francisco

November 18-19, 2009November 18-19, 2009

THE DEFINITION OF THE DEFINITION OF A PROFESSIONA PROFESSION

““The body of persons engaged The body of persons engaged in an occupation, e.g., the in an occupation, e.g., the medical profession or the medical profession or the education profession”education profession”

AN UNPLEASANT TRUTHAN UNPLEASANT TRUTH

Any profession that allows the Any profession that allows the competence of its members to be competence of its members to be determined by procedures over which it determined by procedures over which it has little or no influence is a profession has little or no influence is a profession fully deserving of the low esteem with fully deserving of the low esteem with which, over time, its members are sure which, over time, its members are sure to be held.to be held.

UNPLEASANT TRUTH NO. 2UNPLEASANT TRUTH NO. 2

Any profession that allows the Any profession that allows the competence of its members to be competence of its members to be determined by procedures over which it determined by procedures over which it has little or no influence is a profession has little or no influence is a profession destined to underachieve its inherent destined to underachieve its inherent potential.potential.

UNPLEASANT TRUTH NO. 3UNPLEASANT TRUTH NO. 3

The education profession allows the The education profession allows the competence of our members to be competence of our members to be determined by procedures over which determined by procedures over which we have little or no influence, namely, we have little or no influence, namely, procedures rooted in students’ procedures rooted in students’ performances on educational performances on educational accountability tests.accountability tests.

WHY HAVE WE BEEN SO SILLY?WHY HAVE WE BEEN SO SILLY?

The chief reason that the education The chief reason that the education profession has allowed the competence profession has allowed the competence of its members to be determined by of its members to be determined by others is because of flat-out, pervasive, others is because of flat-out, pervasive, profound, Sunday-go-to-meeting:profound, Sunday-go-to-meeting:

ASSESSMENT ILLITERACYASSESSMENT ILLITERACY

WHAT IS ASSESSMENT WHAT IS ASSESSMENT LITERACYLITERACY??

Assessment literacy represents the Assessment literacy represents the degree to which someone understands degree to which someone understands the most fundamental measurement the most fundamental measurement concepts that are focally relevant to the concepts that are focally relevant to the way we operate and evaluate our way we operate and evaluate our schools. Assessment literacy does not schools. Assessment literacy does not imply that someone must be able to imply that someone must be able to procedurally implement all of these procedurally implement all of these concepts—only to understand them.concepts—only to understand them.

A NEW BOOK BY JIM WRITTEN FOR A NEW BOOK BY JIM WRITTEN FOR CORWIN PRESSCORWIN PRESS

When released? Spring/Summer, 2010When released? Spring/Summer, 2010

Possible titles? Possible titles?

A. A. What School Leaders Must Know about What School Leaders Must Know about Educational AssessmentEducational Assessment

B. B. Twenty Crucial Understandings about Twenty Crucial Understandings about Educational MeasurementEducational Measurement

C. C. Seduction in the SandSeduction in the Sand

D. None of the above?D. None of the above?

MY TWO “MOST CRUCIAL” MY TWO “MOST CRUCIAL” ASSESSMENT UNDERSTANDINGSASSESSMENT UNDERSTANDINGS

Why it is that we are evaluating our Why it is that we are evaluating our schools with the wrong accountability schools with the wrong accountability teststests

A recognition that teachers’ use of the A recognition that teachers’ use of the formative-assessment process can formative-assessment process can have a profoundly beneficial impact on have a profoundly beneficial impact on students’ learningstudents’ learning

TWO DOMINANT KINDS OF TWO DOMINANT KINDS OF ACCOUNTABILITY TESTS IN THE U.S.ACCOUNTABILITY TESTS IN THE U.S.

““Augmented” versions of traditional, Augmented” versions of traditional, nationally standardized achievement nationally standardized achievement tests.tests.

Standards-based tests ostensibly Standards-based tests ostensibly assessing students’ mastery of a assessing students’ mastery of a state’s officially approved content state’s officially approved content standards.standards.

BOTH OF THESE TYPES OF BOTH OF THESE TYPES OF ACCOUNTABILITY TESTSACCOUNTABILITY TESTS

Tend to measure what students bring Tend to measure what students bring to school rather than how effectively to school rather than how effectively those students have been taught once those students have been taught once they arrive.they arrive.

WHERE IS THE VALIDITY EVIDENCE?WHERE IS THE VALIDITY EVIDENCE?WHERE ARE THE VALIDITY WHERE ARE THE VALIDITY

ARGUMENTS?ARGUMENTS?

If the inference at issue is whether If the inference at issue is whether students’ scores on accountability students’ scores on accountability tests accurately reflect the success tests accurately reflect the success with which those students have been with which those students have been taught, then where is the evidence that taught, then where is the evidence that today’s accountability tests can today’s accountability tests can accomplish this measurement mission accomplish this measurement mission satisfactorily?satisfactorily?

FORMATIVE ASSESSMENTFORMATIVE ASSESSMENT

Formative assessment is a planned Formative assessment is a planned process in which assessment-elicited process in which assessment-elicited evidence of students’ status is used by evidence of students’ status is used by teachers to adjust their ongoing teachers to adjust their ongoing instructional procedures or by students instructional procedures or by students to adjust their current learning-tactics.to adjust their current learning-tactics.

(Me, 2008)(Me, 2008)

THE EVIDENTIARY BASISTHE EVIDENTIARY BASIS

In a research review based on 250 empirical In a research review based on 250 empirical studies of classroom assessment that had studies of classroom assessment that had been drawn from more than 680 published been drawn from more than 680 published investigations, Paul Black and Dylan Wiliam investigations, Paul Black and Dylan Wiliam concluded:concluded:

“ “The research reported here shows The research reported here shows conclusively that formative assessment does conclusively that formative assessment does improve learning.” (improve learning.” (Assessment in Assessment in Education, Education, 1998)1998)

CLINCHERS—FOR MECLINCHERS—FOR ME

Two Other Quotes from the Research Review:Two Other Quotes from the Research Review:

The student gains in learning triggered by The student gains in learning triggered by formative assessment were “amongst the formative assessment were “amongst the largest ever reported for educational largest ever reported for educational interventions.” interventions.”

““Significant gains can be achieved by many Significant gains can be achieved by many different routes, and initiatives here are not different routes, and initiatives here are not likely to fail through neglect of delicate and likely to fail through neglect of delicate and subtle features.”subtle features.”

LET’S LOOP BACK 40+ YEARSLET’S LOOP BACK 40+ YEARS

TO THE 1960s, B.D.TO THE 1960s, B.D.

OPTIONAL MEANINGS OFOPTIONAL MEANINGS OF B.D.B.D.

A. An era of benighted distressA. An era of benighted distress

B. Before DylanB. Before Dylan

C. Both of the aboveC. Both of the above

A RAPID REVIEWA RAPID REVIEW

1.1. For both professional survival and to For both professional survival and to achieve greater good, achieve greater good, all all educators educators must become assessment literate.must become assessment literate.

2.2. One focus for assessment-literate One focus for assessment-literate educators should be to improve the educators should be to improve the accountability tests used to judge us.accountability tests used to judge us.

3.3. A second task for assessment-literate A second task for assessment-literate educators is to foster wider use of the educators is to foster wider use of the formative-assessment process.formative-assessment process.

WHAT CAN WHAT CAN YOU YOU DO?DO?

1.1. You can personally enhance your own You can personally enhance your own assessment literacy.assessment literacy.

2.2. You can work solo, with colleagues, You can work solo, with colleagues, or with associations to demand more or with associations to demand more appropriate accountability tests.appropriate accountability tests.

3.3. You can, with word and deed, You can, with word and deed, encourage more teachers, schools, encourage more teachers, schools, and districts to use the formative-and districts to use the formative-assessment process.assessment process.

YOU REALLY SHOULD!YOU REALLY SHOULD!