CONSTRUCTING PERFORMANCE-BASED ASSESSMENTDr. Carlo Magno

ADVANCE ORGANIZER

Performance based assessment Authentic assessment Portfolio assessment Assessment “for” learning

OBJECTIVES

1. Distinguish performance-based assessment with the traditional paper and pencil tests.

2. Construct tasks that are performance based.

Design a rubric to assess a performance based task

TERMS

Authentic assessment Direct assessment Alternative assessment Performance testing Performance assessment Changes are taking place in assessment

METHOD

Assessment should measure what is really important in the curriculum.

Assessment should look more like instructional activities than like tests.

Educational assessment should approximate the learning tasks of interest, so that, when students practice for the assessment, some useful learning takes place.

WHAT IS PERFORMANCE ASSESSMENT? Testing that requires a student to create an

answer or a product that demonstrates his/her knowledge or skills (Rudner & Boston, 1991).

FEATURES OF PERFORMANCE ASSESSMENT

Intended to assess what it is that students know and can do with the emphasis on doing.

Have a high degree of realism about them. Involve: (a) activities for which there is no

correct answer, (b) assessing groups rather than individuals, (c) testing that would continue over an extended period of time, (d) self-evaluation of performances.

Likely use open-ended tasks aimed at assessing higher level cognitive skills.

PUSH ON PERFORMANCE ASSESSMENT

Bring testing methods more in line with instruction.

Assessment should approximate closely what it is students should know and be able to do.

EMPHASIS OF PERFORMANCE ASSESSMENT

Should assess higher level cognitive skills rather than narrow and lower level discreet skills.

Direct measures of skills of interest.

PROBLEMS OF IMPLEMENTATION

Experience in constructing and using performance tests is lacking among many educational measurement specialist.

Performance tests take more time to construct, administer, and score than objective tests.

Standardization, reliability, and validity will be difficult to apply

GUIDELINES TO ESTABLISH VALIDITY OF PERFORMANCE TESTS: Consequences: Do teachers teach

differently? What do students learn? Fairness: There is no guarantee Transfer and generalizability: Extent of small

tasks generalized to larger tasks. Cognitive complexity: no guarantee that high

level cognitive skills are tapped. Content quality: limited sampling of content

is possible

GUIDELINES TO ESTABLISH VALIDITY OF PERFORMANCE TESTS: Content coverage: number of tasks chosen is

small. Meaningfulness: evidence that the

assessment is meaningful for students Cost and efficiency: time consuming and

costly to construct, administer, and score thn objective forms of assessment.

TERMS

Performance-and-product

The emphasis is on the students’ ability to perform tasks by producing their own work with their knowledge and skills.

Alternative assessment Method that differs from conventional paper-and-pencil tests, most particularly objective tests.

Authentic assessment Direct examination of student’s ability to use knowledge to perform a task that is like what is encountered in real life or in the real world.

CHARACTERISTICS OF PERFORMANCE-BASED ASSESSMENT Students perform, create, construct, produce, or do

something. Deep understanding and/or reasoning skills are needed

and assessed. Involves sustained work, often days and weeks. Calls on students to explain, justify, and defend. Performance is directly observable. Involves engaging in ideas of importance and substance. Relies on trained assessor’s judgments for scoring Multiple criteria and standards are prespecified and

public There is no single correct answer. If authentic, the performance is grounded in real world

contexts and constraints.

LEARNING TARGETS

Skills Communication and presentation skills Ex: Speaking1. Speaking clearly, expressively, and audibly

a. Using voice expressivelyb. Speaking articulately and pronouncing words

correctlyc. Using appropriate vocal volume

2. Presenting ideas with appropriate introduction, development, and conclusion

1. Presenting ideas in an effective order2. Providing a clear focus on the central idea3. Providing signal words, internal summaries, and

transitions

3. Developing ideas using appropriate support materials

a) Being clear and using reasoning processesb) Clarifying, illustrating, exemplifying, and

documenting ideas

4. Using nonverbal cuesa. Using eye contact b. Using appropriate facial expressions, gestures, and

body movement

5. Selecting language to a special purposea. Using language and conventions appropriate for the

audience

Psychomotor skills Fine motor: cutting papers with scissors, drawing

a line tracing, penmanship, coloring drawing, connecting dots

Gross motor: Walking, jumping, balancing, throwing, skipping, kicking

Complex: Perform a swing golf, operate a computer, drive a car, operate a microscope

Visual: Copying, finding letters, finding embedded figures, identifying shapes, discrimination

Verbal and auditory: identify and discriminate sounds, imitate sounds, pronounce carefully, blend vowels

Products Write promotional materials Report on a foreign country Playing a new song

VARIATION OF AUTHENTICITY

Relatively authentic

Somewhat authentic

Authentic

Indicate which parts of a garden design are accurate

Design a garden Create a garden

Write a paper on zoning

Write a proposal to change fictitious zoning laws

Write a proposal to present to city council to change zoning laws

Explain what would you teach to students learning basketball

Show how to perform basketball skills in practice

Play a basketball game.

CONSTRUCTING PERFORMANCE BASED TASKS

1. Identify the performance task in which students will be engaged

2. Develop descriptions of the task and the context in which the performance is to be conducted.

3. Write the specific question, prompt, or problem that the student will receive.

• Structure: Individual or group?• Content: Specific or integrated?• Complexity: Restricted or extended?

COMPLEXITY OF TASK

Restricted-type task Narrowly defined and require brief responses Task is structured and specific Ex:

Construct a bar graph from data provided Demonstrate a shorter conversation in French about

what is on a menu Read an article from the newspaper and answer

questions Flip a coin ten times. Predict what the next ten flips of

the coin will be, and explain why. Listen to the evening news on television and explain if

you believe the stories are biased. Construct a circle, square, and triangle from provided

materials that have the same circumference.

Extended-type task Complex, elaborate, and time-consuming. Often include collaborative work with small group

of students. Requires the use of a variety of information Examples:

Design a playhouse and estimate cost of materials and labor

Plan a trip to another country: Include the budget and itinerary, and justify why you want to visit certain places

Conduct a historical reenactment (e. g. impeachment trial of ERAP)

Diagnose and repair a car problem Design an advertising campaign for a new or existing

product

IDENTIFYING PERFORMANCE TASK DESCRIPTION

Prepare a task description Listing of specifications to ensure that

essential if criteria are met Includes the ff.:

Content and skill targets to be assessed Description of student activities

Group or individual Help allowed

Resources needed Teacher role Administrative process Scoring procedures

PERFORMANCE-BASED TASK QUESTION PROMPT

Task prompts and questions will be based on the task descriptions.

Clearly identifies the outcomes, outlines what the students are encourage dot do, explains criteria for judgment.

EXAMPLE OF A TASK PROMPT:

CHARACTERISTICS OF TASKS

1. Should integrate the most essential aspects of the content being assessed with the most essential skills.

2. Should be authentic Realistic Require judgment and innovation Ask the student to do the subject Replicates or stimulates Assess the students ability to efficiently and

effectively use a repertoire of knowledge and skill to negotiate a complex task

Allows opportunities to rehearse, practice, consult resources, and get feedback and refine performances and products.

3. Structure the task to assess multiple learning targets

4. Structure the task so that you can help students succeed.

5. Think through what students will do to be sure that the task is feasible

6. The task should allow for multiple solutions7. The task should be clear8. The task should be challenging and stimulating

to students9. Include explicitly stated scoring criteria as part

of the task10. Include constraints in completing the task

PERFORMANCE CRITERIA

What you look for in student responses to evaluate their progress toward meeting the learning target.

Dimensions of traits in performance that are used to illustrate understanding, reasoning, and proficiency.

Start with identifying the most important dimensions of the performance

What distinguishes an adequate to an inadequate demonstration of the target?

QUESTIONS TO ASK:

What are the attributes of good writing, or good scientific thinking, or good collaborative group process, of effective oral presentation? More generally, by what qualities or features will I know whether students have produced an excellent response to my assessment task?

What do I expect to see if this task is done excellently, acceptably, or poorly?

Do I have samples or models of student work, from my class or other sources, that exemplify some of the criteria I might use in judging this task?

What criteria for this or similar task exist in my state curriculum framework, my state assessment program, my district curriculum guides, my school assessment program?

What dimensions might I adapt from work done by natural curriculum councils, by other teachers?

EXAMPLE OF CRITERIA

Learning target: Students will be able to write a persuasive paper

to encourage the reader to accept a specific course of action or point of view.

Criteria: Appropriateness of language for the audience Plausibility and relevance of supporting

arguments. Level of detail presented Evidence of creative, innovative thinking Clarity of expression Organization of ideas

RATING SCALES

Indicate the degree to which a particular dimension is present.

Three kinds: Numerical, qualitative, combined qualitative/quantitative

Numerical Scale Numbers of a continuum to indicate different

level of proficiency in terms of frequencyor quality

Example:Complete Understanding 5 4 3 2 1 No understanding

No organization 5 4 3 2 1 Clear organization

Emergent reader 5 4 3 2 1 Fluent reader

Qualitative scale Uses verbal descriptions to indicate student

performance. Provides a way to check the whether each

dimension was evidenced. Type A: Indicate different gradations of the dimension Type B: Checklist

Example of Type A: Minimal, partial, complete Never, seldom, occasionally, frequently, always Consistent, sporadically, rarely None, some, complete Novice, intermediate, advance, superior Inadequate, needs improvement, good excellent Excellent, proficient, needs improvement Absent, developing, adequate, fully developed Limited, partial, thorough Emerging, developing, achieving Not there yet, shows growth, proficient Excellent, good, fair, poor

Example of Type A: Checklist

Holistic scale The category of the scale contains several

criteria, yielding a single score that gives an overall impression or rating

Examplelevel 4: Sophisticated understanding of text indicated with constructed meaninglevel 3: Solid understanding of text indicated with some constructed meaninglevel 2: Partial understanding of text indicated with tenuous constructed meaninglevel 1: superficial understanding of text with little or no constructed meaning

EXAMPLE HOLISTIC SCALE

Analytic Scale One in which each criterion receives a separate

score.

Example

Criteria Outstanding5 4

Competent 3

Marginal2 1

Creative ideas

Logical organization

Relevance of detail

Variety in words and sentences

Vivid images

RUBRICS

When scoring criteria are combined with a rating scale, a complete scoring guideline is produced or rubric.

A scoring guide that uses criteria to differentiate between levels of student proficiency.

EXAMPLE OF A RUBRIC

Rubrics should answer the following questions: By what criteria should performance be judged? Where should we look and what should we look

for to judge performance success? What does the range in the performance quality

look like? How do we determine validity, reliability, and

fairly what scores should be given and what that score means?

How should the different levels of quality be described and distinguished from one another?

GUIDELINES IN CREATING A RUBRIC

1. Be sure the criteria focus on important aspects of the performance

2. Match the type of rating with the purpose of the assessment

3. The descriptions of the criteria should be directly observable

4. The criteria should be written so that students, parents, and others understand them.

5. The characteristics and traits used in the scale should be clearly and specifically defined.

6. Take appropriate steps to minimize scoring frame

PORTFOLIO ASSESSMENT: EXPLORATION

Have you ever done a portfolio? Tell me about this experience. Did you enjoy

it? What elements did you include in your

portfolio? Are the materials placed in the portfolio

required?

WHAT ARE PORTFOLIOS?

Purposeful, systematic process of collecting and evaluating student products to document progress toward the attainment of learning targets or show evidence that a learning target has been achieved.

Includes student participation in the selection and student self-reflection.

“A collection of artifacts accompanied by a reflective narrative that not only helps the learner to understand and extend learning, but invites the reader of the portfolio to gain insight about learning and the learner (Porter & Cleland, 1995)

CHARACTERISTICS OF PORTFOLIO ASSESSMENT

Clearly defined purpose and learning targets Systematic and organized collection of

student products Preestablished guidelines for what will be

included Student selection of some works that will be

included Student self-reflection and self-evaluation Progress documented with specific products

and/or evaluations Portfolio conferences between students and

teachers

A PORTFOLIO IS:

Purposeful Systematic and well-organized Prestablished guidelines are set-up Students are engaged in the selection of

some materials Clear and well-specified scoring criteria

PURPOSE OF PORTFOLIO

Showcase portfolio: Selection of best works. Student chooses work, profile are accomplishments and individual profile emerges.

Documentation portfolio: Like a scrapbook of information and examples. Inlcudes observations, tests, checklists, and rating scales.

Evaluation portfolio: More standardized. Assess student learning with self-reflection. Examples are selected by teachers and predetermined.

ADVANTAGES OF PORTFOLIO Students are actively involved in self-evaluation and

self-reflection Involves collaborative assessment Ongoing process where students demonstrate

performance, evaluate , revise , and produce quality work.

Focus on self-improvement rather than comparison with others

Students become more engaged in learning because both instruction and assessment shift from teacher controlled to mix of internal and external control.

Products help teachers diagnose learning difficulties clarify reasons for evaluation Flexible

DISADVATNTAGES

Scoring difficulties may lead to low reliability Teacher training needed Time-consuming to develop criteria, score

and meet students Students may not make good selections of

which of which material to include Sampling of student products may lead to

weak generalization Parents find the portfolio difficult to

underdstand

STEPS IN PLANNING AND IMPLEMENTING PORTFOLIO ASSESSMENT

1. Determine the the purpose2. Identify physical structure3. Determine sources of content4. Determine sources of content5. Determine student reflective guidelines and scoring

criteria6. Review with students7. Portfolio content supplied by teacher and/or student8. Student self-evaluation of contents9. Teacher evaluation of content and student self-

evaluation10. Student-teacher conference11. Portfolios returned to students for school

PURPOSE

Based on specific learning targets Ideal for assessing product, skill, and reasoning

targetsUses: Showcase portfolio-to illustrate what students

are capable of doing Evaluation of portfolio-standardization of what to

include For parents-what will make sense to parents

“Provide specific attention to purpose and corresponding implications when implementing a portfolio.”

PHYSICAL STRUCTURE

What will it look like? How large will the portfolios be? Where are they stored so that students can

easily access them? Will it be in folders or scrap books? How will the works be arranged in the

portfolio? What materials are needed to separate the

works in the portfolio?

SOURCES OF CONTENT

Work samples Student and teacher evaluationsGuidelines: Select categories that will allow you to meet

the pupose of the portfolio. Show improvement in the portfolio Provide feedback on the students on the

procedures they are putting together Provide indicator system

SELF-REFLECTIVE GUIDELINES AND SCORING

Establish guidelines for student self-reflection and the scoring criteria

Scoring guidelines are explained to the students before they begin instruction

IMPLEMENTING PORTFOLIO ASSESSMENT

Review with students: Explain to students what is involved in doing a portfolio.

Begin with learning targets Show examples Give opportunities to ask questions Provide just enough structure so that they

can get started without telling them exactly what to do.

Selection of content will depend on the age and previos experience of students

Students and teachers decide together what to include with nonrestrictive guidelines

SOME ORGANIZATION

Include table of contents Brief description of activities Date produced Date submitted Date evaluated

STUDENT SELF-EVALUATIONS

Reflective and self-evaluation activities need to be taught.

Some guide questions for students: Can you tell me what you did? What did you like best abut this sample of your

writing? What will you do next?

Self-reflective questions: What did you learn from writing this piece? What would you have done differently if you had

more time? What are your greatest strengths and weaknesses in

this sample? What would you do differently if you did this over?

PEER EVALUATIONS

Analysis and constructive, supportive criticism of strategies, styles, and other concrete aspects of the product.

Can include comments or a review by parents

Teacher evaluations: Checklist of content Portfolio structure evaluation: selection of

samples, thoroughness, appearance, self-reflection, and organization.

Evaluation of individual entries: use rubrics Evaluation of entire content: use rubrics

STUDENT-TEACHER CONFERENCES

Conference is conducted with sudents before returning the portfolio

Scheduled throughout the school year; some have it monthly

Clarify purposes and procedure with students, answer questions and establish trust

Give guidelines to prepare for each conference Allow the students to do most of the talking Have students compare your reflections with

theirs Weaknesses and areas for improvement need to

be communicated –show them what is possible for progress

STUDENT-TEACHER CONFERENCES

At the end of the conference there is an action plan for the future

Limit the conference to no more than 10 minutes

Students are encouraged to take notes Focus on one or two major areas of each

conference-helps to have a thoughtful discussion

Wrapping up on assessing student learning

Effect of Previous Practices: rank students on achievement by graduation

New Expectation: Assure competence in Math, Reading, Writing, etc.

Implications?Assessment and

grading procedures had the effect of helping some students succeed now must serve to help all students succeed.

MISTAKEN BELIEFS ABOUT HOW TO USE ASSESSMENT TO SUPPORT SCHOOL IMPROVEMENT:  1. High-stakes tests are good

for all students because they motivate learning

2. If I threaten to fail you, it will cause you to try harder

3. If a little intimidation doesn’t work, use a lot of intimidation

4. The way to maximize learning is to maximize anxiety

5. It is the adults who use assessment results to make the most important instructional decision.

PROFOUND MISTAKETeachers and leaders don’t need

to understand sound assessment practices – the testing people will take care of us.  COUNTER BELIEF

They do need to understand sound assessment practices.

ASSESSMENT LEGACY

 1. Assessment has been far more a matter of

compliance than of teaching and learning2. Disregard of the information needs students and

teachers who make the most frequent and highest impact decisions

3. Assessment that drive as many students to give up in hopelessness as they spur to more learning

4. And we fail to provide practitioners with the assessment understandings needed to help