EDU 385Educational Assessment in the

Classroom

Session 11Performance Assessment

Content Objectives

•Understand the advantages and limitations of Performance Assessments

•Understand how to write, observe, record, and evaluate a performance task

Language Objectives1. Describe how performance assessments differ from paper-and-pencil testing

2. Identify the strengths and limitations of performance assessments

3. Write intended performance outcmes for a performance assessment

4. Distinguish between restricted and extended performance assessment

5. Describe the general procedure for making a performance assessment

6. Describe the use of a student project for performance assessment

7. Prepare a plan for arranging, observing, recording, and evaluating a performance task

8. Construct a holistic scoring rubric for some performance outcome

Academic Vocabulary

•Performance Assessment

•Restricted-Response Item

•Extended-Response Item

•Scoring Rubric

Bell Work

Performance Assessments• Use when paper and pencil tests are inadequate to

assess students performance skills

• When more direct assessments are needed to assess actual performance

• Science - laboratory skills

• English and Foreign-Language - Communication skills

• Mathematics - Problem Solving skills

• Other Performance Subjects: art, music, industrial arts. business, economics, home economics, physical education

Components of Performance SkillsKnowledge Component

Skills Component

Affective Component

Example:

Assessing Writing Ability: concerned about vocabulary, grammar, as well as interests and attitudes that support good writing

Assessing Laboratory Skills: names and uses of equipment and procedures to follow; as well as attitude of care and safety in handling equipment and making accurate measurements

Performance Skills

Writing Performance Objectives for Skills and Products

1. State each general objective so that it clearly describes the skill or product to be assessed

2. List specific performance outcomes for each objective that are most relevant to a successful performance or a satisfactory product

3. List enough specific performance outcomes to clarify what is meant by an effective performance

4. State the specific performance outcomes in terms of observable dimensions of the skill or product

5. State the specific performance outcomes so that they are clear to students

Restricted & Extended Performance

• It may be desirable to measure specific skills (i.e., Restricted-Performance Tasks) before putting them in a more complex Performance

• Restricted-Performance can help diagnose problems

• (e.g., student having difficulty using lab equipment)

• Limiting scope of the task, makes it easier to focus observation and judge response

• Extended Performance Tasks are less structured and broader in scope

• Give students greater freedom and opportunity for self-assessment and self-improvement

• Helps develop students’ independent learning skills

Examples

Restricted Performance Tasks

-Write a one page report describing a field trip-Give a one-minute speech on a given topic-Read aloud a brief selection of poetry-Construct a graph form a given set of data-Demonstrate how to use a measuring instrument

Extended Performance Tasks

-Design and conduct an experiment-Design and build a wood or metal productt-Write a short story-Repair a malfunctioning motor-Paint a picture-Demonstrate a physical or musical performance

Steps in Preparing Performance Assessments

Example - - Demonstrates Skill in Oral Reporting1. Specifying the performance outcomes2. Selecting the focus of the assessment (Procedure, product, or both)3. Selecting an appropriate degree of realism4. Selecting the performance situation5. Selecting the method of observing, recording, and scoring6. Speaks at an appropriate rate7. Presents ideas in an organized manner 8. Uses appropriate language9. Maintains interest of the group

Example - - or realistic set of Extended Performance Tasks-Designs and conducts an experiment-Writes and accurate account of the study-States valid conclusions-Writes a critique of the procedure and findings-Presents and defends the study in class

Specifying the performance outcomes

Restricted Performance commonly use verbs such as: identify, construct, and demonstrate

Specifications of performance outcomes typically include a job or task analysis

Demonstrate Skill in Oral Reporting1. Stands in natural manner2. Maintains good eye contact3. Uses appropriate facial expressions3. Uses gestures effectively4.

Action Verbs Illustrative Instructional ObjectivesIDENTIFY: Selects the correctobjects, parts of the object, procedure, or property (typicalverbs: identify, locate, select,touch, pick up, mark, describe)

-Select the proper tool,-Identify the parts of a typewriter,-Choose correct laboratory equipment,-Select most relevant statistical procedure,-Locate an automobile malfunction,-Identify a musical selection,-Identify experimental equipment needed-Identify specimen under the microscope

CONSTRUCT: Make a product to fit a given set of specifications(typical verbs: construct, assemble, build, design, draw, make, prepare)

-Draw a diagram for an electrical circuit,-Design a pattern for making a dress,-Assemble equipment for experimental study,-Prepare a graph,-Construct a weather map,-Prepare an experimental design

DEMONSTRATE: Performs aset of operations or procedures(typical verbs: demonstrate, drive, measure, operate, perform, repair, set up)

-Drive an automobile-Measure the volume of a liquid-Operate a filmstrip projector-Perform a modern dance step-Repair a malfunctioning TV set-Set up laboratory equipment-Demonstrate taking a patient’s temperature-Demonstrate the procedure for turning an automobile

Typical Action Verbs and Illustrative Instructional Objectives for Restricted Performance Outcomes

Selecting the focus of the assessment

• Assessing the Procedure

• Type of performance that doesn’t result in a product (speaking, reading aloud, physical skills, musical performance)

• Procedures (early part of performance) and products later

• (e.g., typing - proper use of touch system later neatness, accuracy, and speed)

• No product evaluation is feasible

• Procedure is orderly and directly observable

• Correct procedure is crucial to later success

• Analysis of procedural steps improves a product

• Assessing the Product

Selecting the focus of the assessment• Assessing the Product

• When Procedures are sufficiently mastered - Assess Product

• Use specific criteria to used to determine the quality of the product

• Performance Assessment should focus on product when

• Different procedures can result in equally good product

• Procedure not observable (take-home work)

• Procedural steps have been mastered

• Product qualities can be clearly identified and judged

Selecting an appropriate degree of realism

• Strive for Performance Assessments that approximate real-world conditions

• As much realism as possible

• Factors to consider

• Nature of the instructional objectives (does paper/pencil give you the information you need?)

• Sequence of instruction (early in unit paper/pencil may be enough and later use performance tests

• Constraints as time, resources, equipment may limit the use of performance assessments

Degree of Realism

Have students solve this story problem: “If you

bought a toy for 69 cents and gave the clerk $1.00, Low Realismhow much change would you get back?”

Demonstrate the problem situation with actualmoney and ask each student to judge whether the change is correct

Pair off the students, give them actual money,and have them role-play the problem situation

Set up a mock toy store and have each student High Realismdemonstrate the “purchase” of a toy with actual money

Selecting the performance situationPaper-and-Pencil Performance-Emphasis on Application of knowledge and skill in simulated setting

-Action verb construct frequently used (construct weather map, bar graph, diagram an electrical circuit, floor plan, item of clothing

Identification Test-Biology: identify specimens placed at various stations in classroom

-Math: identify correct problem-solving procedures

-English: identify best expression to be used in writing

Structured Performance Test-A structured performance test under standard, controlled conditions

-Examples: making a prescribed measurement, adjusting a microscope, following a safety procedure

-Set performance standards (minimal level of acceptable performance)

-Specify clear and definite criteria for judging procedure or product

-To improve reliability ask students to use descriptive terms (not subjective)

Some Common Standards for Judging Performance

TYPE Examples

RateSolve ten addition problems in two minutesType 40 words per minute

ErrorNo more than two errors per typed pageCount to 20 in Spanish without error

TimeSet up a laboratory equipment in five minutesLocate an equipment malfunction in three minutes

PrecisionMeasure a line within one-eighth of an inchRead a thermometer within two-tenth of a degree

QuantityComplete 20 laboratory experiementsLocate 15 relevant references

Quality (rating)Write a neat, well-spaced business letterDemonstrate correct form of diving

Percentage Correct

Solve 85 percent of the math problemsSpell correctly 90 percent of the words in the word list

Steps RequiredDiagnose a motor malfunction in five stepsLocate a computer error using proper sequence of steps

Use of MaterialBuild a bookcase with less than 10 percent wasteCut out a dress pattern with less than 10 percent waste

SafetyCheck all safety guards before operating machineDrive automobile without breaking any safety rules

Characteristics of a Good Student Project1. It focuses on multiple learning outcomes

2. It includes the integration of understanding, skills, and strategies

3. It is concerned with problems and activities that relate to out-of-school life

4. It involves the active participation of students in all phases of the project

5. It provides for student self-assessment and independent learning

6. It requires performance skills that are generalizable to similar situations

7. It is feasible within the constraints of the students’ present knowledge, time limits, and available resources and equipment

8. It is both challenging and motivating to students

9. It is fair and doable by all students

10. It provides for collaboration between the student and the teacher

Selecting the performance situationSimulated Performance-Attempts to match performance to a real situation

-Math: use of computers to solve lifelike problems (simulated performance);Social studies: role play a jury trial, parliament meeting; science, vocational, business: simulate portions of an actual job performance

Work Sample-Students perform actual tasks that are representative of total performance to be measured

-Work sample approach of assessing student performance widely used in occupations involving performance skills and often can be duplicated in school settings

Student Project-Comprehensive type of performance combining academic, communication, and thinking skills is solving an unstructured real-world problem or unique product

Outline for Problem-Solving a Project:

1. Establishing criteria and standards-Problem Selection, Research Skills, Report Writing, Product Design and Construction, & Oral Presentation

2. Selecting and stating the problem-Students should be free to select a problem that interests them with checks and guidance from the teacher

3. Locating and selecting resources

-May include computer searches, interviewing knowledgable people, books -Selects Appropriate Resource Material

1. Has a variety of resources been selected?

2. Is the resource material relevant to the problem?

3. Do the resources provide various possible solutions to the problem?

4. Does the resource material include evidence supporting the suggested solutions?

5. Is there enough resource material to provide for valid conclusions?

Outline for Problem-Solving a Project:

4. Writing the report-Writes a clear effective Report

1. Has the problem been clearly stated?

2. have the study procedures been adequately described?

3. Has the material from various sources been analyzed, compared, and evaluated?

4. Have the findings been integrated into a well-organized report?

5. Have the findings been integrated into a well-organized report?

6. Does the summary include the main points?

7. Are the conclusions in harmony with the findings and the limits of the study?

8. Does the report exhibit good reasoning ability?

Characteristics of Sound Performance Criteria

1. They describe the components that are most crucial to satisfactory completion of the performance (e.g., beware of peripheral activities that are trivial

2. They focus on observable aspects of the performance (e.g., “Follows safety procedures”, NOT “Demonstrates safety consciousness”)

3. They apply in various contextual settings (e.g., “skill in computation” is applicable in all contexts

4. They represent aspects of performance that experts would agree are necessary for a successful performance (e.g., “good organization” would be recognized by experts as basic in all types of writing)

5. They are stated in terms that are readily understood and usable by students in evaluating performance (e.g., for self-evaluation and peer evaluation)

6. They are in harmony with the instructional objectives and the use to be made of the assessment results (e.g., criteria used in judging writing skills and their improvement over time)

Outline for Problem-Solving a Project:

5. Designing and completing a research study or making a product

-Criteria to consider: Use of Proper Procedures, Adequacy of Interpretation of Results, Validity of the Conclusions

-Criteria should be known to students before beginning the project

-Helpful to put criteria on a rating form

6. Orally presenting and defending the project

-May have each student describe the procedures and findings to a group of students, parents, or members of the community

-Final presentation gives students practice in public speaking and defending their work

-Provides multiple outcomes: research skills, writing skills, speaking skills, thinking skills, self assessment skills, in some cases vocational skills

Selecting the Method of Observing and Recording, and Scoring

Systematic Observation and Anecdotal Records

-Anecdotal Record

-Checklist

-Rating Scales

-Holistic Scoring

-Scoring Rubrics

Anecdotal Record• A brief objective description of some significant

event(s)

• Observed behavior

• Setting in which it occurred

• A separate interpretation of the event

• Most useful when

• Focus on the meaningful events

• Recorded soon after the event

• Contain enough information to be understood later

• Observed event and its interpretation kept separate

Check Lists

• A list of measurable dimensions of a product or performance with a place to record a simple “yes” or “no” judgement

• Construction of a Check List involves the following steps:

• List the procedural steps or product characteristics to be evaluated

• Add common errors to the list, if such is useful in diagnosing poor performance

• Arrange the list in some logical order (e.g., sequence of steps)

• Provide instructions and a place for checking each item

• Add a place for comments at the bottom of the form, if needed

Rating Scales• Rating scale is similar to a check list

• The main difference is rating scales provides opportunity to mark the degree which the element is instead of simply “present-absent”

• May be based on the frequency an event occurs: (e.g., always, sometimes, never)

• The general quality of a performance (e.g., outstanding, above average, average, below average)

• The degree of acceptable performance (e.g., completes task quickly, slow in completing task, cannot complete task without help

Sample Rating ScaleDirections: Rate each of the following items by circling the appropriate number. The numbers represent the following values: 5--outstanding; 4--above average; 3--average; 2--below average; 1--unsatisfactory

PROCEDURE RATING SCALE

How effective was the students performance in each of the following areas?

5 4 3 2 1 a) Preparing a detailed plan for the project5 4 3 2 1 b) Determining the amount of material needed5 4 3 2 1 c) Selecting the proper tools5 4 3 2 1 d) Following the correct procedures for each operation5 4 3 2 1 e) Using tools properly and skillfully5 4 3 2 1 f) Using materials without unnecessary spoilage5 4 3 2 1 g) Completing the work within a reasonable amount of time

PRODUCT RATING SCALE

To what extent does the product meet the following criteria

5 4 3 2 1 a) The product appears neat and well constructed5 4 3 2 1 b) The dimensions match the original plan5 4 3 2 1 c) The finish meets specifications5 4 3 2 1 d) The joints and parts fit properly5 4 3 2 1 e) The materials were used effectively

Constructing a Rating Scale for Performance

• List the procedural steps or product characteristics to be evaluated

• Select the number of points to use in the scale and define them by descriptive terms or phases

• Arrange the items on the rating scale so that they are easy to use

• Provide clear, brief instructions that tell the rater how to mark items on the scale

• Provide a place for comments, if need for diagnostic or instructional purposes

Sample Rating Scale Form for a Student ProjectDirections: Rate each item by circling the appropriate number. The numbers represent the following values: 4—Excellent; 3—good; 2—satisfactory ; 1—weak

Selecting and Stating the Problem4 3 2 1 a) Is the problem in harmony with the students present knowledge and skill?4 3 2 1 b) (add others)

Locating and Selecting Resources4 3 2 1 a) Has a variety of resources been selected?4 3 2 1 b) (add others)

Writing the Report4 3 2 1 a) Has the Problem been clearly stated?4 3 2 1 b. (add others)

Conducting a Research Study 4 3 2 1 a) have proper procedures been followed?4 3 2 1 b) (add others)

Building a Product4 3 2 1 a) Did the product match the specifications4 3 2 1 b) (add others)

Oral Presentation of Project4 3 2 1 a) Did the oral presentation reflect understanding of the problem studied?4 3 2 1 b) (add others)

Holistic Scoring• Based on the overall impression of the

performance or product rather that individual elements

• Typically between 4 and 8 points are given

• Example - The reader will read each writing sample quickly and place them in a pile ranging from 1 to 4

• Good writing is more than the sum of individual elements

• Then each section can be further evaluated

• May be useful to find a “Anchor Paper”

Scoring Rubrics

• List the criteria to keep in mind when making an overall judgement of a product or performance

Sample Scoring Rubric for Writing4--Interesting throughout Flows smoothly, good transitions Well organized topic Good use of mechanics and sentence structure

3--Interesting most of the time Flows smoothly but some poor transitions Organized but some weakness Minor mechanical errors

2--Interest lowered by lapses in focus Flow is interrupted by many poor transitions organization weak, strays from topic Some serious mechanical errors

1--No clear focus Jerky and rambling Poorly organized Many mechanical errors and weak sentence structure

GENERAL SCORING RUBIC FOR A PSYCHOMOTOR SKILLEXCELLENT Uses procedures rapidly and skillfully

Explains function of each step in procedure Modifies procedure to fit changing conditions

GOOD Use procedure correctly but with some hesitation Gives general explanation of steps in procedure Modifies procedure but needs some instructor

guidance

ACCEPTABLE Uses procedure correctly but slow and clumsy Explanation of procedure is limited Modifies procedure but only after demonstration by

instructor

INADEQUATE Fails to use procedure correctly Explanation of procedure shows lack of

understanding Uses trail and error in adjusting procedure