EDU 385 - Session 11 Performance Assessment

7/27/2019 EDU 385 - Session 11 Performance Assessment

1/41

EDU 385

Educational Assessment in the ClassroomSession 11

Performance Assessment


2/41

Content Objectives

Understand the advantages and limitations ofPerformance Assessments

Understand how to write, observe, record, and

evaluate a performance task


3/41

anguage ect ves

1. Describe how performance assessments differ from paper-and-pencil testing

2. Identify the strengths and limitations of performanceassessments

3. Write intended performance outcmes for a performanceassessment

4. Distinguish between restricted and extended performanceassessment

5. Describe the general procedure for making a performanceassessment

6. Describe the use of a student project for performanceassessment

7. Prepare a plan for arranging, observing, recording, andevaluating a performance task

8. Construct a holistic scoring rubric for some performance


4/41

Academic Vocabulary

Performance Assessment

Restricted-Response Item

Extended-Response Item

Scoring Rubric


5/41

Bell Work


6/41

Performance Assessments Use when paper and pencil tests are inadequate to assess

students performance skills

When more direct assessments are needed to assess actualperformance

Science - laboratory skills

English and Foreign-Language - Communication skills

Mathematics - Problem Solving skills

Other Performance Subjects: art, music, industrial arts.

business, economics, home economics, physical education


7/41

Components of Performance Skills

Knowledge Component

Skills Component

Affective Component

Example:

Assessing Writing Ability: concerned about vocabulary, grammar,as well as interests and attitudes that support good writing

Assessing Laboratory Skills: names and uses of equipment andprocedures to follow; as well as attitude of care and safety inhandling equipment and making accurate measurements

Performance Skills


8/41

Writing Performance Objectives for Skills andProducts

1. State each general objective so that it clearly describes the skill

or product to be assessed

2. List specific performance outcomes for each objective that aremost relevant to a successful performance or a satisfactoryproduct

3. List enough specific performance outcomes to clarify what ismeant by an effective performance

4. State the specific performance outcomes in terms of observabledimensions of the skill or product

5. State the specific performance outcomes so that they are clearto students


9/41

Restricted & Extended Performance

It may be desirable to measure specific skills (i.e., Restricted-Performance Tasks) before putting them in a more complexPerformance

Restricted-Performance can help diagnose problems

(e.g., student having difficulty using lab equipment)

Limiting scope of the task, makes it easier to focus observation andjudge response

Extended Performance Tasks are less structured and broader inscope

Give students greater freedom and opportunity for self-assessmentand self-improvement

Helps develop students independent learning skills


10/41

Examples

Restricted Performance Tasks

-Write a one page report describing a field trip-Give a one-minute speech on a given topic-Read aloud a brief selection of poetry-Construct a graph form a given set of data

-Demonstrate how to use a measuring instrument

Extended Performance Tasks

-Design and conduct an experiment

-Design and build a wood or metal productt-Write a short story-Repair a malfunctioning motor-Paint a picture-Demonstrate a physical or musical performance


11/41

Steps in Preparing Performance Assessments

Example - - Demonstrates Skill in Oral Reporting

1. Specifying the performance outcomes

2. Selecting the focus of the assessment (Procedure, product, or both)3. Selecting an appropriate degree of realism4. Selecting the performance situation5. Selecting the method of observing, recording, and scoring6. Speaks at an appropriate rate7. Presents ideas in an organized manner

8. Uses appropriate language9. Maintains interest of the group

Example - - or realistic set of Extended Performance Tasks

-Designs and conducts an experiment-Writes and accurate account of the study-States valid conclusions-Writes a critique of the procedure and findings-Presents and defends the study in class


12/41

Specifying the performance outcomes

Restricted Performance commonly use verbs such as: identify,construct, and demonstrate

Specifications of performance outcomes typically include a job or

task analysis

Demonstrate Skill in Oral Reporting1. Stands in natural manner2. Maintains good eye contact3. Uses appropriate facial expressions3. Uses gestures effectively4.


13/41

Action Verbs Illustrative Instructional Objectives

IDENTIFY: Selects the correctobjects, parts of the object,procedure, or property (typicalverbs: ident ify, loc ate, select,touc h, pick up, mark, describe)

-Select the proper tool,-Identify the parts of a typewriter,-Choose correct laboratory equipment,-Select most relevant statistical procedure,-Locate an automobile malfunction,-Identify a musical selection,-Identify experimental equipment needed

-Identify specimen under the microscopeCONSTRUCT: Make a productto fit a given set of specifications(typical verbs: construct, assemble,build, design, draw, make, prepare)

-Draw a diagram for an electrical circuit,-Design a pattern for making a dress,-Assemble equipment for experimental study,-Prepare a graph,-Construct a weather map,-Prepare an experimental design

DEMONSTRATE: Performs aset of operations or procedures(typical verbs: demonstrate, drive,measure, operate, perfo rm , repair, set

up)

-Drive an automobile-Measure the volume of a liquid-Operate a filmstrip projector-Perform a modern dance step-Repair a malfunctioning TV set

-Set up laboratory equipment-Demonstrate taking a patients temperature

Typical Action Verbs and Illustrative Instructional Objectives forRestricted Performance Outcomes


14/41

Selecting the focus of the assessment Type of performance that doesnt result in a product

(speaking , reading aloud, phy sical ski l ls , musical

performance)

Procedures (ear ly p art of perform ance) and p roducts later

(e.g., typ ing - pro per use of touch s ys tem later neatness,

accuracy, and speed)

No pro duct evaluat ion is feasible

Procedure is order ly and direct ly observable

Correct procedure is cruc ial to later su ccess

Analys is of procedural steps improves a product


15/41

Selecting the focus of the assessment

Assessing the Product When Procedures are sufficiently mastered - Assess

Product

Use specific criteria to used to determine the quality of the

product

Performance Assessment should focus on product when

Different procedures can result in equally good product

Procedure not observable (take-home work)

Procedural steps have been mastered

Product qualities can be clearly identified and judged


16/41

e ec ng an appropr a e egreeof realism

Strive for Performance Assessments that approximate real-

world conditions

As much realism as possible

Factors to consider

Nature of the instructional objectives (does paper/pencilgive you the information you need?)

Sequence of instruction (early in unit paper/pencil may beenough and later use performance tests

Constraints as time, resources, equipment may limit theuse of performance assessments

D f R li


17/41

Degree of Realism

Have students solve this story problem: If you

bought a toy for 69 cents and gave the clerk $1.00, Low Realismhow much change would you get back?

Demonstrate the problem situation with actualmoney and ask each student to judge whetherthe change is correct

Pair off the students, give them actual money,and have them role-play the problem situation

Set up a mock toy store and have each student High Realismdemonstrate the purchase of a toy withactual money

t t t t


18/41

e ect ng t e per ormance s tuat onPaper-and-Pencil Performance

-Emphasis on Application of knowledge and skill in simulated setting

-Action verb construct frequently used (construct weather map, bar graph, diagram anelectrical circuit, floor plan, item of clothing

Identification Test

-Biology: identify specimens placed at various stations in classroom

-Math: identify correct problem-solving procedures-English: identify best expression to be used in writing

Structured Performance Test-A structured performance test under standard, controlled conditions

-Examples: making a prescribed measurement, adjusting a microscope, following asafety procedure

-Set performance standards (minimal level of acceptable performance)

-Specify clear and definite criteria for judging procedure or product

-To improve reliability ask students to use descriptive terms (not subjective)

S C St d d f J d i P f


19/41

Some Common Standards for Judging Performance

TYPE Examples

RateSolve ten addition problems in two minutesType 40 words per minute

Error No more than two errors per typed pageCount to 20 in Spanish without error

TimeSet up a laboratory equipment in five minutesLocate an equipment malfunction in three minutes

PrecisionMeasure a line within one-eighth of an inch

Read a thermometer within two-tenth of a degree

QuantityComplete 20 laboratory experiementsLocate 15 relevant references

Quality (rating)Write a neat, well-spaced business letterDemonstrate correct form of diving

Percentage CorrectSolve 85 percent of the math problemsSpell correctly 90 percent of the words in the word list

Steps RequiredDiagnose a motor malfunction in five stepsLocate a computer error using proper sequence of steps

Use of MaterialBuild a bookcase with less than 10 percent wasteCut out a dress pattern with less than 10 percent waste

Safety Check all safety guards before operating machineDrive automobile without breaking any safety rules

Characteristics of a Good Student Project


20/41

Characteristics of a Good Student Project

1. It focuses on multiple learning outcomes

2. It includes the integration of understanding, skills, and strategies

3. It is concerned with problems and activities that relate to out-of-school life

4. It involves the active participation of students in all phases of the project

5. It provides for student self-assessment and independent learning

6. It requires performance skills that are generalizable to similar situations

7. It is feasible within the constraints of the students present knowledge, time limits, and

available resources and equipment

8. It is both challenging and motivating to students

9. It is fair and doable by all students

10. It provides for collaboration between the student and the teacher


21/41

Selecting the performance situation

Simulated Performance

-Attempts to match performance to a real situation

-Math: use of computers to solve lifelike problems (simulated performance);Social studies: role play a jury trial, parliament meeting; science, vocational, business:simulate portions of an actual job performance

Work Sample-Students perform actual tasks that are representative of total performance to bemeasured

-Work sample approach of assessing student performance widely used in occupationsinvolving performance skills and often can be duplicated in school settings

Student Project-Comprehensive type of performance combining academic, communication, andthinking skills is solving an unstructured real-world problem or unique product

Outline for Problem Solving a Project:


22/41

Outline for Problem-Solving a Project:

1. Establishing criteria and standards

-Problem Selection, Research Skills, Report Writing, Product Design andConstruction, & Oral Presentation

2. Selecting and stating the problem

-Students should be free to select a problem that interests them with checks andguidance from the teacher

3. Locating and selecting resources-May include computer searches, interviewing knowledgable people, books-Selects Appropriate Resource Material

1. Has a variety of resources been selected?

2. Is the resource material relevant to the problem?

3. Do the resources provide various possible solutions to the problem?

4. Does the resource material include evidence supporting the suggestedsolutions?

5. Is there enough resource material to provide for valid conclusions?


23/41


4. Writing the report-Writes a clear effective Report

1. Has the problem been clearly stated?

2. have the study procedures been adequately described?

3. Has the material from various sources been analyzed, compared, andevaluated?

4. Have the findings been integrated into a well-organized report?

5. Have the findings been integrated into a well-organized report?

6. Does the summary include the main points?

7. Are the conclusions in harmony with the findings and the limits of the study?

8. Does the report exhibit good reasoning ability?

Ch t i ti f S d P f C it i


24/41

Characteristics of Sound Performance Criteria

1. They describe the components that are most crucial to satisfactory completion of theperformance (e.g., beware of peripheral activities that are trivial

2. They focus on observable aspects of the performance (e.g., Follows safety

procedures, NOT Demonstrates safety consciousness)

3. They apply in various contextual settings (e.g., skill in computation is applicable in

all contexts

4. They represent aspects of performance that experts would agree are necessary for a

successful performance (e.g., good organization would be recognized by experts asbasic in all types of writing)

5. They are stated in terms that are readily understood and usable by students inevaluating performance (e.g., for self-evaluation and peer evaluation)

6. They are in harmony with the instructional objectives and the use to be made of theassessment results (e.g., criteria used in judging writing skills and their improvementover time)

Outline for Problem Solving a Project:


25/41


5. Designing and completing a research study ormaking a product

-Criteria to consider: Use of Proper Procedures, Adequacy of Interpretation ofResults, Validity of the Conclusions

-Criteria should be known to students before beginning the project

-Helpful to put criteria on a rating form

6. Orally presenting and defending the project-May have each student describe the procedures and findings to a group ofstudents, parents, or members of the community

-Final presentation gives students practice in public speaking and defending theirwork

-Provides multiple outcomes: research skills, writing skills, speaking skills,thinking skills, self assessment skills, in some cases vocational skills

S l ti th M th d f Ob i d


26/41

Selecting the Method of Observing andRecording, and Scoring

Systematic Observation and Anecdotal Records

-Anecdotal Record

-Checklist

-Rating Scales

-Holistic Scoring

-Scoring Rubrics


27/41

Anecdotal Record

A brief objective description of some significant event(s)

Observed behavior

Setting in which it occurred

A separate interpretation of the event

Most useful when

Focus on the meaningful events

Recorded soon after the event

Contain enough information to be understood later

Observed event and its interpretation kept separate


28/41

Check Lists

A list of measurable dimensions of a product or performance with a place torecord a simple yes or no judgement

Construction of a Check List involves the following steps:

List the procedural steps or product characteristics to be evaluated

Add common errors to the list, if such is useful in diagnosing poorperformance

Arrange the list in some logical order (e.g., sequence of steps)

Provide instructions and a place for checking each item

Add a place for comments at the bottom of the form, if needed

Rating Scales


29/41

Rating Scales

Rating scale is similar to a check list

The main difference is rating scales provides opportunity tomark the degree which the element is instead of simplypresent-absent

May be based on the frequency an event occurs: (e.g.,

always, sometimes, never)

The general qual ity o f a performance (e.g., ou tstand ing ,

above average, average, below average)

The degree of acceptable performance (e.g., completes taskqu ickly, slow in complet ing task, canno t complete task

wi thou t help

amp e a ng ca e


30/41

amp e a ng ca eDirections: Rate each of the following items by circling the appropriate number. Thenumbers represent the following values: 5--outstanding; 4--above average; 3--average;2--below average; 1--unsatisfactory

PROCEDURE RATING SCALE

How effective was the students performance in each of the following areas?

5 4 3 2 1 a) Preparing a detailed plan for the project5 4 3 2 1 b) Determining the amount of material needed5 4 3 2 1 c) Selecting the proper tools

5 4 3 2 1 d) Following the correct procedures for each operation5 4 3 2 1 e) Using tools properly and skillfully5 4 3 2 1 f) Using materials without unnecessary spoilage5 4 3 2 1 g) Completing the work within a reasonable amount of time

PRODUCT RATING SCALE

To what extent does the product meet the following criteria

5 4 3 2 1 a) The product appears neat and well constructed5 4 3 2 1 b) The dimensions match the original plan5 4 3 2 1 c) The finish meets specifications5 4 3 2 1 d) The joints and parts fit properly5 4 3 2 1 e) The materials were used effectively


31/41

Constructing a Rating Scale forPerformance

List the procedural steps or product characteristics to beevaluated

Select the number of points to use in the scale and definethem by descriptive terms or phases

Arrange the items on the rating scale so that they are easy touse

Provide clear, brief instructions that tell the rater how to markitems on the scale

Provide a place for comments, if need for diagnostic orinstructional purposes

Sample Rating Scale Form for a Student Project


32/41

Sample Rating Scale Form for a Student Project

Directions: Rate each item by circling the appropriate number. The numbers represent

the following values: 4Excellent; 3good; 2satisfactory ; 1weak

Selecting and Stating the Problem

4 3 2 1 a) Is the problem in harmony with the students present knowledge and

skill?

4 3 2 1 b) (add others)

Locating and Selecting Resources

4 3 2 1 a) Has a variety of resources been selected?


Writing the Report4 3 2 1 a) Has the Problem been clearly stated?

4 3 2 1 b. (add others)

Conducting a Research Study

4 3 2 1 a) have proper procedures been followed?


Building a Product

4 3 2 1 a) Did the product match the specifications


Oral Presentation of Project

4 3 2 1 a) Did the oral presentation reflect understanding of the problem

studied?4 3 2 1 b add others


33/41

Holistic Scoring

Based on the overall impression of the performance orproduct rather that individual elements

Typically between 4 and 8 points are given

Example - The reader will read each writing sample quicklyand place them in a pile ranging from 1 to 4

Good writing is more than the sum of individual elements

Then each section can be further evaluated

May be useful to find a Anchor Paper


34/41

Scoring Rubrics

List the criteria to keep in mind when making an overalljudgement of a product or performance

Sample Scoring Rubric for Writing


35/41

Sample Scoring Rubric for Writing4--Interesting throughout

Flows smoothly, good transitionsWell organized topic

Good use of mechanics and sentence structure

3--Interesting most of the timeFlows smoothly but some poor transitionsOrganized but some weaknessMinor mechanical errors

2--Interest lowered by lapses in focusFlow is interrupted by many poor transitionsorganization weak, strays from topicSome serious mechanical errors

1--No clear focus

Jerky and ramblingPoorly organizedMany mechanical errors and weak sentence structure


36/41

GENERAL SCORING RUBIC FOR A PSYCHOMOTOR SKILL

EXCELLENT Uses procedures rapidly and skillfullyExplains function of each step in procedure

Modifies procedure to fit changing conditions

GOOD Use procedure correctly but with some hesitation

Gives general explanation of steps in procedure

Modifies procedure but needs some instructor guidance

ACCEPTABLE Uses procedure correctly but slow and clumsy

Explanation of procedure is limited

Modifies procedure but only after demonstration by

instructor

INADEQUATE Fails to use procedure correctly

Explanation of procedure shows lack of understanding

Uses trail and error in adjusting procedure


37/41

1. Performance assessment provide direct evidence of valued learning outcomes thatcannot be adequately assessed by traditional paper-and pencil- testing; but they aretime consuming to use and require greater use of judgment in scoring

2. Performance tasks contain knowledge and affective comonents as well as the skill

component. All three components must be considered when planning performanceassessments

3. Writing performance objectives involves stating each general objective so that itdescribes a skil or product and then defining it by a list of specific performanceoutcomes that are relevant, clarify an effective performance and are stated in observableterms that are easily understood by students

4. Restricted performance tasks are highly structured and limited in scope (e.g.,construct a graft). Extended performance tasks are typically less well-structurted andbroad in scope (e.g., design and conduct an experiment)

5. The first step in performance assessment it to specify the intended performanceoutcomes

6. Performance assessment may focus on a procedure (e.g., giving a speech) a product(e.g., a theme) or both (e.g., using tools properly in building a bookcase)

7. In some cases, it may be desirable to emphasize procedure evaluation during theearly stages of instruction (e.g., touch system in typing) and product evaluation later

(typed letter)


38/41

SUMMARY OF PERFORMANCE ASSESSMENT

8. There are varying degrees of realism in performance assessment and the aim is toobtain a high degree of realism as possible within the various constraints operating

(e.g., time, cost, availability of equipment)9. paper-and-pencil performance assessment is useful as a terminal measure in manyareas (e.g., writing, drawing, problem solving) and can serve as a first step towardhands-on performance in others (e.g., procedure for repairing an automobile engine)

10. The identification test is typically concerned with identifying the tools, equipment,and prcedures needed for a performance task and serves as indirect measure ofperformance or as an instrucitonal device to prepare students for actual performance

11. A structured performance test provides for an assessment under standard controlledconditions (e.g., locating a malfunction in electronic equipment). The tools, eqipment,conditions, and standards are performance are all carefully prescribed

12.Performance assessment based on simulated performance (e.g, drivers training

simulator) and the work sample (e.g., analyze business data on a computer) has a highdegree of realism. Many performance skills is laboratory courses, business education,and industrial education can be evaluated at this level.

13. Student projects provide for the assessment of multiple learning outcomes (e.g.,research, writing, speaking, thinking, and self-assessment skills) are adaptable tovarious areas of instruction and typically use realistic problems

SUMMARY OF PERFORMANCE ASSESSMENT


39/41

SUMMARY OF PERFORMANCE ASSESSMENT14. A problem-solving project includes the establishing of criteria and standards forassessing the project, selecting and stating the problem, locating and selectingresources, writing the report, designing, and completing a research study or making aproduct (e.g., model), and orally presenting and defending the project

15. Observing students in natural settings and keeping anecdotal records can aid inevaluating aspects of performance, that are likely to be overlooked by structuredmethods (e.g., work habits)

16. Rating scales provide for analytic scoring, direct attention to the performancedimension to be observed, and provide a conenient form on which to record the

udgement

17. Holistic scoring rubrics and product scales are especially useful where globaludgments are being made (e.g., creative writing , works of art)

18. Improving performance assessments involves making clear what is to be observed,how it is to be observed, how the observations are to be recorded, and ow the results are

to be scored and used. In addition, any procedure that contributes to more objectiveobservations and records will aid in increasing the reliabiltiy and meaningfulness of theresults.


40/41

Now Go Forth

andDo Good Things


41/41

Documents

EDU 385 - Session 11 Performance Assessment