Teacher Assessment Magno_2

7/31/2019 Teacher Assessment Magno_2

1/24

The Development of the Peer

Assistance and Review Form (PARF)Carlo Magno

De La Salle University, Manila


2/24

Assessing Teacher Performance

Students assess teachers

*A peer (fellow teacher/supervisor) assessing theteacher

External personnel assessing the teacher


3/24

Peer Review of Teaching

When a teacher is observed by another teacherfor a specific purpose, it is called in literature aspeer assistance and review (Goldstein, 2004)

peer review (Kerchner & Koppich, 1993),

teacher peer coaching (Bruce & Ross, 2008)


4/24

What peer review of teaching? involving teachers in the summative [also

formative] evaluation of other teachers (Goldstein,

2004, p. 397) evaluating ones peers which allow the assessmentof ones teaching by another person who has similarexperience and goals

a structured approach for building a community in

which pairs of teachers of similar experience andcompetence observe each other teach, establishimprovement goals, develop strategies to implementgoals, observe one another during the revisedteaching, and provide specific feedback (Bruce &

Ross, 2008)


5/24

Purpose

Hiring of teachers

Clinical supervision

Modeling for new teachers

Promotion

Rehiring

Common uses of Teacher Performance Results


6/24

Developmental Approach

Teachers performance from peer reviews shouldbe conceptualized with the aim of helpingteachers attain success in their teaching ratherthan point out mistakes of teachers (Oakland &Hambleton, 2006; Stiggins, 2008).

It is described as a constructive process wherethe peer aims to provide assistance to a lessexperienced teacher in improving theirinstruction and handling of students.


7/24


8/24

Conceptual Framework

Learner-centered principles

perspectives that allow the teachers ability tofacilitate the learners in their learning, thelearning in the programs, and other processes thatinvolves the learner

Danielsons Components of Professional Practice

identified aspects of the teachers responsibilitiesthat have been documented through empiricalstudies and theoretical research promotingimproved student learning.


9/24

Conceptual Framework

Danielsons Components of Professional Practice

Planning and preparation

Classroom environment

Instruction

Professional responsibility

The LC and components of professional practicefits well together in a model (Magno &Sembrano, 2010).


10/24


11/24

Method Rubric Construction (analytical rubric)

Domain 1: Planning and Preparation1a. Demonstrating knowledge of content and

pedagogy1b. Demonstrating knowledge of students1c. Selecting instructional goals

1d. Demonstrating knowledge of resources1e. Designing coherent instruction1f. Assessing student learning

Domain 2: The Classroom Environment2a. Creating an environment of respect and

rapport2b. Establishing a culture for learning2c. Managing classroom procedures

2d. Managing student behavior2e. Organizing physical space

Domain 3: Instruction3a. Communicating clearly and accurately3b. Using questioning and discussion

techniques3c. Engaging students in learning3d. Providing feedback to students3e. Demonstrating flexibility and

responsiveness

Domain 4: Professional Responsibilities4a. Reflecting on teaching4b. Maintaining accurate records4c. Communicating with families4d. Contributing to the school and district4e. Growing and developing professionally4f. Showing professionalism


12/24


13/24

Item Review

FGD

Review checklist

External reviewers


14/24

Pretesting Initial pretesting

Developed a manual how to use the PRPF and

how to conduct proper observation Orientation for raters and ratees 2 raters are assigned for a ratee (primary and

secondary raters)

183 ratees were completed Final pretesting

175 ratees were completed Same procedure


15/24

PARF results (Reliability)Initial pretesting (N=183, 89

items)

Internal consistency for raters:.98 and .97

Planning and Preparation:.94 & .93

Classroom Environment: .93& .92

Instruction: .94 & .92

Professional responsibility:.93 & .91

Final pretesting (N=175, 59items)

Internal consistency for raters:.97 and .96

Planning and Preparation:.94 & .93

Classroom Environment: .88& .88

Instruction: .93 & .90

Professional responsibility:.92 & .76


16/24

PARF results (Rater agreement)

Initial pretesting (N=183, 89items)

=.47,p


17/24


18/24

PARF results (GRM) Initial pretesting (N=183,

89 items) Final pretesting (N=175,

59 items)

Figure 3. Test Information Function of PRPF for

the Primary Raters

Figure 4. Test Information Function of the PRPFof the Secondary Rater

Figure 1. Test Information Function of PRPF for

the Primary Raters

Figure 2. Test Information Function of the PRPFof the Secondary Rater


19/24

Item Maps

Figure 6. Item Map of the PRPF for the Primary Rater Figure 7. Item Map of the PRPF for the Secondary Rater

Item Maps


20/24

PARF results (Factorial Validity)

Initial pretesting (N=183, 89 items)

A four factors structure (3 parcels each factor)

2=8829.23, df=3734,

PGI=.97,

Bentler-Bonnett Normed Fit Index=.96,

Bentler-Bonnett Non-Normed Fit Index=.96.


21/24

PARF results (Construct Validity)

2=169.56,df=98,

standardizedRMS=.02,

PGI=.95,

GFI=.90,

Bentler-BonettNFI=.97,

BentlerCFI=.99,

Bollens

Rho=.96.


22/24

Discussion

Consistent results were obtained in thepreliminary and final pilot testing in terms of the

internal consistencies, concordance of raters,convergence of factors, and factor structure.

Theoretical construction of Danielsons

Components of Professional Practice andLearner-centeredness.

Very few items turn out to be bad fit.

Precision of measurement


23/24

Problems:

Majority of scores given were very high

Low discriminating power of the items

Recommendations:

High cut off scores

Include a wide array of faculty in the assessment

Strict and objective raters


24/24

Contituous Project

Versions following the same framework:

Students rating for teachers (rating scale)

Screening tool for teacher applicants (checklist)

Community Service classes (Ratings scale)

Culinary Laboratory classes (rubric)

Seminar classes (rating scale) Faculty Advisers (rating scale)

Documents

Teacher Assessment Magno_2