Moderation of Overall Teacher Judgments

2011

Purpose of the workshop: To provide support and guidance to better understand the processes and systems needed for the moderation of overall teacher judgments.

Outline of the Day•Set the scene re: moderation•Consider the place of the OTJ in moderation•Look at systems and processes for making consistent overall teacher judgments•The process of moderation•Develop in-school systems to support moderation and especially student learning

Standards in N.Z.Three options: - A test or range of assessments from standardised tools - A set of criteria or - To describe what you want the students to be doing at each year and illustrate them - based on Royce Sadler’s work

Sadler (1988, p.80) Described three elements that comprise teacher judgment of student work. The elements include:

• teachers attending to the learning students produce

• appraising this work against a reference framework

• and making explicit response such as feedback or judgment on the learner’s work.

Overall teacher judgment has an added complexity in that the judgment process is applied to a range of evidence.

Therefore in New Zealand

• The NZC Standards are not a test, they rely on skilled teachers understanding their students and their students’ learning needs.

• Forming OTJs and moderating them are high level professional skills, requiring knowledgeable skilful teachers.

• Our model needs to involve moderation

• Supports the intent of the NZC • vision + principles

Word Burst

In pairs or threes think about the words in the pack then…….•Choose a word that interests you and say why•Choose a word that describes your classroom practice•Choose a word that describes how you work with a colleague•Park any words that you’re not sure about

Making Overall Teacher JudgmentsKey aspects to developing consistency of OTJs:

• Understanding what constitutes an OTJ (and having systems/processes to promote consistent understanding)

• Developing common understandings of the curriculum levels, the ‘standards’ and the reference materials which help describe them

• Having clear criteria and exemplars of student work demonstrating achievement of standards

• Using moderation processes to ensure appropriate and consistent judgments are made.

Bev’s First Video

Use your Recording Frame and look for examples of:

•Multi-dimensionality•Cross Curricular•Student Voice•Levelling•Observations•Learning conversations

Making an O.T.J.•In pairs share how this process compares with what you already do.•What are the implications for your practice?•Record on the recording frame your next steps for making a sound OTJ.

Critiquing the EvidenceLook at the pack of evidence you have brought:Is there enough evidence from a range of sources to show:

• Multi-dimensionality• Cross curricular• Student voice• Observations• Learning conversations• Does it include information from an

assessment tool?

Making an OTJ •Using Vini’s or Sau’s evidence and the reference points previously discussed to decide if he is Above/At/Below or Well Below the Year level standard. •Record on the template provided your judgment for each piece of evidence and your overall ‘Best Fit’.

What might this look like in reading and Mathematics?

Use the templates provided to brainstorm the forms of evidence you might gather in reading and mathematics.

Points to note re Mathematics?

•Independently and most of the time – can do it with very little scaffolding •Language is different but it means the same as the reading and writing standards •Key point is to be looking for the application of knowledge & strategy to solve problem

eg additive thinking – Student applies their additive thinking to solve problems within the three mathematical strands and across the curriculum.

•Teachers need to provide the opportunities for students to show what they can do notice - recognise - respond

.

Points re Mathematics?What might count as evidence:

• Gloss tool – assesses the strategy number frame work which aligns with the NS - (Use of tool needs to be moderated)

• PAT• E-asTTle• Modelling book evidence that they have done alongside the

teacher• Students books/worksheets as well as models,graphs an

diagrams.• Learning conversations

Student’s place in the process – • Ability to articulate their learning and their next steps • Teachers need to consider how their practice supports

students to discuss their learning, explain & justify their mathematical thinking.

.

One school’s system for recording information

Summary: Consistency is developed by… • Discussing what sources of evidence you currently collect to inform teaching

and learning decisions• Critiquing whether these sources of evidence are adequate• Developing a process/template for the school (or areas within the school –

e.g. junior and senior school) so that teachers are using similar types of evidence to inform OTJs.

• Developing common understandings of ‘standards’ and the reference materials which help describe them.

• Having clear criteria and exemplars of student work demonstrating achievement of standards

NZC, National Standards illustrations, Literacy Learning Progressions, Number Framework,

The big picture -Moderation Moderation is the process of teachers sharing

their expectations and understandings of standards with each other in order to improve

the consistency of their decisions about student learning and achievement.

The process where teachers compare judgments to either confirm or adjust them.

Moderation processes require:

• Staff culture and systems for on-going professional learning

• Theoretical and content knowledge of the NZC and the standards

• Familiarity with the relevant support material

Teachers need to be safe to:• express and clarify thinking• ask questions• explore solutions• adapt thinking after listening to informed ideas of

others• tolerate and appreciate differences in perspectives• view differences as opportunities to deepen

knowledge base

This is what a moderation process could look

like?• Select the children whose evidence is conflicting or who sit on the cusp.

• Each teacher from the team/syndicate brings evidence of one of those

students.

• Using evidence, the National Standards and other curriculum resources,

each teacher makes an independent judgment about each student.

• Teachers discuss their judgments and use evidence, the National

Standards, and other curriculum resources to come to an agreed judgment.

• The team selects one student for whom it has reached agreement and

take to school-wide moderation.

Each teacher from a team/syndicate

brings evidence of one of

those students

Select the children whose evidence

is conflicting or who sit on the cusp.

The team selects one student’s

results for which they have

come to agreement and take to whole-school (vertical)

moderation.

Teachers compare independent OTJs and come to an agreed judgment for each student using evidence / NS / resources.

An example of a moderation process

Each teacher uses evidence/ NS/resources to make an independent OTJ for each

student

Moderation Listen to the following conversations and consider:•Consistency•Reliability•Validity•Bias•Teacher knowledge

Moderation

In your level groups use this process to explain your OTJ from Vini’s evidence or the evidence you have brought

Discuss differences and underlying reasons

Decide as a team your agreed OTJ (overall team judgment)

Across school moderation

In vertical groups of three share your team’s explanation of their OTJ.

Recording the consistency with which teachers are making independent OTJs

Student Expert marker Other teachers

1 = agreement0 = non

agreementS1 @ @

Tchr 1

1S2 @ B 0S3 WB WB 1S4 AS @ 0S1 @ @

Tchr 2

1S2 @ @ 1S3 WB B 0S4 AS AS 1S1 @ AS

Tchr 3

0S2 @ B 0S3 WB WB 1S4 AS AS 1

7/12

% AGREEMENT 58%

Making EffectiveOTJs

Attending to componentsof moderation

Understanding and using moderation

High quality learning and teaching

Clear about OTJsCommon understandingsClear criteria and exemplarsUse of moderation processes

Ultimatepurposeof moderation

Interpersonal skillsTheoretical /content

knowledgeFrame of reference

Culture of professional learning

Confirm or adjust quality

judgments

A framework for Moderation

Deepening teacher knowledge

Action PlansUsing your Next Steps sheet develop an action plan for:

• Term 2 2011• The rest of 2011• 2012

Consider teacher/team/management processes.