Embed Size (px)
Citation preview
Coding Manuals as Lenses into Teaching Expertise:
Measuring teaching expertise from the point of classroom
observations
OECD symposium June 2014
Kirsti Klette, University of Oslo
•Understanding Teaching Quality: Building better measures for understanding Classroom teaching – trends and developments •Understanding Teaching Quality: Building better measures for understanding Classroom teaching – trends and developments
Background
• Understanding teaching qualities point to a
long history in the educational sciences
• However – understanding / or measuring
teaching qualities have proven difficult
• Field suffering from fragmentation; theoretical
/ methodological wars; and the lack of
common instruments
Analyzing teaching quality:
Teachers matters!
All studies trying to understand classroom
teaching and learning underscore the role of
teachers
Recent studies suggest
Teachers represent the most important factor
in student achivement, outside family and
education
Different approaches/ traditions for
analyzing teachers:
WHO teachers are:
their personalties, values and beliefs but also
their competences/ qualifications and formal
training
WHAT teachers do: instructional strategies,
content coverage; cognitive activation,
classroom management, interaction patterns
Methodological instruments available:
Teacher properties
Knowledge Practices Values/
Beliefs
Student properties
Values/
Beliefs Practices Knowledge
Questionnaires
Observations
Interaction patterns
Artefacts/ Artefact Collection
Portefolios
Technologies
Tests
Assignments/ Tasks
The problem
• Weak communication between the
theoretical approaches/traditions
• Weak discussion between how the different
methodological approaches support
analyses and enable findings
Studies of knowledge growth and
developments in other disciplines (Foray, 2003; 2006; Nelson 2003; Knorr Cetina, 1997; 2007;OECD, 2000)
Underscore:
• Infrastructures that stimulate research &
development (R & D)
• Supporting Technologies
(instruments, tools, knowledge objects that
can travel /support these developments
(Science Illuminated Technologies - SIT)
• Degree of Codification
Comparing knowledge growth across the
disciplines Nelson(2003) for example argues:
• ”This is not to say that there is no
understanding about the principles of good
teaching. But these have been known for
generations. And it is not clear that we know
more than 100 years ago. This clearly stands
in sharp contrast with other arenas of human
know-how, like information processing, and
communication or transport”
Recent developments in
• Technological developments in video
design (easy storage, on-line streaming; miniaturized /
portable equipment, that can be remotely controlled, analytical
tools)
• Methodological developments (integrated
research design; nested design/ combining qualitative and
quantitative analyses)
• Coding manuals as ways of measuring
teaching qualities (commonalities across these
protocols when it comes to examining features of teaching)
II Coding manuals as measures of
teachers expertise:
The role of codification
Observation manuals have been a part of the
methodological toolkit for measuring qualities of
instruction for decades, the field has also been
suffering from “paradigm wars”, fragmentation
and local production of instruments.
Common protocols a step forward?
• Such common tools for research developed
within an integrated methodological design
could help researchers making progress in
aggregating knowledge about the impact of
different teaching approaches across settings
and subjects areas
Why coding protocols - the role of
codification:
– link the analyses to criteria
– makes the analyses explicit
Strengthen:
– transparency (and thus communication)
– critical re-examination
– subject of comment and addition
– comparison across studies
– transfer
– accumulation of knowledge
III Review of Observation Protocols
• A number of current observation protocols
have been designed to focus upon elements
of classroom instruction that may be
consistent across different grade levels and
content area, examining a series of features
that could be considered generic elements of
teaching
Examples of such protocols
• CLASS (Classroom Assessing Scoring System,
Pianta et al. 2006)
• FFT (Framework For Teaching, Danielsson, 2009)
• PLATO-manual, Grossman et al., 2009
• Mathematical Quality of Instruction (Hill, 2010)
• IPN Video study scoring manuals (Seidel & Prenzel)
• LPS study (Clark et al., 2006)
• Pythagoras Study (Lipovsky et al.2009)
• CPV Video study scoring manuals (Janik et al.2009)
• PISA+ manuals, (Klette et al., 2005; Ødegaard 2007)
CLASS manual: Supportive climate
The Framework for Teaching manual
The Framework for Teaching:
Components of Professional Practice
Domain 1: Planning and Preparation Domain 2: The Classroom Environment
· Demonstrating Knowledge of Content
and Pedagogy Demonstrating
· Knowledge of Students
· Setting Instructional Outcomes
· Demonstrating Knowledge of Resources
· Designing Coherent Instruction
· Designing Student Assessments
· Creating an Environment of Respect and
Rapport
· Establishing a Culture for Learning
· Managing Classroom Procedures
· Managing Student Behavior
· Organizing Physical Space
Domain 4: Professional Responsibilities Domain 3: Instruction
· Reflecting on Teaching
· Maintaining Accurate Records
· Communicating with Families
· Participating in a Professional
· Community
· Growing and Developing Professionally
· Showing Professionalism
· Communicating with Students
· Using Questioning and Discussion
· Techniques
· Engaging Students in Learning
· Using Assessment in Instruction
· Demonstrating
Looking across the protocols:
There are serious commonalities across these
protocols when it comes to examining features
that are considered to be generic elements of
teaching, such as:
• instructional format
• instructional clarity
• teacher-student interaction
• classroom environment/climate
• cognitive challenge
Reflect a shared understanding of key
features of teacher expertise linked to
classroom teaching
• Instructional clarity
• Cognitive challenge
• Discourse features
• Supportive climate
Two illustrative studies:
PISA + video study/ manuals (Klette, 2009)
PLATO manual (Grossman, 2013)
PISA + Video Study A video Based Classroom Study in Mathematics,
Science and Language Arts
We developed coding manuals focusing on:
• instructional format
• instructional activities
• discourse features
Data:
• Videorecordings/ (ca 152 lessons)
–Three camera solution: teacher; class; students
• Video Recorded Student Interviews mathematics/ science (75 interviews)
• Copies of students’ work
• Teacher Interviews (17 teacher interviews)
Three weeks in each classrooms
PISA+ Study A Video Based Classroom Study in
Mathematics, Science & Language Arts (Grade 9)
http://www.pfi.uio.no/forskning/forskningsprosjekter/pisa+/index.html
Instructional activities
Teaching for the
whole class
Individual work
Work in groups
0
10
20
30
40
50
60
70
80
Mathematics Science Norwegian
Whole class
Individual
Group
Instructional format lessons
Videograph analysis transferred to SPSS
%
Wholeclass instruction:
Instruction monologue
Instruction dialogue
Question – Answer
Wholeclass discussion
Student presentation
Task management
Misbehavior
Messages
Whole class Instruction (Science Classrooms)
0 10 20 30 40
Messages etc.
Misbehavior etc.
Task management
Student pres.
Wholeclass
discus.
Question/answer
Instruction
dialogue
Instruction
monologue
Features of whole class instruction in percent
of filmed time in PISA+ science classrooms.
(Klette, 2009;Ødegaard & Klette, 2012)
Teaching activities in science classrooms (Ødegaard & Arnesen, 2006)
Teaching activities:
- task management
- activate prior knowledge
- topic appetizer
- summing up
- going over the ’do now’
- new content knowledge
- practical work
0
10
20
30
40
50
Task m
anag
emen
t
Acitiv
ate
prev
. kno
wledg
e
Topic a
ppetizer
Summ
ing
up
Going
ove
r do
know
Going
ove
r hom
ewor
k
New
con
tent
kno
wledg
e
Practical w
ork
Teaching activities in science
0
10
20
30
40
50
Task m
anag
emen
t
Acitiv
ate
prev
. kno
wledg
e
Topic a
ppetizer
Summ
ing
up
Going
ove
r do
know
Going
ove
r hom
ewor
k
New
con
tent
kno
wledg
e
Practical w
ork
PLATO manual (Grossman, 2009)
Do classroom practices make a
difference?
• How do classroom practices of more effective
teachers differ from those of less effective
teachers?
Development of the PLATO manual
• Based on research in effective practice in ELA
– Agnostic with regard to particular curricular approach
• Able to capture multiple content domains within ELA
– Reading, writing, literature, grammar, speaking, listening
• Built around 4 primary underlying constructs
THE PLATO ELEMENTS
• Modeling
• Strategy Use and Instruction
• Feedback
• Accommodations for Language Learning
Instructional Scaffolding
• Intellectual Challenge
• Classroom Discourse
• Text-Based Instruction
Disciplinary Demand
• Representation of Content
• Connections to Personal/ Cultural Experience
• Connections to Prior Knowledge
• Purpose
Representations and Use of Content
• Behavior Management
• Time Management Classroom
Environment
Diagnostic function of PLATO:
A snapshot of instruction
1
1,5
2
2,5
3
3,5
4
Diagnostic function of PLATO
1
1,5
2
2,5
3
3,5
4
IV Summing up:
• Strong commonalities across the manuals
• Review of literature: Teaching quality linked to:
Instructional Clarity
Cognitive challenge
Discourse features
Supportive climate
Coding manuals as instruments for
instructional improvement
Observation Instruments:
Provide a common technical vocabulary for describing instruction
Decompose teaching into different components
Develop instruments that can travel (SIT)
Allow for comparison of quality of instruction across
classrooms
