Embed Size (px)
Citation preview
OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes
School Evaluation in the
Flemish Community of Belgium
Brussels, 7 December 2011
Claire Shewbridge Analyst, Education and Training Policy Division [email protected]
Structure of the presentation
• The OECD review in general and the OECD review of
school evaluation in the Flemish Community
• Policy trends in the Flemish Community and the analytical
approach used by the OECD review team
• The OECD review team’s assessment of strengths and
challenges in the current approach to school evaluation
• The OECD review team’s policy recommendations to build
on the current approach to school evaluation in the
Flemish Community
• Further information and OECD review outputs
THE REVIEW IN GENERAL AND IN
THE FLEMISH COMMUNITY
The OECD review of School Evaluation in the
Flemish Community of Belgium
OECD reviews of evaluation and assessment: School evaluation in the Flemish Community of Belgium (1)
• The Flemish Community is one of 12 systems being reviewed
– 12 other systems providing analytical reports, i.e. total 24 systems
– Presenting evidence and including stakeholder views
Country review
strand
OECD review visit Country
report
OECD
Report
Australia June 2010 Y Y
Belgium (Fl.) January 2011 Y Y
Chile November 2011
Czech Republic March 2011 Y
Denmark October 2010 Y Y
Luxembourg June 2010
Mexico First quarter 2012
New Zealand August 2010 Y
Norway December 2010 Y Y
Portugal February 2011
Slovak Republic First quarter 2012
Sweden May 2010 Y Y
Analytical
strand
Country
Report
Austria
Belgium (Fr.) Y
Canada
Finland
France
Hungary
Iceland
Ireland
Korea
Netherlands
Poland
Slovenia Y
OECD reviews of evaluation and assessment: School evaluation in the Flemish Community of Belgium (2)
• Focus of international review is on how to best use evaluation and
assessment to improve school outcomes
– Designing a coherent framework for student assessment, teacher
appraisal, school evaluation and system evaluation
– Paying attention to use of results, evaluation capacity, procedures and
implementation
• Background report developed by University of Antwerp Edubron Research
Group and the Ministry of Education and Training
• 6 days of review, including interviews of all major stakeholders and
visits to Brussels, Antwerp, Vilvoorde and Sint-Niklaas
– Seeking a broad cross-section of information and opinions on school
evaluation policies and how to improve these
• OECD review team = Claire Shewbridge and Deborah Nusche (OECD),
Marian Hulshof (Netherlands) and Louise Stoll (UK/Canada)
POLICY TRENDS IN THE
FLEMISH COMMUNITY AND
OECD’S ANALYTICAL APPROACH
The OECD review of School Evaluation in the
Flemish Community of Belgium
Major policy developments influencing
school evaluation in the Flemish Community
• Inspection focus on “output”
• Differentiated inspection to target schools/areas with greater quality concerns (2009)
• Publication of school inspection reports (2004, 2007)
• Schools legally responsible for their quality (2009)
• Consultation platforms within the Policy on Equal Educational Opportunities (GOK) (2002)
Increased focus on school
quality and reducing inequities
• GOK funding evaluation requirements (2002)
• School innovation projects
• Feedback to schools from National assessment
• Participation Decree (2004) ensures stakeholder voice
• Promotion of “school communities”
• Inspection judges school “policy-making capacity”
Stimulating school self-evaluation
Analytical framework for the review of
school evaluation in the Flemish Community
Governance: a framework for school evaluation
Effective
evaluation
procedures
Competencies
for evaluation /
feedback
Use of
evaluation
results
STRENGTHS AND
CHALLENGES
The OECD’s assessment of the current approach
to school evaluation
Governance: a framework for school evaluation
Effective evaluation procedures
Use of evaluation
results
Competencies for evaluation/
feedback
• Schools have the major responsibility for the school improvement process
• The Ministry fosters a degree of common understanding of basic quality
• An increasingly information rich environment for school evaluation
• Availability of robust student assessment tools for primary schools
• Increased focus on the importance of engaging all stakeholders in school
evaluation
• The Ministry does not mandate or steer school self-evaluation
• School evaluation is not well embedded in a larger vision for evaluation
and assessment
• Miss potential synergies between different evaluation approaches
• Insufficient emphasis on improvement/excellence in attainment targets
• High degree of variation in school policy-making capacity
• Lack of information flow impedes school evaluation efforts
Governance: a framework
for school evaluation
• Considerable use of technology to support self-evaluation
• The Context-Input-Process-Output (CIPO) inspection framework is
empirically grounded and comprehensive
• Collection of evidence during the inspection can stimulate school self-
evaluation activities
• There is a legal basis for inspection, including the examination of
school policy-making capacity
• Lack of clarity around purpose of school self-evaluation and minimum
quality
• Inadequate information base for risk assessment prior to inspections
• Communicating school understanding and uptake of the new
inspection methodology
• Judging schools’ implementation of attainment targets
Effective evaluation
procedures
• Recognition of the importance of a school’s policy-making capacity
• Network support to promote and develop school self-evaluation
capacity
• Identified teacher competencies to support school evaluation
• Emerging collegial relations within and between schools to support
competency development
• Efforts to improve the capacity of the Inspectorate to conduct coherent
inspections
• Variation among school leaders in policy-making capacity
• A need to strengthen educators’ evaluation literacy
• Teachers’ orientation to research and enquiry
• Clarifying and making more uniform inspectors’ judgements on quality
Competencies for
evaluation / feedback
• Students have a voice in school policy, including growing involvement
of students in self-evaluation
• Good balance of school responsibility and external pressure to use
inspection results for improvement
• Publication of inspection reports
• Examples of schools using feedback from national and international
assessments in their self-evaluation activities
• Examples of primary schools using network test results as part of
whole-school evaluation
• Lack of strategic and consistent use of self-evaluation
• There is room for improvement in the public use of inspection results
• Schools only receive Inspectorate profile information when being
inspected
• School self-evaluation results not necessarily shared with inspectors
Use of evaluation
results
Some challenges to implementing school
evaluation for continuous improvement
Lack of objective
information to
assess “output”
prior to inspection
Varying levels of
self-evaluation capacity
among schools.
Lack of strategic use of
evaluation results for
school improvement.
Focus on
minimum only.
Hard to judge
how schools
implement them.
Continuous
improvement
Flemish
learning
objectives
Inspection
focus on
output /
differentiation
Schools
responsible
for quality
！ ！ ！
POLICY RECOMMENDATIONS
The OECD’s assessment of how to build on
the current approach to school evaluation
Governance: a framework for school evaluation
Effective evaluation procedures
Use of evaluation
results
Competencies for evaluation/
feedback
Suggested priorities in going forward
Governance: a framework for
school evaluation
Competencies for evaluation/
feedback
Use of evaluation
results
Effective evaluation procedures
Clarify the
goals of school
evaluation and
how different
types of
evaluation fit
together
• Further clarify common goals and expectations with a view to encouraging
excellence and continuous improvement
• Strengthen consistency and coherence of different elements of school
evaluation
– Articulate appropriately with teacher appraisal
– Reinforce links with school leadership appraisal
– Better integration between self-evaluation and inspection
• Promote the use of evaluation and assessment tools by schools for
improvement
– Comparative evaluation of externally developed self-evaluation tools for
schools
– Promote examples of effective practice
• Continue to embed self-evaluation requirements in new policies/programs
– Schools should link this to school quality assurance/improvement plan
– Provide guidelines on how to map this to CIPO inspection framework
Governance: a framework
for school evaluation
Suggested priorities in going forward
Governance: a framework for
school evaluation
Competencies for evaluation/
feedback
Use of evaluation
results
Effective evaluation procedures
Clarify the
goals of school
evaluation and
how different
types of
evaluation fit
together
Increase the
objectivity of
evaluation
procedures and
ensure they promote
improvement and
excellence
• Evaluate potential to develop set of criteria for learning progressions
– Reference to assess student progress in different subjects
• Clarify the criteria for inspection judgements on the quality of
education
– Define clear criteria for all components used to determine quality
– School self-evaluation capacity is a core component of its quality
• Go further in improving the inter-rater reliability of inspection reports
– Concrete rating scale; build on inspection of school groups
• Strengthen the commitment of inspection and schools to the
implementation of Flemish attainment targets
• Extend collegial practice both within and among schools
– Further develop and coordinate emerging critical friendship
• Promote further involvement of students in self-evaluation activities
– Critical feedback for learning, teaching and school improvement
Effective evaluation
procedures
Suggested priorities in going forward
Governance: a framework for
school evaluation
Competencies for evaluation/
feedback
Use of evaluation
results
Effective evaluation procedures
Clarify the
goals of school
evaluation and
how different
types of
evaluation fit
together
Continue to invest in
school leader and
teacher capacity to
conduct evaluation
and use its results
for improvement
Increase the
objectivity of
evaluation
procedures and
ensure they promote
improvement and
excellence
• Further strengthen professional development for effective school self-
evaluation
– Prioritise support to building school policy-making capacity
– Conduct external review of PBD tools and services
• Recognise and strengthen key role of school leaders in self-evaluation
– Flemish framework for leadership competencies
– Develop new leadership roles
– Refine leadership training
– Ministry project on school evaluation competency development
• Increase teacher understanding of policy-making capacity
– Strengthen links between teaching quality and self-evaluation
– Build competencies in data use, research and innovation
– Fund collaborative teacher research projects with cross-school peer
review
Competencies for
evaluation / feedback
Suggested priorities in going forward
Governance: a framework for
school evaluation
Competencies for evaluation/
feedback
Use of evaluation
results
Effective evaluation procedures
Clarify the
goals of school
evaluation and
how different
types of
evaluation fit
together
Continue to invest in
school leader and
teacher capacity to
conduct evaluation
and use its results
for improvement
Increase the
objectivity of
evaluation
procedures and
ensure they promote
improvement and
excellence
Increase the use of
available information
(collected at school
and central level) in
both internal
and external
school evaluation
• Strengthen information flow on key indicators from and to schools
– Provide schools with access to information in Data Warehouse
– Schools to provide performance information to the Inspectorate
– Improve availability and use of objective output measures
• Ensure regular feedback to schools on key CIPO inspection
framework indicators
– Help schools with clear goals and measurable objectives
– Work with schools to build shared language of quality/ evaluation
• Promote the availability and use of appropriate self-evaluation
resources
– Comparative overview of tools available to schools
– Assess school demand for other self-evaluation tools
• Devise ways to improve public use of inspection results
– Simplified language; dedicated website; reports for school groups
Use of evaluation
results
Ensure the quality of teaching and learning
is at the heart of school evaluation
• Increase the use of available information in both internal and external school evaluation
• Increase the objectivity of evaluation procedures and ensure they promote improvement and excellence
• Continue to invest in school leader and teacher capacity to conduct evaluation and use its results for improvement
• Clarify the goals of school evaluation and how different types of evaluation fit together
Improvement
Teacher appraisal
/development
Self-evaluation /
inspection
Support to
schools (PBD)
Leadership
/evaluation
competencies
Collaboration
Learning
progressions
Inspection criteria
Tools for
improvement
School access to
central data
School performance
data to
Inspectorate
Quality of
teaching
and
learning
Outputs available on the Review’s
website:
www.oecd.org/edu/evaluationpolicy
OECD Reviews of Evaluation and
Assessment in Education:
Australia
Denmark
Norway
Sweden
Country Background Reports:
9 out of 24 available
Literature reviews /conceptual
papers including:
formative and summative
assessment; teacher evaluation;
school evaluation; education
standards; use of results...
