Evidence of Quality and Equity in Education

Péter RadóBelgrade, 06.12.2011.

A sample evidence 1.

A sample evidence 2.

A sample evidence 3.

A sample evidence 4.

A few questions:(Assuming that these data are available )

• Is there a demand for these data among the key players of education?

• Do you have access to the information allowing the interpretation of these data? What would it tell about the quality of educational services?

• Do you have the capacity to transform the conclusions drawn from these data to actual design of policies and implementation strategies?

• Can you imagine a minister of education explaining this statistical analysis in a 30 seconds TV interview?

• Can you imagine convincing middle class parents about mixing their children with disadvantaged children on the basis of this analysis?

• Can you imagine convincing teachers about forming heterogeneous classrooms instead of forming easily taught homogeneous classrooms?

• Can you imagine municipalities reorganizing their local school systems in order to ensure equal distribution of disadvantaged students on the basis of this analysis?

Questions emerging from the questions

1. What can generate demand for evidence?

2. How to produce evidence?

3. How to determine quality and equity relevant evidence needs?

4. How to ensure the interpretation of evidence?

5. What are the traps to be avoided?

6. What is the best use of evidence? (Influencing the quality of the policy discourse.)

Generating demand for evidence

From dumb centralization to intelligent post-bureaucratic governance

School management cycle

Inputs Processes Outputs

Managing local school network

Local and regional self-government

Governance of education

Central government agencies

Defining expectations, granting power, deploying procedural mandates

Verifying performance

Generating demand for evidence: governance and management

Making education a complex adaptive system by:

• making schools autonomous learning organizations accountable for the quality of their services and the equity of their outcomes,

• decentralization, deregulation, procedural regulation,

• mandatory multilevel short- and medium term planning,

• Intelligent professional accountability systems with strong focus on educational outcomes produced by whole schools,

• ensuring the openness of decision making at all levels.

Generating demand for evidence: policy-making

The normative approach to EBPM ↔ the very contextual nature of quality and equity →

„Intelligent policy making”

Learning by open deliberation: enrichment of reasoning with the tacit and practical knowledge of practitioners + evidence informing the actors of the policy discourse

Setting the policy agenda(selection and communication)

Planning

Approval(managing policy consultations, preparing decisions, decision

making)

Implementation(management and internal

monitoring)

Evaluation(revision and correction)

Analysing and structuring the

problems

Forecasting, evaluating alternatives, supporting

planning

Counselling

External monitoring

External evaluation

The public policy process Supporting the actors of policy-making

Information production in education 1.

Quality evaluation embedded to a performanace management

system• Connecting goals/targets with

quality evaluation instruments• Student performance assessment

informing (external and internal) school evaluation

• Connecting the pillars of by quality evaluation by an integrated indicator/reporting system

• Strengthening accountability by intervention on the basis of quality evaluation information

• Balancing accountability and organisational learning in schools

Information production in education 2.

The only reliable measure of strengths and weaknesses is international comparison.

→ Enabling information systems for international referencing (EU OMC indicators and benchmark, OECD „Education at a glance” indicators, participation in international student achievement surveys, connecting internal and international assessment systems, etc.)

→ Ensuring the contextual relevance of international comparative information by further analysis and comparative research (See: European educational performance patterns)

Determining quality and equity related information needs

Connecting the results of meta-evaluation with the instruments of quality evaluation

The characteristics of effective schools General effectiveness enhancing factors

Achievement orientation and high expectations Educational leadership Consensus and cohesion among staff Curriculum quality and opportunity to learn School climate Evaluative potential Parental involvement Classroom climate Effective learning time Structured instruction Independent learning Differentiation Reinforcement and feedback

Scheerens et alia. 2003.

Interpretation: research and communication

• Research for deeper understanding: e.g. what makes heterogeneous schools more capable to compensate for disadvantages?

• Typical problem: research substituting information production versus research for the interpretation of information produced on a regular basis.

• Communication: transforming evidence to clear policy messages. → „Simplexity” (Ora Ito) E.g. „Children in schools with heterogeneous intake develop better.”

A few possible traps

• The trap of measurement: what we measure becomes a problem, what we don’t measure remains invisible and ignored

• „Economism”: considering the indicator equal with the indicated fact, policies aiming at „improving indicators” instead of solving problems

• Growing complexity → the fear of loosing control → escaping from complexity by maintaining (or returning to) centralization created simplicity

• Striving for the „optimum” instead of striving for the „possible” (ignoring power, influence, prejudice, whim, etc. by focusing only on evidence)

• Evidence as legitimacy substitute

Finally…

What determines the quality of the policy discourse:the technology and references of policy-making

What has changed in Hungary between 2010 May and July? Politics, nothing else.