Context Quality Evaluation

Fitness for PurposeWhy and How toEvaluate Doctoral programsA Swiss experience

Jacques Lanarès

Context Quality Evaluation

Evaluation of Doctoral Programs •Context and Intentions

•Quality approach

•Evaluation of Programs

Context Quality Evaluation

Evaluation of Doctoral Programs •Context and Intentions

•Quality approach

•Evaluation of Programs

Context Quality Evaluation

Inter-university project to•Improve the training

researchers

•Increase cross fertilization

•Help development of doc. prgs

Context Quality Evaluation

Framework• Specifications and Call

• Political and pedagogical

selection

• Financed

• Evaluation required

Context Quality Evaluation

Required caracteristics• Inter-institutional & Inter-disciplinary

• Represents a scientific field• Create a training environment• Training for research

Context Quality Evaluation

Evaluation of Doctoral Programs •Context and Intentions

•Quality approach

•Evaluation of Programs

Context Quality Evaluation

A process to regulate « tensions »

Teacher’s priorities

Student’s Needs

Expectationsprofessional

Fields

Context Quality Evaluation

A process to regulate « tensions »

Teacher’s priorities

Student’s Needs

Expectationsprofessional

fieldssurveys

InvestigationsProfessionalBodies

investigationsprevious students

Context Quality Evaluation

Authorities

StudentsTeachers

Context Quality Evaluation

Authorities

StudentsTeachers

Indicators

Context Quality Evaluation

Authorities

StudentsTeachers

Peers

Context Quality Evaluation

Authorities

StudentsTeachers

Surveys

Context Quality Evaluation

Authorities

StudentsTeachers

Context Quality Evaluation

Quality Culture • Fitness for Purpose

• Responsibility

• Participation

• Reflexivity

• Balance between Autonomy and Accountability

Context Quality Evaluation

Fitness for Purpose

• Project evaluation grid

• Questions of evaluation

Context Quality Evaluation

To try to avoid paternalism

•Self evaluation

•Constructive feedback from colleagues

Context Quality Evaluation

To Encourage Participation

• Involvement various “actors”

• Inter-programs meeting

Context Quality Evaluation

•

Support reflexivity with questions of the canvas / guide

Context Quality Evaluation

Evaluation of Doctoral Programs •Context and Intentions

•Quality approach

•Evaluation of Programs

Context Quality Evaluation

Objectives of Evaluation• Was the program conform to planning ?

• To what extent match students expectations ?

• Is there an added value perceived ?• What can we learn ?

Context Quality Evaluation

Principles•Learning organisation logic

•Financing not linked to

results

•Involvement of all partners

Context Quality Evaluation

Evaluation Process

Working out

“Gathering”

Intermediate

Evaluation

FinalEvaluatio

n

Material

• Final Questionnaire•Evaluation by directors•Counsellors global report

Results GlobalVision

FinalReport

T = 0 T= 18 months T = 24 months

T = 36 months

•Questionnaires

•Canvas

•Meeting

•Distribution

•Treatment

•Self-evaluation

•Meeting share practice

Context Quality Evaluation

Report•Goals•Results (part./object./act.)

•Organisational context•“side” effects•Global evaluation

Context Quality Evaluation

Some results of evaluation• No accurate indicator of effect• Progress observed in thesis work(acceleration)

• Better professional integration & relations

• Increased ability to argue

Context Quality Evaluation

What did we learn ?• Interest of sharing practices

• Usefulness to support the reflexive

process

• Drive to learn not so strong….

• Evaluation culture has to be

developed

Context Quality Evaluation

Experience alone does not lead to learning

Context Quality Evaluation

What did we learn ?• Interest of sharing practices

• Usefulness to support the reflexive

process

• Drive to learn not so strong….

• Evaluation culture has to be

developed

Context Quality Evaluation

What did we learn ?• Interest of sharing practices

• Usefulness to support the reflexive

process

• Drive to learn not so strong….

• Evaluation culture has to be

developed

Context Quality Evaluation

Evaluation Process

Working out

“Gathering”

Intermediate

Evaluation

FinalEvaluatio

n

Material

• Final Questionnaire•Evaluation by directors•Counsellors global report

Results GlobalVision

FinalReport

T = 0 T= 18 months T = 24 months

T = 36 months

•Questionnaires

•Canvas

•Meeting

•Distribution

•Treatment

•Self-evaluation

•Meeting share practice

Context Quality Evaluation

What is important is not the dancer

It is the dance …