FOSTERING AND ASSESSING CREATIVITY AND CRITICAL THINKING ... … · – Professional development plans for faculty – Development and testing of monitoring instruments Medium term

FOSTERING AND ASSESSING CREATIVITY AND

CRITICAL THINKING IN EDUCATION:

HIGHER EDUCATION STRAND

Stéphan Vincent-Lancrin

Centre for Educational Research and Innovation (CERI) OECD Directorate for Education and Skills

Framework and objectives of the project

3

Skills and education for innovation « 21st Century Skills»

Innovation

Skills

Education and

training

What skills should education systems foster?

Technical skills Know-what and know-how

Behavioural and social skills

Self-confidence, energy, perseverance, passion,

leadership, collaboration, communication

Skills in thinking and creativity Creativity, critical thinking, observation, imagination, curiosity, ability to make connections, metacognition...

A model of change based on evidence

Scale up

5

Prototype Cycles

Development: • Rubrics • Pedagogical activities • Professional development plans • Evaluation instruments

Development/Pilot phase (K-12: 2015-18) (HE: 2018-2021

Validation phase (K-12: 2019-22)

Analysis &

Quick Design

Scale-up phase (country-led)

Systemic change: • Primary education • Secondary education • Tertiary education • Teacher training

Validation

Round 1 (2015-16): Brazil, France, India, Hungary, Netherlands, Russia, Slovakia, Thailand, United States

Round 2 (2016-17): Brazil, France, India, Hungary, Russia, Spain, Thailand, Wales (UK), United States

Primary and secondary education: Teams in 11 countries, with about 17,000 students, 650 teachers and 330 schools participating

Need for a common language and shared understanding of what these skills actually mean, as the basis for guidance to promote them

Skills that are not assessed are not taught consistently. To assess is to give salience, but teachers need to be able to teach what they assess

There is generally little space for students to develop and demonstrate creativity and critical thinking as part of disciplinary learning. Formal acknowledgement in curricula is not enough

Start a process of change: pilot, prototype and develop pedagogical resources as a proof of concept for other teachers – before validation and possibly scale up

Start a systemic process of a change: try to align primary and secondary education, tertiary education, teacher education

Motivation

Can we articulate a common language around these skills, useful for teachers, accepted internationally and aligned with the work in schooling?

Can we develop an exemplary bank of pedagogical resources to show what it means to teach and assess creativity and critical thinking as part of existing curricula?

Can we identify a development/progression scale for these skills?

What are the key contextual factors that matter for their development?

What are the effects of using the developed pedagogical rubric and toolkit on teachers’ practices and beliefs, and on students’ attitudes, academic achievement, and standardised measures of creativity?

Policy research questions

Higher education strand: proposal and next steps

Intervention: Use of a rubric on creativity and critical thinking to improve teaching practice (develop exemplary pedagogical activities, assessments, and teacher professional development plans)

– Common objective, but pedagogical freedom

Domains of intervention (to be defined with participants): – Teacher education – Professional domain (Engineering, Business) – Academic domain – Vocational domain

Student groups

– Undergraduate students (for non-teacher education) – Large introductory courses and/or interdisciplinary projects

aiming to address global challenges

A controlled pedagogical intervention to improve teaching in higher education

OECD rubric on creativity and critical thinking (comprehensive version)

CREATIVITY

Coming up with new ideas and solutions

CRITICAL THINKING

Questioning and evaluating ideas and solutions

INQUIRING

IMAGINING

DOING

REFLECTING

• Empathise, observe, describe relevant experience and information

• Explore ideas

• Understand context/frame and boundaries of the problem

• Challenge assumptions, check accuracy, analyse gaps in knowledge

• Make connections, integrate other disciplinary perspectives

• Stretch and play with unusual/risky/radical ideas

• Identify strengths and weaknesses of evidence, arguments and claims

• Find and compare different perspectives on the problem

• Produce, perform or envision something that is personally novel

• Propose own product/opinion justified on logical, ethical or aesthetic criteria

• Assess the novelty of solution and of possible consequences

• Acknowledge uncertainty/limits of chosen solution/position

To design new pedagogical activities

To improve existing pedagogical activities

To develop new rubrics (domain-specific, expanded, simplified, for self-assessment, etc.)

To assess student work

To keep in mind the importance of these competences and teach/learn them explicitly (metacognition)

Uses of the rubrics

A pedagogical toolkit

A. The toolkit Rubrics for assessment

Dimensions

Levels of progression

Pedagogical activities

Specific for each domain

Designed to test

dimensions and levels of progression

Set of exercises

Specific for each domain

Designed to prepare for the

assessment

Portfolio of student work

For each domain

As examples of different skill levels

To show what teachers do (or could do) to create space for the learning of creative and critical thinking

Different types of pedagogical activities Different domains

Different lengths

Different types of pedagogies or pedagogical techniques

They are not lesson plans: just meant to inspire teachers and make visible the kind of teaching and learning environment that can stimulate students’ creativity and critical thinking

They complement professional development

Pedagogical activities

ACADEMIC YEAR

Teaching and assessment of

C&CT skills

Data collection POST:

• Subject-specific tests • Questionnaires

Data collection PRE:

• Subject-specific tests • Questionnaires

INTERVENTION GROUPS: USE OF RUBRIC + CONTEXTUAL DATA


C&CT skills


C&CT skills


C&CT skills

CONTROL GROUPS: OTHER INTERVENTION + CONTEXTUAL DATA

The evaluation framework for the intervention

Qualitative MONITORING:

• Interviews • Focus groups

Participation process for countries/institutions – Country nominations – Open call to participate (and possible selection)

Types of institutions

– Ideally, at least 3 institutions per country – Preferably different types of institutions to initiate systemic change

(research-intensive/prestigious HEI; average HEI; short 2-year cycle)

Length of the project: 4 years (2018-21)

Expected sample size per domain and country (negotiable): • 500 students in the intervention group, 10 faculty • 1000 students in the control group

A controlled pedagogical intervention to enhance teaching excellence in higher education

What remains to be defined based on interest and feasibility

Domains of the interventions – 2 types of activities per institution?

• Revisit teaching of an undergraduate course in 1 or 2 domains • Develop a project-based activity

– Teacher education

• Do the same as in other domains • Design a joint teacher education programme (and then implement it in

the different countries)

How will we measure the effects on achievement? – Use some standardised tests of creativity, critical thinking and knowledge? – Develop common standards on how to evaluate them in different fields?


What will be common to all participants – All will have the same rubric as a starting point and the same

objective – All will use the OECD instruments to measure the effects of the

interventions and follow the same research protocol – All will document what they do through the same templates

What will be different across participants – They will develop and implement their own projects and

pedagogies – They will develop and implement their own professional

development plans

18


Budget – Each institution to cover the domestic costs of intervention and data

collection, and participation in international meetings (twice a year) – Expected contribution of participating institutions to international

costs will depend on number of domains and institutions (likely around EUR 10-15k per year depending on institution size – EUR 40-60k for 4 years)

– We can ask your ministries if they would be interested in contributing to the international costs if you request it explicitly

Benefits for participating institutions/countries

– Institutional development and improvement of teaching and learning – Professional development of involved faculty (and research output for

coordinators) – Participation in an international community of practice – Contribution to the international common good


Immediate benefits for higher education sector – Development of an international community of practice – Rubrics to design courses and assess students – Bank of pedagogical resources and examples – Professional development plans for faculty – Development and testing of monitoring instruments

Medium term benefits for higher education sector

– Review of university entrance exams – Development of examples of summative assessments on creativity

and critical thinking – Better understanding of competency-based education – Stronger culture of evidence-based innovation processes


What do institutions (and faculty) have to do?

Use the rubric and other pedagogical resources to – Take part in an induction session about the project and the rubric – Develop or improve lesson plans, pedagogical projects or activities fostering

creativity and critical thinking (or implement suggested pedagogical projects or activities )

– Add creativity and critical thinking in the asssessment of their student work – Document and share some of their lessons or pedagogical activities/projects

Give feedback on – Rubrics and their use – Pedagogical activities developed by other institutions – Student work of other institutions

Participate in monitoring work – Answer twice 6-page questionnaires (pre- and post-) – Answer quick questionnaire on use of rubric (intentisity of implementation) – Possibly take part in a focus group to report on their experience

Data might be collected through a tablet application (or computer)

(tbc)

What faculty (in treatment group) are expected to do

Liaise with OECD Secretariat – Participate in webinars and 2 yearly face-to-face meetings in Paris – Give feedback on proposed pedagogical materials – Give feedback on monitoring instruments – Liaise with other relevant participating institutions as appropriate

Support participating faculty in using the pedagogical resources – Organise an induction session about the project and the rubric – Meet participating faculty several times – Support documentation of pedagogical work in OECD template

Coordinate the collection of observational data

– Fill in a control file provided by OECD Secretariat – Administer the pre- and post-questionnaires to students and faculty – Administer pre- and post- tests to students – Have focus groups and interviews with some faculty and students – Observe some pedagogical activities – Write a summary report of field work as input to final international report

What coordinators are expected to do

24

Timeline of the project

2018 • February-June: recruitment of institutions • September: 1st face-to-face meeting in Paris • June-December: design of intervention protocol

2019 • January-February: Start of first round of field work • August-October: Start of second round of field work

2020 • January-June: Continuation of the second round of field

work • June-December: Analysis of the results of the intervention

2021 • Completion of the analysis of the results • Publication of international report and dissemination of

resources produced during the project

• February-June 2018: – Contacting and enrolling institutions/countries – Identifying what interested institutions would like to do

• May-June 2018:

– Commitment of enough participating institutions/countries to start – Design of intervention and participation protocol

• June-September 2018:

– Design of research protocol

• 6-7 September 2018: – First face-to-face meeting – Discussion of design and first decisions on what we will do

• September-December 2018:

– Finalisation of research protocol and first survey instruments – Start of first round of field work asap

25

Timeline in the next few months

where are we at with interest?

• Expressions of interest: 80 institutions (29 countries)

• Final responses to questionnaire: 39 institutions (17 countries)

27

Interests of institutions to participate in the project

Country Expressions of interest

Responses to questionnaire

Australia 1 2 Austria 1 0 Belgium 7 2

Brazil 1 0 Canada 6 4

Chile 2 0 China 7 5

Colombia 1 0 Finland 2 1 France 3 0 India 1 0



Ireland 2 5 Israel 2 0 Italy 2 1

Japan 3 1 Korea 1 0 Latvia 2 1 Mexico 5 1

Netherlands 2 1 Norway 1 0 Portugal 11 6 Romania 3 3



Russia 1 1 Slovakia 1 1

Spain 3 1 Thailand 2 0 Turkey 3 3 United

Kingdom 3 0 United States 1 0

Total 80 39

28

Interests of institutions for the strands of the project

18

11

10

Number of interested institutions for the two strands of the project

Both

Undergraduate highereducationTeacher education

Based on questionnaire responses to the question: Are you interested in participating in the teacher education strand, undergraduate higher education strand, or both?

29

Interests of institutions for the domains of intervention in the higher education strand

12

8 7

6 6 5

4 4 4 3 3

2 2 2 2 1 1 1 1 1 1 1

0

2

4

6

8

10

12

14

Number of occurrences of domains, disciplines or fields of intervention suggested by responding institutions

• How to deal with diversity of disciplines?

– Examples of domain-specific pedagogical projects

– Define a pedagogical project that can be adapted

by all institutions in all different domains

– Design collaboratively a common project that can

be taught the second year

30

Proposals for higher education strand

• How to take advantage of homogeneity?

– Examples of pedagogical activities

– Examples of pedagogical projects

– Design collaboratively a common course

• First year: joint design of a course in some specific

teacher education domain • Second year: Contextualised implementation across the

network

31

Proposals for teacher education strand

questions

Do you have clarification questions? What design suggestions do you have to make the project

(even) more interesting to your institutions? What domains would you like to participate in (if relevant)?

What would be your preferred approach in terms of

pedagogical activities? – Working on some undergraduate courses? Developing a separate

pedagogical project involving the development of students’ creative and critical thinking skills?

How soon can you commit (also financially)? How soon could you realistically start in the field?


Documents

FOSTERING AND ASSESSING CREATIVITY AND CRITICAL THINKING ... … · – Professional development plans for faculty – Development and testing of monitoring instruments Medium term