View
215
Download
1
Category
Preview:
Citation preview
Assessment and Teaching of 21st Century Skills
Claire Scoular
Assessment Research Centre
University of Melbourne
2nd April 2014
San Jose, Costa Rica
2
Outline
1. Background of ATC21S
2. ATC21S: The Latin American Chapter
3. Costa Rican results and outcomes
4. The future of the project in Costa Rica
SECTION 1 : Background of ATC21S
4
CHANGE BEGINS WITH GLOBAL PARTNERSHIP
ATC21S is a joint effort to mobilize international, educational, governmental and business communities to bring 21st-century skills into the classroom.
5
ATC21S GOAL
To develop new assessment approaches matched to new 21C skills and to advise systems, schools, and teachers on the uses of assessment data to help students develop higher order performances
6
ATC21S PROCESS
Phase 1
Conceptualize C21 skills and
education output needs
Phase 2
Skill identification
and hypotheses
Phase 3
Task development
Phase 4
Pilot studies and trials
Phase 5
Dissemination scale and policy
formation
SECTION 1 : Phase 1Conceptualize C21 Skills
8
PREPARING STUDENTS TO SUCCEED IN THE 21ST CENTURY
• Changes in employment trends
• Increase in workforce by 2050
• For Latin America expected increase of 26% (Wang)
9
PREPARING STUDENTS TO SUCCEED IN THE 21ST CENTURY
e are surrounded by technology, and kids are digital natives. They go into the classroom, and they don’t see the same use of technology in their education as they might in their outside lives. If education doesn’t start to address that issue, there will be a disconnect in what kids see in their lives growing up and when they get out into the workplace, which is one of the motivations for us to join this project.”
• — Michael Timms, National Project Manager ATC21S for USA
“We are surrounded by technology, and kids are digital natives. They go into the classroom, and they don’t see the same use of technology in their education as they might in their outside lives. If education doesn’t start to address that issue, there will be a disconnect in what kids see in their lives growing up and when they get out into the workplace, which is one of the motivations for us to join this project.”
— Michael Timms, National Project Manager ATC21S for USA
10
TRANSFORM EDUCATION FOR THE 21ST
CENTURY
11
PHASE 1: CONCEPTUALIZING THE SKILLS
Assemble experts
Define methods
Explore practical and technical needs in the
classroom
Create a new framework
12
SECTION 1 : Phase 2Theoretical and Conceptual Framework
14
SELECTED SKILLS
WAYS OF WORKING
AND THINKING
Collaborative problem solving
TOOLS FOR WORKING AND
LIVING IN THE WORLD
ICT digital literacy
Personal and social responsibility through social networking
15
PROCESS OF ASSESSMENT DEVELOPMENT
Formulation of tasks
Teacher view Cognitive laboratory Pilot Trial
Formulation of developmental continua
Identify the dimension/s Characterise the evidence Draft three profile levels
Definition of constructs
Collaborative Problem-solving ICT Literacy
16
COLLABORATIVE PROBLEM SOLVING: Definition
16
Collaborative problem solving as a 21st Century Skill may be seen as a combination of its component parts. These are collaboration, problem solving and the new technologies currently believed to characterise the 21st century
• Collaboration – working together with a common goal
– Communication– Cooperation
• Problem solving– Knowledge – Strategies
17
Purpose, functions and outcomes
• Why?– When different expertise, knowledge or skills are
both needed and available
• What?– Formulate and express ideas; argue and defend
– Value, review, evaluate
• Outcomes– Problem solution
– Enhanced skills and knowledge
18
COLLABORATIVE PROBLEM SOLVING FRAMEWORK
18
Collaborative problem solving
Social
ParticipationPerspective
takingSocial
regulation
Cognitive
Task regulation
Knowledge buiilding
19
Social skills strands
Participation
Action
Interaction
Task completion
Perspective taking
Adaptive responsiveness
Audience awareness
Social regulation
Negotiation
Self evaluation
Transactive memory
Responsibility initiative
20
Cognitive skills strands
Task regulation
Resource management
Information collection
Systematicity
Ambiguity tolerance
Organisation
Goal setting
Knowledge building
Knowledge acquisition
Represents relationships
Identifies consequences
Hypothesises
21
ICT Literacy
Intellectual Capital
Social Capital
Producer
Consumer
SECTION 1 : Phase 3Task Development
23
Process
Theoretical Framework
Task Design Score
Interpret
&
Report
Intervention
24
25
Laughing Clowns
25
26
Indicative Behaviours
Skill Behaviour in task
Interaction- Chatting with partner
Presence of chat before allowing partner to make a move
Audience awareness- Adapts contributions to increase
understanding for partner
Number of ball moves attempted before stopping and waiting for partner to respond
Reponsibility initiative- Takes responsibility for progress for the
group task
Communicating with partner before first half of balls have been used
Resource management Realises that balls are meant to be shared and only uses allocated half
Relationships - Identifies patterns between information
Both students coming to agreement on how their machine works
27
Balance Beam
28
Indicative Behaviours
Skill Behaviour in task
Action- Active in scaffolded tasks
Student A passes B a mass
Task completion – undertaking part of task individually
Follows instructions, moves 100g to position 4
Responsiveness – responding to contributions of others
Realising that some masses cannot balance. If Student A resends 50 or 500, B returns it immediately
Systematicity- Implements possible solutions to a problem
Trying out all combinations of masses on different beam positions
Solution - Correct answer
Number of successful balances achieved (3 optimum)
29
Skill progressions
SOCIAL
LEVELS PARTICIPATION
Level 4 The student initiates and promotes interaction with their partner; sharing resources,
information and ideas.
The student perseveres in solving the task as shown by repeated attempts and/or
multiples strategies
Level 3 The student is actively participating in the task in familiar and unfamiliar contexts.
The student interacts with their partner more (i.e. responds to communication cues
and requests) and shares information and/or resources with partner.
Level 2 The student is actively participating in the task when it is familiar and/or scaffolded.
There is limited interaction with partner and this is mainly prompted by task
instructions.
Level 1 The student participates in the task but is working quite independently
Task activity is usually directed by system instructions
30
Reporting to Students: Learning Readiness
Real time reports based on empirically developed learning progressions. They will identify a point of readiness to learn for each student.
This type of report can be linked to teaching interventions associated with readiness to learn as indicated by the black bar in the spine of the chart.
SECTION 1 : Phase 4Pilot studies and trials
SECTION 1 : Phase 5Dissemination & Policy Formation
33
34
Dissemination
Resources will now be placed in the public domain
Atc21s.org
35
Trialing Tools
Concept Checking
Cognitive Laboratories
Pilots
Trials
Teacher pedagogy workshops
SECTION 2 : THE LATIN AMERICAN CHAPTER
37
Objectives and Benefits to Costa Rica
• Validate the assessment through pilot and trials
• Results could continue the agenda and lead Latin America in 21st Century skills
• Contributing to large international project
• Gain ideas about future implementation of 21C skills
• Evaluate curriculum
• Regional dialogues with peers as well as feedback from stakeholders in the global initiative
38
ASSESSMENT OF COMPETENCIES OF THE 21ST
CENTURY
Three exemplary contributions:
– Language adaptions
– Customization of ICT Literacy Tasks
– Localized website
39
LANGUAGE ADAPTIONS
40
LANGUAGE ADAPTIONS
41
Componente cognitivo
Descripciones de nivelNivel 5 Resolución refinada de problemas
En este nivel, los estudiantes pueden reconocer errores previos y transferir y usar ese conocimiento en las secciones más complejas de las pruebas. Reconocen qué recursos han causado errores previos y los evitan. Están desarrollando confianza en el manejo de las pruebas y esto les permite corregir o superar las respuestas/acciones de sus parejas. Los estudiantes comparten con sus parejas recursos apropiados y usan estrategias consistentes que han identificado como efectivas en pasos anteriores de la solución del problema. Todo el planeamiento y la mayoría de las estrategias implementadas se realizan dentro del contexto del equipo.
Nivel 4 Trabajo eficiente
En este nivel, los estudiantes comienzan a resolver problemas de manera colaborativa y a usar estrategias de cooperación más efectivas. En este momento pueden simplificar el problema y revisar varias veces la ejecución de secciones previas de las pruebas para determinar si la regla es transferible a las páginas o secciones siguientes. En este nivel, el intercambio de información entre estudiantes incrementa y se hace más eficiente. Su secuencia y el juicio y la error acciones requieren pocos intentos y por lo tanto se concluyó en una cantidad óptima de tiempo / intento. Las completan en un tiempo óptimo. Las acciones parecen estar bien pensadas y planeadas, y cada acción parece tener un propósito. Pueden identificar causas y efectos y usar estrategias adecuadas para lograr una vía de solución correcta. El estudiante dirige y guía a su pareja hacia acciones y recursos apropiados.
Nivel 3 Planeamiento y ejecución estratégicos
En este nivel, los estudiantes pueden completar tareas de menor dificultad de manera correcta e independiente. Son capaces de completar el último paso o sección y ayudar a su pareja en este proceso. Incrementan su trabajo juntos al planear estrategias de trabajo, establecer objetivos y compartir sus recursos. El estudiante tiende a enfocarse en los recursos relevantes y desecha otros que no fueron beneficiosos en intentos anteriores. Son capaces de llegar a respuestas de común acuerdo cuando esto es necesario. En pruebas con varias secciones el estudiante tiende a identificar estrategias y recursos exitosos y aplicar las mismas reglas repetidamente. El estudiante continúa intensificando la investigación secuencial y el comportamiento sistemático en las siguientes secciones de mayor dificultad. Esto lleva a intentos por probar hipótesis junto con
LANGUAGE ADAPTIONS
42
ICT TASK CUSTOMIZATION
Poetry
43
Antarctic Trek
ICT TASK CUSTOMIZATION
44
LOCALIZED WEBSITE
45
Trialing the assessment tools
Trials 2011 (data collection CPS)
16 elementary and high schools
90 11-year-old students, 222 13-year-olds, and 188 15-year-olds
Trials 2013 (ICT and system testing)
9 high schools
776 students
SECTION 3 : SPECIFIC COSTA RICAN OUTCOMES
47
ALL COUNTRIES COMBINED
Valid CPS construct
Item Separation reliability 0.999
Person separation reliability 0.814
Regression variable -0.182
48
ALL COUNTRIES COMBINED
Two separate dimensions (social & cognitive)Social Cognitive
Separation reliability 0.998
Person separation reliability
0.754 0.770
Mean Latent Ability 0.560 -0.824
Correlation between dimensions
0.788
49
Summary of data in Costa Rica
• 575 students were scored
• Sample only not representative• Focus is to validate the test not to compare
countries
50
COUNTRY COMPARISON
CostaRica AllotherCountries
MeanofLatentAbility(StDError) -0.121(0.021) 0.145(0.01)
EAP-PVreliability(SimilartoCronbach’salpha) 0.711 0.783
ReliabilityofItemSeparation 0.993 0.999
The psychometric properties of the tasks in Costa Rica were similar as in other countries, indicating that these tasks can be completed in Costa Rica with little to no language, cultural, or country specific bias.
51
Differences in the difficulty of the items were minor or non-existentOut of 150 items only two were more difficult for CR than other countries
Difficulty = Frequency
Could be explained by time constraints
Hexagons - Answering the final answer correctly
Small Pyramids – Responding to questions from partner
52
Sunflower - presence of chat when growth appears
Survey (self assessment) - Replying to chats
Two items were easier for CR than other countries
However, there are many items that assess each skill and these differences are based on single items so they are not representative of overall ability in a specific skill
There is no particular task that is more difficult for Costa Rica than the other countries.
SECTION 4:IMPLICATIONS AND FUTURE DIRECTIONS
54
Now we have…
Share results for teaching
Assessments to test skills
Two skill constructs
55
Now we need…
Developmental assessments
PD for teaching
Resources for teaching
Strategies for teaching
56
Expected to teach subject content and develop 21st
century skills
• assess and intervene in both key learning area and general capabilities
Required to think beyond discipline expertise
• current teacher training mainly discipline-based pedagogy
• most effective methods of teaching skills in one discipline may not be same in another
Domain specific or general curriculum
57
Literacy Numeracy
ICT CapabilityCritical and Creative
Thinking
Personal and Social Capability
Ethical Understanding
Intercultural Understanding
Science
History
Math
English
LEARNING
AREASGENERAL CAPABILITIES
Australian Curriculum and Reporting Authority, ACARA (2013).
EMBEDDED INTO CURRICULUM : Australian Curriculum
58http://www.moe.gov.sg/media/press/2010/03/moe-to-enhance-learning-of-21s.php
EMBEDDED INTO CURRICULUM : Singapore Curriculum
59CONVIVENCIA
VIDA COTIDIANA Y ARTES INDUSTRIALES
SEXUALIDAD
ÉTICA, ESTÉTICA Y CIUDADANÍA PRIMARIA(MÚSICA, ARTES PLÁSTICAS, EDUCACIÓN FÍSICA,
CÍVICA Y ESTUDIOS SOCIALES)
MATEMÁTICAS
ESPAÑOL
ÉTICA, ESTÉTICA Y CIUDADANÍA SECUNDARIA
(MÚSICA, ARTES PLÁSTICAS, CÍVICA, EDUCACIÓN FÍSICA)
EMBEDDED INTO CURRICULUM : Costa Rica
60
SUSTAINABILITY
61
62
63
64
Poetry
65
Antarctic Trek
66
Why engage in the assessments?
Establish a baseline for academic standards
and
Influence curricula at local, regional, national and global levels
Influence curricula at local, regional, national and global levels
Generate new approaches to evaluation and teaching
© Copyright The University of Melbourne 2014
Claire Scoularc.scoular@unimelb.edu.au
Esther Caree.care@unimelb.edu.au
www.unimelb.edu.au/arcwww.atc21s.orgwww.actrc.org
Recommended