Evaluación ExternaInstituto Franklin-UAH

Dr. Julio Cañero

Marta Walliser, PhD.

1. Welcome

Framework for the evaluation: Spain • Orden 9932/2012

• Decreto 126/2012 (2 de diciembre 2013)

• Disposición Tercera establece:

• Los centros presentarán alumnos de 2º y 4º EPO.

• Los centros realizarán la evaluación en colaboración con una entidad externa de su elección y de acuerdo con un procedimiento supervisado por dicha entidad.

• Las entidades evaluadoras deberán ser entidades reconocidas por su experiencia en la enseñanza de idiomas.

• Las pruebas que realicen estas entidades deberán ser adecuadas a la edad de los alumnos y conformes a los niveles lingüísticos acordes con el Marco Común Europeo de Referencia para las lenguas.

• Se evaluarán las destrezas orales: Listening & Speaking. (Anexo 1)

Marco de la Evaluación • En su punto 3 se explicitan las excepciones:

• Alumnos que se hayan incorporado tardíamente al programa bilingüe.

• Alumnos de compensación educativa con adaptación curricular significativa.

• Alumnos con nnee o alguna discapacidad que dificulte la realización de las pruebas.

• Otros alumnos que el centro, de manera razonada y justificada, considere que NO reúnen las condiciones para la realización de la prueba. (Anexo III)

• En el 4 se describe el procedimiento de la evaluación :

• Durante el 2º trimestre los alumnos comunicarán a la DG de Becas el nombre de la Entidad Externa (IF-UAH) así como los alumnos de 2º y 4º curso que van a examinarse, igualmente se debe de comunicar aquellos alumnos que quedan exentos de la prueba (punto 3). La prueba se realizará durante el tercer trimestre del curso escolar, preferentemente durante el mes de Mayo La comunicación se remitirá en el Anexo II. (Nota: en esta presentación únicamente nos referimos a los alumnos que serán examinados por el Instituto Franklin-UAH).

• En el 5 indica la comunicación de los resultados:

• Los resultados de las pruebas se remitirán a la DG de Becas antes del comienzo de las actividades lectivas del siguiente curso escolar, conforme a los Anexos IV-A y IV-B.

• Los anexos deberán ser cumplimentados en soporte papel y certificados por el centro y la entidad evaluadora. Adicionalmente los anexo se remitirán en fichero de hoja de cálculo formato.xls a la subdirección General de Enseñanza Privada y concertada.

New Opportunities for Assessment Centers

- EU has generated new alternative scernarios for the development of new evaluation and accreditation centers.

- CERCLES (European Confederation of Languages Centres in Higher Education)

- ACLES (Asociación de Centros de Lenguas en la Enseñanza Superior.)

Our concerns

CERF FrameworkCommunicative Methodology

Testing of Young Learners.

CEFR Instrument

• Common European Framework of Reference for Languages (CEFR) provide an excellent common reference cognition and reflection. State outcomes that should drive teaching and doing ….

• “I CAN…..”

Language Assessment vs CLIL

Language assessment only evaluates language learning

A1 descriptorsSegún el Consejo de Europa, el Marco Común Europeo de Referencia para las Lenguas define las destrezas orales para el nivel A1 de la siguiente manera.

• El nivel A1 es un nivel básico en el que el alumno adquiere aprendizajes básicos del idioma, puede entender y expresar ideas concretas.• Enmarcado por unas situaciones concretas que el alumno llega a dominar y una habla controlada. • El alumno puede participar en conversaciones de forma básica siempre y cuando los otros participantes están dispuestos a hablar de forma lenta y clara,

repitiendo lo que ha dicho si es necesario. • El alumno necesita tiempo para entender y para hablar. • Reconoce formas gramaticales y fonéticas, pero puede no reproducirlas correctamente. • Se hace entender y es capaz de continuar una conversación básica. • Utiliza expresiones sencillas y personales.

• Vocabulario: • Tiene un vocabulario básico que describe su entorno más cercano.• Vocabulario referente a su familia, colegio, amigos, gustos, comidas, el tiempo o números (1 al 20)

• Gramática• Tiene la capacidad de hacer frases.• Puede hacer descripciones básicas

• Fluidez • Puede articular frases con dificultad,• El discurso se interrumpe para pensar en la idea que quiere expresar,• El discurso tiende a ser entrecortado.

• Comunicación: • Puede hacer frases. • Tiende a incorporar palabras en español cuando habla / preguntar vocabulario en español ¿cómo se dice ….. En ingles? • Puede comunicarse siempre que su interlocutor hable despacio y con claridad y esté dispuesto a cooperar.

A2 Descriptors• Según el Consejo de Europa, el Marco Común Europeo de Referencia para las Lenguas define las destrezas orales para el nivel A2 de la siguiente manera. Aunque el alumno ya maneja estructuras y vocabulario básicos, sigue sus capacidades siguen regidos por situaciones concretas y cotidianas. El alumno empieza a coger soltura con el idioma. Ya puede

tener una comunicación básica si la otra parte se esfuerza para ayudar mantener la conversación. Interactúa con otros miembros de una conversación. También empieza a entender ideas menos concretas y/o sobre situaciones menos conocidas. El alumno ya domina ciertas estructuras y los puede usar sin tender que detenerse. Se hace entender con conversaciones directas y sencillas.

• El nivel AV2 es el segundo nivel de la escala MCER. Siendo un nivel básico el alumno refuerza y consolida los conocimientos adquiridos anteriormente, ampliando su capacidad de comprensión, expresión , fluidez y comunicación. un nivel básico en el que el alumno adquiere aprendizajes básicos del idioma, puede entender y expresar ideas concretas.

• El alumno domina y expresa correctamente situaciones y contextos familiares. Enmarcado por unas situaciones concretas que el alumno llega a dominar y una habla controlada. • El alumno puede participar en conversaciones aportando detalles y descripciones concretas. • El alumno participa en las conversaciones de forma significativa incorporando ideas y detalles originales. . • Aumenta la fluidez y concreción en las aportaciones comunicativas. • Conoce y reproduce formas gramaticales y fonéticas. • La calidad de descripciones y comunicaciones son más precisas y detalladas. .

• Vocabulario: • Tiene un vocabulario básico que describe su entorno más cercano.• Vocabulario referente a su familia, colegio, amigos, gustos, comidas, el tiempo o números (1 al 20)

• Gramática• Tiene la capacidad de hacer frases.• Incorporan descripciones básicas elaboradas.• Se empiezan a consolidar verbos (aunque todavía se comenten errores 3º persona)• Hace referencia temporal.• Incorporación del –ing forms.

• Fluidez • Puede articular frases de forma fluida (a nivel básico)• El discurso fluidez y coherencia, puede ser original.

• Comunicación: • Puede hacer frases básicas, con nivel de detalle. • Se reduce significativamente la interferencia con el español Si tiene alguna pregunta la realiza en inglés, -- How do you say ….. In English? • Puede comunicarse con su interlocutor espontáneamente y sin tanto apoyo.

CEFR

CEFR

CEFR The CEFR: transparent, coherent and comprehensive• The result of over twenty years of research, the Common European Framework of Reference for Languages: Learning, teaching, assessment (CEFR) is

exactly what its title says it is: a framework of reference. It was designed to provide a transparent, coherent and comprehensive basis for the elaboration of language syllabuses and curriculum guidelines, the design of teaching and learning materials, and the assessment of foreign language proficiency. It is used in Europe but also in other continents and is now available in 39 languages.

• Six levels of foreign language proficiency • The CEFR describes foreign language proficiency at six levels: A1 and A2, B1 and B2, C1 and C2. It also defines three ‘plus’ levels (A2+, B1+, B2+).

Based on empirical research and widespread consultation, this scheme makes it possible to compare tests and examinations across languages and national boundaries (see the section “The CEFR and language examinations: a toolkit”). It also provides a basis for recognizing language qualifications and thus facilitating educational and occupational mobility.

• The CEFR’s illustrative scales of “can do” descriptors are available in a bank of descriptions together with many other related descriptors. • The CEFR is much more than proficiency scales• The CEFR’s scales of foreign language proficiency are accompanied by a detailed analysis of communicative contexts, themes, tasks and purposes as

well as scaled descriptions of the competences on which we draw when we communicate. This helps to explain why the CEFR is increasingly used in teacher education, the reform of foreign language curricula and the development of teaching materials (in this connection see the results of a survey carried out in 2006 among Council of Europe member states).

• Using the CEFR in specific contexts• The CEFR does not offer ready-made solutions but must always be adapted to the requirements of particular contexts, for example, the teaching

and learning of Romani and of French Sign Language. The need for careful interpretation and adaptation is especially acute when the CEFR’s descriptive apparatus and proficiency levels are used to explore the communicative needs of adult migrants and to guide the assessment of their proficiency in the language of their host community (see the relevant studies)

Communicative Methodology

Communicative language teaching (CLT), or the communicative approach, is an approach to language teaching that emphasizes interaction as both the means and the ultimate goal of study

Communicative Methodology Principle Application

Authentic materials are used Teacher uses newspaper columns, job advertisements, weather reports, menus, catalogues…

One function can have many different forms Students can communicate for a specific purpose in many ways

Students need to learn cohesion and coherence Teacher uses activities such as scrambled sentences

Students should be given the opportunity to express their opinions Games are useful as are activities where students must communicate and receive feedback (did the listener/reader understand?)

Errors are tolerated to a certain extent Other students and teacher ignore errorsEncourage cooperative relationships among students; opportunity to negotiate meaning

Teacher uses strip stories; students work together to predict next picture

The social context of the communicative is essential Teacher uses role-plays

Learning to use language forms appropriately is important Teacher reminds student of the role they are playing or the particular situation they are in and how that impacts on the communication; teacher encourages students to develop independent learning skills

Teacher acts as advisor/facilitator Teacher moves from group to group, offering advice and answering questions; teacher collaborates with students to select goals, content and processes

Speakers have choices in communication of what to say and how to say it Students and teacher suggest alternative forms that could be used

Students should be given opportunities to develop strategies for interpreting language as it is actually used by native speakers

For homework, students are asked to listen to a debate on the radio or watch one on television

Testing Young Learners

- Ages 6 to 12 (covers primary school in many contexts). This is the least visited and researched age group of testing.

- There is not specific material for Testing YL and for teaching YL - Most assessment materials have demonstrated to be limited, since they have been designed for second language student, not students in

bilingual English schools.

Testing Young Learners

Child & Age Development (Piaget)Abstract thinking

Capacity to articulate Capacity to justify and back up opinions.

Testing Young Learners

- Child development - Skills in mother tongue- Goals of the test - What do we test in young learners

ISSUES WITH CEFR

• CEFR is not specifically intended for children or young learners.• Materials are not designed following the communicative approach. • It is difficult for schools to decide what students should take (A1 and A2). • Levels and student development has to be contemplated and adapted to

the current descriptions of the CEFR

Test vs Assessment

• ASSESSMENT: umbrella term covering any systematic means of discovery and record how well someone is able to do something.

• TEST: Formal evaluation, students working and being checked for a specific use of language.

What is assessment for?

- Should enable teachers to notice patterns of learning.- Be aware of what the students CAN DO.

- Support the shaping of metacognitive skills- Get information as to what to expect from learners

Role of the examiner

Tell me what you see?

Student response

Role of the examiner

Role of the examiner

Role of the examiner

Role of the examiner

Benjie & Frankie 2015

•BENJIE (A1)

•FRANKIE (A2)

Sources of information

• European Accreditation Models (ACLES)• British Council • ALTE (Association of Language Testes in Europe) • Consultations with Assesment Experts of Partner Institutions• Prior experience of the Assessment Process• Experience of Schools, Teachers and Language Assistants.

BENJIE (A1)

Similar Structure of the Exercises

• LISTENING PART (25 pts)

Match the names (5 pts)Answer the questions (7 pts) Match the right picture (7 pts) Listen, color and Draw (6 pts)

• SPEAKING PART (25 pts)

Describe the pictures (5 pts)Differences (7 pts) Hoobies and daily routines (7 pts) Conversation on diverse familiar

topics (6 pts)

FRANKIE (A1)

Similar Structure of the Exercises

• LISTENING PART (25 pts)

Match the names (5 pts)Answer the questions (7 pts) Match the right picture (7 pts) Listen, color and Draw (6 pts)

• SPEAKING PART (25 pts)

Describe the pictures (5 pts)Differences (7 pts) Hoobies and daily routines (7 pts) Conversation on diverse familiar

topics (6 pts)

Issues we are working on

• Test Design: we are working on integrating communicative methodology in the test design.

• Generating materials for Test preparation • Working with the role of the Examiners for student evaluation success • Gathering Data as to how integrate the test results into the school as

a whole.

Issues to be considered

• We are still working on the data and development instruments to be able to have a better interpretation of the results.

• Speaking 85% • Listening 95%

Result analysis

• We can conclude that students do very well on the listening part of the test for different reasons.

• We can conclude that students do well on the speaking part of the test as well.

• The truth is that is difficult to have a full analysis of the results because we still do not have enough comparative data.

• The important issue is: What are we going to do this data? How are we going to integrate this information in our schools?

Issues to be considered

• Different profile of schools • Test taking experience of students and teachers• Test taking preparation • Context of school and school culture on assessment • Issues of discipline• Attitude of the teachers and administration towards the assessment

itself.

On top of that

A British Council Survey reveals a depressing but not surprising use of the information revealed by the results:

• 1) they are not used productively in the school• 2) the information does not go beyond the quantitative results

Do schools take into consideration Language Assessment results when evaluating students?

• Never 27• Rarely 16 • Sometimes 10• Quite often 3• Often 2• Always 2

Recommendations for the future

We should work on test results from an institutional perspective, not only as a way to measure the students performance.

Bilingual Model in Spain

Faculty Development

CLIL

Teaching Assistants

Sistemic Approach

• How and who should do this? • What is our role in this systemic process? • How do analyze our data? • Do we want to do it?• What for?

Questions for the future?

Thank you for your time and attention