Enhancing Effective English Language Assessment in Secondary Schools Enhancing Effective English Language Assessment in Secondary Schools Teacher Conference,

Enhancing Effective English Language Assessment in Secondary Enhancing Effective English Language Assessment in Secondary

Schools Schools Teacher Conference, Hong Kong, April 22nd, 2006

Towards more effective assessment in Hong Kong

secondary schools: Findings from the chalkface

Chris Davison

Faculty of Education, University of Hong Kong

From a testing to an assessment culture

“ Assessment is the practice of collecting evidence of student learning in terms of knowledge, skills, values and attitudes through observation of student behaviour when carrying out tasks, tests, examinations, etc”.(Curriculum Development Council, 2000)

From a testing to an assessment culture Assessment practices in Hong Kong

traditionally aimed to select students for education or employment (Biggs, 1995), but major educational reforms underway, with school-based and standards-referenced assessment being introduced as way to improve learning and teaching (Education Commission, 2000; CDC, 2000; HKEAA 2003).

Official adoption of UK Assessment Reform Group’s distinction between assessment for learning vs assessment of learning


Concerted move to adopt school-based assessment (Stiggins & Conklin 1992; Black & Wiliam 1998; Brookhart 2003):

Integral part of teaching and learning Teacher-mediated, evolving Multi-modal (observation, inquiry, analysis,

test, etc) Co-constructed and dialogic Context-dependent Oriented towards individual improvement

rather than system-wide comparisons

Integral part of teaching and learningIntegral part of teaching and learning

Teacher-mediated and evolvingTeacher-mediated and evolving

Multiple and varied formatsMultiple and varied formats

Multiple and varied formatsMultiple and varied formats

Co-constructed and dialogicCo-constructed and dialogic

Context-Context-dependentdependent

Individually-orientedIndividually-oriented


http://cd.emb.gov.hk/basicguide/BEGuideeng0821/chapter05.html“Based on the beliefs that every student is unique and possesses the ability to learn, and that we should develop their multiple intelligences and potentials … there should be a change in assessment practices and schools should put more emphasis on 'Assessment for Learning' as an integral part of the learning, teaching and assessment cycle …


… In other words, teachers should use assessments (e.g. as simple as effective verbal questioning, observation of student behaviour) and provide immediate feedback to enhance student learning in everyday classroom lessons. The focus is on why they do not learn well and how to help them to improve rather than just to use assessments to find out what knowledge students have learned”


Supported by the development of standards-referenced assessment frameworks and "authentic" alternative school-based assessment activities/tasks embedded within the instructional program, as well as more formal summative school-based assessment (eg. HKCEE 2007).

Such assessment seen as complementing the recent adoption of a more outcomes-oriented English syllabus.


BUT … experience elsewhere (and in Hong Kong with TOC and TAS) demonstrate conclusively that the shift from a "culture of testing" to a "culture of assessment" (Gipps, 1994) is fraught with difficulties - theoretical , socio-cultural and, above all, practical - thus must be carefully monitored, evaluated and supported.

Our research (studies), 2004-?

Aims: To stimulate, support, document and

evaluate the development of effective formative and summative assessment practices in English language teaching, with a particular focus on the HKCE 2005-07.

To identify and describe factors which may facilitate and/or hinder the connection of formative (and summative) assessment and feedback with learning and teaching

Our research (studies), 2004-?

13 interlinked projects, part of a coherent and systematic programme of research and evaluation and professional development, funded by HKEAA, QEF, UGC and the Faculty of Education, HKU, http://web.hku.hk/~sbapro

Data collected from all Form 4 English teachers and students in all HK secondary schools through questionnaires, individual and group interviews, focus groups, classroom observations, video-recordings, self-reflections and retrospective analysis.

The research (studies), 2004-?

Research school-based, co-participatory and action-oriented:“Action research [rejects] the concept of a two-stage process in which research is carried out first by researchers and then in a separate second stage the knowledge generated from the research is applied by practitioners. Instead, the two processes of research and action are integrated” (Somekh, 1995)

Results consistently demonstrate that there needs to be more attention to developing teacher assessment (un)readiness, and to system-level support.

Hall, George, and Rutherford’s (1977) Stages of Concern Model is one of best known individual-oriented models found in the educational literature which can be used to track the concerns of teachers adopting or implementing edcational innovations.

Hall et al. define concern as “the composite representation of the feelings, preoccupation, thought, and consideration given to a particular issue or tasks” (p.5).

The contribution of theory: The stages of concerns model

Two key assumptions are made in the model:

Assumption 1: Teacher concern is a multi-dimensional construct.7 distinct stages of concern assumed to exist irrespective of the nature of the educational innovation …Self-concerns: Awareness, Informational,

Personal. Task concerns: Management. Impact concerns: Consequence,

Collaboration, Refocusing.



Stage 1(Awareness)

Little concern about or involvement with the innovation is indicated.

Stage 2(Informational)

A general awareness of the innovation and interest in learning more detail about it is indicated. The person seems to be unworried about herself/himself in relation to the innovation. She/he is interested in substantive aspects of the innovation in a selfless manner such as general characteristics, effects, and requirements for use.


Stage 3(Personal)

Individual is uncertain about the demands of the innovation, her/his inadequacy to meet those demands; and her/his role with the innovation. This includes analysis of her/his role in relation to the reward structure of the organization, decision making, and consideration of potential conflicts with existing structures or personal commitment. Financial or status implications of the program for self and colleagues may also be reflected.

Stage 4(Management)

Attention is focused on the processes and tasks of using the innovation and the best use of information and resources. Issues related to efficiency, organizing, managing, scheduling, and time demands are utmost.

Stage 5(Consequence)

Attention focuses on impact of the innovation on students in her/his immediate sphere of influence. The focus is on relevance of the innovation for students, evaluation of student outcomes, including performance and competencies, and changes needed to increase student outcomes.

Stage 6(Collaboration)

The focus is on coordination and cooperation with others regarding use of the innovation.

Stage 7(Refocusing)

The focus is on exploration of more universal benefits from the innovation, including the possibility of major changes or replacement with a more powerful alterative. Individual has definite ideas about alternatives to the proposed or existing form of the innovation.


Assumption 2: Teacher concern is a developmental construct.

Initially, a teacher’s self-concerns are expected to be the most intense. As the teacher becomes more comfortable, task concerns begin to dominate. Finally, impact concerns become most important and the teacher starts to demonstrate a strong sense of professionalism and control. A teacher can move back and forth across several stages as they develop.



Analyzing teachers’ stages of concern and their paths of development helps us to evaluate teachers’ “assessment readiness” and the nature of support they require. However, teachers are not unfettered free agents, but strongly influenced by their own personal histories and beliefs (including perceptions of self-efficacy), and the implicit and explicit discourses and practices at the institutional and societal level: “All action is socioculturally mediated, both in its production and in its interpretation” (Ahearn, 2001, p. 112).

Teachers’ assessment

orientation and decision-making

processes

SITUATIONAL FACTORS

eg. Time, trust, collegial support, student numbers/characteristics, perceived impact of assessment, etc

EXTERNAL PRESSURES

eg. Syllabus and exam requirements, other educational reforms, parental/community expectations, etc

ASSESSMENT PRACTICES

(Davison, 2004, in progress, adapted from McMillan 2003)

TEACHER KNOWLEDGE, BELIEFS AND ATTITUDESeg. Role of education; nature of assessment, teaching and learning; nature of (English) language and language learning, prior experiences of assessment/change, etc

Findings Although a range of assessment

activities are being practiced in the schools, they are mostly summative assessments for grading purposes.

Assessment activities like dictations, formal examinations are more common than portfolios, student-teacher conferencing and/or student-student conferencing, and self and/or peer assessment.

Alternative assessment activities are usually tried out on a more informal basis and account for a small percentage of marks.

Findings In terms of assessment practice, teachers

are still more likely to assess discrete skills (e.g. speaking, writing, reading, listening) rather than integrated skills (e.g. writing notes before speaking/oral presentation/discussion).

Students tend to have more experiences of individually-based assessment rather than pair/group, self and peer assessment.

The focus of assessment is upon the final product rather than the learning process.

Findings There is a wide range of teachers’

concerns about school-based assessment in both its formal and informal modes.

Our data suggest about 20-30% of teachers are still exhibiting Stage 3-type concerns about “alternative” assessments, about another 20-30% are at Stage 5, or higher, with the rest still demonstrating characteristics associated with Stage 4, but with clear shift in the upward direction, especially after the experience of active group experimentation and sharing.

Findings

Examples (background, beliefs/understandings, attitudes, practices, changes):

Peggy: not yet professionally trained, rather confused about fundamental concepts (eg. formative vs continuous assessment, training vs teaching), feeling totally overloaded, very uncertain, constantly seeking reassurance, has become more rather than less anxious over the last six months, needs much stronger support.

Findings

Steven: qualified, but fairly “traditional”, positive about direction of change, but focus of “teaching” on logistics, workload and time issues, still strongly influenced by current practice (eg. video-records show SBA group interactions still very stilted and unnatural, students encouraged to use memorized phrases and reliant on notes), not convinced he is “doing it properly”, but quick to change when has opportunities to discuss and compare with peers.

Findings

Mandy: well qualified and experienced, very enthusiastic and creative, video-records show students confident and engaged, activities tailored to suit specific context, feels positive about student development, main concerns are about lack of systematic and school-wide approach, and misunderstandings of colleagues and in community, and the effect on students of such uncertainty.

Findings All teacher informants concur that the

aims of assessment proposed by EMB are good for learning and teaching, but they think that their school’s assessment programme is not effective enough in meeting the new demands, schools and community still preoccupied with ‘recording data about students’ progress for ranking/comparative purposes, including for promotion’.

Teachers argue that schools have put too much emphasis on this aim, at the expense of assessment for learning

The need for more school-level support Planning:

“The assessment activities are not well-planned and coordinated”

“From 4 teachers have never met together – there is no common planning time”

Provision of feedback: “Students cannot get detailed individual

comments about their own performance”“Only marks or grades are given to the students.

Teachers don't have time to talk to students individually”

The need for more school-level support

The need for more formative assessment:

“Currently, almost all assessment activities are traditional and summative. They can give students some feedback on their progress but not quite effective in helping them to develop”

“Formative assessment tends to be neglected by both students and parents as they only believe studying for the exam is the ultimate goal of learning”


The need for (much) more whole school understanding and collaboration:

“I can try out some new methods in my own class provided that they don't interfere too much with the normal curriculum”

“It's hard. Many of these policy and practices are old and traditional. Unless I get most of my colleagues to agree to the new change, it will be difficult”

“The school is pre-dominantly composed of teachers who do not strongly believe in the effectiveness of assessment policy and practice. I'm not a heavy weight in the English panel”


The need for (much) more whole school understanding and collaboration:

“The management level of our school should update and equip themselves with policy for whole school assessment and to what extent SBA be incorporated into the curriculum.

“More seminars should be given to school authority so that we don't have to fight for support when carrying out the new assessment policy”

“There is not much an individual can do”

Conclusions

Need to continue to negotiate ways to develop assessment readiness which take into account the school community, not just the teacher, as the key unit of change, by:

Reconceptualizing assessment not as a top-down change imposed upon schools, but as a process of dialogue between teachers, learners and the wider school community.

Conclusions

Providing teachers and schools with the opportunity to share evolving assessment beliefs and practices in order to develop a sense of ownership, common understanding of the assessment process, and a more critical but informed perspective on practice:“Fundamental changes in school assessment practices need to be planned, discussed, shared, negotiated and agreed by all teachers in each school. A corresponding assessment policy and mechanisms to bring it about need to be worked out at whole-school and classroom levels … parents (need to be informed) of the rationale underpinning the change in assessment practices…” (CDC, 2003)

Conclusions

Providing opportunities for teachers, students, parents and administrators to experience how assessment for learning “looks and feels very different” (Leung, 2002) to traditional exam-oriented

assessments of learning.

Conclusions Acknowledging that for most teachers

understanding of theory only emerges through practice, and practice must not only be theoretically sound, but also totally practical, perhaps the greatest challenge of all: “At the heart of change for most teachers is the issue of whether it is practical … in the ethic of practicality amongst teachers is a powerful sense of what works and what doesn’t, of which changes will go and which will not – not in the abstract, or even as a general rule , but for this teacher in this context”. (Hargreaves, 1994: 12)

Documents

Enhancing Effective English Language Assessment in Secondary Schools Enhancing Effective English Language Assessment in Secondary Schools Teacher Conference,