Peer Tutoring Project (PTP) (July 2002 – October 2002)[1] Knowledge building discourse in the Peer Tutoring Project: Knowledge Forum database (http://cite-kf.cite.hku.hk:40003/, and http://cite-kf.cite.hku.hk:40005 Guest login: guest1, guest1)

The Problem Changing the predominant school culture from teaching and learning to Knowledge Building requires deep changes in both teachers and students. Given the pressures of the set school curriculum, the high stake public examinations and the heavy workload confronting teachers, are there scalable ways of bringing about deep cultural changes and development of KB abilities and facilitation skills in students and teachers?

The Peer Tutoring Project and the Assessment for Better Learning Scheme were attempts to change the learning culture of Hong Kong students in a scalable setting. The preliminary analysis of the data collected indicated some levels of success in this endeavour, as well as highlighted some significant hurdles. One important hurdle was the lack of alternative models of assessment beyond traditional modes of testing. One consequence was the failure in demonstrating the pedagogical superiority of KB in leading to better learning outcomes. Hence, innovations in assessment is necessary for scalable introductions of KB in the school curriculum.

Assessment of students’ learning outcomes and the accountability problem Achievement tests set by teachers to find out the mastery of key topic contents at the end of PTP However, assessment items used were rather traditional, aiming to test students’ understanding which would be

better learnt through rote Result: failed to demonstrate the superiority of learning outcome from deeper engagement in KB In some cases, students who did well in KB in Knowledge Forum scored lower in paper-and-pencil achievement test

than would otherwise be predicted by their prior achievement, creating an accountability challenge

Teachers’ engagement as a community of practice During the PTP project, the teachers played more of the role of “observers” in that their engagement was mainly

observing and evaluating student’s work on the database without any direct involvement in the knowledge building Teachers’ contributions were mainly restricted to administrative reminders or for the purpose of ice breaking Later on, when teachers’ awareness of the nature of knowledge building was rising, their involvement and facilitation

on the knowledge platform was also increasing especially at the second activity As we have emphasized on the scaffolding and facilitation of teachers as critical factor to the advancement of KB, it is

crucial to provide channels to teachers in order to bootstrap the pedagogical innovation and let continual improvement flourish. One of the great leaps forward from the teacher practitioners is their intention to participate in the IKIT Scholars Program. By joining in this program, teachers will be acting as researchers and strive for barrier breaking to knowledge building. Teachers will design new learning activities and work at the cutting edge of knowledge innovation. It is hoped to trigger the change of learning culture from bottom-up approach and in a more scalable setting

Preliminary Findings:

The assessment issue The preliminary analysis suggested that the assessment items & criteria suggested were not innovative at all. Students

do not seem to be able to conceptualize formal assessment as something outside of the conventional content oriented reproduction model that they were familiar with from years of schooling. The assessment items set by students are even more conservative than those by the teachers.

The students seem to compare assessment items primarily on the basis of the conceptual difficulties (simpler v.s. advance knowledge) of the questions instead of the deeper levels of understanding or application of ideas.

Challenges ahead• Developing methods of assessment appropriate for an ‘idea-centered’ curriculum aiming to foster knowledge building

capacities• Explore the various kinds of cognitive, metacognitive and affective learning outcomes that may develop during the

process of KB and to develop methods of assessment that can be used for formative and summative purposes• Explore further on models of scalable ways of promoting KB in the school curriculum • Foster and support the establishment of a community of teacher practitioners engaged in breaking new barriers in

promoting KB in the school curriculum

Reference: Bereiter, C., & Scardamalia, M. (1993). Environments for collaborative knowledge building. Toronto: Centre for Applied Cognitive Science Ontario

Institute for Studies in Education.Lamon, M., Reeve R. & Scardamalia M. (2001). Mapping Learning and the Growth of Knowledge in a Knowledge Building Community. American

Educational Research Association Meeting 2001. Seattle, Washington.Law N. & E. Wong(2003) “Development Trajectory in Knowledge Building: An Investigation” Designing for Change in Network Learning

Environments. Ed. B. Wasson, S. Ludvigsen and U. Hoppe. Dordrecht: Kluwer Academic Publishers, 2003.Rahikainen, M., Lallimo, J. & Hakkarainen, K. (2001) Progressive Inquiry in CSILE Environment: teacher Guidance and Students’ Engagement.

Conference of Computer-supported Collaborative Learning 2001.Scardamalia, M. (2002). Collective Cognitive Responsibility for the Advancement of Knowledge. IKIT, OISE, University of Toronto. Available

http://db.ikit.org:37495 (accessed 2002, Jul 11, 2002).

Evaluation criteria Mean of all groups

Accessibility of principle

Community knowledge, collective responsibility

•All group members shared responsibility for contributing regularly to advance the work•Ideas provided should contribute to the collective goal and of value to others

2.07 * 1

Democratizing knowledge

•All members contribute without over-dominating, valuing others’ contribution•There is a relatively even contribution of notes from each member

1.88 2

Idea diversity •The variety of ideas contributed by the group of students•The ability to provide additional, relevant information which may not be directly related

1.62 3

Epistemic agency •Theory construction: the group shared the responsibility for the advancement of knowledge•Theory refinement: members compare and contrast the ideas contributed by each other

1.63 4

Knowledge building discourse

•Idea revision, knowledge refinement and idea-improving process observed•Good use of KF features: reference, annotation, scaffold, etc. for scholarly communication

1.33 5

Improvable ideas •Give comments and critique to others’ contributions, to improve own & others’ ideas•The existence of theory refinement, revision and continual improvement

1.26 6

Real ideas, authentic problems

•The ability of students to turn their own ideas or problems into a researchable question •The quality of the investigation

0.79 7

Rise above •The use of rise-above notes to advance discussion•Quality of rise-above notes: good synthesis of ideas and indicate direction for further work

0.77 8

Constructive uses of authoritative sources

•Use authoritative sources•Build-on and supplement other sources•Keep a critical stance on information sources•Generate bibliographies for the referenced sources

0.37 9

Embedded and transformative assessment

•Ability to perform internal assessment, self-evaluation and review on progress•Contribute notes to reflection journal•Ability to correct, fine-tune and improve study plan

0.37 9

Overall AimTo explore scalable approaches to changing learning culture in an examination-oriented settingObstacles tackled Provide a general operational framework to conduct KB within the curriculum Create a scheme that allow schools and teachers to form a community while allowing schools maximum

flexibility for curriculum arrangements Create a scheme that would provide a general scaffolding framework that will direct the focus of the learning

activities towards KB Provide evidence that KB will lead to better learning outcomes as measured by conventional & KB criteria Develop alternative methods of assessment (including summative) that can highlight the superiority of KB

pedagogies

Table 2 Rubrics for evaluating each group’s level of performance in relation to the knowledge building principles and the score for the mean of the entire groups in the PTP

Law N. & E. Wong. “Developmental Trajectory in Knowledge Building: An Investigation”. Designing for Change in Networked Learning Environments: Proceedings of the International Conference on Computer Support for Collaborative Learning 2003. Ed. B Wasson, S. Ludvigsen & U. Hoppe. Dordrecht: Kluwer Academic Publishers, 2003.

*The score ranges from 0 (minimum) to 3 (maximum)

Achievements

• Students were able to learn to use KF for Knowledge Building even with minimal facilitation from teachers

• There is a trajectory of growth and development in KB, showing that some principles are more difficult to reach than others

• Students and teachers were generally positive about the experience and many were able to articulate to some extent the difference between KB and their conventional learning, and to be able to point out the personal learning gain from the experience

Assessment for Better Learning Scheme (ABL) (April 2003 – present)

THE UNIVERSITY OF HONG KONG

Exploring a scalable mechanism for changing learning culture in an examination-oriented settingBy Nancy Law, Allan Yuen, Elaine Wong and Johnny Yuen