5
Eager weavers: Designing assessment for an online environment Aileen Wyllie * Faculty of Nursing, Midwifery and Health, University of Technology, Sydney, Australia article info Article history: Accepted 20 November 2010 Keywords: Blended learning Peer review Online assessment abstract As more and more Australian universities move their courses into blended learning environments (BLEs), it is important that strategies be devised to monitor teaching practices and student learning. In 2008, ve large Australian universities were given an ALTC (Australian Learning and Teaching Council) grant to address the area of teacher peer review (PR). This paper shows evidence from one of the universities (The University of Technology, Sydney) where a case study was undertaken to optimise teacher peer review in a BLE. The project involved an innovative approach of interdisciplinary PR between an academic from the Faculty of Nursing, Midwifery and Health and an academic from the Faculty of Arts and Social Science (FASS). As a result of this collaboration, the online assessment in the nursing subject was radically changed with a greater percentage of the marks awarded to online activities. The design took more up fronteffort by the coordinator, as is usually the case in online preparation, but results have shown that students who had previously relied heavily on directions from their tutors throughout the semester, were now taking on more responsibility for their own learning and for the learning of fellow students. Ó 2010 Elsevier Ltd. All rights reserved. Introduction In many Australian universities there is a strong trend to develop subjects in a blended mode of delivery. This gives greater exibility for students to access information and for teachers, when designing assessment tasks for students. Assessment is a driving force for student engagement in a subject. It dominates student experiences of learning as well as consuming a large proportion of teaching time. It is therefore important to develop accurate and effective review processes that provide quality teacher feedback. A peer review process was used in this study to give teacher-to- teacher feedback when designing an online assessment. Peer review (PR) between teachers is not new; however its use in a Blended Learning Environment (BLE) is relatively new. The paper provides evidence from a case study where the use of teacher PR resulted in an improved and successful online student assessment. The use of undergraduate student online discussions is growing, though with mixed results. Achieving a large group undergraduate discussion with quality interactive student participation showing individual thinking, remains a challenging assessment to design. It was my objective when I joined this group. My pairing with an academic from the Faculty of Arts and Social Science (FASS) has had a profound impact on the re-design of my assessment. During our feedback sessions I was introduced to the concept of the role of weaver: a facilitator role for online discussion groups. Using a model adapted from Wang (2008) I re-designed my online col- laboratories for 2009, to allow each student (450 in total) the experience of facilitating an online discussion group. Results have been most encouraging and have shown that the design better engaged students in the content and that students took on more responsibility for their learning and for fellow students. Background Peer review Peer review has been described as the process of making judgements about the quality of learning and teaching which usually involves a colleague observing/examining a learning experience and learning environment and providing feedback(Pelliccione et al., 2009 p.2). According to Pelliccione et al. (2009) feedback leads to reection, followed by discussion, with the ultimate aim of improving student learning. The technique of performing a PR has often been referred to as peer observation(McMahon et al., 2007) and has been widely used for face-to-face teaching review (Menzies et al., 2008) where many resources have been developed (Bernstein, Burnett, Goodburn, & Savory, 2006). There is some evidence, espe- cially in the UK, that PR is used more frequently in higher education (Pelliccione et al., 2009). Teachers are seeking formative feedback as a means of changing practice (Dirndorfer-Anderson et al., 2009) * Tel.: þ61 02 9514 5204; fax: þ61 02 9514 5049. E-mail address: [email protected]. Contents lists available at ScienceDirect Nurse Education in Practice journal homepage: www.elsevier.com/nepr 1471-5953/$ e see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.nepr.2010.11.011 Nurse Education in Practice 11 (2011) 99e103

Eager ‘weavers’: Designing assessment for an online environment

Embed Size (px)

Citation preview

Page 1: Eager ‘weavers’: Designing assessment for an online environment

lable at ScienceDirect

Nurse Education in Practice 11 (2011) 99e103

Contents lists avai

Nurse Education in Practice

journal homepage: www.elsevier .com/nepr

Eager ‘weavers’: Designing assessment for an online environment

Aileen Wyllie*

Faculty of Nursing, Midwifery and Health, University of Technology, Sydney, Australia

a r t i c l e i n f o

Article history:Accepted 20 November 2010

Keywords:Blended learningPeer reviewOnline assessment

* Tel.: þ61 02 9514 5204; fax: þ61 02 9514 5049.E-mail address: [email protected].

1471-5953/$ e see front matter � 2010 Elsevier Ltd.doi:10.1016/j.nepr.2010.11.011

a b s t r a c t

As more and more Australian universities move their courses into blended learning environments (BLEs),it is important that strategies be devised to monitor teaching practices and student learning. In 2008, fivelarge Australian universities were given an ALTC (Australian Learning and Teaching Council) grant toaddress the area of teacher peer review (PR). This paper shows evidence from one of the universities (TheUniversity of Technology, Sydney) where a case study was undertaken to optimise teacher peer review ina BLE. The project involved an innovative approach of interdisciplinary PR between an academic from theFaculty of Nursing, Midwifery and Health and an academic from the Faculty of Arts and Social Science(FASS). As a result of this collaboration, the online assessment in the nursing subject was radicallychanged with a greater percentage of the marks awarded to online activities. The design took more ‘upfront’ effort by the coordinator, as is usually the case in online preparation, but results have shown thatstudents who had previously relied heavily on directions from their tutors throughout the semester,were now taking on more responsibility for their own learning and for the learning of fellow students.

� 2010 Elsevier Ltd. All rights reserved.

Introduction

In many Australian universities there is a strong trend todevelop subjects in a blended mode of delivery. This gives greaterflexibility for students to access information and for teachers, whendesigning assessment tasks for students. Assessment is a drivingforce for student engagement in a subject. It dominates studentexperiences of learning as well as consuming a large proportion ofteaching time. It is therefore important to develop accurate andeffective review processes that provide quality teacher feedback. Apeer review process was used in this study to give teacher-to-teacher feedback when designing an online assessment. Peerreview (PR) between teachers is not new; however its use ina Blended Learning Environment (BLE) is relatively new. The paperprovides evidence from a case study where the use of teacher PRresulted in an improved and successful online student assessment.

The use of undergraduate student online discussions is growing,though with mixed results. Achieving a large group undergraduatediscussion with quality interactive student participation showingindividual thinking, remains a challenging assessment to design. Itwas my objective when I joined this group. My pairing with anacademic from the Faculty of Arts and Social Science (FASS) has hada profound impact on the re-design of my assessment. During our

All rights reserved.

feedback sessions I was introduced to the concept of the role of‘weaver’: a facilitator role for online discussion groups. Usinga model adapted from Wang (2008) I re-designed my online col-laboratories for 2009, to allow each student (450 in total) theexperience of facilitating an online discussion group. Results havebeen most encouraging and have shown that the design betterengaged students in the content and that students took on moreresponsibility for their learning and for fellow students.

Background

Peer review

Peer review has been described as ‘the process of makingjudgements about thequalityof learningand teachingwhichusuallyinvolves a colleagueobserving/examining a learningexperience andlearning environment and providing feedback’ (Pelliccione et al.,2009 p.2). According to Pelliccione et al. (2009) feedback leads to‘reflection, followed by discussion, with the ultimate aim ofimproving student learning’. The technique of performing a PR hasoften been referred to as ‘peer observation’ (McMahon et al., 2007)and has beenwidely used for face-to-face teaching review (Menzieset al., 2008)wheremany resources have been developed (Bernstein,Burnett, Goodburn, & Savory, 2006). There is some evidence, espe-cially in the UK, that PR is used more frequently in higher education(Pelliccione et al., 2009). Teachers are seeking formative feedback asa means of changing practice (Dirndorfer-Anderson et al., 2009)

Page 2: Eager ‘weavers’: Designing assessment for an online environment

A. Wyllie / Nurse Education in Practice 11 (2011) 99e103100

especially if the PR is complemented by students’ evaluations(Alexander & Golja in McKenzie et al., 2008). To date in the onlineenvironment, PR has been used to review learning objects (Taylorand Richardson, 2001) online courses and course materials (Woodand Friedel, 2009). Less has been developed for online and BLEswhere PR of teaching presents particular opportunities and chal-lenges. In a recent paper Bennett and Barp (2008) point out that .

“Much remains to be explored, researched and documented as tohow and how far ‘online-ness’ impacts on the peer observationprocess, the experience and the benefits for participants. Theevidence is that distinct strategies, processes and models areprobably needed to provide guidance for transferring peer obser-vation online (p.265). Even with clear guidance on where to lookand what to focus on, online-ness affects what you can ’see’, howeasily you can understand what is going on, and potentially pres-ents ‘more’ for you to observe” (p.567).

Since the early 1990’s, Marshall (1994) has been advocating theneed to recognise the changing nature of learning environments,especially those that involve assessment.

Assessment is well recognised as the driving force in moststudents’ learning (Ramsden, 2003). For most students the assess-ment is the subject . assessment defines what students regard asimportant, how they spend their time and how they come to seethemselves as students and then as graduates (Brown & Knight inDirndorfer-Anderson et al., 2009), though assessment has beensurprisingly absent from the PR literature d perhaps reflecting anemphasis on teacher performance reviews. This paper is an exampleof how PR assisted in the transformation of an online assessmentfrom something to be ‘tolerated’ into something more likely to beembraced by students.

Outline of the ATN project

The project was a two-year initiative concerning a collaborativeAustralian Learning and Teaching Council project across the fiveAustralian Technology Network universities (University of Tech-nology, Sydney; Curtin University of Technology; University ofSouth Australia; RMIT University and Queensland University ofTechnology).

The project aimed to:

� create, trial and evaluate processes and resources to supportscholarly PR of teaching and learning in blended learningenvironments

� enable the use of PR for both formative feedback andimprovement, and for recognition and reward (McKenzie et al.,2008)

A co-productive, action-research approach (McKenzie et al.,2008) was used, involving teams of six academics at each partneruniversity in the development and trialling of PR frameworks,protocols and resources. Institutional team members were invitedacross a range of disciplines and blended learning contexts,including entirely online, as well as mostly face-to-face with someonline support. The action-research cycles involved team membersengaged in reciprocal PR of aspects of teaching in BLEs in order todevelop, trial and refine a common framework and protocols,which were built by integrating literature on effective teaching(Biggs and Tang, 2007; Ramsden, 2003); learning in electronic orBLE (Bennett and Santy, 2009); scholarly work from Glassick et al.(1997); PR and peer observation literatures (Van Note Chism andChism, 2007) and the promotions criteria and related teachingdescriptions in the five partner universities.

The information from the above literature was combined withfeedback from the PR teams to modify the framework and proto-cols. This resulted in a resource pack based on the six points belowand formed the basis for the review of my 2008 subject:

1. Clear goals for students’ learning and the design of the learningenvironment

2. Current & relevant preparations: includes consideration ofcontent, processes and student needs that are informed byscholarship

3. Appropriate methods and implementations: thoughtfullychosen, applied effectively and modified in response tostudents’ feedback

4. Effective communication: with students, teaching team, andother colleagues

5. Important outcome: students’ learning and engagement, otherintended and unintended outcomes, possible scholarlypresentations or publications

6. Reflective critique: including use of feedback and reflection forimprovement (modified from Glassick et al., 1997; McKenzieet al., 2008)

The case study: ‘old’ assessment under review

Design

The online assessment under review was set within a coreundergraduate nursing subject (Organisational Relationships) with450 students enrolled. The subject integrated three components:

� Power and policy in health service� Relationships and the workplace� Preparing the new graduate for transition to the workplace

The review was conducted by using three stages (ongoingcycles, Lewin: action-research). There were three reviewers: twonursing colleagues and an academic colleague from the Faculty ofArts and Science (FASS). Each PRwas conducted according to a four-step process developed in the project. Only the PR from the FASScolleague is discussed in this paper.

Pre-review briefingOne face-to-face meeting of approximately an hour, accompa-

nied by the briefing template established a structure for the casestudy. The focus was a discussion of the reviewee’s (my) desiredgoals, aspects of teaching to be reviewed and areas of focus for thereview. It also developed the reviewer’s understanding of thecontext.

The goals stated by the reviewee were:

“I am interested in knowing how well the design of the onlinecollabatories are suited for student learning and for the subjectoverall; especially the assessment”.

When outlining the focus for the review, the reviewee statedthat:

“. one of the concerns I have is that the collabatories are only‘owned’ by the students as a means of fulfilling the assessment task. they were designed to encourage independence. at present theteachers still ‘set the stage’, ‘creates the environment’ and ‘guidesthe process’ for these students who are just about to graduate”

A further 2-h meeting took place once the reviewer had studiedwhat was going to be reviewed (collabatories and relevant infor-mation to students and grading criteria). Completing the six part

Page 3: Eager ‘weavers’: Designing assessment for an online environment

A. Wyllie / Nurse Education in Practice 11 (2011) 99e103 101

ATN framework criteria required the reviewee to analyse in somedetail what they actually want to elicit from the review: aninvaluable part of the PR process, as Dirndorfer-Anderson et al.(2009) discovered.

The ‘Review’ using the six part framework criteriaThe review took place over the course of some weeks. The

experience of the reviewer at designing assessment for BLE wascrucial to this section. During these sessions the reviewer askedquestions of the reviewee and then completed further aspects ofthe framework. One comment by the reviewer for the ‘AppropriateMethod for Student Learning’ criteria illustrates how the experi-ence of the reviewer was invaluable to the critique of the students’2008 performances:

“. even though the collabatories are encouraging student learningthere are only a few students who take initiative in leadingdiscussions . the majority rely on or wait for teacher guidance”(reviewer)

DebriefingThe debriefing meeting which lasted for approximately 1 h was

an important formative opportunity which:

� enabled the reviewee to reflect� provided a space for the reviewer to offer supportive andconstructive feedback

� allowed the reviewer and reviewee to discuss and take notesfor future changes

Additional points of clarification between the reviewer andreviewee followed as required via email or phone.

ReportingThe reviewee was presented with a full report (Briefing

Template; Teacher’s response to Framework Criteria; Reviewer fullresponses for Framework Criteria) and a two page SummaryReport. The report will be used as summative evidence for futureapplications for performance reviews, promotion, teaching awardsetc. I have already used my Summary Report for performancedevelopment meetings with my academic supervisor.

The intervention: changes to be made to the online assessment

As a result of the review there were major changes to thestudent collaboratories for 2009:

� increased value given to working within the collaboratories(now 60%)

� each student given the experience and responsibility of‘leading’ a group and ‘weaving’ the ‘conversations’ online d

then evaluating their experiences (larger part of the assess-ment 60%)

� substantive information to assist the student in being ‘weavers’(Wyllie, 2010; Wang, 2008; Pallof and Pratt, 2005; Salmon,2000)

� teacher role as supportive mentor of the current weaver andalso positioned in the group as a participant

The ‘weaver’ role was an innovative approach to the design ofthe assessment. This was based on sessions withmy reviewer, workby Pallof and Pratt (2005) and Gallagher-Lepak et al. (2009). Therole expectations were that students would demonstrate thefollowing abilities:

� comfort with technology� confidence with online facilitation� comfort with a reasonable degree of chaos & conflict� the ability to communicate clearly� able to create a safe learning space for the group� the ability to nurture the development of relationships� able to promote self-organisation & empowerment� demonstrate knowledge of the topics and of questioningtechniques to stimulate group discussion

Wang’s (2008) model was used to streamline the above abilitiesand assess the students’ performance. Successful facilitation meantthat students were able to:

a) provide information and effective questions. Summarise keypoints (Intellectual Role)

b) keep discussion focused, monitor regularly, set up rules andnorms, invite missing members (Managerial Role)

c) set the tone, be encouraging, invite responses and acknowledgecontributions (Social Role)

d) start new threads, familiarity with the system and assist thestudents in using the system (Technical Role)

Results

Student evaluations of the re-designed assessment

Feedback was sought from the students at the end of thesemester. Ethics approval for the action research was given by theuniversity human ethics committee. A letter was sent to allstudents enrolled in the subject with two email reminders askingfor their participation in an anonymous questionnaire. There wasa 56% response rate. The questionnaires captured both qualitativeand quantitative data (See Table 1). Qualitative data gave feedbackabout the role of the ‘weaver’ and the quantitative questions askedabout the role of weaver as a useful tool in their learning.

The students provided rich feedback about their experiences.The following are just a few examples (of many) about how theycoped with the challenge of being a weaver, as it relates to themodel adapted from Wang (2008).

Student comments on the ‘intellectual role’ d getting the questionsright

“I had to consider the questions for the discussion carefully, in orderto encourage group members to be interested in the topic and toopen their minds for the discussion. So, the questions should beeasy to understand and explore. related to clinical. I learned theimportance of phrasing questions”

Another student stated .

“. instead of just one question to have many questions to giveparticipants more choice for discussion. you need to spread outthe questions as people can be overwhelmed”

The managerial role e demonstrate leadership in the situation

“Keep the conversation flowing and integrating the participants’responses so as to make the forum a community discussion ratherthan responding to each individual post”“I have comes to the conclusion that in the future it could provebeneficial to play the role of devil’s advocate and deliberately takean opposing view to provoke further discussion”

Page 4: Eager ‘weavers’: Designing assessment for an online environment

Table 1Quantitative student feedback 2009.

QuestionTotal group number 450.Number of replies ¼ 252 replied(return rate 56%)

Agree/strongly agree Disagree/strongly disagree Not answered/not appropriate

With regard to the technique of asking questions my level ofunderstanding increased

152(63%) 80(33%) 10(4%)

The role of weaver helped me to reflect on my writing skills 135(54%) 113(45%) 4(1%)The role of weaver helped me to reflect on my communication skills 140(56%) 108(43%) 2(1%)Able to put into practice my learning about group dynamics a98(40%) 136(56%) 10(4%)I had a pattern of day and time that I would (as weaver) log onto

the discussion board

a63(26%) 171(70%) 9(4%)

I kept to this pattern during my time as weaver a44(19%) 164(71%) 22(9%)There was sufficient support given to the weaver role by the tutor 170(68%) 80(31%) 2(1%)I have gained insight into the role of facilitation of an online

discussion group194(77%) 58(23%) 0

Overall I found it useful for my practice to develop an understandingof being supportive to my colleagues

165(66%) 83(33%) 4(1%)

a Results below 50% show studies accessing the LMS at times of their convenience & that this assessment did not reflect a group assessment task.

A. Wyllie / Nurse Education in Practice 11 (2011) 99e103102

The social role e sense of community

“Unique experience and unlike anything I had done through mydegree. It allowed every student an opportunity to facilitate andmanage important and thought provoking conversations, which Ibelieve contributed to a sense of community and ownershipamongst peers”

The technical role e comfort with technology and system

“As the effective communication is an essential part of our practicewith the fast development of computerised communication tech-nology, work and learning through the online medium has becomea vital skill”

This student went on to state .

“I lack computer skills . recognising that my career path willinvolve computer skills I have enrolled in a computer course”

Students also had to give summaries during their time as‘weaver’

One example:

“My time as the weaver has come to an end and I would like tothank everyone who has contributed. Some common themes thathave come out of this discussion include . It was interesting tohear from ‘S’where she said. and ‘S’ had a similar experiences. ‘Z’suggested that . ‘W’ outlines the .. ‘K’ reiterates that . ‘S’believes that and a good suggestion from ‘M’ to approach. ‘E’ hasalso experienced . it seems from the discussion that there aremany resolutions to this issue . “

Other general comments of note

Some students experienced isolation

“feeling of isolation in the role as weaver. I felt disjointed frompeople. I like to see people’s faces. I found it a great challengeremaining fluent and passionate about the issues being discussed.”“. all contact with lecturer and classmates occurs only in writing.Strangely enough this gave me a feeling of isolation even whenthere were a lot of posting in front of me on my screen .”

Some enjoyment

“really enjoyed the concept of the weaver it kept you focussed onthe whole discussion and not just your input”“refreshing change”

Discussion

This paper explored Peer Review as a process in itself and forimproving online student assessment. Through the ATN Teamproject, two subject coordinators from different disciplines (socialsciences and nursing) were given a unique opportunity to havefocussed conversations on improving learning assessment tasks forsubjects in a blended learning mode. The PR structure was effectivefor these sessions, resulting in an innovative assessment leading toincreased levels of student involvement and satisfaction.

In the re-designed assignment, students led online discussiongroups with confidence and displayed leadership skills importantfor sustaining their professional careers. Questions chosen by thestudents becamemore streamlined and focussed towards the topic.Students came to realise that when acting as ‘weaver’, that allowingparticipants’ time to reflect on one or twowell chosen questionwasmore helpful to the discussion than bombarding the boards withlots of questions. Given other comments on the subject survey,students discovered ‘relevance’ to personal learning and involve-ment in the learning of colleagues. There was added depth to the‘student-to-student’ engagements and even more important: anemphasis on critical discussion, rather than continual agreementamong participants which can have a detrimental effect onlearning. Agreement in itself is not problematic; but can besymptomatic of a superficial online discussion (McConnell, 2002).

While the assignment was not immediately seen as a collabo-rative learning task, both weaver and participant needed oneanother in pursuing a ‘profitable’ discussion. The participants hadto respect the current ‘weaver’ and the ‘weaver’ relied on the grouptaking part in the discussion. With this reciprocal relationship,goodwill was fostered and the conversation flowed better. Collab-oration also had the benefit of all community membersapproaching the issue as a learning event, where they developed ordiscovered new strengths. In achieving this I believe that theassessment helped students to move away from a dependence ontutors as the only source of judgement about the quality of theirlearning. It is likely that this experience can be transferred to otherlifelong learning situations and contexts. Equipping themwith suchskills could be a key to the so-called ‘learning society’ (Boud, 2000).This is in line with other scholars, who argue that students can take

Page 5: Eager ‘weavers’: Designing assessment for an online environment

A. Wyllie / Nurse Education in Practice 11 (2011) 99e103 103

responsibility for their own learning, given proper scaffolding andstructure (Wozniak and Silveira, 2004).

The openness of this assessment task appeared to be animportant element for student learning. Most assessment tasks areset between student and tutor. In this assessment, the work of all(weaver and student and tutor participants) takes place in an openenvironment. The tutors remained responsible for the ‘formal’grading and feedback to all students, however it was noticeable asthe semester wore on, that some ‘weavers’ language adopteda ‘teacher speak’ style where feedback became part of their ques-tioning technique. This had the effect of building a more cohesivegroup where trust flourished and ‘informal’ feedback became justas or more important than the topic under discussion.

The ‘weaver’ role was carried out by the majority of the studentswithout toomuch anxiety. Most problems came from students whoinitially baulked at the management role. When this happenedtherewas intervention by the tutor until the student could be put ontrack. The student questionnaires evaluated well and what I foundwas even more satisfying . that the role of ‘weaver’was ‘enjoyed’.

For the coordinator (me)problemswere fewduring the semester,mainly reflecting the time spent in preparing the students andrelevant resources/information for the discussion. This preparationallowed teachers to spend more time being involved/observingcommunity learning, rather than being relegated to the usualmanagerial facilitation role. One of the strengths of online work isthat it makes the discussion the focus of attention and since it isrecorded, everyonehad a chance to give their views at a time of theirchoosing.

The PR framework provided a useful visualisation tool forteacher reflection. An opportunity for interdisciplinary conversa-tion with an experienced colleague (critical friend) provided in the‘structured place’ of the PR and resulted in an assignment thataligned more strongly with the subject objectives. The case studyand broader project work demonstrates that PRs were mostsuccessful between ‘real peers’ (Dirndorfer-Anderson et al., 2009).McMahon et al. (2007) make the point that the reviewee needs tobe able to control the whole process e from whether they partic-ipate or not, to what is done as a result of the review, and this wasstrongly emphasised in this case study. Because BLEs can becomplex, use of PRs require more engagement and more time.Given increasing student numbers in undergraduate programs andthe move towards blended learning, we have to optimise any newlearning spaces. PR can be a powerful tool, particularly when bothinterest and experience are applied to the task. The case highlightsthat reviewers do not have to be from the same discipline, but thatthe experience of teaching within a BLE, is essential for the processto be effective. There is an urgent need for sound and effectivereview processes of online assessment.

Conclusions

Online learning and teaching is here to stay. The importance ofthis project was to explore how discussion boards can be success-fully used for a large cohort of undergraduate nursing students asa place for learning, social interaction and assessment. The assess-ment will continue to be refined in order to direct the studentstowards active rather than passive learning. It is anticipated that thepeer review process will be part of this refinement process.

Acknowledgements

The author would like to thank Dr Jo McKenzie for the invitationto join the ALTC Team project; Dr Theresa Dirndorfer-Anderson forher time as reviewer and Dr Nicola Parker as a critical friend.

References

Bennett, S., Barp, D., 2008. Peer observation - a case for doing it online. Teaching inHigher Education 13 (5), 559e570.

Bennett, S., Santy, J., 2009. A window on our teaching practice: enhancing indi-vidual online teaching quality though online peer observation and support. AUK case study. Nurse Education in Practice 9, 403e406.

Bernstein, D.J., Burnett, A.N., Goodburn, A., Savory, P., 2006. Making teaching andlearning visible: course portfolios and the peer review of teaching. Anker,Bolton, MA.

Biggs, J., Tang, C., 2007. Teaching for quality learning at university: what the studentdoes, third ed. Open University Press, Maidenhead, UK.

Boud, D., 2000. Sustainable assessment: rethinking assessment for a learningsociety. Studies in Continuing Education 22 (2), 151e167.

Dirndorfer-Anderson, T., Parker, N., McKenzie, J., 2009. Assessing online collabo-ratories: a peer review of teaching & learning. ATN Assessment Conference2009 Proceedings.

Gallagher-Lepak, S., Reilly, J., Killon, C.M., 2009. Nursing student perceptions ofcommunity in online learning. Contemporary Nurse 32 (1e2), 133e146.

Glassick, C.E., Huber, M.T., Maeroff, G.I., 1997. Scholarship assessed: evaluation ofthe professoriate (a special report). The Carnegie Foundation for the Advance-ment of Teaching, San Francisco, US.

Marshall, B., 1994. Learning from the academy: from peer observation of teaching topeer enhancement of learning and teaching. Journal of Adult TheologicalEducation 1 (2), 185e204.

McConnell, D., 2002. The experience of collaborative assessment in e-Learning.Studies in Continuing Education 24 (1), 73e92.

Menzies, G., Pratt, J., Thorp, S., Docherty, P., 2008. Piloting a peer feedback programat UTS. Professional development project. International Teachers ProgramBoccioni, University.

McKenzie, J., Pelliccione, L., Parker, N., 2008. Developing Peer Review of Teaching inBlended Learning Environments: Frameworks and Challenges. Paper Presentedat the in Hello! where Are You in the Landscape of Educational Technology?Proceedings Ascilite Melbourne 2009. http://www.ascilite.org.au/conferences/melbourne08/procs/mckenzie-j.pdf.

McMahon, T., Barrett, T., O’Neill, G., 2007. Using observation of teaching to improvequality: finding your way through the muddle of competing conceptions,confusion of practice and mutually exclusive intentions. Teaching in HigherEducation 12 (4), 499e511.

Pallof, R., Pratt, K., 2005. Collaborating online: learning together in community.Jossey-Bass, San Francisco.

Pelliccione, L., Dixon, K., Siragusa, L., Howitt, C., Atweh, B., Dender, A., Swaine, J.,2009. Academic peer review: enhancing learning environments for globalgraduates. In: Teaching & learning for global graduates. Proceedings of the 18thannual teaching learning forum. Curtin University of Technology, Perth,pp. 29e30. http://otl.curtin.edu.au/tlf/tlf2009/refereed/pelliccione.html.

Ramsden, P., 2003. Learning to teach in higher education. Routledge, London, UK.Salmon, G., 2000. E-moderating: the key to teaching and learning online. Kogan

Page, London.Taylor, P.G., Richardson, A.S., 2001. Validating scholarship in university teaching:

constructing a national scheme for external peer review of ICT-based teachingand learning resources (technical report). Department of Education Trainingand Youth Affairs, Canberra, Australia (Commonwealth of Australia).

Van Note Chism, N., Chism, G.W., 2007. Peer review of teaching: a sourcebook,second ed. Anker Pub. Co, Bolton, Mass.

Wang, Q., 2008. Student-facilitators’ role in moderating online discussion. BritishJournal of Educational Technology 39 (5), 859e874.

Wood, D., Friedel, M., 2009. Peer review of online learning and teaching: harnessingcollective intelligence to address emerging challenges. Australasian Journal ofEducational Technology 25 (1), 60e79.

Wozniak, H., Silveira, S., 2004. Online discussion: promoting effective student tostudent interaction. In: Atkinson, R., McBeath, C., Jonas-Dwyer, D., Phillips, R.(Eds.), Beyond the Comfort Zone. Proceedings of the 21st ASCILITE Conference,Perth, pp. 956e960. http://www.ascilite.org.au/conferences/perth04/procs/wozniak.html. 5e8 December.

Wyllie, A., 2010. Eager ‘weavers’: designing assessment for an online environment.15th April 2010. In: Nursing Education in a Global Community: Collaborationand Networking for the Future. 3rd International Nurse Education Conference,Sydney, Australia.