Journal of Advanced Nursmg, 1995. 22,48-57

Assessing the level of student reflection fromreflective journalsFrances KY Wong RN BSN DNE MAAssistant Professor, Department of Health Sciences, Hong Kong Polytechnic University

David Kember BSc MSc PhDEducational Development Officer, Educational Development Unit, Hong KongPolytechnic University. Hong Kong

LorettaY F Ch\mg RN DipHCE BHSc MEdFormer Lecturer, Department of Health Sciences, Hong Kong Polytechmc University,Hong Kong Current PhD student, Deakin Umversity, Australia

and Louisa Yan CertEd BEd MPhilAssociate Project Co-ordmator, Educational Development Unit, Hong Kong PolytechmcUniversity, Hong Kong

Accepted for publication 20 September 1994

WONG F K Y , KEMBER D , CHUNG L Y F & YAN L (1995) Journal of Advanced Nursing22,48-57Assessing the level of student reflection from reflective journalsThe concept of reflective learning has been widely adopted in many of the nursingcurricula today Reflective leaming is of particular relevance to the education ofprofessionals, as it encourages students to integrate theory with practice, appreciatethe world on their own behalf, and turn every experience into a new potentiallearning experience While nurse educators have widely accepted the educationalbenefits of reflection, research into reflective learning is hampered by the lack ofreliable and widely accepted methods for assessing whether reflection takes placeand the level of any reflection This study attempted to develop and test codingsystems for written reflective journals based on two well-known models of reflectivethinking The reflective ]ovu:nals submitted by the students were subjected to contentanalysis at two levels The findings of this study suggest that student writing can beused as evidence for the presence or absence of reflective thinking The process ofallocating students to three categories of non-reflector, reflector and critical reflectorwas straightforward and reliable Identifying textual elements within journals andallocating them to the finer levels of reflection within a more complex model ofreflective thinking was, however, more problematic and considerably less reliable

INTRODTJCTTON Real-world practice does not often present well-definedprohlems to the practitioners Rather, the world is com-

Professional education should aim at preparing pro- posed of happenings that need to he sorted and organizedfessionals who can ultimately apply theoretical knowledge hefore they hecome meaningful The real world isto practice in the real world This is not easy, however djmamic, and each client encounter is unique (Schon

1987) There are no pre-set solutions to real-life prohlemsCorrespondence Frances Wong Assistant Professor Deportment of Health The practitioner needs to see the world on their OWn hehalfSciences, Hong Kong Polytechnic University HongKong (Schon 1987), and come up With their OWn interpretation

48 © 1995 Blackwell Science Ltd

Assessing student reflection

of the reahty The challenge of educators, therefore, is tohelp individual learners to deal with everyday clinicalsituations in a competent manner This concern is particu-larly relevant to the education of professional nurses, whoare required to exercise autonomous and expertjudgement Reflection has heen identified as an effectuallearmng strategy that can achieve the aim (Boud et al1985, Schon 1987, Jarvis 1987, 1992, Powell 1989,Mezirow 1991, McCaugherty 1992, Murphy & Atkms 1994)

Schon (1987) has challenged the traditional professionalschools for not preparing students for competence in real-life practice He asserts that the educational preparationof professionals should he centred on enhancing the prac-titioners' ahility to reflect Reflection in learning is rec-ommended in pre-registration, as well as post-registration,nurse education, for it can facilitate the integration oftheory and practice {Leino-Kilpi 1990, McCaugherty 1992,Snowhall et al 1994) In pre-registration nurse education,turning expenence mto learning is found to he particu-larly valuahle in the clinical practicum (Saylor 1990,McCaugherty 1992, Murphy & Atkins 1994) Reflectionhelps to turn each experience into a potential new learningsituation (Jarvis 1992) It enhances the development ofprofessional competence

Lack of empirical research

We have ohserved m the literature review that there ismore discussion of the concept of reflection, or the extentof its use, than empirical research reports on the assess-ment of reflection in learning As reflective learning ishecoming an emphasis m the contemporary nursmg curric-ula, educators need to estahlish means to monitor its effectOtherwise reflection will just hecome a slogan

Student journals have heen a common method used toencourage and record reflection in learmng (Bowman1983, Hahnemann 1986, Nicassio 1992) Wnting aboutexperiences may enahle practitioners to make explicit theknowledge that is implicit m their actions (Hahnemann1986, Atkms & Murphy 1993) Reflective journals can hea valid tool for assessing students' accomplishment ofleammg The establishment of an approach to evaluatethe level of reflection is essential to the successfulimplementation of reflective learning m the educationof professional nurses

REVIEW OF THE LITERATURE

The concept of reflection has heen widely discussed in theliterature of teacher education since the 1980s (e g Fellows& Zimpher 1988, Ross 1989, Smyth 1989) The nursmghterature has recently joined m the discourse (e gPowell 1989, McCaugherty 1992, Atkins & Murphy 1993,Snowhall et al 1994) A number of scholars has attemptedto propose defliutions and models of reflection (e g Boud

et al 1985, Schon 1987, Mezirow et al 1990, Mezirow1991)

Reflection has heen defined as

An important human activity in which people recapture theirexpenence, think about it, mull it over and evaluate it

(Boud et al 1985)

The central point of reflection m learmng is expenenceReid (1993) points out that reflection can he firom eitheran interest m examining an experience then drawing ontheoretical concepts, or testmg out theoretical conceptsthrough expenence Reid's view corresponds with Kolh &Fry's (1975) experiential leammg model The modeldescnhes how concrete experience is used as basis forobservation and reflection, from which ahstract conceptsand generalizations are formed The new hjrpotheses andimplications of concepts guide the choice of new expen-ence, when the assumptions can be tested in the new situ-ations (Kolh & Fry 1975) Kolb & Fry (1975) have descnbeda cyclical relationship between new and old expenenceFellows & Zimpher (1988) also see that m reflection

Past experience and understandings are linked with presentexperience to lead to new understandings and appreciations

Experience alone, however, is not the key to learning(Boud et al 1985) Learning from experience involvesreflection, which is a generic term used as a 'shorthanddescription' for a numher of ideas and activities (Boudet al 1985) The reflective process involves both feelingsand cognition, which are rlosely interrelated and inter-active (Boud et al 1985, Mezirow et al 1990) The feelingsmay be positive or negative They cue the individual torespond at the initial stage of the reflective processIntensive personal feelings are also present towards theend stage of the reflective process As an outcome ofreflection, the individual has the will to act upon hisor her convictions suhsequent to transformative learning(Mezirow et al 1990)

The commitment to action becomes the life-force of theindividual (Boud et al 1985) The cognitive activitiesinclude making inferences, discnminating and associatingrelationships, and validating assumptions (Boud et al1985, Mezirow et al 1990) Reflection generally involveshigher-order mental processes at the conscious level Itenables one to correct distortions and errors m one'sbeliefs (Mezirow et al 1990) This learning process ofmaking a new or revised interpretation of the meaning ofan experience, fosters subsequent understanding, appreci-ation, and action (Fellows & Zimpher 1988, Mezirow et al1990, Mezirow 1991)

Single-loop learning

Reflective practice may begin at the point where prac-titioners take a fresh look at their routine practice, and

© 1995 Blackwell Science Ltd, Joumal of Advanced Nursing, 22, 48-57 49

FKY Wong etal

question that which is taken for granted (Jams 1992)When goals, values and strategies are simply taken forgranted, Argyns & Schon (1974) called it 'single-loopteaming' Such learmng operates withm a closed and self-confixmmg world of understandmgs 'Double-loop learn-ing' occurs when one questions the role of the framing andlearning systems The expenence is then reframed andevaluated (Argyns & Schon 1974) Reid (1993) has givena personal account of her successful expenence facilitatingreflective practice among a group of nurses, who made theremark 'but we're domg it already'' Practitioners can con-tinue to develop their practice through reflection once theystart challengmg their conventional practice

To challenge the established and habitual pattems ofexpectation, to question tbe validity of a long-taken-for-granted meanmg perspective, is to become cnticallyreflective (Mezirow et al 1990, Mezirow 1991) Neitheracting according to routines/habits nor carefully plannedactions are reflective actions (Boud et al 1985, Jarvis 1987,Schon 1987, Mezirow et al 1990)

Reflective learning is found to be very valuable to pro-fessional education Scbon (1987) has urged that

Professional education should he redesigned to combine theteaching of applied science with coaching m the artistry of reflec-tion-in-action

Saylor (1990) also asserts that the ability to 'reflect uponone's repertoire of knowledge and experience' is essentialfor professional competence The nurse educator canfacilitate students' use of clinical expenence m enhancingprofessional growth by adopting a reflective practicum(Murphy & Atkins 1994)

Reflection m learning is useful in post-registration nurseeducation as well The post-registration nurses are adultlearners who possess a reservoir of experience as aresource for learning These students have an immediacyto integrate theory into practice (Jarvis 1987) Leino-Kilpi(1990) has included self-reflection m nurse teacher edu-cation Snowball et al (1994) have explored the use ofreflection as a learning tool when supervismg health carestudies students wnting their dissertation They found thatthe need to integrate theory and practice dunng theprocess provided teachers and students alike withan opportunity for using reflection as a tool to enhanceprofessional practice

Time-space

An important way to enhance learmng is to strengthen thelink between tbe learning expenence and the reflectiveactivity which follows it The allocation of specific tunefor reflecbon, and allowing adequate time-space wouldenhance the reflective process (Boud et al 1985) Wnting,brainstorming and dialogue are the commonly recom-mended strategies to promote reflective learning among

students (Boud et al 1985, Hahnemann 1986, Smyth 1989,Mezirow et al 1990, Mezirow 1991, Nicassio 1992)

The response of the learner to new expenence isdetermined significantly by past expenences and one'sperception of the world History of success and adequatepreparation m particular areas contnbute to one's newlearmng in a positive manner (Boud et al 1985) Tbe out-comes of reflection include a new way of doing something,tbe clarification of an issue, the development of a skill ortbe resolution of a problem New perspectives on experi-ence or changes m behaviour are developed (Boud et al1985, Mezirow et al 1990) Mezirow (1991) suggestsconfining the concept of reflection to Dewey's (1933)definition 'Reflection means validity testing'

CONCEPTUAL FRAMEWORK

Several potential models were explored to see whetherthey provided a suitable framework for a coding schemeThe reflective process portrayed by Boud et al (1985) andMezirow (1991) was eventually used as the conceptualframework m this study We found that the two modelsoffered a clear description of the conception, and the pro-cess, of reflection Also, their work correspond with tbedescription of a reflective curriculum m the coursephilosophy of the post-registration Bachelor of Sciencem Nursing (Honours) [BScN(Hons)] course offered at tbeHong Kong Pol)rtechnic University

Boud et al (1985) have depicted a model of reflectiveprocess m which they have suggested the key elementsof the process Mezirow (1991) has distinguished non-reflectors, reflectors and cntical reflectors The followingparagraphs are a brief review of their work

The model of reflective process by Boud et al (1985)explains that as an individual encounters an experience,he or she responds Tbe reflective process is initiated whenthe individual returns to the expenence, recollecting whathas taken place and replaying tbe expenence There-evaluation takes place There are four elements m tbeprocess of re-evaluation

1 association — relating new data to that which isalready known,

2 mtegration — seeking relationships among tbe data,3 validation — determining the authenticity of the ideas

and feelmgs that have resulted,4 appropnation — makmg knowledge one's own

Outcomes

The outcomes of reflection may include development ofnew perspectives or changes m behaviour The s3Tithesis,validation and appropnation of knowledge are part of tbereflective process, but they can be outcomes as well Boudet al (1985) explam that tbe elements are separated just

50 © 1995 Blackwell Science Ltd, Joumal ofAdvanced Nursing, 22, 48-57

Assessing student reflection

to draw attention to the various features in the processThe elements do not proceed m a hnear sequence, nor arethey independent of each other There can be omission ofsome stages, or compression of some of the elements attimes

Mezirow (1991) argues that not every act of becomingaware of thoughts or feelings is reflective Two kmds ofaction have been identified as 'non-reflective' One ishabitual action, such as tj^mg or dnvmg a car The secondone IS thoughtful action, which involves

A selective review of prior learning rather than a deliberateappraisal or reappraisal of it

(Mezirow 1991)

Reflection involves the assessment of the assumptionsimplicit m beliefs (Mezirow et al 1990) It involves theconfirmation, addition or transformation of ways of inter-preting expenence (Mezirow 1991) Premise reflection isthe term used by Mezirow to refer to challenging the val-idity of presuppositions m pnor learning (Mezirow et al1990) When there is transformation of meaning, premisereflection has occurred (Mezirow 1991)

leamers The course emphasized the value of practise,which was important in realizmg the theory learned

The design ofthe NAE course was that the lecture inputwas complemented by tutorial sessions Students, ingroups of five to six, were required to devise a teachingplan m an identified area They would then need to pro-vide theoretical justification for their design of the planTutorial support was arranged mbetween lectures to facili-tate students m working out their plans Towards the endof the semester, students discussed their teaching planswith the class, and tested out their thoughts Eventuallystudents had to carry out the plan at their working placeThe target leamers could be their clients, or student nursesm their practice areas After the implementation phase,each individual student had to wnte a reflective paper

A 2-hour session had been included at the beginning ofthe course to introduce the concept of reflection and howthe process can facilitate their learning from expenenceDuring the course, the students had received help theywould have normally obtained, for instance, clarifying therequirement ofthe assignment, and clarifying expectationsof the course

THE STUDY

This study intended to initiate a procedure for assessingthe level of reflection from written papers The only refer-ence we were able to discover m the nursing literature todetermining reflection levels from written reflective jour-nals was by Powell (1989) However, we were unable tomake use of this work as the paper gave no details of thecoding procedures We also suspected that the processneeded to be treated with a measure of scepticism as thepaper made no mention of any tests for reliability andvalidity on the codmg process

Bums (1994) has recently described a process for theassessment of student nurses based upon three levels ofreflection descnbed by Goodman (1984) The assessmentexercise includes learning contracts, mentors and agrading viva, so the processes are difficult to transfer toassessing reflection levels in written journals

In this study we used the theoretical work by Boud et al(1985) and Mezirow et al (1990) as the basis for estimatingthe quality of reflection The project attempted to formu-late reproducible procedures for momtormg reflection

Context ofthe study

This study took place among a group of registered nurseswho were undertaking a unit called 'The nurse as an edu-cator' (NAE) m the BScN(Hons) programme at the HongKong Polytechnic Umversity The NAE course was a30-hour unit It aimed at developmg among the post-registration nurse students an appreciation of the impor-tance of education m nursing, targetmg clients or nurse

Methodology

Before the study was started, the purpose and design wereexplained to the students The students were reassuredthat the results of coding would not m any way affect theircourse assessment mark All of the students agreed toparticipate in the research

The coding scheme is explained below The reflectivepapers submitted by the students were subjected to contentanalysis In-depth interviews were conducted to illumi-nate the thought process of a 'critical reflector' emd a 'non-reflector' The features of the two groups of students dem-onstrating the level of reflection at either end ofthe processwere then exeunined

Content analysis

A coding scheme was developed to analyse students'reflective papers The coding scheme was used to help thecoders to tie research questions and conceptual interestsdirectly to the data (Miles & Huberman 1984) Theresearchers used the reflection process descnbed by Boudet al (1985) as a base Five scnpts were sampled for pilot-ing and refining the coding scheme Dunng the process,criteria were identified for each coding category The categ-ones for coding are descnbed in Appendix A Contents ofscnpts that were mainly related to administrative and per-sonnel issues were excluded from the coding m this studyThey were not content-specific to the teaching expenence,and not relevant to the reflective process

Five general pnnciples of coding were developed dunngthe pilot stage (Wong & Chung 1993)

© 1995 Blackwell Science Ltd, foumal of Advanced Nursing, 22, 48-57 5 1

PKY Wongetai

1 The initial stage of the reflective process, as depicted byBoud et al (1955) was called 'returmng to exf»enence'This stage was not coded because returning to expen-ence was the requirement of all students in thisassignment

2 Repeated points or arguments were coded only once3 Coding was only valid with evidence or substantia-

tion4 Coding had to be supported by textual data Inter-

pretative speculation was not accepted5 Using quotes from hterature to illustrate or substantiate

one's own expenence was amenable to codingHowever, quotes that merely demonstrated textbookknowledge were not coded

Rehahility ofthe coding

Five coders were involved in establishing and checkingthe reliability of the content analysis codes Three of thefive coders proceeded with initial content analysis at thefirst level of coding They were the pnncipal investigatorof the research, who was also the subject co-ordinator ofthe NAE course, a nursing lecturer from a tertiary institute,and a nurse tutor from a hospital-based school of nursingEach coder would separately code each script Two levelsof coding were used At the first level entire reports wereexamined for paragraphs that appeared to provide evi-dence that the student had utilized one of the elements ofBoud et al's (1985) model of reflection, namely attendingto feelings, association, integration, validation, appropri-ation or outcomes of reflection The coders first had tolocate paragraphs that appeared to indicate that reflectionhad taken place, and then to determine which element ofthe reflective process it referred to

The second level of coding was to allocate each studentto one of three categories of non-reflector, reflector andcntical reflector, derived from the work of Mezirow et al(1990) In this second level of coding, the unit of codingIS the student, as opposed to the first level which hasparagraphs of text as the unit of coding

The normal approach to estimating the reliability ofcoding IS to use the formula

Reliability =number of agreements

total number of agreements plus disagreements(Miles & Huberman 1984)

When this formula is used to calculate reliability values,the segments of text to be coded are normally well definedEssentially the formula is generally used to detenmnewhether coders agree as to which of several categones adetermined case should be placed in

This formula could, therefore, be used m its normalsense, for the second level of coding Agreement as towhether students were non-reflectors, reflectors or cntical

reflectors was high The reliability coefficient was 0 88when codmg was performed completely independentlyAfter discussion of cases, it was normally possible to reachfull agreement

Attempting to use the reliability formula for the firstlevel of coding was more problematic The coders wererequired to reach agreement on two aspects of the codmgprocess Firstly, ldentifymg the same paragraph withmquite extensive reports, and secondly allocating the ident-ified paragraph to the same element from Boud et al's(1985) model of reflection

A further difficulty m the coding was that the journalscovered a variety of topics The coders were trymg tointerpret a discussion of a teachmg experience to distilout evidence of levels of reflection The codmg was cer-tainly not of direct statements about reflection or reflec-tive processes The data being coded were therefore oftennot directly related to the categones

In view of these caveats on the use of the conventionalmeasure of coding reliability, it is not surprising that thelevel of reliability was lower than that for the secondlevel of coding The lmtial three coders who establishedthe coding procedure managed to reach agreement levelsof between 0 5 and 0 75 However, two further codersfound it more difficult to apply them with the same levelof reliability Both preferred to restrict coding to thebroader second level categones, as they foimd it difficultto differentiate the fine distinctions between the elementswithm Boud et al's (1985) model from the journalmaterial

Coding according to Boud's model

There were 45 scnpts included m this study The textualmatenal was initially classified mto eight subcategonesattendmg to feelings, association, integration, validation,appropriation, outcome of reflection A total of 100reflective elements was coded Most of the reflectionelements appeared to be dispersed in the first three categ-ories The component of association seemed to mostprominent The elements of validation, appropriation andoutcome of reflection appeared to occur less frequentlyIllustrative text of each subcategory is depicted m thefollowing exemplars The student codes have beenspecially treated to disguise the identity of individualstudents

Attending to feelings

I am frustrated with the present ward practice and interpersonalrelationship I hope the onentation style will be changed

(Student NKD)

NKD IS trying to remove obstructive feelmgs and utilizepositive feelmgs during the expenence

52 © 1995 Blackwell Science Ltd, foumal of Advanced Nursing, 22, 48-57

Assessing student reflection

Association

Most of them [the learners] were not familiar with the [bloodpressure] measuimg techmque the practical session went over-time because their ability was overestimated It revealed that agood result needed a detailed assessment and plannmg

(Student SCH)

SCH has lmked the old knowledge of the need to carry outcomprehensive assessment of the teaching situation to theexpenence of overrunning of time diu-ing teaching

Integration

Blood chemistry may seriously impair the [renal] patients' abilityto understand and retain information introducing computer-assisted learning enables patients to work at their own pace,allowing them to cover material again and again withoutembarrassment or fear that they are taking up too much of theeducator's time

[Student MCY)

Here MCY is seeking the relationship of altering bloodchemistry of renal patients and their learning needs, andMCY came up with an idea for the provision of individ-ualized computer-assisted learning

Validation

The results were very strange the ['bright'] patient returned with3 kg weight gam, while the latter [labelled to be 'slower learner']could keep the body weight gam below 1 5 kg The teachmgprogramme I planned already included the necessary knowledge,skills and attitude, the objective needs of the patients, to form thebehavioural objectives These three components are interrelatedand interacted to constitute some behaviour changes the benefitand barriers are subjectively perceived by an individual, whichmay not be the same as we think they should be In order toexplore the patient's subjective feeling, good communi-cation between the patient and the health educator should beestablished

(Student OLK)

OLK has found inconsistency between expected outcome(prior knowledge) and reality (new appreciations) Therewas discrepancy m patients' learning outcomes in spite ofa rather comprehensive teachmg programme QLK probedfurther and learned that the benefit and barriers perceivedby the patient were evaluated subjectively The appraisalof the perception of the pabent by the health educator wasfound to be essential

Appropnation

Holistic approach tends to be high sounding We found it difficultto apply into our programme at the beginning We finally pickedup some hints after we looked into it from different dimensionsFor instance, the learner-centred design of activities, the encour-

agement of participation with learners' relatives and the effort tomeet individual needs, all were the ways towards the hohsticconcept The project therefore gave me the ideas to apply holisticeducation on the earth

(Student OCW)

OCW bas made ways to overcome the difficult situationand apply the 'holistic' concept m the health educationprogramme Now the concept of 'holistic approach' hasbecome real to her

Outcome of reflection

Working as a health nurse for a few years I have the worry that Imay have adopted a routine pattem of behaviours m deliveringhealth education To involve the clients in evaluating the edu-cational process was a new expenence I learned not to takedaily health education activities as routine tasks

(Student FLL)

FLL has transformed the perspective in health educationAfter the teaching expenence, FLL has taken a fresh lookat health education, rather than taking it as part of thework routine as a health nurse

Classification according to Mezirow's model

The next level of analysis was to place the students intoone of three broader categories derived from Mezirowet al's (1990) model of reflection non-reflectors, reflectorsand critical reflectors Non-reflectors showed no evidenceof any of the elements of reflective thinking Reflectorsshowed evidence of attending to feelings, associationand/or integration but did not demonstrate any cnticalchanges of perspective The critical reflectors had featuresof a reflector, but also sbowed evidence that they hadchanged their perspective

FEATURES OF NON-REFLECTORS

Non-reflectors showed no evidence of any of the reflectiveelements m Boud et al's (1985) model Six students(13 3%) were classified as non-reflectors in this studyTheir work was examined and there were some commonfeatures could be identified among the non-reflectors (NR)

First, the NRs were very descriptive m their papersThey tended to report on the happenmg, rather thanre-visitmg the expenence and analysing it The paperswere filled with 'thoughtful action' in Mezirow et al's(1990) term Some of the NRs made precise observationsand recordings of their teaching expenences However,there was no inference or analysis of the expenenceSecond, the NRs tended to make assumptions withouttrying to test them out for therr validity A view was usu-ally presented without supportive evidence nor specificreferencmg to the expenence Third, the NRs tended to

© 1995 Blackwell Science Ltd, Journal of Advanced Nursing, 22, 48-57 53

FKY Wongetal

adopt a relatively straightforward way of viewing the situ-ation There was little awareness of the contextual factorsThe thinking tended to be concrete, 'as-a-matter-of-fact',and there was nunimal evidence of ahstract thinkingFourth, some of the NRs wrote the refiective paper like anyother academic paper Those papers appeared to he ratherimpersonal They descnbed ideas substantiated with refer-ences, as if they had never gone through the teachingexperience required of them

Student YCY was mterviewed to illuminate the thoughtprocess when the reflective paper was composed Theinterview was conducted usmg the teaching expenencedepicted in the refiective paper as a hase for discourseThe thought of YCY was characterized by the presence oflinear, simple logic of 'if-then' For instance, YCYexplained that 'if they are able to imderstand what you tellthem, then they will follow advice then it will improvetheir health ' These propositions were quite prescriptivefor YCY, and there was no demonstration of any effort tovalidate these assumptions YCY found that the NAE pro-gramme had taught her to wnte an educational plan,adopting the three leammg domains Subsequently YCYshowed careful planning of the teaching programme forher patients The effort to equip oneself to he a 'good' nurseeducator was evident m YCY, so much so that the valueand effect of patient education was seldom questioned

FEATURES OF REFLECTORS

This group of students demonstrated reflection merely atone or more of the first three levels, l e attending to feel-ings, association and integration The group included thelargest number, 34 (75 6%) out of a total of 45 studentsThe reflectors were able to relate their experiences andturned them mto new leammg opportunities

They could identify relationships between prior knowl-edge and/or feelings with new knowledge and/or feehngsThey were ahle to modify what was known to newsituations, arrived at insights and possibly ideas of someongmality However, unlike cntical reflectors, they didnot demonstrate effort m validatmg assumptions, signs ofmaking knowledge one's own, nor transformation mperspectives

FEATURES OF CRITICAL REFLECTORS

Table 1 shows the codix^ results of the 5 (111%) cnticalreflectors They were students who had attained reflectionat the level of validation, appropnation, and/or outcomeof reflection Incidentally, this group of students also dem-onstrated reflection at other levels The five pieces of workof the cntical refiectors (CR) were examined Vanouscommon features were identified from the wntten papers

First, the CRs always returned to the expenence m thediscussion There was an attempt by the CRs to continually

Table 1 Coding results of cntical reflectors

Suhcategones

Attending to feelmgsAssociationhitegrationValidationAppropnationOutcome of reflection

Student

MCY

112100

OCW

123010

OLK

103301

FIX

020101

OWF

051100

examine the expenence and themselves m a cnticalmanner Second, the CRs were able to frame the problemin context The CRs tended to adopt a wide and multi-dimensional perspective m dealing with the issue at handAs they were pursumg alternative views or possihilities,they drew on a number of resources, including pnorknowledge, existing information and the literature Third,the CRs tended to be courageous in trying out differentmethods, and were amenable to change The CRs declmed'hahitualization' of practice, and did not take things forgranted

Student OLK was interviewed In the interview OLKexplained how at first she made assumptions, then lateron confirmed or disconfirmed the propositions The testingof discrepancy between assumption and realify was delib-erate In the process, OLK drew on a number of resources,vaned from personal expenence, knowledge attained fromprevious course work, to the use of relevant literature OLKfound that the required assignment has encouraged her tobe 'like a scientist', to apply vigorous strategies to venfythings, regardless of whether they were seemingly right ornot The practise of such thought process was not new,although less senous, to OLK in her personal hfe OLKrecogmzed the personal element m reflection, particularlywhen the issue concerned 'daily expenence or one's ownphilosophy'

DISCUSSION

Reflectivity of individual students

The data have revealed that the post-registration nurse stu-dents m this study have demonstrated reflectivity mainlyat the attending to feelings, association, and mtegrationlevel This level of reflectivify appears to be antecedent tothe next level of validation, appropriation, outcome ofrefiection The second level corresponds with the 'criticalreflectivity'as described by Mezirow etai (1990) Studentswho attain the cntical reflectivify level have all achievedat the non-cntical level Students who have achieved atthe non-critical level do not necessanly manage to achieveat the cntical level

54 © 1995 Blackwell Science Ltd, fournal of Advanced Nursing, 22, 48-57

Assessing student reflection

This phenomenon seems to support Boud et al's (1985)model, which suggests that 'the [reflective] process maytend to proceed m the sequences we have descnbed'There is the other group of students who have not heenable to demonstrate evidence of the reflective elementsThis does not mean that this group of students have notgained from the expenence They have leamed to deviseplans, implement thoughtful actions However, they werenot able to turn the expenence into another potential learn-ing opportunity

Table 2 reveals that expenence, m terms of number ofyears of workmg, does not appear to have an effect on thelevel of reflectivity among students m this studyExperience m this sense does not necessanly imply learn-ing One might go through the day-to-day routmes, habits,and even thoughtful action, without critically examimnglife events, achieving personal knowledge and attainingperspective transformation

Analysis of student reflection using Boud's model

Conceptually the first three components of Boud's modelcan be separated from the latter three, which correspondwith the features of 'premise reflecbon' depicted byMezirow et al (1990) However, as suggested by Boud et al(1985) themselves, the elements of the reflective processare not simple and lmear Some stages might be omitted,and some of the elements could be compressed This canbe illustrated by examimng the result of OLK m Table 1For mstjmce, OLK has manifested no evidence of associ-ation, while she showed several instances of validationThe interpretation of the coders to an extent might affectthe coding results The following quote, taken from thescnpt of the same student, helps to illustrate the point

My proposal to make use of this health education programme toguide the health teachmg to patients with similar condition hadbeen turned down by the outcome itself When I re-exammed theprogramme, I gained further insight about health education It wasabout personal value and attitude change

In the first instance, the coders coded '3 ' twice, for theamval of insights by the student on the relationship ofhealth teaching and personal value, as well as health teach-ing and attitude change Then the coders decided that theparagraph should be given a coding of '4' The student hadgained insights in health education, and the student was

Table 2 Years of working experience of critical reflectors (CR),reflectors (R) and non-cntical reflectors (NR)

CR NR

MeanRange

987-15

10 65-20

10 75-15

'testmg the mtemal consistency between new appreci-abons and pnor knowledge or behefjs' (Boud et al 1985)by re-examming the programme Later on in the text, thestudent manges to demonstrate her efforts m testmg outthe inconsistencies This example tends to realize Boudet al's (1985) claim that 'it is not possible to regard them[the elements and stages of the reflective process] as dis-tmct and unrelated ' However, it may also suggest that finecoding to levels may not be particularly useful

Our findings are not consistent with those of Powell(1989) She sinalysed interview transcnpts by searchmg fortextual elements correspondmg to vanous levels of reflec-tion Subjects were given scores indicating how manytextual elements were allocated to each reflection level Asix-category coding scheme was based on Mezirow'searlier work (1981), while a five-category coding schemewas based on Colaizzi's (1973) work

There was no mdication in Powell's paper that theremight be any degree of uncertainty over the 'scores' Norwas there any discussion of the certainty with whichtextual elements in the transcnpts could be identifiedand attributed to categories corresponding to particularreflection levels However, as the paper reported neitherprocesses for verifying the reliability of the coding norany reliability data, we feel that the results of this studymust be viewed with considerable caution

The distmction between non-reflectors and reflectorscan be achieved using Boud et al's (1985) model Asdiscussed above, the fine differentiation of levels mightbe difficult The discnmmation from the first three levelsof reflection (attending to feehngs, association, inte-gration) from the other higher three levels (validation,appropriation, outcome of reflection) was, however,possible Special features could be identified from thesethree categones the non-reflectors, reflectors and cnticalreflectors

The use of reflective papers as study material

The reflective paper is a piece of assignment work as apartial fulfilment of the NAE course Students might havesome reservation m exposmg all their thoughts, in casethe 'negative expenence', or 'testing of inconsistencies'nught indicate their weakness m plannmg, and operatingof the teaching plan, and thus would affect their assess-ment mark The expression of the reflective process thestudents underwent might have been inhibited by the wnt-mg capability and the command of English of individualstudents One might suspect that there is a possibility thatstudents can reflect without putting it m their journalsThe findings of the interviews revealed that the students'thought processes could be elaborated m the mterviewHowever, the interviewees mamtamed at the samereflective level as they presented themselves m the wnttenpaper This suggests that wntten matenal can be used as

© 1995 Blackwell Science Ltd, foumal of Advanced Nursing, 22, 48-57 55

FKY Wongetai

a reliable tool to distinguish the primary categones ofreflectors

CONCLUSION

The study suggests that student writing m reflective jour-nals can be used as evidence for the presence or absenceof reflective thinking The process of allocating studentsto the three categones of non-reflector, reflector and criticalreflector was straightforward and reliable Further, alloca-tion of students to these three categones revealed manycommon and distinct features of the journals withm eachcategory

Identifying textual elements withm journals and allocat-ing them to the finer levels of reflection withm the Boudet al (1985) model was more problematic and considerablyless reliable As an exercise by itself we would have to saythat we do not feel that the levels of reliability we couldachieve were sufficient to recommend the process forfurther use by others

As a step towards establishing the three broader categor-ies, however, the coding process could be justified Eventhough individual coders might identify different textualelements within the journals, or might disagree over theprecise level of the aspect of reflection, there was noresulting disagreement as to the allocation of the studentto the three categones of non-reflector, reflector and cnticalreflector

We believe that we have established that the writing inreflective journals can be used to diagnose whether stud-ents are reflecting on their practice, and whether thisreflection is cntical in nature Numerous courses mnursing education, as well as many fields of professionaleducation, are now orientated towards promoting reflec-tive thinking A large proportion of these courses usereflective journals to encourage reflective thinking Theability to analyse scripts for evidence of reflective thinkingIS, therefore, important Unless it can be done in a rehableway it IS difficult to establish whether the courses areachieving their aims and encouraging reflective thinking

Precursor to further research

Establishing a reliable and widely accepted method fordetermining whether and to what level reflection takesplace IS an important precursor to further research intoreflective teaching and leammg A method for determiningthe extent and quality of reflection is necessary to examinethe influence of teaching initiatives, demographic charac-tenstics or expenmental variables

For example, if courses or the cumculum are redesignedto encourage reflective leaming, the teacher needs to havea measure of reflective thinking to detemune whether theinnovation does improve the promotion of reflection Thishas implications for the theoretical and practical curricu-

lum m both pre- and post-registration nursing coursesCompanson of the reflective practice of different classesof students with vanous course designs can only take placeif there is a means of examinmg their reflective thinkingChanges m hospitai practices and management to promotereflection upon practice can only be evaluated if there isa method for determining the extent to which reflectionoccurs

In this study we believe we have established a reliablecind usable method as clarifying students as non-reflectors,reflectors and cntical reflectors Categonzmg elements oftext according to elements within a complex method ofreflection was more problematic We mtend to take thisstudy further to see whether other categonzation systemsor models of reflective thinking prove to be more usefulor more reliable for analysing reflective writing We believethat we have established an appropriate procedure forresearch of this t}rpe, through the use of independentjudges and determination of reliability

Acknowledgements

We are grateful to Manan Wong, Linda Chan and Eva Kanfor their contribution m the data analysis process, andSally Chan and Sophia Chan for their participation inteaching the course The study was funded by researchgrants from the Department of Health Sciences, Hong KongPolytechnic University and from the Research Committeeof the Hong Kong Polytechnic University

References

Argyns C & Schon D (1974) Theory into Practice Jossey-Bass,San Francisco

Atkins S & Murphy K (1993) Reflection a review of the literatureJournal of Advanced Nursing 18(8), 1188-1192

Boud D, Keogh R & Walker D (1985) Reflection TurningExperience into Learmng Kogan Page, London

Bowman R F (1983) The personal student journal mirror of themmd Contemporary Education 55(1), 25-27

Bums S (1994) Assessing reflective leaming In ReflectivePractice in Nursing The Growth ofthe Professional Practitioner(Palmer A , Bums S & Bulman C eds), Blackwell ScientificPublications, Oxford, pp 20—34

Colaizzi P F (1973) Reflection and Research m Psychology APhenomenological Study of Leaming Hunt Publications, NewYork

Dewey J (1933) How We Think Regnery, ChicagoLemo-Kilpi H (1990) Self-reflection m nursmg teacher and

education Journal of Advanced Nursing 15(2), 192-195Fellows K & Zimpher N L (1988) ReflecUvity and the instruc-

tional process a definitional comparison between theory andpractice In Images of Reflection m Teachers Education(Waxman H C , Freiberg WJ, Vanghan IC & Weil M eds).Association of Teacher Educators, Reston, pp 1-19

Goodman J (1984) Reflection and teacher education a case studyand theoretical analysis Interchange 15(3), 9-26

56 © 1995 Blackwell Science Ltd, Journal of Advanced Nursing, 22, 48-57

Assessing student reflection

HahnemannBK (1986) Journal vin-iting a key to promoting criU-cal thinking m nursmg students foumal of Nursing Education25(5), 213-215

Jarvis P (1987) Adult Leammg in the Social Context CroomHelm, London

Jarvis P (1992) Reflective practice and nursmg Nurse EducationToday 12(3), 174-181

Kolb DA & Fry R (1975) Towards an applied theory ofexperiential learning In Theones of Group Processes (CooperC L ed ), John Wiley, London, pp 33-57

McCaugherty D (1992) Integrating theory and practice SeniorNurse 12(2), 36-39

Mezu-ow J (1981) A cntical theory of adult learning andeducation Adult Education 32(1), 3-24

Mezirow J (1991) Transformative Dimensions of Adult LearningJossey-Bass, San Francisco

Mezirow J & Associates (1990) Fostering Cntical Reflection mAdulthood A Guide to Transformative and EmancipatoryLearning Jossey-Bass, San Francisco

Miles M B & Huberman A M (1984) Qualitative Data AnalysisSage, California

Murphy K & Atkins S (1994) Reflection with a practice-led cur-nculum In Reflecting practice in nursing The growth of theprofessionai practitioner (Palmer A M , Bums S & Bulman Ceds), Blackwell Scientific Publications, Oxford, pp 10-19

Nicassio F (1992) Swamplog a structured journal for reflection-in-acUon The Writing Notebook 9(3), 13-18

Powell J H (1989) The reflective practitioner in nursing Journalof Advanced Nursing 14(10), 824-832

Reid B (1993) 'But we're doing it already'' Exploring a responseto the concept of reflective practice in order to improve its facili-tation Nurse Education Today 13(4), 305-309

Ross D D (1989) First steps in developing a reflective approachJournal of Teacher Education 40(2), 22-30

Saylor C (1990) Reflection and professional education art,science, and competency Nurse Educator 15(2), 8-11

Schon D [1987] Educating the Reflective Practitioner Jossey-Bass,San Francisco

Smyth J (1989) Developing and sustaining critical reflection inteacher education Journal of Teacher Education 40(2), 2-9

Snowball J , Ross K & Murphy K (1994) Illuminatmg dissertationsupervision through reflection Journal of Advanced Nursmg19(6), 1234-1240

Usher R & Bryant I (1989) Adult Education as Theory Practiceand Research Routledge, London and New York

Wong F K & Chung L C (1993) Re-visiting theory and practicethe Hong Kong experience — an evaluative study of reflectivelearning among post-registration nurses Proceedings of theFirst Nationai Nursing Conference Australia EffectiveCollaboration Effective PracUce in Nursing Charles StuartUniversity, Australia, pp 406-424

APPENDIX A

Coding scheme of reflective process (derived from Boud et al's(1985) model)

Elements ofreflective

Code process

1 Attending tofeelings

2 Association

3 Integration

4 Validation

5 Appropriation

6 Outcome ofreflection

Criteria Remarks

Utilizing positivefeelings

Removingobstructing feelings

Lmkmg of prior Relating the oldknowledge, feelings and the newor attitudes with Making way fornew knowledge, the newfeelings or attitudes

Discovenng priorknowledge, feelingsor attitudes that areno longerconsistent withnew knowledge.feelmgs or attitudes

Re-assessing priorknowledge, feelingsor attitudes andmodify toaccommodate newknowledge, feelingsor attitudes

Seeking the nature Relating the oldof relationships of and the newprior knowledge. Emergingfeelmgs or attitudes originalitywith newknowledge, feelingsor attitudes

Amving at insightsTesting for internalconsistencybetween newappreciations andpnor knowledge orbeliefs

Making knowledgeone's own

New knowledge.feelings or attitudeentering into ownsense of identity

New knowledge.feelmgs or attitudesbecoming asignificant force inown life

TransformaUon inperspectives

Change in behaviourReadiness forapplication

Commitment toaction

I 1995 Blackwell Science Ltd, Joumal of Advanced Nursing, 22, 48-57 57