This article was downloaded by: [Universitat Politècnica de València]On: 24 October 2014, At: 16:26Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Africa Education ReviewPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/raer20

Accountability in teacher education:Positioning pre-service teachers asevaluators of their performanceLungi Sosiboa

a Faculty of Education & Social Sciences, Cape Peninsula Universityof Technology, South AfricaPublished online: 23 Dec 2013.

To cite this article: Lungi Sosibo (2013) Accountability in teacher education: Positioning pre-serviceteachers as evaluators of their performance, Africa Education Review, 10:sup1, S159-S182, DOI:10.1080/18146627.2013.855441

To link to this article: http://dx.doi.org/10.1080/18146627.2013.855441

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Accountability in teacher education:Positioning pre-service teachers asevaluators of their performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Lungi SosiboFaculty of Education & Social Sciences,Cape Peninsula University of Technology,South AfricaEmail: SosiboL@cput.ac.za

AbstractIn the United States and elsewhere, the term ‘accountability’ is a buzzword in thediscourses on initial teacher preparation. Various accountability measures which arebelieved to reinforce the integrity and effectiveness of initial teacher education programmesand to serve as evidence of the pre-service teachers’ competences are in place in teacherpreparatory programmes. They include state-regulated initial certification for new teachers,high stakes state-required teacher testing for initial licensing and voluntary professionalaccreditation of teacher preparatory programmes and institutions. However, in countriessuch as South Africa, the concept of accountability in initial teacher preparation is seldomused. This action research study analyses the growth and development of pre-serviceteachers who, through the use of a debriefing form, underwent reflection and praxis whichfacilitated their growth and development. The debriefing form, which the pre-serviceteachers used to score their performance, became the instrument of data collection. Thescores that the pre-service teachers assigned themselves were analysed quantitatively. Inthis article, it is argued that by scoring themselves, pre-service teachers were takingaccountability for the outcomes of their performance. The findings suggest that with propertraining on the objectives and use of the debriefing form, pre-service teachers can growand develop as responsible individuals who can account for their actions and for theoutcomes of their actions.

Keywords: accountability, reflection, praxis, performance, outcomes

ISSN 1814-6627 (print), 1753-5921 (online)DOI: 10.1080/18146627.2013.855441University of South Africa Press

Africa Education Review 10Supplementary issue S1, 2013

pp. S159 – S182

S159

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

Introduction

Challenges related to Teaching Practice evaluations administered through observa-tions and criteria-based evaluation forms have been documented (Gordon, 2009;Leshem and Bar-Hama, 2008; O’Leary, 2004; Scriven, 1983 in Kulik, 2001).Scriven (1983, in Kulik, 2001) criticizes expert visits to the classrooms due to theirtendency to alter teaching and the inherent biases in evaluators that he claimsskew their observations. In a study conducted by Gordon (2009) on the TeacherEducation Programmes (TEPs) in a University of Technology in South Africa,the Teaching Practice coordinator of those programmes, alluding to the TeachingPractice evaluations administered through observations and criteria-based formsexpressed the same sentiment that ‘[Teaching Practice] assessments are subject-ive and therefore vary considerably’ (123). Leshem and Bar-Hama (2008) concurthat observation-based evaluations are subjective as they are based on theobserver’s own teaching approach. Nonetheless, although critical of observa-tions, O’Leary (2004: 14) asserts that ‘traditionally, classroom observation hasoccupied a prominent role in terms of its use as a tool by which to judge andsubsequently promote good practice’. This assertion alludes to the fact that,despite their challenges, observations and criteria-based evaluation forms are stillwidely used in pre-service teacher education.

The action research (AR) reported in this article is an extension of a 2011qualitative study in which I investigated students’ experiences on observations andcriteria-based forms as Teaching Practice evaluation instruments. In that qualitativestudy, I used in-depth semi-structured interviews to collect data from four focusgroups of between four and five student teachers from second, third and fourthyear Bachelor of Education (BEd) that was offered in the Further Education andTraining (FET) TEP in the Faculty of Education of a selected South African highereducation institution (HEI). My objective had been to understand whether theseevaluation instruments affected students’ performance and how. The purposeof the AR was to pilot an alternative Teaching Practice evaluation instrument (thedebriefing form) in order to determine if it could be used as a tool that allowedstudents to reflect and take accountability for their performance.

In the TEP under study, students’ final Teaching Practice scores were and arestill determined through a range of measures; including observations, criteria-based evaluation forms and a teaching-practice portfolio. Scores obtainedthrough observations and criteria-based forms are assigned by mentors anduniversity supervisors/evaluators and later collated and added to those of theTeaching Practice portfolio. These scores constitute a major component of thestudent teachers’ summative scores.

Lungi Sosibo

S160

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

Action research cycle

The current AR adopted the five phases of the AR cycle as developed bySusman (1983). These included diagnosing; action planning; taking action;evaluating the consequences of an action and specifying learning. These phasesare reported below.

DiagnosingDuring the interviews conducted in the 2011 study, students cited a number ofchallenges that they felt hindered their performance during Teaching Practiceevaluations. These included evaluators’ diverse philosophical standpoints andpersonalities, dissimilar contexts and limited resources, as well as the evaluationprocess. A brief summary of each of these issues is presented below.

Students reported that the evaluators’ diverse ideological standpoints made itdifficult for them to understand what constituted ‘good’ or ‘effective’ teaching.They mentioned that Teaching Practice administered evaluations set differentand conflicting evaluation standards which made evaluations subjective andbiased. They also held that evaluators made different interpretations of theevaluation forms, leading to a difficult for them in meeting the evaluators’expectations as these differed according to their interpretations. They furthercontended that some evaluators imposed their teaching styles on them, arguingthat this distorted the observation and evaluation outcomes.

Students also highlighted concerns related to material and human resources,arguing that the evaluators seemed oblivious to the diverse school contextswhen observing or evaluating them. Evidently, the availability or lack ofeducational resources impacted profoundly on the students’ marks as, accordingto them, those students in affluent schools tended to outperform those inimpoverished schools. In addition, students complained that evaluators assignedmarks as if the school contexts were similar. This issue raises questions aboutwhether different evaluation standards should be used for different schoolcontexts, or whether a one-size-fits-all approach should be adopted andevaluation instruments adapted to suite different contexts. Regarding humanresources, students commended the mentors for the skills and behaviours theytaught them but agreed that some mentors were not supportive.

Students also unequivocally expressed dissatisfaction with the broad evaluationcriteria and complex performance indicators on the evaluation forms, complain-ing that such elaborate criteria confused them and impeded their optimalperformance. They expressed a belief that the evaluators did not know which

Accountability in teacher education

S161

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

aspects to observe or which performance indicators to focus on duringevaluations. Furthermore, they voiced a great deal of discontent with theevaluators who merely made checks on the evaluation forms without providingdetailed feedback, and those who purportedly made inaccurate comments,claiming that such behaviour subjected them to unnecessary failure.

AnalysisFrom the pre-service teacher’s utterances, it could be inferred that observationsand criteria-based evaluation forms presented a number of challenges. Aconclusion could be drawn that the students regarded these evaluation measuresas subjective rather than objective means of evaluating the quality of theirperformance, which in their view distorted the outcomes of their performance.It might be possible that factors mentioned above did indeed eclipse theevaluation process, thus adversely affecting scoring reliability and validity(Worthen, 1993). However, it might also be possible that rather than takingaccountability for the outcomes of their performance, students were simplyshifting it to external factors or extenuating circumstances beyond which theymight have had no control. The latter situation highlights the inevitability of theneed for more valid and reliable alternative means of supplementing theevaluation instruments and of making students accountable for the outcomes oftheir performance during Teaching Practice evaluations.

Action planningIn response to the need expressed above, I developed a debriefing form to serveas an alternative Teaching Practice evaluation and reflection tool that wouldallow Teaching Practice evaluators to hold students accountable for theTeaching Practice evaluation outcomes and place accountability squarely intheir hands. In this study, it was argued that by evaluating or consenting toevaluate their performance, students were taking accountability for the out-comes of their performance and vice versa. The next section describes how thedebriefing form enabled the reflective and accountability process to unfold.

Taking actionCook-Sather (2010: 557) states that literature on the student voice highlights therarity of the inclusion of students as pedagogical consultants; as the generalassumptions about students and even college students is that ‘they are neitherable to offer nor interested in offering insights about teaching and learning’. Inorder to counter this assumption, the debriefing form was administered to make

Lungi Sosibo

S162

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

the students ‘pedagogical consultants’ (Cook-Sather, 2010: 557). The objectivewas to provide them with a more valid and reliable evaluation tool that wouldfeature them prominently as ‘primary actors or protagonists [rather than] assubjects to be acted upon by others’ (Cook-Sather, 2010: 556), thus compellingthem to take accountability for the outcomes of their performance duringTeaching Practice evaluations. Additionally, by actively engaging students inevaluating their own performance, I was democratizing the Teaching Practiceevaluation process and relegating power to the students, thus making evalua-tions transparent. Cook-Sather (2006) and other proponents of student voicework (Beuschel, 2008; Rudduck and McIntyre, 2007; Thiessen and Cook-Sather, 2007) agree that student inclusion is commensurate with the principlesof equity and democracy.

Before I developed the debriefing form, I invited a group of five evaluators torespond to a short survey which required them to identify four or five keyevaluation constructs that would constitute the theoretical framework for thedebriefing process. The idea was not to replace the existing evaluation form butto simplify it by limiting the variables on which debriefing would be done. Thisway, the variables would be as understandable and meaningful as possible toboth evaluators and students (Loughran and Corrigan, 1995).

As shown in Appendix 1, the debriefing form contains information such as thestudents’ names and year of study; pre-debriefing scores as determined bystudents prior to reflection; students’ reflection/feedback regarding what workedor did not work and how that which did not work could be improved, takinginto consideration the four variables which served as the evaluation framework(item 3: i–iv); evaluators’ reflection/feedback on what worked or did not workand how it could be improved; students’ comments/questions; students’ post-debriefing scores; and a summary which reflects the evaluators’ scores andstudents’ pre- and post-debriefing scores, names and signatures of studentteacher and evaluator, as well as the date on which the debriefing was done.

The score sheet (Appendix 2) which was used together with the debriefing formconsists of eight categories, namely, students’ fictitious names, year of study,evaluators’ scores, students’ pre-debriefing scores, students’ post-debriefingscores, variations between evaluators’ and students’ pre-debriefing scores,variations between students’ pre-debriefing and post-debriefing scores, as wellas variations between evaluators’ and students’ post-debriefing scores. Thesescores were the focus of the AR and were used to determine the extent to whichstudents could be held accountable for their performance. Students wererequired to keep the debriefing forms as an integral component of their

Accountability in teacher education

S163

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

Teaching Practice portfolios as they served as an accountability tool. Theprocedures followed in using the debriefing form and score sheets are explainedin more detail in the methodology section.

Theoretical framework

In this AR the focus was on understanding the extent to which the debriefingform could serve as a reflection tool that would enable students to takeaccountability for Teaching Practice evaluation outcomes. Therefore, thereflection theory was deemed appropriate for this study. Dewey (1933) iswidely recognized as someone who came up with this concept in the twentiethcentury. He regarded reflection as problem-solving or thinking about solving aproblem, which involved action chaining. Thus, according to him, reflection isan active and deliberative cognitive process which involves reflective thinkingand reflective action. Schön (1983, 1987) presents two forms of reflection, thatis, reflection-in-action, which he describes as reflection that happens whileaction (e.g. teaching) is still occurring; and reflection-on-action, which hedescribes as reflection that occurs after the event. Clearly, Schön’s definition ofreflection is intrinsically related to action. According to him, through reflectionand action, professionals are bound to make rational judgements about how tomodify their actions and find new ways of doing them while in action(reflection-in-action) or after the action has occurred (reflection-on-action).

With regard to teacher education, Zeichner (1990) and Hall (1985) claim thatemphasizing reflection too soon in their preparation turns novice teachers offand become difficult to sustain. The assumption is that the neophytes tend toperceive it as a worthless distraction that takes their attention away frommastering the content and teaching skills they are particularly anxious about.McNamara (1990) and Noffke and Brennan (1988) concur and recommend adevelopmental approach that allows students an opportunity to acquire meta-teaching and metacognitive skills before they are required to practice reflection.Although I support the developmental approach, I contend that reflection shouldbe an integral component that is incorporated in all the teaching skills and notbe seen as a separate entity, regardless of the students’ level of study as thiswould enhance their holistic growth and development.

Accountability in teacher education

This section provides a definition and pertinent literature on accountability, whichis the focus of this AR. The Reader’s Digest Great Encyclopedic Dictionary

Lungi Sosibo

S164

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

(1966) describes the verb ‘to account’ as ‘to give a rational explanation’, ‘to beresponsible’ and ‘to answer’; and the adjective ‘accountable’ as ‘liable to be calledto account’ and ‘responsible’. The Random House Thesaurus, College Edition(1984) defines accountability as ‘answerability,’ ‘responsibility,’ and ‘liability’.The Business Dictionary defines accountability as ‘the obligation of an individualor organization to account for its activities, accept responsibility for them, and todisclose the results in a transparent manner’. It would thus appear that in literaturethe terms accountability, answerability, and responsibility are used interchange-ably. Cook-Sather (2010: 555) concurs, that ‘to be responsible is to be answerableor accountable for something within one’s power, control, or management’.

Emmanuel and Emmanuel (1996) claim that accountability contains threeessential components. The first is the loci of accountability which includesdifferent parties that can be held accountable or hold others accountable. InTeaching Practice, these parties could be evaluators, mentors, student teachersor learners. The second contains the domains of accountability which areactivities for which parties can be held accountable, which in this study, is items3 (i–iv) on the debriefing form. The third component constitutes formal andinformal procedures of accountability for evaluating compliance, which in thisstudy is the debriefing form. Although Emmanuel and Emmanuel (1996) werediscussing accountability in the context of Health Care, the three componentsthey mentioned fit snugly in teacher education.

The term accountability became a buzzword in the USA as a result of the NoChild Left Behind (NCLB) Act of 2001 and the heightened criticism levelledagainst higher education for low quality teacher production. Consequently, therewas urgency for teacher education programmes to put accountability measuresin place in order to ensure high quality graduates. With the NCLB legislationcame numerous calls for diversified and more valid and reliable accountabilitymeasures and more authentic assessments of evaluating higher education ingeneral and pre-service teachers’ performance specifically (Palomba and Banta,1999). In the context of teacher education, these measures were seen asreinforcing the integrity and effectiveness of initial teacher education pro-grammes, as well as serving as evidence of the pre-service teachers’competences that they knew (content) and were able to do (pedagogy) whatthey ought to know and be able to do as determined by a set of specificevaluation criteria (Cochran-Smith, 2001: 20). Among these measures werestate-regulated initial teacher certification, high stakes state-required teachertesting for initial licensing, and voluntary professional accreditation of teacherpreparatory programmes and institutions (Cochran-Smith and Zeichner, 2005).

Accountability in teacher education

S165

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

However, with time these measures came under heavy scrutiny and criticism asquestions were asked about the validity and reliability of the standardized tests(The Coalition for Effective Schools, 1990, in Evans, Daniel, Micovch, Metzeand Norman, 2006). In reaction to this criticism, a number of locally developedaccountability measures that supplement the mandatory tests were developed,such as electronic portfolios for documenting student teacher performance,using learners as pedagogical consultants in student teacher development,improved data systems, student teacher interviews, to name but a few (Cook-Sather, 2010; Ludlow, Mitescu, Pedulla, Cochran-Smith, Cannady, Enterlineand Chappe, 2010; Militello, Schweid and Sireci, 2010; Ballard and Bates,2008; Reusser, Butler, Symonds, Vetter and Wall, 2007; Evans et al., 2006).

In the context of South African teacher education, the Higher Education QualityCouncil (HEQC) is a body regulated by the Council on Higher Education(CHE), which is a statutory body of the Department of Higher Education andTraining (DHET) aimed at improving the standards in teacher education andother university offered programmes through formal quality assurance mea-sures. The HEQC uses desktop reports developed by universities throughreflection on their programmes and processes, as well as through site visitationsby HEQC cadres. Although this quality assurance process may help inmaintaining standards and enhancing accountability on the part of thestakeholders, not much research has been conducted on its impact. There isstill a need for more research in this area and also to determine the extent towhich this process enhances accountability among both teacher educators andneophytes.

Methodology

The purpose of this AR was to pilot the debriefing form. Hence, as a newinstrument that was still being tested, it was administered to a randomly-selectedsample of only 20 students at second, third and fourth year levels of the FETBEd degree. For a pilot study, this sample was considered adequate. First yearswere excluded as they did not have formal Teaching Practice evaluations. Sincementors were known to inflate students’ scores, they were also excluded fromthe study. Only the scores allocated by university-based evaluators were used inthis study as these were generally considered more reliable than those assignedby mentors. In order to prevent a conflict of interest on the part of thedebriefing-form developer and for purposes of triangulation, the form wasadministered by three different evaluators, including two who had notparticipated in its development but had contributed in refining it. The validity

Lungi Sosibo

S166

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

and reliability of the form were tested by the five evaluators who hadparticipated from its inception.

Before the students went out to do Teaching Practice, they were informed aboutthe purpose and importance of the debriefing forms as supplementary evaluationinstruments that would enable them to be coached by evaluators and to reflecton their performance. The forms were distributed to them as part of theirTeaching Practice evaluation packs. As consumers, they were charged with theresponsibility of handing them to the evaluators before the latter left theevaluation sites. Since the importance of the form had been emphasized to thestudents and it was assumed that they needed assistance and guidance, it wasbelieved that they would take this responsibility seriously or else they wouldmiss this opportunity. In order to ensure a thorough understanding, all theTeaching Practice coordinators were asked to explain and clarify anymisconceptions about the debriefing forms to the students. Students were alsofamiliarized with the four variables on which the debriefing would be based.They were also given an opportunity to experiment with the forms during themicro lessons. Evaluators were also briefed about the purpose and format of theforms.

The debriefing forms and the score sheets served as primary data-collectioninstruments as all the reflections and scores were recorded therein. Specifically,both qualitative and quantitative data were collected in the current AR in theform of students’ and evaluators’ reflection feedback and their evaluationscores. However, only quantitative data were used to determine the extent of thestudents’ accountability. That is, a comparison among the scores of theevaluators and students’ pre- and post-debriefing scores were the focus of thisstudy. Further research needs to focus on the qualitative reflection feedbackdone by both parties. Even though the qualitative data were not used in the AR,it is believed that the reflection process had an impact on the manner in whichstudents allocated their post-debriefing scores.

The debriefing process followed several chronological steps after the studentshad finished teaching the lessons. It began when the evaluators asked studentsto allocate themselves a pre-debriefing score based on their own judgments oftheir teaching performance. This score was recorded in the debriefing form. Byallocating a score, students were assumed to be committing themselves andtherefore taking accountability for their performance. The next phase requiredthem to respond to the three questions pertaining to their performance (items 2a,b and c on the debriefing form), taking the four constructs which served as theframework into consideration (item 3: i–iv on the debriefing form). The

Accountability in teacher education

S167

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

evaluators then gave them feedback regarding their teaching performance asguided by items 3a, b and c on the debriefing form and by the same fourvariables. Students were then given a chance to comment or ask questions to theevaluators regarding their feedback (item 5 on the form), followed by themassigning themselves a post-debriefing score as informed by both their andevaluators’ feedback. These scores were also recorded in the debriefing form.

After all the scores were recorded and the reflection by both parties had beencompleted, the students and evaluators signed and dated the debriefing form. Itwas only after both parties had signed the form that the evaluators disclosed thescores they had assigned to the students. When the debriefing sessions werecompleted, I collected the debriefing forms from the two evaluators and collatedall the data, and then recorded the quantitative data in the score sheet, as shownin Appendix 2.

Several ethical considerations were made in this AR. For instance, fictitiousnames were used in order to conceal the participants’ identity. They also had tosign the consent forms before they participated in the study. They were alsoinformed about their right to decline or withdraw from participation at any time.

Evaluating

The scores collected were analysed quantitatively using the ANOVA test,scatterplots and correlations. Generally, the students adjusted their post-debriefing scores, bringing them closer to those of the evaluators and resultingin a lower mean between the two sets of scores. For instance, the mean of −2.68between the evaluators and students’ pre-debriefing scores indicated thatoverall, the students had scored their performance 2.6 higher than the evaluator,changed to −1.21 after the debriefing. This meant that after the debriefing, theyscored themselves only 1.21 higher than the evaluator. It was, however, obviousfrom these means that even though they adjusted their post-debriefing scores,the adjustment was marginal and the majority of them still scored themselveshigher than the evaluator.

A mean of −1.85 was found between the students’ post- and pre-debriefingscores, meaning they scored themselves 1.85 higher during pre- than post-debriefing. Nonetheless, this mean was still higher, suggesting that they did notchange their marks dramatically after the debriefing. Similarly, the median scorealso changed as a result of the debriefing. For instance, the median of −4.00was found between the evaluator and students’ pre-debriefing scores. Thismedian decreased dramatically to −1.00 between the evaluator and students’

Lungi Sosibo

S168

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

post-debriefing scores. Nonetheless, the median remained relatively highat −3.00 between the students’ post- and pre-debriefing scores, suggestingthat even though they decreased their post-debriefing scores, the adjustment wasnot dramatic. The median score was also slightly to the left, suggesting that thestudents’ scores were generally higher than those of the evaluators even after thedebriefing.

The standard deviation of 6.26 between the students’ pre-debriefing andevaluators’ scores indicated a pretty large variation (range between −15.00and 8.00) between the two sets of scores. The standard deviation of 4.54between the students’ post-debriefing and evaluators’ scores showed that thevariation was still high (ranging between −11.00 and 7.00) but not as high asbefore the debriefing. However, although the standard deviation was less afterthe debriefing, it was clear that the students still scored themselves higher thanthe evaluators. The standard deviation of 3.55 between the students’ post-and pre-debriefing scores indicated that the variation between these scoresdecreased (the range decreased from between −15.00 and 8.00 to between−7.00 and 5.00), suggesting that the debriefing marginally decreased students’scores and changed their attitudes.

Graphically, these distributions could be shown as follows:

The scatterplots tested the correlations between the evaluators and students’ pre-and post-debriefing scores, that is, the extent to which students’ scores were

0.0

1.3

2.5

3.8

5.0

-15.0 -8.8 -2.5 3.8 10.0

Histogram of Diff_Eval_Stud_Pre_Debr

Diff_Eval_Stud_Pre_Debr

Cou

nt

Figure 1: Histogram of difference between evaluators’ and students’ post-debriefingscores (SD b/n-15 & 8)

Accountability in teacher education

S169

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

close to the goal standard (evaluators’ scores) in accordance with their levels ofstudy. As can be seen in the graphs below, there was a relationship. As shown inFigure 3 below, during the pre-debriefing the students’ scores were all over theplace, indicating that the relationship between their scores and those of the

50.0

60.0

70.0

80.0

50.0 57.5 65.0 72.5 80.0

Stud_Pre_Debrief_Score vs Eval_Score

Eval_Score

Stu

d_P

re_D

ebrie

f_S

core

Year_of_Study234

Figure 3: Scatterplots of students’ pre-debriefing and evaluators’ scores

0.0

1.3

2.5

3.8

5.0

-15.0 -8.8 -2.5 3.8 10.0

Histogram of Diff_Eval_Post_Debr

Diff_Eval_Post_Debr

Cou

nt

Figure 2: Histogram of difference between students’ and evaluators’ and students’ post-debriefing scores (SD b/n-11& 7)

Lungi Sosibo

S170

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

evaluators was not close. This relationship was confirmed by the comparisonbetween the evaluators’ mean scores of 56, 58 and 72 and the students’ pre-debriefing mean scores of 66, 57 and 74 for second, third and fourth years,respectively, which indicated that the second years were way off the goalstandard when compared with their third and fourth year counterparts. Thus, aconclusion can be drawn that during the pre-debriefing, the second years werethe worst in terms of scoring themselves and third years were worse off thantheir fourth year counterparts in terms of scoring themselves. With regard to thethird years, it is possible that the mean score might have been affected by thesmall sample and not necessarily by their inability to score themselves.

Nonetheless, once debriefed, their scores came closer to the evaluators’, asshown in Figure 4 below. When compared with the evaluators’ mean scores of56, 58 and 72 for second, third and fourth years respectively, one could see animproved correlation between the evaluators’ and students’ post-debriefingmean scores of 63, 56 and 72 per study level. In fact, the fourth yearsoutperformed the lower two levels in terms of evaluating themselves duringpost-debriefing, as shown by the extent to which most of their scores are closerto the goal standard (evaluators’ scores) in Figure 4 below. However, the secondyear students’ scores were still far from the goal standard. This situation is

50.0

60.0

70.0

80.0

45.0 53.8 62.5 71.3 80.0

Stud_Post_Debrief_Score vs Eval_Score

Eval_Score

Stu

d_P

ost_

Deb

rief_

Sco

re

Year_of_Study234

Figure 4: Scatterplots of students’ post-debriefing and evaluators’ scores.Note: Scores are scattered all over the place, students’ scores are closer to evaluators’suggesting low correlation

Accountability in teacher education

S171

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

confirmed by the evaluators’ mean score of 56 and second years’ mean scoreof 63.

The analysis of variance (ANOVA) tested whether there were differences amongthe different levels of study. The graphs below summarize the findingssuccinctly. Figure 5 shows that all the groups overrated themselves duringpre-debriefing; as shown by the mean of −2.68 for all the groups. However, thesecond years were the worst off with a mean of −8.16, which indicates that theysignificantly overrated their performance.

After the debriefing, the mean improved to −1.21 for all the groups. However,the second years’ mean of −5.57 is great, compared to those of 2.6 for the thirdyears and 0.42 for the fourth years, as shown in Figure 6 below.

In all the instances, the probability levels of 0.12, 0.05 and 0.27 were found tobe not significant due to the small size sample.

Specifying learning

Several conclusions could be drawn from the analysis of the findings. One isthat the debriefing worked. Students willingly adjusted their scores upward ordownward after the debriefing, which brought their scores closer to the standardgoal (evaluators’) against which they benchmarked their evaluations of their

-15.00

-8.75

-2.50

3.75

10.00

2 3 4

Box Plot

Year_of_Study

Diff

_Eva

l_S

tud_

Pre

_Deb

rief

Figure 5: Mean difference b/n evaluators’ and students’ pre-debriefing scoresaccording to level of study

Lungi Sosibo

S172

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

teaching performance. After the debriefing, there was a high correlationbetween the evaluators’, especially the final-year students’ scores. Thiscorrelation may be due to chance, but it may also be strongly associated withtheir familiarity with the evaluation process and reflective practice. The oppositeis true for especially the second year students. Their lack of experience andfamiliarity with evaluations and with the reflection process may have led to thelow correlations between their and the evaluators’ scores. This may supportthe argument I made earlier, that reflection should be integrated in teacherpreparation right away and not later, as argued by Zeichner (1990), Hall (1985),McNamara (1990) and Noffke and Brennan (1988).

Notably also is that even though the students adjusted their scores after thedebriefing, the adjustments were not drastic but they effectively brought thescores closer to the evaluators’. Similarly, the relative change was not much.Students with higher scores remained with higher scores and vice versa. Therewere no instances where high scores dramatically changed to low scores andvice versa although overall, the second years considerably overrated theirperformance. These phenomena were obvious in the three different statisticalanalysis (ANOVA, scatterplots and correlations) undertaken and may be anindication that the debriefing influenced them to score themselves objectively.

-15.00

-8.75

-2.50

3.75

10.00

2 3 4

Box Plot

Year_of_Study

Diff

_Eva

l_P

ost_

Deb

rief

Figure 6: Mean difference b/n evaluators’ and students’ post-debriefing scoresaccording to level of study

Accountability in teacher education

S173

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

It can be assumed that by consenting to evaluate their own teachingperformance, students were committing themselves and taking accountabilityfor the outcomes of their performance. In other words, they were takingresponsibility for the things they did, how and why they did them the way theydid. They were also taking account for what they were or were not able to do.What about those students who responded with ‘Don’t know’? This response isdisconcerting but it would have been more so if it had come from the fourthyear students who are about to become certified as teachers as it would point toa lack of accountability and commitment on their part and to them as non-reflective practitioners. Fortunately, these utterances were made by two second-and one third-year students who may not be as advanced and experienced asfourth years are thought to be.

Several interpretations could be made about the students who gave ‘don’t know’responses. This could indicate that these students were refusing to takeresponsibility for the outcomes of their performance. Another explanation couldbe that these students were unable to judge their performance prior to thedebriefing. It could also be that they lacked confidence, as in the case of thesecond-year student whose evaluator score was 58%. The other assumptioncould be that they were reluctant to assign themselves low scores, judging bythe low evaluators’ scores of 45% and 48% for the two students doing secondand third years of study. However, this behaviour was more pronounced on thepart of the student teacher who declined to allocate both pre- and post-debriefing scores.

Although the qualitative data were not used in this action research, it is hopedthat a paradigm shift occurred in students in terms of taking accountability andfocusing on themselves as responsible for the outcomes of their ownperformance instead of pointing at external forces. Granted, some of the factorsthat students blamed for the outcomes of their performance in the qualitativestudy cannot be changed overnight, such as the state of impoverished schools. Itis therefore important that they learn early on to focus on themselves and howthey could deal with the mitigating factors in their teaching endeavours ratherthan shift the blame. Doing so would allow them to grow and develop asreflective teachers and help them to learn to adapt to diverse teaching andlearning environments.

Conclusion

This action research sought to determine the extent to which the debriefing formcould serve as a tool for making students accountable for the outcomes of their

Lungi Sosibo

S174

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

performance in Teaching Practice evaluations. The decision to use the debrief-ing form was taken after it was inferred that students shifted accountability ofthe outcomes of their performance to external factors such as contexts,subjectivity and the evaluation process. In this study it was argued that byconsenting to evaluate their own performance, students were taking accountab-ility for the outcomes of their performance. Indeed, the majority of themevaluated themselves using the debriefing form, thus consenting in takingaccountability. Nonetheless, second and third year students proved that theywere not yet in a position to evaluate themselves accurately using the debriefingform but their fourth year counterparts proved to have attained this competence.Be that as it may, this cannot be said with certainty as chance might have had animpact on the success of the fourth years in evaluating themselves successfully.The results of this study cannot be generalized to other populations due to thesmall sample size. It is important to replicate this study using a larger sample soas to make the results generalize able. In addition, it would be important toconduct a longitudinal study with a bigger sample of students from their secondto fourth year in order to determine their growth and development, as well as tosee if the fourth years retain the accuracy in self-evaluation or whether theoutcome was by chance. Only then can valid conclusions be drawn about thevalidity and effectiveness of the debriefing form as an accountability instrument.Perhaps this could result in TEPs investing less on fourth year Teaching Practiceevaluations as these students would have acquired competence in self-evaluation. Furthermore, further studies should be conducted in order todetermine if there are differences in how students evaluate their performancebased on factors such as race, gender, school context, etc. It would also behelpful to conduct a qualitative study after the form has been used for severalyears, in order to investigate the students’ perceptions of this form and todetermine if they have further concerns about observations and criteria-basedforms as Teaching Practice evaluation measures. Additionally, it would beimportant to use the qualitative data collected in this AR in order to determine ifthe students’ reflective feedback was commensurate or not with the scores theyassigned themselves.

References

Ballard, K. and Bates, A. 2008. Making a connection between studentachievement, teacher accountability, and quality classroom instruction.The Qualitative Report, 13(4): 560–580.

Beuschel, A.C. 2008. Listening to students about learning. A report from theCarnegie Foundation for the Advancement of Teaching. Strengthening pre-

Accountability in teacher education

S175

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

collegiate education in community colleges. Available at: http://www.carnegiefoundation.org/publications/index.asp (Accessed 22 May 2012).

Cook-Sather, A. 2010. Students as learners and teachers: taking responsibility,transforming education, and redefining accountability. Curriculum Inquiry,40(4): 555–575.

Cook-Sather, A. 2006. Sound, presence and power: “Student voice” ineducational research and reform. Curriculum Inquiry, 36(4): 359–390.

Cochran-Smith, M. 2001. Constructing outcomes in teacher education: policy,practice and pitfalls. Education Policy Analysis Archives, 9:11. Available at:http://epaa.asu.edu/ojs/article/view/340/466 (Accessed 12 March 2011).

Cochran-Smith, M. and Zeichner, K.M. eds. 2005. Studying teacher education.The Report of the AERA on Research and Teacher Education.

Dewey, J. 1933. How we think: a restatement of the relation of reflectivethinking to the educative process. Boston: D.C. Health.

Emmanuel, E.J. and Emmanuel, L.L. 1996. What is accountability? Medicineand Public Issues, 124(2): 229–239.

Evans, S., Daniel, T., Mikovch, A., Metze, L. and Norman, A. 2006. The use oftechnology in portfolio assessment of teacher education candidates. Journalof Technology and Teacher Education, 14(1): 5–27.

Gordon, A. 2009. Cutting and pasting: changing the fabric of teacher educators’work at CPUT. In Opportunities & challenges for teacher educationcurriculum in South Africa, ed. G. Kruss. Cape Town: HSRC.

Hall, G.E. 1985. A stages concern approach to teacher education. Paperpresented at the 69th Annual Meeting of the American EducationalResearch Association, Chicago, IL. March 31–April, 1985.

Kulik, J.A. 2001. Student ratings: validity, utility, and controversy. NewDirections for Institutional Research, 109: 9–25.

Leshem, S. and Bar-Hama, R. 2008. Evaluating teaching practice. ELT Journal,62(3): 257–265.

Loughran, J. and Corrigan, D. 1995. Teaching portfolios: a strategy fordeveloping learning and teaching in pre-service education. Teaching andTeacher Education, 11: 565–577.

Ludlow, L., Mitescu, E., Pedulla, J., Cochran-Smith, M., Cannady, M.,Enterline, S. and Chappe, S. 2010. Journal of Education for Teaching, 36(4): 353–368.

McNamara, D. 1990. Research on teachers’ thinking: its contribution toeducating student teachers to think critically. Journal of Education forTeaching, 16(2): 147–160.

Militello, M., Schweid, J. and Sireci, S.G. 2010. Formative assessment systems:evaluating the fit between school districts’ needs and assessment systems’

Lungi Sosibo

S176

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

characteristics. Educational Assessment, Evaluation and Accountability, 22(1): 29–52.

Noffke, S. and Brennan, M. 1988. The dimensions of reflection: a conceptualand contextual analysis. Paper presented at the Annual Meeting of theAmerican Education Research Association [AERA], New Orleans, LA,April 5–9, 1988.

O’Leary, M. 2004. Inspecting the observation process: classroom observationunder the spotlight. IATEFL Teacher Development SIG, 1(4): 14–16.

Palomba, C.A. and Banta, T.W. 1999. Assessment essentials: planning,implementing, and improving assessment in higher education. San Fran-cisco, CA: John Wiley & Sons.

Reusser, J., Butler, L., Symonds, M., Vetter, R. and Wall, T.J. 2007. Anassessment system for teacher education quality improvement program.International Journal of Educational Management, 21(2): 105–113.

Rudduck, J. and McIntyre, D. 2007. Improving learning through consultingpupils. London: Routledge.

Schön, D. 1983. The reflective practitioner. New York: Basic Books.Schön, D. 1987. Educating the reflective practitioner. San Francisco, CA:

Jossey-Bass.Stryk, L.R. and McCoy, L.H. 1993. Pre-service teachers’ use of videotape for

self-evaluation. The Clearing House, 67(1): 31–34.Susman, G. 1983. The business dictionary. [Online] Available at: http://www.

businessdictionary.com/definition/accountability.html#ixzz21j2PeodG(accessed on 6 November 2013).

The Coalition for Effective Schools. 1990. Performances and exhibitions: Thedemonstration of mastery. Horace, 6(3): 1–12.

The Reader’s Digest Great Encyclopedic Dictionary, Including Funk &Wagnalls Dictionary. 1966. Pleasantville, NY: Reader’s Digest Association.

The Random House Thesaurus, College Edition. 1984. New York: RandomHouse.

Thiessen, D. and Cook-Sather, A. eds. 2007. International handbook of studentexperience in elementary and secondary school. Dordrecht: Springer.

Worthen, B.R. 1993. Critical issues that will determine the future of alternativeassessment. Phi Delta Kappan, 74: 444–457.

Accountability in teacher education

S177

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

APPENDIX 1.ABBRIDGED DEBRIEFING FORM (to be used in discussion betweenevaluator and student teacher after lesson has been presented)

1) NAME OF STUDENT: ____________________

STUDY LEVEL: __________

2) Pre-debriefing scores (determined by student before the debriefing):_______%

3) Student feedback:

In answering the questions that follow, please reflect on the following variables:(i) connecting theory and practice; (ii) methods/approaches (includinginstructional materials, assessment/questioning techniques, and learnerparticipation); (iii) social justice (i.e. dealing with learners from diversecontexts); and (iv) mastery of subject matter/content).

What, in your opinion:

a) Has worked, and why?

b) Has not worked and why?

c) How could you have done (b) better?

4) Evaluator feedback:

a) Evaluator feedback 1: These are the aspects in which, in my opinion, youhave performed well, because:

b) Evaluator feedback 2: These are the aspects in which, in my opinion, youhave not performed well, because:

c) This is how you could improve on the aspects in which you have notperformed well:

5) Student comments/questions:

6) Post-debriefing scores (determined by student after the debriefing):______%

7) Summary:

Evaluator’s score: _____% Student’s Pre-debriefing score: ____%

Lungi Sosibo

S178

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

Student’s Post-debriefing score____%

Student’s Printed Name and Signature: ________________

Date: ________________

Evaluator’s Printed Name and Signature:___________________

Date: ________________

Accountability in teacher education

S179

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

Appendix 2. The score sheet.

VARIATIONS BETWEEN

1 2 3 4 5 6 7 8

Studentname(fictitiousnames)

Year ofstudy

Evaluators’scores

Students’pre-

debriefingscores

Student’spost-

debriefingscores

Evaluatorsand

students’pre-

debriefingscores

Students’pre- andpost-

debriefingscores

Evaluatorsand

students’post-

debriefingscores

AsandaMayoli

2 50 65 58 −15 −7 −8

FrancoisDu Pre

3 57 50s 50s ? 0 ?

BonganiMtshali

2 68 75 70 −7 −5 −2

BonangMotsa

4 79 70–75 75 −9 0–5 4

LucasMitchell

3 62 56–60 56–60 2–6 0 2−6

DorothyFeltman

4 66 65 65 1 0 1

AdrianFortuin

4 71 69 69 2 0 2

LungiSosibo

S180

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

Appendix 2 (Continued)

VARIATIONS BETWEEN

1 2 3 4 5 6 7 8

Studentname(fictitiousnames)

Year ofstudy

Evaluators’scores

Students’pre-

debriefingscores

Student’spost-

debriefingscores

Evaluatorsand

students’pre-

debriefingscores

Students’pre- andpost-

debriefingscores

Evaluatorsand

students’post-

debriefingscores

LynetteMeyer

2 55 65 61 −10 −4 −6

Moira Bailey 2 45 Don’t know Don’t know ? 0 ?

AngeliqueAdams

4 72 76 73 −4 −3 −1

MonaMeiring

2 59 60 61 −1 1 −2

AnthonyBassadien

4 71 75 70 −4 −5 1

Duke Gema 3 58 50 55 8 5 3

MerriamAdonis

3 66 70 65 −4 −5 1

Accountabilityin

teachereducation

S181

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014

Appendix 2 (Continued)

VARIATIONS BETWEEN

1 2 3 4 5 6 7 8

Studentname(fictitiousnames)

Year ofstudy

Evaluators’scores

Students’pre-

debriefingscores

Student’spost-

debriefingscores

Evaluatorsand

students’pre-

debriefingscores

Students’pre- andpost-

debriefingscores

Evaluatorsand

students’post-

debriefingscores

Felicia Smit 4 76 80 77 −4 −3 −1

Brian Simms 3 48 Don’t know 50 ? ? −2

AbdullahIbrahim

4 70 79 73 −9 −6 −3

MarilynSmith

2 62 70 73 −8 3 −11

PapidouAlama

2 58 Don’t know 65 ? ? −7

ElaineMakhanya

2 52 60 55 −8 −5 −3

LungiSosibo

S182

Dow

nloa

ded

by [

Uni

vers

itat P

olitè

cnic

a de

Val

ènci

a] a

t 16:

26 2

4 O

ctob

er 2

014