Evaluation of reflective practice in teacher education

This article was downloaded by: [Moskow State Univ Bibliote]On: 05 February 2014, At: 10:41Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

European Journal of Teacher EducationPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/cete20

Evaluation of reflective practice inteacher educationEsther Belvisa, Pilar Pinedaa, Carme Armengola & Victoria Morenoa

a Social and Systematic Pedagogy Department, AutonomousUniversity of Barcelona, Barcelona, Spain.Published online: 14 Sep 2012.

To cite this article: Esther Belvis, Pilar Pineda, Carme Armengol & Victoria Moreno (2013)Evaluation of reflective practice in teacher education, European Journal of Teacher Education,36:3, 279-292, DOI: 10.1080/02619768.2012.718758

To link to this article: http://dx.doi.org/10.1080/02619768.2012.718758

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/loi/cete20

http://www.tandfonline.com/action/showCitFormats?doi=10.1080/02619768.2012.718758

http://dx.doi.org/10.1080/02619768.2012.718758

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

Evaluation of reflective practice in teacher education

Esther Belvis*, Pilar Pineda, Carme Armengol and Victoria Moreno

Social and Systematic Pedagogy Department, Autonomous University of Barcelona,Barcelona, Spain

Teacher education based on ‘reflective practice’ consists of observing, analysingand reflecting on teacher performance in order to improve professional practice.This article presents the results of an evaluation of a programme on mathematicsteaching carried out using reflective practice. It was targeted at 284 teachers invarious stages of the education system, using an approach that combines quanti-tative (surveys) and qualitative (interviews and reports) methods. While thestudy shows that the education programme generates high levels of satisfaction,pedagogical appropriateness and learning, its achievements in effectiveness aremoderate. Although, in general, what is learned through teacher education isimplemented in the classroom, it is done so in an isolated fashion, in otherwords, individually and without becoming a part of the culture of the school.The results show little evidence of its impact on student learning. The articleoffers proposals for improving the education programme, while, at the sametime, analysing the usefulness of the approach and evaluation tools employed.

Keywords: continuous teacher education; reflective practice; evaluation ofeducation; education results; transfer; impact of education

1. Education in reflective practice and the evaluation models

Continuous teacher education is a key element to ensuring the quality of teachingas well as providing appropriate responses to the changing demands of our society.The investment made in teacher education is considerable, both from public organi-sations as well as from individual centres and teachers. Questions remain as to howmuch we know about the usefulness of this investment in teacher education and towhether it improves teaching practices or teacher education or has any effect onschools.

Teacher education in reflective practice focuses on improving teacher perfor-mance as a result of reflection on their practices. As indicated by Lisle (2006),‘reflective practice in education is “learning-in-practice”’ (118). Authors who haveundertaken research on reflective practice agree with its capacity to transform teach-ing and students’ learning processes (Parsons and Stephenson 2005; Van Soest andBohle 2005; Johansson, Sandberg and Vuorinen 2007; Kavaliauskiené et al. 2007).Therefore, assessing the results of teacher education in reflective practice is a keyelement to identify its effectiveness.

Teacher education processes based on reflective practice explore the nature ofteaching and learning through observation and subsequent interpretation. It also uses

*Corresponding author. Email: [email protected]

European Journal of Teacher Education, 2013Vol. 36, No. 3, 279–292, http://dx.doi.org/10.1080/02619768.2012.718758

� 2013 Association for Teacher Education in Europe

Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

10:

41 0

5 Fe

brua

ry 2

014

specific and appropriate techniques for each situation in the same classrooms whereteaching practice is taking place (Esteve 2004). Teachers, in order to reflect anddeepen their understanding of their role, can use what happens in each session. Thisshould lead to an improvement of professional practice. Ultimately, actual teachingpractice and experience are the starting point of this type of education.

Despite the importance of identifying the results of teacher education in class-rooms and in schools, very few studies in this regard have been published. InSpain, Biscarri (1993) analysed teacher education of early childhood professionalsand deepened the existing link between motivation and professional behaviour.However, he did not address the issue of evaluating this training. Other authorshave also investigated the needs and motivations of teachers with regard to teachereducation (Colen and Defis 1997; Martínes and Sauleda 2004; Albertín and Sufiarre2005). Nevertheless, their analyses have not addressed the evaluation of trainingresults. Other authors who tackle the issue of evaluation confirm that there are veryfew studies that provide data on the results that teacher education generates in theclassroom and in schools (Flecknoe 2002; Pineda et al. 2007).

Thus, there exists a clear need for evaluating the outcomes of teacher educationand identifying a solid and functional model that may serve as a starting point. Thecontinuous education evaluation models, which are most prominent in our context,include Pineda’s (2002) holistic evaluation model and Kirkpatrick’s (2005) four-level model (satisfaction, learning, transfer and impact of training). These modelsstress the importance of assessing learning and state that the identification of teachereducation outcomes in terms of transfer, that is, the application of learning to theworkplace, is the central element of evaluation. Swanson (1996), in his perfor-mance-learning-satisfaction evaluation model, emphasises the importance of detect-ing what needs to be improved in professional practice in order to evaluateeducation results in terms of learning and changes in the workplace. Furthermore,he establishes an interesting link between needs, learning and outcomes, which isessential to evaluate education. This link refers to the importance of needs assess-ment to generate learning and the need to assess learning in order to identify spe-cific outcomes of workplace training.

Several authors have studied the transfer of education, focusing more on factorsthat determine such transfer rather than the development of specific models toassess outcomes. In this regard, they highlight the contributions of Baldwin andFord (1988), Noe (1996) and especially Holton (1996, 2005). Regarding impactassessment, Phillips (1990) made an interesting contribution with regard to measur-ing economic impact. This is adaptable to contexts where qualitative or non-eco-nomic impacts such as teacher education are being measured. In our study, we startwith these contributions to evaluate the results of a teacher education programme,specifically, mathematics training through reflective practice.

2. Research design

The case that was evaluated was the Teacher Education Programme for TeachingMathematics organised by the Catalonian Government’s Education Department. Theaim of this programme is to improve mathematics-teaching practice through the useof reflective practice. A total of 284 subjects participated in the training activity:172 early childhood and primary education teachers and 112 secondary educationteachers.

280 E. Belvis et al.

Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

10:

41 0

5 Fe

brua

ry 2

014

The programme consisted of teachers reflecting on their own educational prac-tice while sharing among colleagues and in small groups. This reflection, whichshould be done collectively and in a systematic way, started with observations andended with strategic planning for performance improvement.

The objectives of the teacher education programme are as follows:

(1) to generate strategies in the classroom to improve the development of stu-dents’ mathematics skills, specifically through the use of methodologies thatpromote the use of problem situations to emphasise the instrumental charac-ter of mathematics and its connection to the natural, social and cultural envi-ronment,

(2) to acquire tools and strategies for dynamic group participation in the mathe-matics classroom,

(3) to learn about different group dynamics participation strategies in the mathe-matics classroom,

(4) to acquire tools and strategies for integrating Information and Communica-tion Technology (ICT) in mathematics teaching practice.

The methodology used in the investigation followed an evaluation approach. Inother words, its purpose was to assess the quality of a particular practice in accor-dance with established criteria in order to focus on its improvement (McMillan andSchumacher 2005, 24–5).

A mixed-type methodological perspective that applied both quantitative andqualitative methods was used. The combination of methods was done in a comple-mentary way. The results obtained from one method were used to develop, improve,illustrate or clarify the results obtained through other methods, which were imple-mented consecutively.

Our evaluation model is part of the Holistic Model (Pineda 2002). This modelwas chosen since it places a special emphasis on the evaluation of the pedagogicaldimension of training and analyses the coherence of the training process from itsdesign to the transfer and the impact of training on the organisation.

The Holistic Model specifies several levels of evaluation: satisfaction, learning,pedagogical appropriateness, transfer, impact and cost-effectiveness. In our case,due to the nature of teacher education, all the evaluation levels were applied exceptfor the last one: cost-effectiveness. Figure 1 illustrates the tools used to evaluateeach level as well as the point in time at which they were implemented.

Each level of the evaluation consisted of a series of indicators and variablesthat, along with the evaluation criteria – relevance, adequacy and satisfaction –allowed for the design of tools to collect information. Table 1 shows the indicatorsand variables that corresponded to each level of the evaluation.

Regarding the evaluation tools used, the participant questionnaire focused on thelevel of satisfaction towards the training. It was a questionnaire with variousclosed-response items. This instrument was validated using the procedure of expertopinion. It was administered to teachers participating in the programme that had justbeen completed, with a sample of 88.4% individuals who had completed the train-ing (251 subjects).

The trainer completed a report at the end of training. The report included her/hisevaluation of the participants’ learning and of the pedagogical appropriateness of

European Journal of Teacher Education 281

Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

10:

41 0

5 Fe

brua

ry 2

014

the training programme. From the set of courses carried out, nine reports were col-lected using various criteria to ensure representativeness.

Three types of interviews were undertaken: an interview with participants, aninterview with participants’ colleagues and an interview with the management teamof the school. These interviews allowed for the gathering of detailed and in-depthinformation on the learning acquired by participants, transfer to the classroom, theconditions under which such transfer occurred and the impact generated by the tea-cher education. Interviews with colleagues and centre management were used onlyto assess the levels of transfer and impact. It should be noted that in order to createthese instruments it was necessary to take into account the learning objectives ofthe programme as well as the expected behavioural objectives in accordance withthe skills that an expert in reflective practice must develop and be able to use.These instruments were designed after evaluating the achieved learning. Theresearch approach adopted was that of progressive feedback and continuous meth-odological revision based on the results obtained.

The interviews were conducted three months after completion of the programmeand were recorded in audio format. The sample consisted of 16 schools located inCatalonia, Spain, evenly distributed according to the province and the educationlevel of the participant. In each school, three interviews were conducted: with theteacher participant, with a colleague of the said participant and with a representativeof management. To ensure the relevance of the interview results, participants wererequired to have spent a certain amount of time working at the centre in order tosee if there was a change in teaching performance as a result of the teacher educa-tion. Also, colleagues and board members interviewed could not have a direct linkto the programme that could result in a possible bias in their responses.

3. Results

Results are related to the level of learning, transfer and impact, which allows foridentifying how the outcomes of teacher education affect teachers’ professionalpractice as well as the general improvement of the classroom and centre climateand performance.

At the end of training

Three months after training

Interview with participant

1. SATISFACTION

2. LEARNING

3. PEDAGOGICALAPROPRIATENESS

4. TRANSFER

5. IMPACT

Interview with colleagues

Interview with management

Trainer’s report

Questionnaire for participant

Figure 1. Evaluation model.


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

10:

41 0

5 Fe

brua

ry 2

014

3.1. Learning

To evaluate the level of learning of the teachers participating in the programme, thetechniques that were used to obtain information included the trainer’s report and theparticipant interviews. In both cases, attempts were made to analyse the degree towhich the programme objectives were met. The achievement of the education objec-tives were high or very high, according to the teacher trainer, although it is worthnoting that the objectives related to the use of reflective practice strategies with stu-dents – objectives 1 and 3 – presented a higher degree of learning than thoserelated to strategies for reflection on teaching practice and the introduction of newinformation technologies (see Figure 2). This factor raised two related issues. Onthe one hand, the ability of trainers to generate learning in the trainees and, on theother, the implications this may have had on the subsequent transfer and impactwithin the centres. In this regard, individual monitoring assessed learning

Table 1. Indicators, variables and criteria according to the evaluation levels.

Dimensions Indicators Variables

1. Satisfaction Objectives Achievement;Adaptation to needs

Contents Difficulty;Innovation

Methodology MethodsMaterials Didactic resourcesTrainer Good understanding of the

contents;Ability to communicate

Management Information received;Timetable and schedule;Facilities

Climate Workplace environmentApplication and globalvalue

Implementation forecast;Usefulness in practice

2. Learning Objectives Achievement3. Pedagogical

appropriatenessTraining design Objectives; Contents; Methods;

Didactic resources; EvaluationManagement Information received; Timetable;

Schedule; FacilitiesImplementation Trainer; Methods;

Didactic resources; Evaluation4. Transfer Application Applied learning

Organizationalconditioning

Support and follow-up;Availability of resources;Group climate

Pedagogical conditioning Training design;Orientation towards transference

Personal conditioning Participant preparation;Transfer motivation

5. Impact Quantitative results Specific learning;Student achievement

Qualitative benefits Workplace environment;Student motivation;Engagement in learning


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

10:

41 0

5 Fe

brua

ry 2

014

achievements. For this, activities, the portfolio and the moodle (Modular Object-Oriented Dynamic Learning Environment) were used as evaluation tools.

Results obtained from the participant questionnaire showed how 71% of the par-ticipants considered that the level of learning obtained was sufficient. It must benoted that participants were already familiar with much of the learning included inthe training. When achievement was unsatisfactory it was argued that there was alack of time for the proper development of the teacher education. According to thisgroup, the objective related to the acquisition of tools and strategies to incorporatesystematic reflection in teaching practice and work habits was the most successfullyachieved. The objective related to the introduction of new information technologiesto the teaching of mathematics was less successfully achieved. This may be due toa lack of knowledge and mastery of these areas by the teacher, as well as the factthat in teacher education design information technologies are treated as a methodo-logical tool and not as a support tool for training.

3.2. Transfer

The outcomes regarding the level of transfer that occurred as a result of teachereducation were drawn from interviews carried out with the teachers participating inthe training, their colleagues and their managers. Triangulation processes were usedduring analysis in relation to the main issues in education transference: application,organisational conditioning, pedagogical conditioning and personal conditioning. Inthis sense, the idea was to analyse how the teacher education programme had beenunderstood and implemented in schools.

In teacher education it is imperative to carry out a plan with the followingobjective: teachers shall implement changes or improvements in performance in theclassroom as a result of the learning obtained through training. In this way, thereflective cycle is fulfilled. The change outlined in the action plans focused on fouraspects: group work and problem solving, linking mathematics with the environ-ment, skills linked to mathematics work and teaching and learning strategies.

25% 62% 13%

13% 74% 13%

22% 56% 22%

78% 22%

100%0% % of trainers

Objective 4: To acquire tools and strategies to incorporate ITC in theteaching of mathematics

Objective 3: To understand different intervention strategies and groupdynamics in the mathematics classroom

Objective 2: Acquire tools and strategies to incorporate the systematicreflection on and about teaching practice within professional habits

Objective 1: Build strategies in the classroom to enhance thedevelopment of the competence of mathematics students, particularly

regarding methodologies that promote the use of problematic situationsto emphasize mathematics' instrumental character

Very low Low High Very High

Figure 2. Degree to which the objectives were achieved.


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

10:

41 0

5 Fe

brua

ry 2

014

Figure 3 shows that the majority of changes in the introduced professional per-formance were those that referred to ‘learning guide’, ‘stimulate the learners in rea-soning, comparison and the development of actions and examples’ organisingexperiences and ‘using inductive methods’. By inductive method we mean the anal-ysis of practice within their own context, going from the specific to the general.Thus, improving teachers’ performance through teacher education has a positiveeffect on the quality of guidance and structure of student learning by using induc-tion and self-reasoning.

Some evidence of implementation is: the introduction of discovery and manipu-lative activities; rethinking the way to manage classes; the discovery of mathematicsin everyday life; specific group work; work from centres of interest and globalisedwork; an increase in attention paid to students; and so on. It is also necessary toadd that the vast majority of teachers (75%) believe that the introduced changesendure over time.

In interviews, 81.3% of the teachers stated that they implemented their actionplan as well as all the planned changes. A total of 18.8% said that they only par-tially implemented the plan and that they left some of the planned changes for alater stage. Although 87.5% felt that the introduced changes would endure overtime, they also believed that education outcomes could be improved by proposingthat the action plan become effective at the classroom, department and school leveland not be limited just to the classroom, since what has been observed is that theeffect is both specific and limited. In order to improve results, it would be appropri-ate to extend the training and have it go into greater depth. The participant wouldmake a commitment to the school and to the programme by formalising an actionplan within their classroom or while being monitored by a tutor. This would

12,5%

25%

25%

25%

31,3%

43,8%

50%

50%

62,3%

0 10 20 30 40 50 60 70 80 90 100

Building theory from practice and vice versa

Create an empathetic climate

Incentivise the co-construction of knowledge

Make the participants aware of the structures theyunconsciously make use of

Attend to their individual needs

Utilise inductive methods

Structure the experiences

Encourage learners towards reasoning, comparing anddeveloping actions and examples

Guide learning

Graph 2. Changes introduced in the participants' teaching practice

% of participating teaching staff

Figure 3. Changes introduced in the participants’ teaching practice.


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

10:

41 0

5 Fe

brua

ry 2

014

improve both results and the implementation of learning achieved and would guar-antee the continuity of the process.

Factors that facilitate or hinder transference were also evaluated. Results showedthat in teacher education in reflective practice, transference was primarily linked toa good work atmosphere in the teacher education sessions as well as to the school’sculture. The predisposition of the schools when faced by changes was positive inalmost all the cases (93.8%), as was teacher education support and innovation.

Other studies (Biscarri 1993; Early et al. 2004; Pineda 2007) show that theeffectiveness of teacher education depends on the teacher education culture. Out-comes improve if the members of the school believe that teacher education has animportant role to play. However, it must be emphasised that this would have to betied to a request for training by the staff or departments. It has been observed thatschools that achieved the greatest level of transfer had more than one person partici-pating in training and a certain sensitivity to impart it.

Another element to promote in order to bring about this change in the culture ofthe schools is the person responsible for teacher education. The person in charge ofteacher education in each school will identify the collective needs, design anddevelop the centre’s teacher education plan and monitor its implementation andeffects on the centre. This is critical to increase the efficiency of teacher education.

Among the most often mentioned obstacles is the lack of time due to workoverload and the limitations with regard to introducing innovations. This is commonin fixed programming. Another major obstacle is the ‘complexity of transferringlearning’, which implies a major change in the traditional way of working andrequires a high degree of involvement and prior preparation. This is also com-pounded by the instability of the teaching teams of the school and the breakdownof the working groups. Teams of teachers often change from one course to another,which may hinder the continuity of changes and innovations that the group wouldlike to introduce. This has often hampered the creation of learning communities,understood as working groups that conduct in-depth studies on various themes andshare their experiences.

Managers and colleagues report that changes occurred in the participant’s teach-ing performance. However, in most cases they state that they did not have evidenceof the changes, which calls into question some of their claims. At the same time,they affirm that at the level of the school there have been almost no improvements.Where there were changes, they were not very significant and the involvement ofother factors was considered such as a greater investment of time and more involve-ment by the teacher participant. The changes that were most reiterated by both man-agers and colleagues were: the introduction of new activities for students, greaterplanning of the sessions and an increase in teacher confidence and interest. It isimportant to note that nearly half of the colleagues indicated a positive attitudinalchange in the participating teacher.

3.3. Impact

Data on the impact of the teacher education were collected through interviews. In87.5% of cases there was an overall improvement in the classroom as a result ofthe training, especially in terms of climate and student and staff motivation. Theparticipating staff, especially those members coming from primary education, statedthat working through experience greatly encourages students and facilitates their


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

10:

41 0

5 Fe

brua

ry 2

014

learning. They also noted that the new activities based on reflective practice areespecially effective with students with attention difficulties as it results in an overallimprovement in the attitude, interest and involvement of these students. Colleaguesand managers also recognised that there was an impact, but to a lesser extent. Nev-ertheless, they also recognised that it is too early to assess the impact, as indicatedin the following statement:

It is still too early to be able to judge. Those who have begun to implement it havebeen doing so in this course, which has only been running for two months. I supposethat at the end of the semester we will be able to evaluate it.

It has also been repeatedly shown that perceptions are created from conversationswith the staff involved. This is because neither the managers nor the colleaguesfrom the schools have a habit of entering the classroom and, therefore, find it moredifficult to check whether there has been an improvement.

A high percentage of participating teachers (75%) stated in interviews that therewas an overall improvement in the participant’s department or group of teachers.They stated that there was an improved didactic orientation towards the material.The department colleagues implemented changes and took advantage of theresources provided by the participating teacher, such as the use of new didacticmaterials and the introduction of working on mathematical problems in an experien-tial way. There was a lower percentage of cycle or department colleagues (43.8%)who considered that there were improvements as a result of teacher education. Incases where some form of impact was observed, the evidence mostly involvedimproved didactic orientation towards the material, as expressed by participatingteachers in interviews.

Only 18.6% of participating teaching staff believed that teacher education hadan impact at the school level. This is explained by the short period of time elapsedsince the completion of the programme. The cycle or department colleagues sharedpractically the same opinion. Finally, at the school level the managers respondedthat there was an improvement in 31.3% of the cases, but, in general, they did notdiscuss the evidence with respect to these improvements. The instability of theschool’s teaching teams and the above-mentioned high-turnover rate complicatedthe achieved impact. The expected cascade effect of teacher education was quitelow. This effect was present in schools with a very high predisposition for this.

Regarding students’ performance, in general, there were fewer failing studentsand passing was less difficult. It was observed that expected learning was more eas-ily acquired. Furthermore, thanks to the more practical content work, this resultedin a greater wealth of learning. However, it is important to note that some partici-pating teachers indicated that the observed improvement in these matters was slightand might be affected by other factors, such as a greater investment of time andinvolvement by the teacher:

… the results from tests performed in the class have also been positive, but it must benoted that there has been a greater investment of time which obviously facilitates abetter understanding of what needs to be learned.

Regarding the dissemination of the teacher education activities, the teaching staffstated that they disseminated various activities across the cycles, or that they


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

10:

41 0

5 Fe

brua

ry 2

014

planned to do so in the future. On this issue, teachers working at the secondaryeducation level felt that reflective practice will not be as well accepted as at theearly childhood and primary education level.

In reference to the awakened interest in reflective practice, an increase in theinterest of colleagues to know about it was noted, especially for those who had theopportunity to experience it or observe it in practice. They stated:

There are teachers who, when seeing the work being carried out in the classrooms ofteachers who have completed the course, are encouraged by seeing that it works.

Only 38% of teaching staff participants’ responded affirmatively when askedabout the formation of a learning community during the teacher education period.Some teacher participants specified that during the teacher education, a learningcommunity feeling was created, however, in more than half the cases the linksbetween the participating teachers were never altogether established.

In cases where this was achieved there was an effort to try to maintain the learn-ing community. This depended entirely on the initiative of participating teachingstaff and took various forms. We present them in the learning community’s orderfrom least to most impact:

• Groups of teachers sporadically maintain contact through electronic means.• Groups of teachers have limited the learning community to the school towhich they belong.

• Groups of teachers have created working groups on reflective practice withthe participating teaching staff in the same area.

• Some of the comments from the teachers who were part of these teams are:‘the point is to create a working group to monitor reflective practice as a con-tinuation for the teachers in the area; we are a small group with high expecta-tions’ and ‘at the end of the workshop, we saw that we all had a commonidea about the type of mathematics that they wanted to work on. We proposedto do a seminar for a second year, opening it up to other people, to otherschool staff. We are getting much more out of this since we now have a foun-dation’.

The teaching staff appreciated the potential of reflective practice to improve teach-ing practice. This encouraged the exchange of ideas and experiences throughdynamic activities that promoted the creation of learning communities. The imple-mentation of the reflective practice process in classrooms and schools requires a sig-nificant investment of resources given the complexity of both the teacher educationand its implementation in centres. Nevertheless, the evaluation results show signifi-cant benefits since reflective practice gives the teachers new skills and strategies toface the challenges of teaching. This makes them more independent and more awareof group processes and the individuals in the classroom.

4. Discussion and conclusion

Reflective practice as a teaching methodology, through constant reflection on one’sown interventions, has enormous potential in improving teaching and learning pro-cesses in classrooms and in optimising teaching practice. The application of this


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

10:

41 0

5 Fe

brua

ry 2

014

methodology in the teaching of certain basic skills to students, including mathe-matic skills, may have beneficial effects in improving their academic results. This isparticularly interesting in contexts such as Catalonia, where the mathematic skills ofstudents are below the average of other Spanish regions according to the PISA2006 survey.

The evaluation of the teacher education programme for reflective practice inmathematics teaching shows that, in general, the results are partially positive andthat some of the teacher education objectives can be met. However, the achievementof learning is uneven across the different objectives. The objective most satisfacto-rily met is related to learning strategies for systematic reflection on professionalpractice and teaching habits. The remaining objectives, orientated towards develop-ing strategies in classroom with regard to the revitalisation of the group as a meansfor developing skills and incorporating ICT, had more modest levels of achieve-ment. Thus, there is a need for finding new strategies in order to obtain betterresults. PISA (OECD 2004, 27; 2009, 21) has already said how mathematic skillsdeveloped in technology-rich environments may improve student learning results. Itmeans that there is a need for using ICT in maths learning, but also for improvingthe knowledge construction theories from practice and vice versa. This interestingarea was not analysed in this study, thus there is a need for further projects toexplore these issues.

The learning results have consequences for subsequent transfer to the classroom.It is lower in those areas where the learning achieved is reduced, for example therevitalisation of the group-class. This result validates Swanson’s (1996) approach asthe theoretical basis for this study, in particular the close link that he identifiesbetween needs, learning and transfer – Swanson considers that transfer occurs whenthe education responds to the needs of the person and when this person learns. Inour case, the non-recognition of the need for group dynamics by some of the partic-ipants, along with other aforementioned factors, have resulted in learning being lowin this area and transfer also being limited.

Some of the learning achieved by teachers attending the teacher education onreflective practice has been transferred, which has produced changes in their profes-sional performance. The most significant changes are found in the strategies toguide and structure student learning and in enhancing the use of intuition and self-reasoning. These skills are very useful in learning mathematics, which the studentdoubtless can transfer to other fields of knowledge, thus improving their ‘learningto learn’ ability, which is so necessary today.

A key element to ensure the transfer of teachers’ learning is the development ofan action plan that aims to incorporate reflective practice in some aspect of teach-ing. In this process, which allows for the transfer of knowledge during teacher edu-cation, the participant has the support of the trainer and other training participants.Most of them implement the plan during and after the programme and note that thisstrategy is very effective in facilitating transfer of teacher education in the educationsector as well as in other sectors.

The study of the factors that influence transfer of teacher education in reflectivepractice provides interesting results that verify its principal theories. The climateand predisposition of schools faced with change as well as the support for teachereducation as an innovative strategy are emphasised as facilitating transfer. As advo-cated in the literature (Lim and Johnson 2002; Early et al. 2004; Holton 2005), cli-mate, colleague support and motivation when facing change are key factors for the


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

10:

41 0

5 Fe

brua

ry 2

014

transfer of teacher education and the transfer of reflective practice is facilitated incentres where these factors are most present.

Some methodological limitations have been detected in the results obtainedregarding transfer. These are inherent in the education sector and may affect otherstudies on the evaluation of the results of teacher education. The first limitation hasto do with the fact that the participants have no direct supervisor to monitor andfollow them as well as take control of the transfer of teacher education to the work-place. This means that data on this level tend to be generic and do not provide solidevidence. A second limitation is that there is a tendency for the participants’ col-leagues and managers to evaluate the school and the participating teaching staffvery positively, causing this information to be of limited usefulness when contrastedwith the results obtained from participants.

Teacher education in reflective practice has generated impact, most especially atthe classroom level, by improving the climate and the motivation of students andteachers. These changes have also had impact at the level of the cycle or group ofmathematics teachers. They have increased interest in both reflective practice andthe introduction of changes by participating teachers in their teaching as well as theresources they use. However, the impact at the school level is smaller. Perhaps thisis because more time is needed for the changes to take effect, or because they havea more direct impact on the area of mathematics and the immediate environment ofthe teachers trained, which makes it difficult to generalise to other areas of knowl-edge.

An interesting impact is noted at the student level: students exhibit less diffi-culty in learning mathematics and, in general, their academic performance improves.This result shows that reflective practice can be a useful methodology for improvingteaching practices in mathematics and other subjects and for increasing student per-formance in those key skills that are less developed. However, it must be noted thatthis result may have been influenced by other factors, such as a greater investmentof time and a higher level of involvement of the teacher participant.

Here we highlight some elements that can improve transfer and impact in theclassroom and in the school such as the participation of more than one teacher fromthe same mathematics department and the presence of an innovative culture withinthe school. Other studies have shown that the effectiveness of teacher educationdepends on the culture of training. If the school places an important emphasis onteacher education, results improve. It must be emphasised that this should be linkedto a request for teacher education by the school staff or departments. In this way, ithas been observed that schools with greater transference have more than one personparticipating in the programme and are sensitive to its importance. Other factors toconsider are changes in the school’s teaching teams and working groups, whichhave in many cases hampered the creation of learning communities among partici-pants.

The evaluation undertaken is the feedback type: the results obtained using thefirst exploratory tools were used to design the subsequent instruments. Thisapproach is very useful for evaluating teacher education results since it allows oneto identify the key factors in transfer of training and to later evaluate them using alarge sample, thereby obtaining valid and reliable results that are generalisable.

In general, results show that this teacher education has enabled teachers to intro-duce strategies for reflective practice in the classroom, thus transforming their teach-ing practice. We hope that this experience will encourage other researchers to go


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

10:

41 0

5 Fe

brua

ry 2

014

further in the evaluation of the results of teacher education processes such as these,which are clearly directed towards innovation in teaching and ultimately to improvestudent learning.

Notes on contributorsEsther Belvis is a researcher interested in the intersections of the fields of education, art andtechnology. She holds a PhD from the University of Warwick and she has published papersin different international journals.

Pilar Pineda has tenure as professor of Economics of Education and Training inOrganizations at the Universitat Autònoma de Barcelona, Spain. She has directed severalresearch projects on evaluation of training, transfer of training, and VET. She is the authorof five books and more than 20 papers on training, education and work.

Carme Armengol has tenure as professor in the Area of Didactics and School Organisationin the Department of Applied Pedagogy at the Universitat Autònoma de Barcelona, Spain.She has directed several research projects centred on improving and developingorganizations.

Victoria Moreno holds a PhD in Educational Sciencies from Universitat Autònoma deBarcelona. She is a trainer and consultant on Evaluation of Training and Human ResourcesDevelopment and a member of the Research Group on Educational Policy and Training atthe UAB.

ReferencesAlbertin, A.M., and B. Zufiarre. 2005. La formación del profesorado de educación infantil:

Una trayectoria desde la LGE hasta la LOE [Early childhood education teacher train-ing: A trajectory from LGE to LOE]. Pamplona: Universidad Pública de Navarra.

Baldwin, T.T., and J.K. Ford. 1988. Transfer of training: A review and directions for futureresearch. Personnel Psychology 41: 63–105.

Biscarri, J. 1993. La formación permanente de los profesores: Motivaciones y condicionan-tes. [Teachers continuing education: Motivations and conditioning] Lleida: Institutd’Estudis Ilerdencs.

Colen, M.T., and O. Defis. 1997. La formación permanente del profesorado de educacióninfantil como punto de partida de la formación inicial. [Continuing education for earlychildhood education teachers as the first step in initial training]. http://www.uva.es/aufop/publica/actas/viii/edinfant.htm.

Early, D.M., D.M. Bryant, R.C. Pianta, R.M. Clifford, T.M. Burchinal Egan, B. Yang, andK.R. Bartlett. 2004. The effects of organizational learning culture and job satisfactionon motivation to transfer learning and turnover intention. Human Resource Develop-ment Quarterly 15: 279–301.

Esteve, O. 2004. Nuevas perspectivas en la formación de profesorado de lenguas: Hacia elaprendizaje reflexivo o aprender a través de la práctica, a VV.AA. Actas I JornadasDidácticas de Español y Alemán como Lenguas Extranjeras [New perspectives in lan-guage teacher education: Towards reflexive learning or learning through practice].Madrid: Instituto Cervantes.

Flecknoe, M. 2002. Measuring the impact of teacher professional development: Can it bedone? European Journal of Teacher Education 25, nos. 2–3: 119–32.

Holton, Elwood F. 1996. The flawed four-level evaluation model. Human Resource Develop-ment Quarterly 7, no. 1: 5–21.

Holton, Elwood F. 2005. Holton’s evaluation model: New evidence and construct elabora-tions. Advances in Developing Human Resources 7, no. 37: 37–54.

OECD. 2004. Marcos teóricos de Pisa 2003. [The theoretical framework of PISA 2003].Madrid: Ministerio de Educación y Ciencia.

OECD. 2009. Learning mathematics for life: A view perspective from PISA. Paris: OECD.


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

10:

41 0

5 Fe

brua

ry 2

014

http://www.uva.es/aufop/publica/actas/viii/edinfant.htm

http://www.uva.es/aufop/publica/actas/viii/edinfant.htm

Johansson, Inge., Anette Sandberg, and Tuula Vuorinen. 2007. Practitioner-oriented researchas a tool for professional development. European Early Childhood Education ResearchJournal 15, no. 2: 151–66.

Kavaliauskiené, G., L. Kaminskiené, and L. Anusiené. 2007. Reflective practice. Assessmentof assignments in English for Specific Purposes. Ibérica 14: 149–66.

Kirkpatrick, James. 2005. Transferring learning to behavior. Training and Development 59,no. 4: 19.

Kirkwood, M., and D. Christie. 2006. The role of teacher research in continuing professionaldevelopment. British Journal of Educational Studies 54, no. 4: 429–52.

Lim, D.H., and S.D. Johnson. 2002. Trainee perceptions of factors that influence learningtransfer. International Journal of Training and Development 6, no. 1: 36–48.

Lisle, Angela. 2006. Maintaining interaction at the zone of proximal development throughreflexive practice and action research. Teacher Development 10, no. 1: 117–43.

Martinez, M.A., and N. Sauleda. 2004. La educación infantil y la formación del profesoradohacia el siglo XXI: Integración e identidad [Early childhood education and teachertraining towards the twentieth Century: Integration and identity] Congreso Internacionalde Educacion Infantil. Córdoba: Servicio de Publicaciones de la Universidad de Córdo-ba.

McMillan, J., and S. Schumacher. 2005. Research in education: Evidence based inquiry.Allyn & Bacon: 24–5 (McMillan y Schumacher, 2005:24–25).

Noe, Raymond A. 1996. Trainees’ attributes and attitudes: Neglected influences on trainingeffectiveness. Academy of Management Review 11, no. 4: 736–49.

Parsons, M., and M. Stephenson. 2005. Developing reflective practice in student teachers:collaboration and critical partnerships. Teachers and Teaching 11, no. 1: 95–116.

Phillips, J. 1990. Handbook of training evaluation and measurement methods. Houston, TX:Kogan Page.

Pineda, P. 2002. Gestión de la formación en las organizaciones [Training management inorganizations]. Barcelona: Ariel.

Pineda, P, X. Ucar, V. Moreno, E. Belvis, V. Garrido, A. Forés, S. Roig, F. Ferrer, and E.Vilaplana. 2007. Evaluación de la calidad de la formación continua en el sector de laeducación infantil: Informe ejecutivo. [Quality assessment in continuing training in thearea of early childhood education: Executive report]. Barcelona: ….

Swanson, R.A. 1996. Evaluation systems in HRD. PLS evaluation system: Sales communi-cation case study. In Proceedings of the Symposium Evaluation Systems in HRD, Min-neapolis, 718–25.

Zoest, Van., R. Laura, and Jeffrey V. Bohl. 2005. Mathematics teacher identity: A frameworkfor understanding secondary school mathematics teachers, learning through practice.Teacher Development 9, no. 3: 315–45.


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

10:

41 0

5 Fe

brua

ry 2

014

Documents

Evaluation of reflective practice in teacher education