A handbook for medical teachers

A HANDBOOK FORMEDICAL TEACHERSFourth Edition

This page intentionally left blank

A HANDBOOK FORMEDICAL TEACHERSFourth Edition

DAVID NEWBLE, BSc(Hons), MBChB,MD, FRACP, DipEdProfessor and HeadDepartment of Medical Education,University of Sheffield, UK, andpreviously Associate Professorin Medicine, The University of AdelaideSouth Australia

ROBERT CANNON, MA(Hons),MEdAdmin, DipTertEdAssociate Professor and DirectorAdvisory Centre for University EducationThe University of AdelaideSouth Australia

Illustrations by Zig Kapelis, MArch, MURP,DipTCPFormerly Senior Lecturer in ArchitectureThe University of Adelaide, South Australia

KLUWER ACADEMIC PUBLISHERS NEW YORK, BOSTON, DORDRECHT, LONDON, MOSCOW

eBook ISBN: 0-306-47506-5Print ISBN: 0-7923-7092-9

©2002 Kluwer Academic PublishersNew York, Boston, Dordrecht, London, Moscow

Print ©2001 Kluwer Academic Publishers

All rights reserved

No part of this eBook may be reproduced or transmitted in any form or by any means, electronic,mechanical, recording, or otherwise, without written consent from the Publisher

Created in the United States of America

Visit Kluwer Online at: http://kluweronline.comand Kluwer's eBookstore at: http://ebooks.kluweronline.com

Dordrecht

CONTENTSPREFACE

CHAPTER ONE: HELPING STUDENTS LEARNIntroductionHow students learnNon-traditional students and their learningUsing new technology and learningLearning more effectivelyGuided reading

CHAPTER TWO: TEACHING IN LARGE GROUPSIntroductionThe context of large group teachingWhat about non-traditional students?Preparing for large group teachingPresenting the large group teaching sessionWhat active learning strategies are available?Using teaching materials and technologyWhen things go wrongEvaluating large group teachingA concluding thought – if you must ‘lecture’Guided reading

CHAPTER THREE: TEACHING IN SMALL GROUPSIntroductionThe importance of small group teachingWhat is small group teaching?Managing a small groupStructure in small group teachingIntroducing stimulus materialsAlternative small group discussion techniquesUsing technology for teaching small groupsEvaluating small group teachingWhen things go wrongGuided reading

CHAPTER FOUR: MAKING A PRESENTATION AT A CONFERENCEIntroductionPresenting a paperPreparing the paperPreparing the abstract and your contribution to the

proceedings of the conferenceWhat you should do on the dayPreparing a conference poster

ix

122567

12

151616171923273133353636

394040414244464650505153

55565656

616264

v

Chairing a conference sessionGuided reading

CHAPTER FIVE: TEACHING PRACTICAL AND CLINICAL SKILLSIntroductionThe attributes of an effective clinical teacherImproving clinical teachingImproving the clinical tutorialAlternatives to traditional clinical teachingTechniques for teaching particular practical and clinical skillsTeaching practical and laboratory classesEvaluating clinical and practical teachingGuided reading

CHAPTER SIX: PLANNING A COURSEIntroductionWho should be responsible for curriculum planning?Course contentStudentsAims and objectivesWriting objectivesChoosing methods and relating objectives to teaching

and learning activitiesRelating objectives to the assessment methodsSequencing and organising the courseOther course design considerationsEvaluating the courseGuided reading

CHAPTER SEVEN: TEACHING IN A PROBLEM-BASED COURSEIntroductionWhat is problem-based learning?Implementing problem-based learningCourse design considerationsTutoringStaff resources for problem-based learningGuided reading

CHAPTER EIGHT: ASSESSING THE STUDENTSIntroductionThe purposes of assessmentWhat you should know about educational measurementAssessment methodsTypes of assessmentEssayShort-answer and simple computation questions

6669

71727273757880848787

89909191939696

100103104106107108

109110110111112118120121

125126126128134134134136

vi

Structured testsObjective testsDirect observationOralStructured clinical/practical assessmentSelf-assessmentThe learning portfolioAssessing students with a disabilityAssessing students as groupsUsing technology in assessmentFeedback to studentsReporting the results of assessmentGuided reading

CHAPTER NINE: PREPARING LEARNING AND TEACHINGMATERIALS USING TECHNOLOGYIntroductionBasic principals in preparing learning and teaching materialsTypes of learning teaching materials and aidsThe overhead projectorThe 35-mm slide projectorThe video projectorThe whiteboard and blackboardVideo and filmPrinted materialPublishing material on the World Wide WebUsing technology in learning and teachingGuided reading

CHAPTER TEN THE EVALUATION OF LEARNING AND TEACHINGIntroductionThe context of evaluationThe evaluation of learning and teachingBringing it all together: a framework for the

evaluation of your teachingGuided reading

APPENDIX ONE: WHERE TO FIND OUT MORE ABOUTMEDICAL EDUCATIONBooks and journalsTraining opportunitiesOverseas travelOrganisations

INDEX

139140148151151153156158159160161162162

165166166168168172174175176179181183184

187188189193

198206

209210212212212

215

vii


PREFACEThe first edition of this book was published in 1983 as aresponse to a concern that few resources were available tothe medical teacher wishing to gain a perspective on basiceducational principles and their application to teaching. Itssuccess led to revised editions published in 1987 and 1994.We have also been delighted to find the book in use inmany countries and there are now several non-Englishversions.

Over these years the climate for educational reform hasimproved considerably. Higher education institutions areresponding to both internal and external pressures toimprove the quality of their courses and the teachingperformance of individual staff members. Institutions areencouraging educational innovation, tackling the problemof rewards and promotion for contributions to teaching,supporting staff development activities, and introducingstaff appraisal and quality assurance procedures. Thus, theneed of individual teachers for practical help in meetingthese changing expectations is becoming increasinglyevident.

Preparing this fourth edition has presented us withnumerous challenges. Clearly much has changed evensince 1994. For example the rapid growth in the use of theInternet and other teaching technologies raises manyquestions. The pressures on institutions to cut costs andto teach ‘more efficiently’ has intensified a move towardsgreater accountability.Participation in some form of ‘teachthe teachers how to teach’ programme is increasinglybecoming mandatory and in some countries may evenbecome an external requirement to be in charge ofuniversity courses. We know our book has been usedsuccessfully as a basic text for such programmes.

In keeping with these changes we have updated allchapters and have attempted to cover recent develop-ments we feel have general application. In doing so wehave remained mindful of the original design of the book tobe easy to read and to focus on the practical needs of themedical teacher. It is not our intent that this be acomprehensive text on medical education. However, wehave continued to provide key references and guidedreading for each chapter and an appendix which lists some

ix

of the more widely read journals and major internationalmeetings on medical education.

We have, as usual, planned this book so that you can godirectly to the topic of immediate interest. However, withthis edition we have commenced with an overview of theconsiderable research that has been forthcoming in the last20 years on how students learn and the importance of thisfor the teacher. It puts up front an emphasis on the manyfactors which influence the effectiveness of our activities asteachers. If we are not aware of these, and do not take theminto account, then much of what we do may be undermined.

Finally we would like to express our appreciation to thestaff at Kluwer Academic Publishers for their continuedsupport. The help of our secretary Carol Icke is specificallyacknowledged and appreciated.

David NewbleRobert Cannon

x

1: HELPING STUDENTS LEARN

INTRODUCTION

If you inspected a copy of the previous edition of our bookyou would notice that this chapter had moved from beingthe last to the first. While this book is intended to be verypractical in its approach we believe there are importantreasons for the early introduction of rather moretheoretical perspectives. The first is that research andthinking about learning is yielding insights into teachingwhich helps us construct practical advice on a firmerfoundation than previously. The second reason is becauseof the fundamental challenge it provides to the moretraditional views and stereotypes that prevail aboutstudents and learning in higher education.

Teachers have been primarily interested in what and howmuch students learn and elaborate assessment methodshave been devised to measure these. But in the last quarterof the twentieth century a considerable body of evidenceaccumulated which suggested that we need to becomemuch more concerned with how our students learn and thecontextual forces that shape their learning. We need toappreciate that some of our students are having difficultieswith their studies arising not just from their lack ofapplication or psychosocial problems, but from specificways in which they study and learn. We must alsoappreciate that many of their difficulties are directlyattributable to the assumptions we make about them, andthe way we teach, organize courses, and conductassessments.

HOW STUDENTS LEARN

Although there has been an enormous amount of researchinto learning over very many years, no one has yet comeup with a coherent set of principles that would adequatelypredict or explain how students learn in any particularcontext. There have been psychological studies, studies inthe neurosciences, in cognitive science, evolutionarystudies, anthropological studies, and even archaeologicalevidence about learning to name a few! The paper byMarchese, available from the Web, provides a fascinating,scholarly and entertaining introduction to all this intellec-tual effort.

2

It was not until 1976 that a landmark study by two Swedishresearchers, Marton and Saljo, shifted the traditionalresearch focus from teachers and teaching onto whatstudents actually think and do in real situations. Theyreported that all students have distinctive approaches tolearning that we now understand are influenced by manyfactors, as shown in Figure 1.1. The chain of events inlearning and the links between them are the focus of muchcurrent research effort and so are likely to be furtherrefined over time. We attempt to summarise currentunderstanding here.

One of the factors influencing learning is studentcharacteristics and these include individual differences,students’ previous learning experiences and currentunderstanding of the subject. Other influences can begrouped under context characteristics. These include,especially, the ethos of the department organizing thecourse and the characteristics of the curriculum. Closelyrelated to this is the teachers’ approach to teaching (acharacteristic we discuss in more detail below).

The effect of these factors is to influence students’perceptions of their context and the learning approachthat is expected of them. Students can be observed to useone of three broad approaches to learning, commonlycalled surface, deep and strategic.

3

Students adopting a surface approach to learning arepredominantly motivated by a concern to complete thecourse or by a fear of failure. In fact, the emotional aspectsof students’ perceptions of their context is beginning toreceive attention and it is emerging that anxiety, fear offailure and low self-esteem are associated with surfaceapproaches. Surface approach students intend to fulfil theassessment requirements of the course by using learningprocesses such as acquiring information, mechanicalmemorisation without understanding it, and reproducingit on demand in a test. The focus is on the material or taskand not on its meaning or purpose. The learning outcomeis, at best, a memorisation of factual information andperhaps a superficial level of understanding.

In contrast, students adopting a deep approach aremotivated by an interest in the subject matter and a needto make sense of things and to interpret knowledge. Theirintention is to reach an understanding of the material. Theprocess of achieving this varies between individualstudents and between students in different academicdisciplines. The operation learner relies on a logicalstep-by-step approach with a cautious acceptance ofgeneralisations only when based on evidence. There isan appropriate attention to factual and procedural detailwhich may include memorisation for understanding. Thisprocess is most prevalent in science departments. On theother hand, the comprehension learner uses a process inwhich the initial concern is for the broad outlines of ideasand their interconnections with previous knowledge. Suchstudents make use of analogies and attempt to give thematerial personal meaning. This process is more evident inarts and social science departments. However, anotherprocess is that used by the so-called versatile learner forwhom the outcome is a deep level of understanding basedon a knowledge of broad principles supported by a soundfactual basis. Versatile learning does not preclude the useof memorisation when the need arises, as it frequently doesin science-based courses, but the students do so with atotally different intent from those using the surfaceapproach.

Students demonstrating the strategic approach to learningmay be seen to use processes similar to both the deep andsurface learner. The fundamental difference lies in their

4

motivation and intention. Such students are motivated bythe need to achieve high marks and to compete withothers. The outcome is a variable level of understandingthat depends on what is required by the course and,particularly, the assessments.

The learning outcomes can be broadly described in termsof quantity and quality of learning. The outcomes we wouldhope from a university or college education are very muchthose resulting from the deep approach. Disturbingly, theevidence we have suggests that these outcomes may notalways be encouraged or achieved by students. Indeed, aswe stress repeatedly, there is good reason to believe thatmany of our teaching approaches, curriculum structuresand, particularly, our assessment methods, may beinhibiting the use of the deep approach and supportingand rewarding the use of surface or strategic approachesto learning. This appears to be particularly so for medicalstudents undertaking traditional curricula (see article byNewble and Entwistle).

NON-TRADITIONAL STUDENTS AND THEIRLEARNING

Medical schools now enrol significant numbers of studentswho do not come directly from high school. Students fromoverseas and older students entering without the usualprerequisites are just two examples of what we might call‘non-traditionalstudents’ in medical education.

There has been something of an explosion in the researchand writing about such students and their learning. Thisliterature is very revealing. In broad terms, it is showing usthat any so-called ‘problems’ with these students are oftenthe result of ill-informed attitudes and educationalpractices, in short, a result of poor teaching. This confirmsthe importance of creating a positive learning environmentrather than seeking fault with students.

Students from different cultural backgrounds

One thing we are sure you will have noticed in yourinstitution or from your reading is that stereotypes areattached to students from different cultural backgrounds.One of these stereotypes is that students, particularly those

5

from Asia, are rote learners. Yet many studies have shownthat these students score at least as well and sometimeshigher than western students on measures of deeplearning. You may also have noticed how there seems tobe a disproportionate number of these Asian students whoreceive academic distinctions and prizes!

This apparent ‘paradox’ – adopting surface approachessuch as rote learning but demonstrating high achievementin academic courses – has been the subject of muchinvestigation. What is emerging is that researchers haveassumed that memorisation was equated with mechanicalrote learning. But memorisation is not a simple concept. It isintertwined with understanding such as when you mightrote learn a poem to assist in the processes of interpretationand understanding. Thus the traditional Confucianheritageway of memorisation can have different purposes. Some-times it can be for mechanical rote learning. But it is alsoused to deepen and develop understanding. The paradoxof these learners is solved when memorisation is seen as animportant part of the process leading to understanding.

We encourage you to read further about these issues, andabout some of the other problematic cultural stereotypes(such as Asian student participation in classes) in theGuided Reading sources listed at the end of the chapter.They will not only help you to assist these students becomemore effective learners but also provide a clearerunderstanding about the processes of learning moregenerally.

Older students

The literature in this area also makes interesting reading. Ittells us is that older students are generally little differentfrom the more traditional entry younger students, andsometimes better in important ways. Figure 1.2 sum-marises some of the key findings presented in Hartley’sbook.

USING NEW TECHNOLOGY AND LEARNING

The literature in this field tends to be of two main kinds: thatwhich has researched the impact of technologies such ascomputers on learning processes and outcomes, and the

6

more general and speculative literature. Given the vastspan of this literature over issues and time, and given thedifferent methods used by researchers, it is difficult todraw too many useful generalisations to help you in yourteaching other than:

research on the impact of technologies such ascomputer-aided learning shows small but neverthelesspositive effects on learning and attitudes;studies of the impact of specific technologies (such asvideo and electronic mail) on learning shows a greatdiversity of outcomes which reflect both the nature andpotentialcapability of the technology and, importantly,the way it is used by teachers and learners;the general literature is pointing to ways in whichtechnology, sensitively used, can contribute to a rangeof improved learning processes and to outcomes suchas enhanced tools for learning; improved flexibility forthose with access to the technology; individualisinglearning; and more student activity.

If you are hoping to find spectacular learning outcomesfrom using the new technologies you may be disappointedat this stage. Equally we believe there are opportunities toaddress many of the ills of education by using moderntechnology to support quality learning.

LEARNING MORE EFFECTIVELY

The concepts outlined above are not only supported by agrowing body of research evidence, but also match thekinds of things good teachers know and do when teachingtheir students. We are now in a better position than wewere in earlier editions of this book to make suggestions

7

and offer practical advice based on the accumulatingresearch evidence and the experiences of practicingteachers in higher education.

Improving the learning environment

This must be considered at various levels. At the broadestlevel is the educational philosophy underlying the wholecurriculum. There may be little you can do about this, butthere is evidence that students from schools usingtraditional teaching practices are more likely to adoptsurface approaches to a greater degree than students fromprogrammes that are more student-centred. You may beable to gauge where the educational philosophy of yourown discipline or curriculum fits and predict the likelyeffect it has on your students’ approach to learning.

At another level, and one where you might be able to exertsome influence, is the structuring of the curriculum. Youshould be aware that the fragmentation of the curriculuminto a large number of courses or course componentstaught by different teachers may be counter-productive tothe development of deep approaches. The time availableto each is limited and so the opportunities for students tocome to grips with the deeper implications and perspec-tives of subject matter are similarly restricted. Suchfragmentation has become increasingly apparent inmedical curricula.

In recent years many different teaching methods havebeen re-discovered or developed not from researchstudies but from the practice and the experience ofthoughtful teachers. Perhaps the one which has made thegreatest impact is problem-based learning which isdiscussed in detail in Chapter 7.

There are other methods and approaches which are morestudent-centred and appear more likely to encouragedeep learning and enhance learning outcomes. Some ofthe principles which these incorporate are shown in Figure1.3. They may be reflected in activities such as researchprojects, peer teaching, case-based learning, learningportfolios and so on. In general, this means that the primaryfocus of the teacher should be to provide a learningenvironment which is stimulating, based on interaction,and which emphasizes the responsibility of the student toparticipate actively in learning activities.

8

The major differences between student-centred learningand conventional teaching are listed in Figure 1.4.

As most teachers reading this book will be working in aconventional institution, it is important to introduce intocourses those measures which might encourage the use ofthe deep approach. Some other measures you canimplement are listed below:

Ensure that the course objectives specify more thanjust facts and technical skills by giving suitableemphasis to higher-level intellectual skills, such asproblem-solving and critical thinking; to workingcollaboratively with others; and to the explorationand development of appropriate attitudes.Introduce teaching activities which require students todemonstrate a deep understanding of the subjectmatter. Do not allow students to ‘get away’ with onlyreproducing factual information and take a genuineinterest in what they say and do as indicators of theirlearning.Reduce the time allocated to didactic teaching to allowmore time for students to work with other people andfor self-directed learning.Decrease the amount of factual material that has to bememorised. Both pressure of time and overloadingwith content is known to encourage the surfaceapproach even in those intending to use the deep

9

approach. These problems are prevalent in manyscience-based medical courses.Spend more contact time in helping students tounderstand and use basic principles and in under-standing the difficulties they may be having. Get intothe habit of expecting students to explain answers toquestions. The frequent use of the word ‘why’ willquickly establish if the answer is based on memorisa-tion or on an understanding of an underlying principle.Evaluate the extent to which students find you or theircontext threatening and take measures to eliminate orreduce this as much as you can, taking care to maintainacceptable levels of intellectual challenge.Most importantly, review the assessment procedures.This is a critical task. If the assessment, course contentand learning methods do not match the course

10

objectives, then one could be the world’s greatestteacher and make little impact on the students’learning. For example, an over-reliance on objectivetests of low level recall (true-false, multiple choice) willalmost certainly encourage the use of surfacestrategies. If you aim to have students understand thesubject, then you must introduce forms of assessmentwhich require them to demonstrate this understand-ing. This may mean the re-introduction of essays, andthe use of research projects, self and peer assessment,and so on.

Modifying teaching approaches

Evidence is being found that there is a relationshipbetween a teacher’s approach to teaching and the qualityof student learning outcomes. Research and thinking aboutteaching over nearly thirty years shows that teachers holdrather different ‘theories’ of teaching and learning whichinfluence their approach to teaching. Very broadly, thereare teachers who believe their job is to cover the subjectsystematically by transmitting content to students. Failureto learn the content is seen to be the fault of the student. Itappears that teachers who have this approach are morelikely to encourage surface learning among their students.

On the other hand there are teachers who consider theirmain role is to assist students’ understanding andconceptual change. They focus on what the students doand what learning outcomes result from their activity.Failure to learn is considered to be just as likely to be dueto some failure in the way in which the curriculum wasplanned and implemented, as it is to be a deficit in studentsor their teachers. Such teachers, who would describe theirteaching as student-focused, are less likely to encouragesurface learning approaches among their students. Westrongly suggest you read the article by Trigwell, Prosserand Waterhouse to deepen your understanding of theseimportant relationships.

The bottom line is that teachers need to be aware of theirapproach and the impact this may have on the learningapproach of their students. We are not in a position tomodify your beliefs and theories here, although we hopethat some of this information may help! However we can

11

suggest you experiment with several of the student-centred strategies described in this book if you wish toencourage high quality student learning.

Improving learning skills

There seems little doubt that good learning and study skillscontribute to improved academic performance, though inthemselves they are not a guarantee of success. Equally,possessing learning skills is now seen as having a lifelongrelevance and not just limited to good grades in an end-of-course examination.

These lifelong learning skills can be developed in yourcourses and include self-organising skills; skill in deeperlearning strategies such as analysis, judgement, synthesisand application; locating, retrieving, interpreting, evaluat-ing and managing information; and the skills of breadth anddepth of vision and the capacity to appreciate theinterrelated nature of knowledge.

It is clear that some students continue to use study skills andapproaches which are inappropriate and ineffective, It isimportant for teachers to identify such students as they mayneed help with specific study skill counselling. Many ofthese students become persistent poor performers and itcan be very rewarding for both teacher and students torealise that a specific remedy is available.

For further information and help with specific study skillcounselling we suggest you look at some of the guides andmanuals that are now available. It might also be helpful toconsult the student counselling service or teaching supportunit at your institution.

GUIDED READING

A very good general text on learning is Learning andStudying, A Research Perspective by J. Hartley, Routledge,London, 1998. This is a particularly helpful reference forteachers as it simply and comprehensively discusseslearning from a range of different research perspectivesand makes practical suggestions on ways in which teacherscan improve learning for their students. It contains sectionsthat review the literature on older students and technolo-gies discussed above.

12

Other useful texts covering the approaches to learningliterature are The Experience of Learning (2nd edition) byF. Marton, D. Hounsell and N. Entwistle (eds), ScottishAcademic Press, Edinburgh, 1997 and Teaching for QualityLearning at University by J. Biggs, Buckingham, OpenUniversity Press, 1999.

For an introduction to practical strategies and theoreticalissues in lifelong learning we recommend C. Knapper andA. Cropley, Lifelong Learning in Higher Education (2ndedition) Kogan Page, London, 2000, and P. Candy, G.Crebert and J. O’Leary, Developing Lifelong LearningThrough Undergraduate Education, Australian Govern-ment Publishing Service, Canberra, 1994. Both containmany exemplars of lifelong learning practices in highereducation.

Recent editions, from around 1997, of the journal HigherEducation Research and Development have includedseveral helpful papers about Asian students. A particularlyrelevant edition is Volume 16, Number 1, April 1997:‘Common misconceptions about students from South-EastAsia studying in Australia’ by D. Chalmers and S. Volet.

A very useful book containing detailed advice on how tostudy is A Guide to Learning Independently by L. Marshalland F. Rowland, Open University Press, UK, 1999. Thoughwritten for students it is a valuable resource for teachers.

Books and articles referred to in this chapter:

F. Marton and R. Saljo (1976). On qualitative differences inlearning: I. Outcomes and process. British Journal ofEducational Psychology, 46: 4-11.

K. Trigwell, M. Prosser, and F. Waterhouse (1999).Relations between teachers’ approaches to teaching andstudents’ approaches to learning. Higher Education, 37: 57-70.

T. Marchese. The Adult Learner in Higher Education andthe Workplace; The New Conversations about Learning.Available:http://www.newhorizons.org/lrnbus_marchese.html

D. Newble and N. Entwistle (1988). Learning styles andapproaches: implications for medical education. MedicalEducation, 20, 162-175.

13


2: TEACHING IN LARGE GROUPS

INTRODUCTION

Large group teaching is often thought of as the same aslecturing. While the lecture is still a very common teachingmethod in most medical schools we want to encourage youto think more creatively about how you might best use yourtime when faced with a large group of students. There aregood educational reasons for moving away from thetraditional approach of ‘lecturing’ to groups of passivestudents to strategies which introduce more activelearning. While the solution to this concern may, in part,involve replacing the notion of large group teaching withalternative approaches, such as small group teaching ordistance learning, we recognize that other factors maypreclude such options. Should this be the case we believethat you can employ a range of techniques in the largegroup situation which will engage your students enthu-siastically in active learning, provide them with immediatefeedback and build a productive and scholarly relation-ship.

Why do we want to support you in this move towardsputting students at the centre of your thinking? Because theevidence continues to mount that, although the lecture is aseffective as other methods to transmit information (but notmore effective), it is not as effective as other methods tostimulate thinking, to inspire interest in a subject, to teachbehavioural skills, or to change attitudes. These are amongthe objectives that many medical teachers aspire to whenthey lecture. On the other hand, if we seriously wish tofoster lifelong learning skills and attitudes among ourstudents, one of the worst things we can do is to encourageand reward the kinds of passivity that the lecture methodcommonly provides.

This chapter seeks to provide you with practical sugges-tion on how you might approach the task if asked to give a‘lecture’.

THE CONTEXT OF LARGE GROUP TEACHING

An important preliminary step in your preparation is tofind out as much as you can about the context of yourteaching in the overall teaching programme or course.Unfortunately this context is often ill defined and may beonly a title in a long list of topics given out by the

16

department or school. However, do try to find out as muchas you can. This means enquiring about such things as:

what students have been taught (and what they mayknow),what the purpose of your teaching session is to be,what resources, such as library materials, are availablefor students,what the assessment arrangements for the course orunit are,what methods have been used to teach students in thepast.

This last point is most important. You may wish, afterreading this chapter, to try out some new ideas withstudents. Beware! Students do appreciate good teachingbut may resent the use of some techniques that seemirrelevant to their purposes, to the course aims, and to theway their learning is assessed. When introducing newlearning and teaching techniques you must carefullyexplain the purpose of them to students. Be prepared forsome resistance, especially from senior students if they donot appreciate the connection between the techniques andthe assessment arrangements.

The course co-ordinator, curriculum committee, head ofdepartment and other teachers in the course are allpotential sources of advice and assistance to you. However,do not be surprised if you are told that you are the expertand that it is your responsibility to know what studentsshould be taught! If this happens you should insist on somehelp to review what happened in the past. To do otherwiseis to teach in an academic vacuum.

WHAT ABOUT NON-TRADITIONAL STUDENTS?

A declining proportion of university students enter directlyfrom local secondary schools. Given the growing numbersof non-traditional students, such as international studentsfrom different cultural and linguistic backgrounds andmature-age students, it is important that you note thecomposition of your class and consider this in yourplanning. What can be done? An essential starting pointis to base your teaching on sound educational principleswhich are likely to be beneficial for all students in yourclass.

17

In addition, however, you can assist your students fromdiverse backgrounds by instituting practices that providemodelling, resources for increased comprehension, andenhanced opportunities for social contact.

Modelling the kinds of learning objectives you have will beimportant for all students including those who come fromcultures where the traditional authority of teachers andauthors is strongly valued and not to be questioned, Forexample, plan to model critical or analytical thinking bypublicly questioning a set text and explicitly demonstratingthrough your own thinking approaches the ways in whichscholars in your discipline test the validity of claims made.

Resources for increased comprehension will be welcomedby the majority of your class, but particularly thosestudents for whom the language of instruction is not theirfirst language, and for hearing and sight-impairedstudents, among others. What can you do? A short listwould include: presenting an overview and structure toeach session; using concrete examples of the principlesyou are teaching; linking one session to the next and to theone preceding it; providing clear, large, legible over-heads or slides, handouts, outlines of the teaching session;audio taping of classes and the maintenance of a tapelibrary; posting class notes on the Internet or an intranet;and indicating supporting references in books and journalsand stating why each reference is important or how itrelates to the topic.

Using technology in teaching large groups can also be avalid strategy to address some of these issues. See belowfor ideas on how you might proceed with this.

Social contact will be achieved in large group teachingwhen you use some of the group-based approachesdescribed here. One of the most under-used resources inhigher education is the students themselves, so plan waysin which you might constructively use the experience andknowledge of particular students or groups of students inyour teaching. Not only will the learning be enriched butan opportunity for some social contact between yourselfand the students will have been provided.

18

PREPARING FOR LARGE GROUP TEACHING

What is the purpose of your teaching?

Having clarified the context of your large group teachingsession you need to ask yourself ‘What is its purpose?’ Thisis a question you should always ask so that you have a clearidea about matching ends with means.

A possible range of answers is given below, many of whichwill overlap.

To encourage thinking skills.Examples: Interpret a set of statistical data; evaluate aresearch proposal; criticize a journal article or medicaltreatment plan; apply earlier learning to a novelsituation.To construct an academic argument.Example: Present the pro and con arguments withrespect to a health policy issue.To present students with information about a subject.Example: Review and comment on the research on aparticular subject.To demonstrate a procedure, a way of thinking, orapproach to problem solving.Examples: Lead students through a line of reasoningabout a problem; demonstrate a clinical or technicalprocedure.

Resolving the purpose of your large group teaching will bea useful benchmark throughout the process of preparation,presentation and final evaluation.

Having clarified the context and purpose to the best of yourability the time has come to get down to some detailedplanning. The best way to start is to write down theoutcomes you hope to achieve in your teaching session(s).We say ‘write down’ advisedly because nothing clarifiesthe mind more than putting pen to paper!

Identify the content

Now set about identifying the content. We suggest you startby initially jotting down the main ideas, theories andexamples that come to mind regarding the central purposeof your teaching session. This should be done sponta-

19

neously, without any particular concern for the order inwhich you may eventually wish to organise your material.Figure 2.1 illustrates a way of doing this that has been foundto be helpful by staff attending our courses for newacademic staff.

The topic (in the example, taken from a medical lecture onhypertension) is placed in the centre of the paper and themain points to be made are written down as indicated.When the main ideas are identified, further points will tendto branch out as you think more carefully about them. Thisprocess is continued until you have exhausted all yourideas. You may at this stage discover that you need to readaround some of the ideas in order to refine them or to bringyourself up to date.

During this exercise you will find that illustrative examplesof key points come to mind. Jot these down also. In additionyou should be on the look out for illustrations from whichyou might prepare audiovisual aids and teaching materi-als. Appropriate jokes or cartoons may be collected duringthis exercise. And, most importantly, make a note of shortactivities or exercises that you can use as a basis for studentactivity.

20

Finalise the plan

Now you must finalise your teaching plan. The roughcontent plan must be transformed into a structure, whichfollows some kind of logical sequence. There is no singlebest way of doing so but you may prefer a formal structurefrom which to work. The important point is to have astructure and make this clear to students when you areteaching.

One such structure is seen in Figure 2.2.

This is the classical content-oriented plan. We hope youmay wish to be more ambitious and use other plans whichhave the potential to more effectively demonstrate howknowledge is discovered and organised in your discipline.These plans require extra thought but done well are likelyto be rewarding to both you and your students.

An example of such an alternative is the problem-focusedplan. It can be structured as follows:

21

Introduction: Presentation of the problem (e.g. as acase)First possible solutionSecond possible solutionThird possible solution etcComparison and appropriateness of possible solutionsSummary and Conclusions

Another adaptation is the academic-argument plan. Theorder in which you place arguments in this structureappears to be critical. The order suggested is:

Introduction: Overview of teacher’s position andsupporting argumentsFirst major point: Presentation of counter-argumentsSecond major point: Discussion/refutation of counter-argumentsThird major point: Arguments in favour of teacher’spositionConclusion: Restatement of teacher’s position

These examples imply that all large group sessions arecomplete in themselves. In reality, of course, this may notbe the situation. Most will be part of a series on a particulartopic or theme. Consequently, they will need to be linkedtogether to provide continuity from one session to the next.In other words, you will need to modify the planssuggested above to suit the demands of your teaching asit proceeds through the series.

The plans illustrate the general question you will have toanswer as you organize your material: ‘How will Isequence the ideas I wish to present?’ Sometimes thesequence might be indicated by the nature of the materialto be presented. However, apparently logical sequencesmay not always be optimal for student learning and youshould give some thought to the ways in which studentinterests, their knowledge and approaches to learningsuggest sequences of presentation. Some possible se-quences are:

Proceed from observations of reality, such as a brief in-class activity (e.g. a case study, an exercise or shorttest, a short film or video, a Classroom AssessmentTechnique [see below], or a demonstration), to

22

abstract ideas, theories and principles. This is some-times called an inductive approach to teaching.Proceed from generalisations to particular examplesand applications. This is a deductive approach – areversal of the inductive sequence outlined above.Proceed from simple ideas and applications to morecomplex ones.Proceed from what students can be expected to knowto what students do not know.Proceed from common misconceptions to explanationand clarification.Proceed from a whole view to a more detailed view.

PRESENTING THE LARGE GROUP TEACHINGSESSION

Having decided what you intend to teach, you must nowgive careful attention to how you are going to present it tothe students. Let us assume that it is to be your first contactwith this group of students. You may wish to obtain theirattention initially by devising an arresting opening. Waysof doing this are limited only by your personality and yourimagination. An appropriate anecdote, a video clip, aquotation or a discussion with a few of the group maygenerate interest.

However, it must be borne in mind that the attention of thestudents ought to be engaged by the material rather thanthe personality of the teacher. The danger of the latter hasbecome known as the ‘Dr Fox effect’ based on anexperiment where an actor (Dr Fox) gave a lecturecomprising meaningless double-talk which fooled experi-enced listeners into believing that they had participated ina worthwhile and stimulating learning experience.

Starting the session

Particular attention needs to be given to the way you begin.For many teachers, this is the most difficult aspect ofteaching a large group. It is essential to decide beforehandexactly how you intend to start. Do not leave this decisionuntil you are facing the students. Perhaps the easiest way tostart is to explain the purpose of the teaching session andhow it is organized. An outline on the board or on atransparency showing your teaching plan is a good way of

23

doing this. Such visual material will take attention awayfrom yourself, give you something to talk to and allow youto settle down. Writing the plan on the board gives studentsa permanent reminder of the structure of your session.

Once you become more confident, other issues should beconsidered. It is good practice to arrive early and chat withsome of the students to establish their level of previousknowledge. Alternatively, you can start by asking a fewpertinent questions, taking care that this is done in a non-threatening manner. Should you establish that seriousdeficiencies in knowledge are present you must be flexibleenough to try and correct them rather than continue onregardless.

Varying the format

You should now give attention to the body of the largegroup session. Student attention must be considered. Apurely verbal presentation will be ineffective and willcontribute to a fall-off in the level of attention. You shouldtherefore be planning ways of incorporating some of thetechniques described in the next section. Figure 2.3. showsus that levels of attention and learning will fall progres-sively. No more than 20 minutes should go by before thestudents are given a learning activity or before theteaching technique is significantly altered. Ways of doingthis include posing questions or testing the students,generating discussion among students and using anaudiovisual aid. These active learning strategies arediscussed in more detail later.

24

Finishing the session

The conclusion is as important as the introduction. Yourclosing comments should also be well prepared. The lastthings that you say are the ones the students are most likelyto remember. This will be the opportunity to reiterate thekey points you hope to have made. You may also wish todirect students to additional reading at this time, but bereasonable in your expectations and give them a clearindication of what is essential and why it is essential asopposed to what you think is merely desirable. A couple ofminutes near the conclusion to allow them to consolidateand read their notes is a worthwhile technique to use fromtime to time.

Rehearsal and check

Some of the best teachers we know find it very helpful torehearse or to try-out some parts of their teaching so thismay be even more important for the less experienced.However, the purpose of the rehearsal should not be tobecome word perfect, and it is impossible to rehearse theoutcomes of activities you give your students. A rehearsalwill often reveal that you are attempting to cram too muchinto the time and that some of your visual aids are poorlyprepared or difficult to see from the rear of the theatre. Thevalue of a rehearsal will be much enhanced if you invitealong a colleague to act as the audience and to providecritical comments and to help you check out projectors,seating, lighting, air conditioning, and other physicalmatters.

In some institutions you will have access to courses onteaching methods. Overcome your natural reticence andenrol. It is likely that one component of the course will giveyou the opportunity of viewing your teaching technique onvideo. The unit running the course may also provide anindividual to come and observe your teaching, giving youthe expert feedback you may not always get from acolleague.

Some personal considerations about anxiety whenteaching large groups

When you are satisfied that you have attended adequatelyto the kinds of things discussed above, you will find it

25

helpful to reflect on some matters of personal preparationfor your teaching. Paramount among these considerationsis dealing with nervousness - both before and during yourlarge group session.

Most teachers, speakers and actors confess to feelinganxious before ‘going on-stage’. However, if you arethoroughly prepared, much of the potential for nervous-ness will have been eliminated. And you should keep inmind that a certain level of anxiety is desirable to ensurethat you perform well!

One writer on higher education, Christine Overall, hasdescribed the commonly experienced anxiety in terms of‘feeling fraudulent’. She suggests a way of managing thisfeeling is to act as if you know what you are doing, and todisplay the confidence and authority to do what you need todo. In the large group session, this may mean looking at theaudience, smiling, handling audiovisual equipment con-fidently, being very clear and firm about instructions foractive learning tasks, knowing what you will say and do atthe beginning and ending of your session, and so on.

Apart from being thoroughly prepared, there are anumber of ‘do’s and don’ts’ to keep in mind.

Do not allow yourself to conjure up visions of mistakesand disasters. Think positively. Imagine an interestedand appreciative audience, achievement of the goalsyou set for yourself and your students, and being incontrol of the situation.Practise some relaxation exercises and deep breath-ing. Consult one of the numerous booklets or cassetteson this topic or attend a relaxation class if you think itmight help.If you can, plan on arriving at the lecture room earlyenough to ensure everything is in order and to allowtime to talk to one or two of the students in the classabout their work. This approach not only serves as avaluable ‘bridge’ between you and your students, butalso can be very helpful in meeting some of their needsand understanding difficulties that you might be able toincorporate into your teaching.

26

WHAT ACTIVE LEARNING STRATEGIES AREAVAILABLE?

Active learning stands in contrast to much of what passesfor ‘learning’ in large lecture classes - it is lively, dynamic,engaging and full of life. It is a basis from which lifelonglearning skills can be developed. Active learning is oftendefined in contrast to the worst of traditional teachingwhere the teacher is active and the student is the passiverecipient. Specifically, active learning occurs when youuse strategies to ensure the session includes elements ofstudent activity such as talking, reading, writing, thinking,or doing something. These activities might be undertakenalone, in pairs of students, or in small groups of up to aboutfour.

There are several levels at which we would encourage youto plan for student activity. At its most basic level, we havealready stressed how variety in the presentation isessential in maintaining attention and therefore thepossibility of engaging with the material.

Variations in your manner and style

It is important that you feel comfortable with the way youpresent your session. However, you should not limit yourmanner and style. Changes in the volume and rate ofspeech, the use of silence, the maintenance of eye contactwith the class and movement away from the lectern tocreate a less formal relationship should all be considered.

Active participation

A powerful way of enhancing learning is to devisesituations that require the students to interact with you orwith each other. Questions are the simplest form ofinteraction. Many teachers ask for questions at the end oftheir presentation but most are disappointed in the studentresponse. Others direct questions at students but unless theteacher is very careful, the dominant emotion will be one offear. It is therefore preferable to create a situation in whichall students answer the questions and individuals are notplaced in the spotlight.

You may wish to prepare a question in the form of amultiple-choice or true-false item that can be projected as a

27

slide or an overhead transparency. Asking for a show ofhands for each alternative answer to the question can checkunderstanding. You should follow up by explaining whyeach alternative is or is not a suitable answer. The timerequired for this will usually be about 5-7 minutes (1-2minutes to answer the question, 4-5 minutes to givefeedback on the correct and incorrect answers).

An alternative is having students ask questions. Again, ifyou ask ‘are there any questions?’ silence is the likelyresponse because individuals do not generally like to bethe focus of attention, particularly in very large classes. Butif you ask students to write a question on paper and turn itin, you can then address some or all of these in a relativelyanonymous and non threatening way.

Small group activity within a large group is uncommonlyattempted even though it is simple to arrange for a largenumber of students in a theatre of any size. Once you try itout you may find it so exciting to hear the steady hum ofstudents actually discussing your subject that you willnever again feel comfortable giving a didactic lecture! Thegeneral approach is to break down the class into smallgroups, using a judicious rearrangement of seating ifnecessary. Small groups of two to four people may beformed among neighbours without any movement whilelarger groups may be quickly formed by two to fourstudents in one row turning to form a group with students inthe row behind. If a substantial amount of discussion time isplanned the groups might best be formed at the beginningof the session and asked to spread themselves out to use upthe whole lecture theatre space. The selection of the mostappropriate grouping will largely depend on what youwish to achieve. Small groups may be asked to discuss alimited topic for a few minutes (sometimes called buzzgroups) or to consider broader topics for a longer periodof time. You may then wish to allow all or some of thegroups to report back to you. This is a very useful exercisewhen problems are given to the students to solve andwhere a variety of different responses can be expected.Some more specific examples of small group activity arenow discussed.

One-to-one discussion is a particularly valuable techniquein the situation where you might wish all the class to

28

consider a very emotive or challenging concept. Thismethod is described in detail in Chapter 3 on Small GroupTeaching.

Reading or problem-solving activities may be introduced.These can involve a combination of individual study andsmall group discussion. There are many variations on thisstrategy.

One example is the situation in which students areprovided with an extract or article from a medical journal,a summary, a quotation, a set of diagrams or a set of clinicalresults. A directed-reading or problem solving task is set.This task should involve the students for 5-10 minutes. Atthe end of this period of individual work students areinstructed to discuss something with the person besidethem. They may, for instance, be asked to compareanswers, draw conclusions, raise issues, identify mis-understandings or make evaluative judgements. Thestudents are then asked for feedback. Depending on thesize of the group, you could ask for reports from all orsome of the pairs, have pairs report to another pair andseek general reports from these larger groups, or have ashow of hands to questions or issues you have identified asyou moved around the class during the discussion phase.Alternatively you could ask students to write responses,then collect these, and collate the information after thesession as a basis for your teaching in the next session.Conclude by drawing ideas together, summing-up, orwhatever is appropriate to the task you set them.

Whatever you do – and this is critical – thoroughly plan theactivity: clearly structure the time and the tasks set, andstick to your plan (unless there are very good reasons tochange). Your instructions, including the time availableand tasks to be carried out, should be clearly displayed ona handout, or on the board, for ready reference during theexercise.

Brainstorming is a technique which can be modified for usein large group teaching. It can be of value at the beginningto stimulate interest in the topic to be discussed. Thestudents are presented with an issue or a problem andasked to contribute as many ideas or solutions as they can.All contributions are accepted without comment or

29

judgement as to their merits and are written on the boardor on an overhead transparency. This approach en-courages ‘lateral’ or ‘divergent’ thinking. One of us hassuccessfully used this technique with a class of 120 at thebeginning of a lecture. The session commenced with arequest for the class to put forward their suggestions inresponse to a question. These suggestions were thencategorised and used as a basis for further discussion in anenvironment where the students had been the initiators ofthe discussion points. Brainstorming is discussed further inthe chapter on small group teaching.

Classroom Assessment Techniques (CATs) are a relativelyrecent innovation that we would encourage you to use withyour students. These techniques stimulate active learningbut most importantly help teachers gather useful informa-tion on what, how much, and how well their students arelearning. The simplest and most popular CAT is the ‘minutepaper’. In this technique, the teacher stops two or threeminutes early and asks students to write anonymous briefresponse to a question such as: “What was the mostimportant thing you learned in this class?” or “Whatimportant question remains unanswered?” Papers arecollected and reviewed by the teacher prior to the nextclass meeting at which time feedback may be given orpoints clarified.

Another CAT is the ‘pro and con grid’. This techniqueassists in the development of thinking skills by encoura-ging students to go beyond initial reactions to an issue. Inresponse to a suitable prompt or question, students writeout a specified number of pros and cons or advantages anddisadvantages. These can then be discussed in smallgroups, analysed in class, or analysed yourself prior to thenext class session. We strongly recommend the book byAngelo and Cross on classroom assessment.

Student note-taking

The research in this area generally supports the view thatnote taking should be encouraged. It is a process whichrequires student attention and activity. The teacher canassist this process by providing a structure for material thatis complex. Diagrams and other schematic representationsmay be more valuable than simple prose.

30

USING TEACHING MATERIALS AND TECHNOLOGY

Educational issues and the technical aspects of preparingand using teaching aids and materials are discussed indetail in Chapter 9. This section will review their use inlarge group teaching for a variety of purposes includingillustrating the structure, providing information andexamples, stimulating interest and activity, and providingvariety. The aids most likely to be used are handouts, theboard, overhead transparencies, slides, videos and,increasingly, on-line Internet sessions.

Handouts must serve a clear purpose and be used duringthe teaching session so that students are familiar with theircontent and simply do not file them away. Handouts may bevaluable as a guide to the structure of your session and inthis case may be very similar in content to the teachingplan. Such a handout should be given out at the beginning.You may wish to use the handout to provide detailedinformation on an area not well covered in standardstudent texts or not covered in detail in your teaching. Suchhandouts might be given out at the end of the session.Handouts may also be used to guide further study and toprovide references for additional reading. Whenever youdistribute handouts, it is essential that you use them in someway with your students.

‘Blackboards’ (which are usually green these days) andwhiteboards are still very widely used and we urge you tolook at the information in Chapter 9 about their preparationand use. Clear, legible and well-planned use of these basicaids is a delight to see and remain valuable allies inassisting you to communicate with your students. They areespecially worthwhile for displaying an outline of yoursession or for recording feedback from students inresponse to questions you may have raised.

The overhead projector is extensively used in teachingand is particularly useful for giving outlines and listing keypoints. A blank sheet of paper can be used to reveal thepoints in sequence. A pen or pencil placed on thetransparency itself should be used to direct the students’attention to the appropriate point rather than using thepointer on the screen. Information may be added with a feltpen to the transparency as the teaching proceeds. We havefound that the value of the overhead is seriously reduced

31

by four common practices. First, too much information isincluded on each transparency. Secondly, the teacherworks through the material too quickly or talks aboutsomething different while students are trying to read andtake notes from the screen. Thirdly, the transparency iscarelessly positioned or is out of focus. Fourthly, and themost common abuse, is that material on transparencies isfar too small to be read by students.

The 35-mm projector is still widely used and some teachersbuild up an extensive collection of slides. However, manyteachers are now incorporating their slides into Power-Point® presentations. Slides too are often misused. Thosecontaining printed material should be kept simple andmust be clearly visible at the back of the theatre with thelights on. Care must be taken when reproducing materialfrom books and journals, which often contain far too muchinformation. Coloured slides of relevant material are idealfor illustrating points and for adding variety and interest.When using slides, avoid turning off the lights for morethan brief periods. The level of attention will rapidly fall,however interesting your slides happen to be.

Computer presentation systems (e.g. PowerPoint®) arerapidly taking over the function performed by both theoverhead and slide projectors. If you teach in locationswhere you are confident of the technology then there aremany advantages in using this aid including ease andflexibility of preparation and the capacity to generatestudent notes derived directly from your presentation. Youcan also incorporate video and sound in your presentationas required. In Chapter 9 we give you more informationabout preparing and using these systems.

A computer presentation is governed by the sameprinciples as those for slides and overheads – clear,legible text and pictures, and use in a room wheresufficient lighting can be left on for student note-takingand activities. If you are not confident of the environment inwhich you are teaching and in case the technology fails, it isstill wise to have overhead transparencies or slidebackups.

Videos, and less often films these days, are best utilised inshort segments. Their use requires more careful planning,

32

as it will be necessary to have a technician to set upequipment. However, the effort is well worthwhile for boththe impact of the content and the variety it introduces. Weuse such material to show illustrative examples andpractical techniques. They may also be used in attemptsto influence attitudes or to explore emotionally chargedissues. A short segment (trigger) can be shown illustratingsome challenging situation and the class asked to react tothis situation. Videos and films for this purpose arecommercially available in some disciplines.

WHEN THINGS GO WRONG

Throughout this book we present the view that things areless likely to go ‘wrong’ if you have carefully preparedyourself for the teaching task. However, unexpecteddifficulties can and do arise, so strategies to deal withthese need to be part of your teaching skills. In ourexperience problems in teaching large groups are likelyto fall into one of the following categories.

Problems with audio-visual materials and equipment. Anequipment failure can be a potential disaster if you haveprepared a computer presentation or a series of slides ortransparencies for projection. Preventive measures in-clude having a thorough understanding of your equip-ment, back-up equipment on hand, and learning to changeblown bulbs or remove jammed slides.

If these measures are of no avail, you will have to continueon without the materials and may do so successfullyprovided that you have taken care to have a clear recordin your notes of the content of your material. Photocopiedenlargements of slides of data are a useful backup here.You may then be able to present some of the informationverbally, on a blackboard or whiteboard, or on anoverhead transparency if the original problem was withthe slide projector. You will not, of course, be able to usethis approach with illustrations and you may have tosubstitute careful description and perhaps blackboardsketches to cover essential material. Whatever you do, donot pass around your materials, which may be damagedand, of course, by the time most of the audience receivethem, they are no longer directly relevant to what you aresaying!

33

Difficulties with your presentation. Losing your place andrunning out of time can be disconcerting. Do not startapologising or communicate your sense of ‘panic’ if thisshould happen. Instead, pause, calmly evaluate yoursituation, decide on a course of action, and continue. Onelecturer we know invites students to check their noteswhile she simply cleans the board as she thinks throughwhat to do next!

Challenges from students. We have deliberately avoidedthe use of the word ‘problem’ in relation to your interactionwith students because the ‘problem’ may be with you (thatis, your manner, your preparation or presentation, forexample) or it could be more in the form of a genuinelymotivated intellectual challenge to what you have beendoing or saying. It is essential to be clear as to exactly whatthe challenge is and why it has occurred before you act.

Many teachers fear confrontation with students in a class.We cannot go into all aspects of classroom managementand discipline here, but we can identify a number ofprinciples and refer you to more detailed discussionselsewhere (McKeachie’s Teaching Tips is a usefulreference).

Disruptive behaviour and talking in class are commonchallenges and must not be ignored, both for the sake ofyour own concentration and for the majority of studentswho are there to learn. Simply stopping talking and waitingpatiently for quiet usually overcomes minor disturbances.If this happens more than once the other students willusually make their displeasure known to the offenders. Ifthe disruption is more serious, you will have to speakdirectly to the students concerned and indicate that you areaware of the offence. But do try initially to treat it withhumour or you may alienate the rest of the class. If theproblem persists, indicate that you will be unable totolerate the situation again and that you will have to askthem to leave. Make sure you do just this if the problem re-emerges, Do so firmly and calmly. If the situation leads toconfrontation, it is probably best if you leave the room. It isremarkable what effect this has on students! Make everyattempt to meet the offenders afterwards to deal with theproblem.

34

We have been appalled at accounts of teachers whoendure the most unreasonable physical and verbal abusein classes and do nothing about it – other than sufferinwardly. There is no need for this and the majority ofstudents will look for firm but fair disciplinary measures.An added measure is to arrange arriving and leavingclasses so that you have time to get to know at least some ofthe students in the class – especially potentially trouble-some ones. The active learning strategies we havesuggested in this chapter are also ways of addressingthese challenges – both by engaging students in their ownlearning (and using up some of their energy in this way!)and giving you a further opportunity to get to know them.Anonymity is a great accomplice in disruptive behaviour!

EVALUATING LARGE GROUP TEACHING

Improving the quality of your teaching in large groups willdepend on a combination of experience and your will-ingness to critically evaluate your performance. Evaluationmay be seen as informal or formal. The informal way mayinvolve asking several students whom you know for theircomments. It may also be undertaken by asking yourself aseries of questions immediately after your teaching, suchas:

How much time was taken to prepare?Were the notes helpful?Were the visual aids clear and easy to read?What steps could be taken to improve preparation andorganisation of the active learning tasks?Did the questions stimulate discussion?What did I learn about students understanding fromtheir questions/comments/written responses to theCATs?Were the purposes of the session achieved? How do Iknow this?

The distribution of questionnaires to the class is a moreformal way of evaluating teaching. Many such forms havebeen designed and can usually be obtained from theteaching unit within your institution. The best way ofobtaining an independent evaluation is to seek the servicesof a teaching unit. They will sit in on your teaching andprepare a detailed analysis and go over it with you later.

35

A CONCLUDING THOUGHT - IF YOU MUST‘LECTURE’…

It may be objected that the crowded curriculum does notallow time for the active learning techniques described inthis chapter. This objection rests on the view that materialhas to be ‘covered’. However, this argument presumes thatstudents will learn the material covered and this isunfortunately not usually the case. We have already shownthat levels of attention to a traditional expository lecturedecline and it is known that other indicators of perfor-mance, such as recall, fall fairly rapidly around 20 minutesafter the lecture begins. Worse, what little is learned in theremainder of the lecture time interferes with under-standing of material presented earlier.

So, if you feel the inclination to ‘lecture’ so that you havecovered the material, perhaps the question you should beasking yourself is ‘Should I be wasting so much timespeaking for 50 minutes?’ In some courses it is the case thata few students regard lectures as an important learningactivity. Further, lectures are perceived as being a meansto pace study, as a way of keeping in touch with thecoursework, and as a supplement to other more importantlearning activities such as practical classes, tutorials andassignments. At worst, lectures are seen as a boring wasteof time relieved only by the skill and daring of the paperplane throwers and other attention seekers!

The challenge is to work out a clear and educationallydefensible rationale for lecturing. Lecturing can only be auseful learning method for students where the techniquesof teaching large groups are appropriately employed. Wehope that this chapter has contributed to dealing with thesechallenges.

GUIDED READING

Almost all books which are concerned with the practi-calities of teaching in higher education will devote somespace to the lecture method or teaching large numbers ofstudents and you will undoubtedly find many of thesehelpful.

Perhaps the most popular book on lecturing is D. Bligh’sWhat’s the Use of Lectures?, Jossey-Bass, 2000. This book

36

gives an overview of the research, useful information onpreparing and delivering lectures, and an interestingsection on alternatives to the lecture.

Another source of practical advice is the HERDSA GreenGuide (No 7, 1992) Lecturing by R. Cannon. It includesmore detail than provided in this chapter on preparation,presentation, evaluation and active learning methods. (It isavailable from the Higher Education Research andDevelopment Society of Australasia, HERDSA, PO Box516, Jamieson, ACT 2614, Australia.)

Another useful book to review for ideas on improvinglearning from lectures is 53 Interesting Things to Do inYour Lecture by G. Gibbs, Technical and EducationalServices Ltd, Bristol, 1992.

There is also a large published literature on active learningin large groups. A good introduction to this concept isPromoting Active Learning by C. Meyers and T. Jones,Jossey-Bass, San Francisco, 1993.

Books and journals referred to in this chapter:

T. Angelo and K. Cross. Classroom Assessment Techni-ques: A Handbook for College Teachers, Jossey Bass, SanFrancisco,1993.

W. McKeachie. McKeachie’s Teaching Tips: Strategies,Research, and Theory for College and UniversityTeachers, Houghton Mifflin Co, Boston, 1998.

J. Ware and R. Williams (1975). ‘The Doctor Fox Effect’: Astudy of lecturer effectiveness and rating of instruction,Journal of Medical Education, 50, 149-156.

37


3: TEACHING IN SMALL GROUPS

INTRODUCTION

Small group teaching can be a most rewarding experiencefor both teacher and student. To achieve success you willneed to plan carefully and develop skills in groupmanagement. You should avoid the common error ofbelieving that constructive discussion in groups willhappen spontaneously. To avoid difficulties you will needan understanding of how groups work and how to apply arange of small group techniques to achieve the goals youset out to achieve.

THE IMPORTANCE OF SMALL GROUP TEACHING

Teaching in small groups enjoys an important place amongthe teaching methods commonly found in medicaleducation for two rather different reasons. The first ofthese can be described as social and the other aseducational. For many students in the university, andespecially those in the early years of their studies, the smallgroup or tutorial provides an important social contact withpeers and teachers. The value of this contact should not beunderestimated as a means for students to meet and dealwith people and to resolve a range of matters indirectlyassociated with your teaching, such as difficulties withstudying, course attendance and so on. Such matters will, ofcourse, assist with the attainment of the educationalobjectives of your course.

Among the educational objectives that you can bestachieve through the use of small group teaching methodsare the development of higher-level intellectual skills suchas reasoning and problem-solving; the development ofattitudes; and the acquisition of interpersonal skills such aslistening, speaking, arguing, and group leadership.Increasingly, students will need these skills so that theycan participate effectively in problem-based learning (seeChapter 7). These skills are also important to medicalstudents who will eventually become involved profession-ally with patients, other health care professionals, commu-nity groups, learned societies and the like. The distinctionbetween social and educational aspects of small groupteaching is rather an arbitrary one but it is important tobear it in mind when you plan for small group teaching.

40

WHAT IS SMALL GROUP TEACHING?

Much of what passes for small group teaching in medicalschools turns out to be little more than a lecture to a smallnumber of students. We believe that small group teachingmust have at least the following three characteristics:

Active participationFace-to-face contactPurposeful activity

Active participation

The first, and perhaps the most important, characteristic ofsmall group teaching is that teaching and learning isbrought about through discussion among all present. Thisgenerally implies a group size that is sufficiently small toenable each group member to contribute. Research andpractical experience has established that between five andeight students is ideal for most small group teaching. Youwill know that many so-called small groups or tutorialgroups are very much larger than this ideal. Although agroup of over twenty students hardly qualifies as a smallgroup it is worth remembering that, with a little ingenuity,you can use many of the small group teaching proceduresdescribed in this chapter with considerable success withlarger numbers of students. Generally speaking, though,you will be looking for a technique which allows you tobreak such numbers down into subgroups for at least someof the time.

Face-to-face contact

The second characteristic of small group teaching is that itinvolves face-to-face contact among all those present. Youwill find it difficult to conduct satisfactory small groupteaching in a lecture theatre or tutorial room with studentssitting in rows. Similarly, long board-room type tables arequite unsuitable because those present cannot see all othergroup members, especially those seated alongside.Effective discussion requires communication which is notonly verbal but also non-verbal involving, for example,gestures, facial expressions, eye contact and posture. Thiswill only be achieved by sitting the group in a circle.

41

Purposeful activity

The third characteristic of small group teaching is that thesession must have a purpose. It is certainly not an occasionfor idle conversation although, regrettably, some teachingin groups appears to be little more than this. The purposesyou set for your small group can be quite wide. Theyinclude discussing a topic or a patient problem anddeveloping skills such as criticising, analysing, problem-solving and decision making. It is highly likely that you willwish the small group session to achieve more than onepurpose. In medical schools, most groups are expected todeal with a substantial amount of content. However, you willalso wish to use the small group approach to develop thehigher intellectual skills of your students and even toinfluence their attitudes. In order to achieve these variouspurposes you will need considerable skills in managingthe group and a clear plan so that discussion will proceedin an orderly fashion towards its conclusion.

MANAGING A SMALL GROUP

Small group teaching is considerably more challenging tomanage than a conventional lecture because you must takeaccount of the students’ behaviour, personalities and theemotional aspects of being in a group. To achieve successwith a small group you must also have a clear under-standing of how a group operates and how it develops. Youhave particular responsibilities as the initial leader of thegroup but your role will vary considerably both within asession and from session to session. For instance, if youadopt an autocratic or authoritarian style of leadership (notan uncommon one among medical teachers) you may wellhave a lot of purposeful activity but there will be a limitedamount of spontaneous participation. You should pre-ferably adopt a more co-operative role where youdemonstrate an expectation that the students will takeresponsibility for initiating discussion, providing informa-tion, asking questions, challenging statements, asking forclarification and so on. A successful group is one that canproceed purposefully without the need for constantintervention by the teacher. This is hard for most teachersto accept but is very rewarding if one recognises that thisindependence is one of the key goals of small groupteaching and is more important than satisfying one’s ownneed to be deferred to as teacher and content expert.

42

In managing a group there are two main factors thathave tobe considered. These are factors relating to the task of thegroup and factors relating to the maintenance of the group.In addition there must be a concern for the needs of eachstudent within the group.

The tasks of the group: clear definition of tasks is somethingthat must be high on the agenda of the first meeting. Thereason for the small group sessions and their purpose inthe course must be explained. In addition, you shouldinitiate a discussion about how you wish the group tooperate, what degree of preparation you expect betweengroup meetings, what role you intend to adopt, what rolesyou expect the students to assume and so on. Because suchdetails may be quickly forgotten it is desirable to providethe students with a handout. The list of headings in Figure3.1 may be helpful:

Maintenance of the group: this refers to the achievement ofa good ‘climate’ for discussion. Ideally it is one that is open,trustful and supportive rather than closed, suspicious,defensive and competitive. It is important to establish thatthe responsibility for group maintenance rests with thestudents as well as with the teacher. The firm but pleasanthandling of the loquacious or dominating students early inthe session or the encouragement of the quiet student areobvious examples of what can be achieved to produce therequired environment for effective group discussion.

The successfully managed group will meet the criteriashown in Figure 3.2.

43

STRUCTURE IN SMALL GROUP TEACHING

We mentioned earlier the need to have a clear plan so thatthe group discussion will proceed with purpose and in anorderly fashion. A structured approach to tasks and to theallocation of the time available is a useful tool for you toconsider. An example of such a structured discussionsession is illustrated in Figure 3.3.

44

This is a structure of a discussion based on a casepresentation. Note that the structure lays out what is to bediscussed and how much time is budgeted. Such a schemeis not intended to encourage undue rigidity or inflexibility,but to clarify purposes and tasks. This may seem to be atrivial matter, but it is one which creates considerableuncertainty for students. Keeping to a time budget is verydifficult. You need to be alert to how time is being spentand whether time for one part of the plan can betransferred to an unexpected and important issue thatarises during discussion.

Another structure, not commonly used in medical educa-tion, is illustrated in Figure 3.4.

This structure includes the principle of ‘snow-balling’groups. From an individual task, the student progressesthrough a series of small groups of steadily increasing size.There are special advantages in using this structure whichare worth noting: it does not depend on prior studentpreparation for its success; the initial individual workbrings all students to approximately the same level before

45

discussion begins; and it ensures that everyone partici-pates, at least in the preliminary stages.

INTRODUCING STIMULUS MATERIALS

A very useful means of getting discussion going in groupsis to use what is generally known as ‘stimulus material’. Wehave seen how this was done in the snowballing groupstructure described previously. The range of stimulusmaterial is really very large indeed. It is limited only byyour imagination and the objectives of your course. Hereare a few examples:

A short multiple-choice test (ambiguous items workwell in small groups).A case history.Video (e.g. short open-ended situation, such as apatient’s reaction to a doctor).A real or simulated patient.Observation of a role-play.Visual materials (e.g. X-rays, photographs, slides,specimens, real objects, charts, diagrams, statisticaldata).An audio recording (e.g. an interview, heart sounds, asegment of a radio broadcast).A student’s written report on a project or a patient.Computer or Web-based material.A journal article or other written material (aninteresting example is provided by Moore where heused extracts from literary works to help studentsunderstand the broader cultural, philosophical, ethicaland personal issues of being a doctor. Examples ofsources of these extracts included Solzhenitsyn’sCancer Ward and Virginia Woolf’s On Being III).

ALTERNATIVE SMALL GROUP DISCUSSIONTECHNIQUES

As with any other aspect of teaching it is helpful tounderstand several techniques in order to introducevariety or to suit a particular situation. Such techniquesinclude:

One-to-one discussionBuzz groupsBrainstormingRole playingEvaluation discussion

46

1. One-to one discussion

This is a very effective technique which can be used with agroup of almost any size. It is particularly useful as an ‘ice-breaker’ when the group first meets, and is valuable forenhancing listening skills. It can also be used to discusscontroversial or ethical issues when forceful individualswith strong opinions need to be prevented from dominat-ing the discussion (Figure 3.5).

Buzz groups

These are particularly helpful to encourage maximumparticipation at one time. It is therefore especially usefulwhen groups are large, if too many people are trying tocontribute at once or, alternatively, if shyness is inhibitingseveral students (Figure 3.6).

47

3. Brainstorming

This is a technique that you should consider when you wishto encourage broad and creative thinking about aproblem. It is also valuable when highly critical groupmembers (including perhaps yourself?) appear to beinhibiting discussion. If used frequently, it trains studentsto think up ideas before they are dismissed or criticised.The key to successful brainstorming is to separate thegeneration of ideas, or possible solutions to a problem,from the evaluation of these ideas or solutions (Figure 3.7)

4. Role playing

This is a powerful and underused technique. It is valuablein teaching interpersonal and communication skills,particularly in areas with a high emotional content. It hasbeen found to be helpful in changing perceptions and indeveloping empathy. It is not a technique to use without

48

some experience so you should arrange to sit in on a roleplay session before using it in your own course. In thisregard, your colleagues teaching psychiatry or counsel-ling should be able to help as will the Green Guide byErnington (Figure 3.8).

5. Plenary sessions

In many group teaching situations, and sometimesconferences, subgroups must report back to the largergroup. This reporting back can be tedious and ofteninvolves only the subgroup leaders who may present avery distorted view of what happened. The plenarysession method may help you solve these problems(Figure 3.9).

49

USING TECHNOLOGY FOR TEACHING SMALLGROUPS

By combining computers and communication technology,you can make fundamental changes to the way you presentand distribute material and interact with your students. Byusing these technologies, the distinctions between largeand small group teaching tend to break down.

However, some examples of the ways in which technologycan be used specifically to support small group teachingare:

By using electronic mail (e-mail) to communicate withone or more studentsThrough electronic discussion groupsBy conferencing techniques using computer, soundand video.

Implementing electronic teaching is different to otherapproaches in many ways. For instance, uninitiatedcolleagues will be totally uncomprehending if you objectto being interrupted when you are working at yourcomputer by saying you are ‘teaching’!

One fundamental difference between face-to-face teachingand being ‘on-line’ is that you will be interacting with whatis known as a ‘virtual’ group. This means that the groupdoes not exist as an entity at any one time or place, but thatit is dispersed both in time (within limits) and place and thatthe group interacts ‘asynchronously’ at different times.Furthermore, there are different rules of behaviour forelectronic communication known colloquially as ‘neti-quette’ and which both you and your students shouldobserve. To learn more about this topic you can search forit on the World Wide Web. A very good overview isprovided at:<http//www.albion.com/netiquette/book/index.html>or in the book Netiquette by Virginia Shea.

EVALUATING SMALL GROUP TEACHING

Evaluation implies collecting information about yourteaching and then making judgements based on thatinformation. Making judgements based on what onestudent says, or on rumour or intuition, is simply not goodenough. You must collect information in a way that is likelyto lead to valid conclusions. However, constant evaluation

50

of small group activities is not recommended as it mayinhibit the development and working of the group.Evaluation may be of two types: informal or formal.

Informal evaluation: this can proceed from a carefulreflection on what happened during your time with thegroup. You may do this by considering a number ofcriteria which you feel are important. For example, youmay be interested in the distribution of discussion amonggroup members, the quality of contribution, the amount oftime you talked, whether the purpose of the session wasachieved, what students have learned, and so on. Ofcourse, your reflections will be biased and it is wise to seekconfirmation by questioning students from time to time.However, the importance of informal evaluations lies inyour commitment to turn these reflections into improve-ments. If you are concerned with your own performance,the assistance of a trusted and experienced colleaguesitting in on the group, or even just a discussion of yourown feelings about the group, may be very helpful.

Formal evaluation: formal approaches to evaluationinclude the use of questionnaires or the analysis of videorecordings of the group at work. Standard questionnairesare available which seek student responses to a setnumber of questions. An example is shown at the end ofthe chapter (Figure 3.10).

Although such standard questionnaires can be useful youmay find it more beneficial to design one that contributesmore directly to answering questions which relate to yourown course and concerns. As questionnaire design can bedifficult, it is recommended that you seek the assistance of ateaching unit. The analysis of videos of your group at workis also a task which would require the expertise ofsomeone from a teaching unit.

WHEN THINGS GO WRONG

You may encounter a variety of difficulties in your groupsessions. For example, while you might decide to ignore asleeping student or an amorous couple in a lecture class,providing it was not disruptive, it would be impossible todo so in a small group. How you resolve these difficultieswith the group is critical. An authoritarian approach would

51

almost certainly destroy any chance of establishing the co-operative climate we believe to be essential. It is generallymore appropriate to raise the problem with the group andask them for their help with a solution.

One of your main roles as a group leader is to be sensitiveto the group and the individuals within it. Research hasidentified a number of difficulties that students commonlyexperience. These are:

Making a contribution to the discussion.Understanding the conventions of group work andacceptable models of behaviour.Knowing enough to contribute to the discussion.Being assessed.

These difficulties frequently get in the way of productivediscussion. They tend to be due to genuine confusion onthe part of students combined with a fear of exposing theirignorance in front of the teacher and their peers. It istherefore essential for you to clarify the purpose of thegroup and the way in which students are to enter into thediscussion. Their previous experience of small groupsessions or ward teaching might lead them to see theoccasion as only a threatening question and answersession. They must learn that ignorance is a relative termand that their degree of ignorance must be recognised andexplored before effective learning can begin. A will-ingness by the teacher to admit ignorance and demon-strate an appropriate way of dealing with it will be veryreassuring to many students.

Confusion in the students’ minds about how they are beingassessed can also cause difficulties. Generally speaking,assessing contributions to discussion is inhibiting andshould be avoided. If you do not have discretion in thismatter then at least make it quite clear what criteria you arelooking for in your assessment. Should you be able todetermine your own assessment policy then the followingare worth considering:

Require attendance at all (or a specified proportion of)group meetings as a prerequisite.Set formal written work, e.g. an essay, a series of shortpapers, a case analysis.Set a group-based task, e.g. keeping an account of thework done by the group.

52

The teacher’s perceptions of group difficulties may notnecessarily match those of the students. A discussion withthe group about how they think things are going or theadministration of a short questionnaire are ways of seekingfeedback.

Once the group is operating it is important to monitorprogress. Be sensitive to the emotional responses of thegroup and to the behaviour of individual students. Thebook by Tiberius is a useful source of additional advice onsuch issues.

GUIDED READING

For a wide-ranging discussion of the purposes andtechniques of small group teaching we suggest you turnto the collection of papers edited by D. Bligh: TeachThinking by Discussion, SRHE/NFER, Nelson, Guildford,UK 1986. This monograph also provides a good introduc-tion to the research literature on small groups. Also SmallGroup Teaching: A Troubleshooting Guide by R. Tiberius,Kogan Page, London, 1999.

Another excellent guide, to both the theory and thepractice of group work, is D. Jacques, Learning in Groups,Kogan Page, London 1991.

Books and journals referred to in this chapter

W. Hill. Learning Thru Discussion. Sage, California, 1982.

W. Bion. Experiences in Groups. Tavistock, London, 1968.

A. Moore (1976). Medical Humanities – A New MedicalAdventure. New England Journal of Medicine, 295, 1479-80.

A. Northedge (1975). Learning Through Discussion at theOpen University. Teaching at a Distance, 2, 10-17.

E. Ernington (1997). Role Play. HERDSA Green Guide No21. (available from HERDSA, PO Box 516, Jamieson, ACT2614, Australia).

53

54

4: MAKING A PRESENTATION ATA CONFERENCE

INTRODUCTION

This chapter may appear to be out of place in a book abouteducation and teaching. However, most medical teachers,at some time, will wish to make a presentation at a scientificmeeting. There are many obvious similarities betweengiving a lecture and presenting a paper. There are alsosignificant differences which may not be quite so obviouswhich made us feel that this chapter might be appreciated.

Poster sessions are growing in popularity at many nationaland international meetings as an alternative to the formalpresentation of papers. We have, therefore, included ashort segment on the preparation of a conference poster.

PRESENTING A PAPER

Though much of the advice given in the chapter onlecturing is just as relevant in this section, the aims of ascientific meeting or conference are different enough towarrant separate consideration. Much of this differencerelates to the restriction on time. It is likely that a strict timelimit will be imposed. If you are in the position to give apaper it is certain that you will have a lot to say, far more infact than can possibly be delivered in such a short time.You will also be caught in the difficult situation of havingmany of the audience unfamiliar with the details of yourarea of interest, some of the audience knowing consider-ably more than you do about the area, and all of theaudience likely to be critical of the content and presenta-tion. These and other factors make the giving of a scientificpaper an anxiety provoking situation, particularly for theyoung and inexperienced lecturer hoping to make a goodimpression on peers and superiors. However, it is also asituation that is amenable to resolution by careful planningand attention to technique.

PREPARING THE PAPER

Much of the advice that follows was admirably dealt withmany years ago in the classic book by Calnan and Barabasentitled Speaking at Medical Meetings, They describedthree stages that you must go through during thepreparation of a short scientific communication.

56

The collection and selection of the data.The arrangement (getting the structure right anddeciding on the most suitable visual aids).Polishing, writing it out and rehearsal.

These same basic headings have been retained in thischapter.

The collection and selection of the data

There is a great tendency for speakers to cram more thanis possible into their papers with the inevitable conse-quence of either speaking too fast or going over time. Theaudience knows that it is not possible to cover everythingin detail and is primarily interested in hearing a shortcohesive account of your research. To achieve this you arenot going to be able to present all your hard-won data. Youare going to have to be very selective and in most instancesyou will have to restrict yourself to one aspect of yourwork.

Your first step should be to write down in one sentence themain purpose of your paper. In other words, what is themain message you wish to get across? Having done this youshould identify the three or four pieces of evidence you willuse to give support to your views. You should keep in mindthat you will only have two or three minutes to describeeach piece of work so that when you are assembling yourdata you must be aware of the need to simplify the resultsinto a more easily digested form (e.g. complex tablesreconstructed into histograms).

The arrangement

The first task is to get the basic plan worked out. Thepresentation will fall into several components.

IntroductionStatement of the purpose of your researchDescription of methods and resultsConclusions

The introduction: this is a vital component of your talk. Itmust set the context of your work for the audience, many ofwhom may not be experts in your field. They may also be

57

suffering the after effects of the previous paper or of a dashfrom another concurrent session venue. You have no morethan two minutes to excite the interest of the audiencebefore they relapse into the mental torpor so prevalent atscientific meetings. You must therefore give a consider-able amount of thought to the introduction. It must besimple, precise and free from jargon. It must start from abroad base so that the audience can identify the point atwhich your research fits into the scheme of things andmake them appreciate the vital importance of your owncontribution.

The statement of purpose: this should take no more than aminute but it is also a vital component of the talk. In thesefew sentences you will have to convince the audience thatwhat you set out to do was worthwhile. It should flow fromthe introduction so that it sounds like a logical outcome ofprevious research.

The description of methods and results: the description ofmethods will usually have to be abbreviated or evenreduced to a mention (‘The so-and-so technique was usedto . . . ’ ) . If the development of a new method is an importantpart of your work then obviously it must be described inmore detail but you must decide whether the mainmessage is to relate to the method or the resultssubsequently obtained.

The results are usually the most important part of thepaper. You will inevitably have spent a lot of time gettingthem together. It is possible that you have alreadyprepared a variety of tables, graphs and charts for thepurpose of publication. Do not fall into the trap of thinkingthat these are suitable for presentation to a live audience.How often have you, for example, sat in a meeting wheresomeone has projected slides of an incomprehensible andillegible table taken straight from a journal!

The conclusions: these must flow naturally from the results.You will be aiming to make one or two clear statementswhich you are able to conclude from your work. It isadvisable to be reasonably modest in your claims.

58

The presentation aids

The second task is to prepare the visual aids. In mostinstances these will be slides, overhead transparencies ora computer-based presentation. Considerable thoughtmust be given to these as their impact and quality maymake or break the presentation. They must complementyour oral presentation, not duplicate it. The technicalaspects of the preparation of visual materials are coveredin greater detail in Chapter 9 but a few specific points areworth mentioning at this time.

Having roughed out the plan of the talk it should bereasonably obvious where a slide or overhead transpar-ency is required. You may need one or two during theintroduction, such as a clinical picture of a patient or anillustration of a previous piece of research. You may nothave time to say much about the method but an illustrationof the technique may be pertinent. If so, simple linediagrams are usually more valuable than photographs.

The display of the results provide you with the greatestchallenge. It is during this part of your paper that the visualmaterial will often be of more importance than the verbalexplanation (‘A picture says a thousand words’). Avoidcomplex tables and where possible convert tables to chartsor simple graphs. Rarely is it appropriate to show massesof individual data, just show the mean or rounded offfigures. If you feel you really must refer to complex data itis better to have this prepared in printed form anddistributed to the audience.

Having prepared the illustrations, check that they areaccurate and legible. As a rough rule, a slide where theinformation can be read with the naked eye will besatisfactory when projected. Lettering on an overheadtransparency must be no less than 5 mm high and for acomputer-based presentation a minimum of 32 points issuggested. Then take them to a large lecture theatre andproject them. Check that they are indeed legible from thefurthermost corners of the theatre. It is also helpful to take acolleague with you to check that the message is clear andthat there are no spelling mistakes.

59

Polishing, writing it out and rehearsal

At this stage you should have a good idea of what youintend to say and the visual aids that you require. It is nowadvisable to write the text of the talk in full. Do not write inthe style you use for journal publications. Pretend you aretalking to an individual and write in a conversational mode,avoiding jargon wherever possible.

As you go along identify the correct position for the visualaids. You may find places where you have not prepared anappropriate illustration. This should be rectified even if it isonly to consist of a couple of key words. Remember,during the talk the visual aids must complement your talkand not distract from it. There must always be an accuratematch between the content of your aid and what you aresaying. If you do not have an aid to illustrate what you aresaying, insert a blank rather than leaving on an irrelevantillustration.

Once you have the rough draft, edit it. Then read it aloud atabout the pace you think you will go during thepresentation. Further editing and alterations will berequired, as almost certainly you will have gone overtime. Some find it a useful ploy at this stage to record thetalk on a tape recorder and listen to the result verycritically.

The next stage is to present the paper to an honest andcritical colleague. The feedback is often extremelyvaluable. At this state you will have finalised the text andaids.

You must now make the decision whether you will read thepaper or not. Most authorities consider that you should bewell enough rehearsed to speak only with the aid of cuecards or the cues provided by your visual aids. If you havea highly visual presentation most of the audience will belooking at the screen so the fact that you are reading is lesscritical. Providing the text is written in conversational style,and you are able to look up from the text at frequentintervals, then reading is not a major sin. The chief risk ofspeaking without a text in a very short presentation isgoing over time which at best will irritate the chair and theaudience, and at worst will result in your being cut off inmid-sentence.

60

Whatever you decide, rehearsal is essential and a dressrehearsal in front of an audience (e.g. the department) aweek or two before the event is invaluable. Not only willyou receive comments on the presentation but you will alsobe subject to questions, the answering of which, in aprecise manner, is just as important as the talk itself.

PREPARING THE ABSTRACT AND YOURCONTRIBUTION TO THE PROCEEDINGS OF THECONFERENCE

The abstract

Most conferences will require you to prepare an abstract,sometimes several months before the meeting. It mayinitially be used to help select contributions and willultimately be made available to participants. Contributorsare often tardy in preparing their abstracts which isdiscourteous to the conference organisers and makes theirtask more difficult.

The abstract should be an advertisement for your paper. Itshould outline the background to the study and summarisethe supporting data and the main conclusions. Quitefrequently abstracts promise what they do not deliver soavoid becoming guilty of false advertising.

The proceedings

Many national and most major international conferenceswill publish proceedings. Should you be presenting apaper as such a conference you will be required toprovide your contribution to these proceedings during theconference or shortly afterwards. It is not appropriate topresent the organisers with the script and slides that youhave just used in your presentation. The contribution to theproceedings should be written in a style consistent withthat used in a journal article. The content should be thesame as in the presented paper but not necessarilyidentical. It is perfectly permissible to expand some areas,particularly with regard to the methods and resultssections, where more details could be included. Thisshould, of course, all be done within the guidelines forformat and length specified by the organisers.

61

WHAT YOU SHOULD DO ON THE DAY

‘There’s many a slip twixt cup and lip’. This sayingprovides a reminder that, however good your preparationfor the presentation of the paper has been, there is stillplenty that can happen to ruin your carefully laid plans.Fortunately, many such problems can be prevented oranticipated. You should find it useful to work your waythrough the checklist in Figure 4.1.

Handling questions

Most conferences have a fixed period of time for questions.In some ways this is the most critical part of thepresentation. Some people in the audience are going totest you out with penetrating questions and how you handlethem will enhance or detract from the impact of yourperformance. This is one of the reasons why we suggesteda full dress rehearsal in front of your department in orderto practice your answering of difficult questions for whichsome participants will be eagerly searching. The followingare some points to remember:

Listen to the question very carefully.If the question is complex or if you suspect that not allthe audience heard it, restate it clearly and succinctly.Answer the question politely and precisely. Sometimesa simple ‘yes’ or ‘no’ will be sufficient. Avoid thedanger of using the question to give what amounts to asecond paper.Be alert to questioners who are deliberately trying totrick you or to use the occasion to display their ownknowledge of the subject.If the question is particularly awkward or aggressivetry to deflect it as best you can. Strategies includeagreeing with as much of what was said as possible,acknowledging legitimate differences of opinion orinterpretation, or suggesting you meet the questionerafterwards to clarify your position. At all costs avoid aheated head-on clash in front of your audience.However, do not be afraid to politely disagree withany questioners, however eminent, when you are sureof your ground. Remember, they may only be testingyou out.

62

63

PREPARING A CONFERENCE POSTER

The conference poster is an increasingly popular alter-native to presenting papers at conferences. You will findthat the poster has several advantages over the traditionalpaper such as:

Allowing readers to consider material at their own rate.Being available for viewing over an extended period oftime.Enabling participants to engage in more detaileddiscussion with the presenter than is the case with theusually rushed paper discussion session.

If the conference organisers have arranged a postersession we suggest that you consider taking advantage ofit. It may provide you with an opportunity to presentadditional material to the conference that would otherwisebe difficult because of limitations on the number ofspeakers.

What is a conference poster?

A conference poster is a means of presenting informationfrom a static display. A poster should include a least thefollowing parts:

A titleAn abstractText and diagramsName of author(s), their address(es) and where theymay be contacted during the conference.

Additional material that you might consider for the poster,or in support of the poster, includes:

Illustrations and photographsExhibits and objectsAudio-visual displays, such as a videoA take-away handout, which might be a printedreduction of your poster.A blank pad, so that when you are not in attendanceinterested readers can leave comments or contactaddresses for follow-up.

64

Preparing the poster

If you decide that a poster is an appropriate way ofpresenting your information, there are a number of thingsyou must take into consideration during its presentation.

Firstly, ascertain from the conference organisers thefacilities and size of space that will be available. Thenproceed to plan the poster. The poster should commu-nicate your message as simply as possible, so do not allowit to become clogged with too much detail. Layout ideascan be gleaned by looking through newspapers andmagazines or, better still, from graphic design books andjournals. If possible, discuss these ideas with a graphicartist. The layout should be clear, logical and suitable forthe material being presented. Try a number of differentrough layouts first and seek the opinion of a colleague todetermine the best. A possible layout is shown in Figure4.2.

Plan to mount components onto panels of coloured card cutto sizes convenient to transport. An alternative is to get thewhole poster photographically enlarged to full size. It canbe carried rolled up in a cardboard or plastic tube.

Text should be large enough to be read at the viewingdistance, which is likely to be about one metre. In this case,we suggest that the smallest letters be at least 5 mm highand preferably larger. Good quality titles and text can beproduced with a word-processor and a high-resolutionprinter. These can be enlarged, as required, photo-

65

graphically or on a photocopier. Consider using colour tohighlight specific points.

Break up the density of text into several discrete parts. Forexample, consider dividing the text into an abstract, anintroduction, a statement of method, results and conclusioneach with its own clear heading. A short list of referencesor of publications arising out of your work might also beappropriate.

Remember, that in preparing your poster you are reallytrying to achieve many of the same things you would wishto achieve with a talk or lecture: attract interest andgenerally communicate effectively.

CHAIRING A CONFERENCE SESSION

Much of the success of a conference will depend on thequality of the chairing of individual sessions. Should thistask fall to you, there are many responsibilities to fulfil.There are three categories of tasks – responsibilities to theorganisers, to the speaker and to the audience.

Responsibilities to the organisers

The organisers of the conference will have approachedyou several months before the event. If you are lucky, theywill also have given you detailed guidelines to follow but ifnot you must, at a minimum, find out:

The time and length of the session.The number and names of the speakers.A copy of the instructions given to speakers withparticular reference to the time allocated for thepresentation and the time allocated for discussion.Whether there are concurrent sessions.

Ideally you will contact the speakers in advance of theconference to ensure they have indeed received instruc-tions and understand their implication, particularly withregard to time. You may find that some are inexperiencedand nervous about the prospect of their presentation andyour advice will be appreciated. Referring the speaker tothe earliest parts of this chapter might be valuable.

66

If early contact has not been made, it is essential to meetwith speakers prior to the session. You must clarify theformat of the session and reinforce your intention to stickrigidly to the allocated time. You should explain themethod to be used to indicate when there is one minuteto go, when time is up and what steps you will take shouldthe speaker continue for longer. This may sound draconianbut, believe us, it is vital. A timing device on the lectern isan invaluable aid to compliance.

Prior to the session, you must also familiarise yourself withthe layout of the venue, the audio-visual facilities and thelighting. In the absence of a technician, you may be calledon to operate the equipment and lighting or to instruct thespeakers in their use,

At the start of the session, announce that you intend to keepto time and do so.

Finally, you must be certain that the session and individualpresentations commence and finish at the programmedtime. This is particularly important when there areconcurrent sessions.

Responsibilities to the speakers

Speakers invariably fall into one of three types.

The well-organised speakers: they will tell you exactlywhat they are going to do and what they require. If you askthem how long they are going to speak, they will tell you inminutes and seconds! You will need to have little concernfor these speakers, but they will expect you be as wellprepared and organised as themselves.

The apprehensive speakers: they will generally be theyounger and less experienced. They will often have a well-prepared paper to present but are in danger of not doingthemselves justice. You can assist such speakers greatly byfamiliarising them with the facilities before the session andencouraging them to practise operating the audio-visualequipment. If this seems beyond them, you may be able totake on this task yourself. Calm reassurance that all will bewell is the message to convey.

67

The confident under-prepared speakers; these areremarkably prevalent and the most dangerous for thechairperson. They will not seek advice and will denyhaving received detailed instructions about their presenta-tion. When you ask them how long they expect to speak,you will receive an off-hand response. They will tell youthat they have not rehearsed their presentation and willalmost certainly go over time. There is little you can do tohelp such people because they are certain they haveeverything under control. However, they can be yourdownfall unless you prepare to intervene. Prior to thesession you must convince them you are serious aboutcutting them off if they speak over time. Unfortunately thisstrategy will often fail and you must be prepared to actimmediately the first time a speaker goes over the allottedtime. After a maximum of 15 seconds grace, rise from yourchair. If the hint is not immediately taken, you have nooption but to politely but firmly stop the speaker. Examplescan be cited of such speakers being physically led off thestage still talking, but such extremes should not arise.Fortunately, you will only have to intervene in such a wayonce in a session and, should it happen, your future as aninvited chairperson is assured.

In the discussion period you must see fair play. Ensure thatquestions are relevant and brief. Do not allow a questionerto make long statements or commence a mini-presentationof their own work. Suggest any significant differences ofopinion be explored informally at the subsequent coffeebreak.

Responsibilities to the audience

The audience has a right to expect several things. Theymust be able to hear the speaker and see the slides. Theymust be reassured that you will keep the speakers to timeto protect their opportunity to ask questions and to allowthem to move to any concurrent sessions. Speakers goingover time is the commonest complaint of participants andthe chair is usually held to blame. During the questionperiod you should ensure that the time is not monopolisedby the intellectual heavies in the front rows. On the otherhand you must also be prepared to ask the first question ifnone is immediately forthcoming from the audience.

68

Finishing off

At the close of the session, thank the speakers and theaudience and remind them of the starting time of the nextsession. You may also have been asked to transmitinformation from the organisers. Particularly importantwould be to obtain completed evaluation forms for thesession if these had been provided.

GUIDED READING

The book we can still recommend for further reading is J.Calnan and A. Barabas’ Speaking at Medical Meetings,Heinemann, London, 1981. This pocket-sized do-it-yourselfguide is not only valuable but also entertaining. It containsmany useful illustrations and good advice about thepreparation of visual aids.

Another manual is P. Race’s Conference Presentations andWorkshops, University of Northumbria, 1986 (availablethrough Amazon.co.uk).

Those interested in the organisation and evaluation ofmedical meetings are referred to a series entitledImproving Medical Meetings, written by D. Richmondand his colleagues, published in the British Medical Journal(1983), 287, pp. 1201-2; 1286-7; 1363-4; 1450-1.

For help with the design of charts and graphs check thereferences at the end of Chapter 9.

An excellent general text on public speaking is C. Turk’sEffective Speaking, Spon, London, 1985. This is acomprehensive reference work that has been written bya university lecturer.

Another comprehensive resource is S. Lucas’ The Art ofPublic Speaking, McGraw Hill, 2000.

69


5: TEACHING PRACTICAL ANDCLINICAL SKILLS

INTRODUCTION

In the first part of this chapter we plan to look at ways ofimproving your clinical teaching. Later in the chapter wewill address the issue of practical and laboratory basedteaching.

While it is increasingly likely that your institution willprovide some form of ‘teach the teachers’ course it isrelatively unlikely that it will specifically address clinicalteaching. It is a fact that clinical teaching is the mostneglected area of all teaching despite being the one wheremore deficiencies have been found than in any other. Theconclusion of one extensive study was that ‘many (clinical)teaching sessions, particularly ward rounds, were hap-hazard, mediocre and lacking in intellectual excitement’. Inone study of medical schools in North America, it wasstated that there were few students who could reporthaving been monitored in the interview and physicalexamination of more than one or two patients and that asurprising number had been awarded their degreewithout ever having been properly supervised in thecomplete data-collecting process of even one patient! It isour experience, with notable exceptions, that a similarsituation can be found in many medical schools in otherparts of the world.

THE ATTRIBUTES OF AN EFFECTIVE CLINICALTEACHER

These have been identified on the basis of the opinions ofexperts, the perceptions of students and from theobservations of actual clinical teaching. Considering thelimited nature of the research there is a remarkableconsistency in the results. It might be helpful to start bychecking yourself against these attributes.

Do you encourage active participation by the studentsand avoid having them stand around in an observa-tional capacity?Do you have and demonstrate a positive attitude toyour teaching?Is the emphasis of your teaching on applied problemsolving?Do you focus on the integration of clinical medicinewith the basic and clinical sciences or do you spend

72

most of the time on didactic teaching of factualmaterial?Do you closely supervise the students as they inter-view and examine patients at the bedside and provideeffective feedback on their performance or do you relyon their verbal case presentations in the teachingroom?Do you provide adequate opportunities for yourstudents to practice their skills?Do you provide a good role model, particularly in thearea of interpersonal relationships with your patients?Does your teaching provide stimulation and chal-lenge?Is your teaching generally patient-orientated or does ittend to be disease-orientated?Are you friendly, helpful and available to yourstudents?

Should your honest answer to some of these questions be‘no’ then you are probably a typical clinical teacher asmany studies have shown that all of these characteristicsare rarely present. Just becoming aware of such attributesshould encourage you to be more critical of yourapproach. The remainder of this chapter will deal morespecifically with the planning and the techniques which canbe introduced to enhance the effectiveness of your clinicalteaching.

IMPROVING CLINICAL TEACHING

If you are a clinical teacher with no responsibilities for theplanning of the curriculum, there may be few educationalinitiatives open to you other than to improve your hospital-based or community-based teaching. What you should aimto do is to try and acquire as many as possible of theattributes described in the previous section. There are nohard and fast rules as to how you can achieve this aim butthe following points may be helpful.

Plan the teaching: it is possible that you will have receivedhighly specific instructions from the medical schoolparticularly if you are teaching in a structured programme(see later). If not, it is worthwhile contacting the departmenthead or the course co-ordinator to see if there are definedobjectives for the part of the curriculum in which yourteaching is placed.

73

If these are not forthcoming, you should draw up your ownobjectives, at least to the extent of writing down what youhope to achieve during the students’ attachment. In doingso you must taken into account your time, the duration ofthe students’ attachment, the number of students and theseniority of the students. You must be realistic about whatyou can achieve and not attempt to cram too much into yoursessions. You should inform the students about the planwhen they start and listen to any comments they may makewhich might reasonably give you cause to modify the plan.Though clinical teaching is essentially opportunistic, beingdependent on the availability of patients, it is wise to keep arecord of the conditions seen during your teaching so thatby the end of the course you have covered a wide enoughrange of illustrative cases. You should, of course, co-ordinate your teaching with other tutors who are involvedwith your group of students.

Set a good example: it is surprising how infrequentlystudents get the chance to watch an experienced cliniciantake a history, perform an examination and subsequentlydiscuss the outcome and plans with the patient. It isnormally impossible to do this on a working hospital wardround because pressure of time means that decision-making is given priority. However, the outpatient depart-ment or a community practice setting often provide betteropportunities.

One of the difficulties students might have under suchcircumstances is the contrast between what you do inpractice and what you expect of the students. This issuemay have to be discussed. The important thing is for thestudents to see you in action, particularly in regard to theway you relate to the patients while at the same timeachieve the medical aims of the encounter. Even in busyclinical situations it is important to demonstrate a concernfor the patient’s feelings.

Involve the student: the need for active participation is therecurrent theme throughout this book and nowhere is itmore important than in the clinical teaching situation. Youshould take every opportunity to ask students to perform.This may range from talking to a patient, checking physicalsigns, presenting the case history, answering questionsand looking up clinical information for presentation at the

74

next teaching session. In general try and make sure all thetasks are directly related to the patients the student hasseen. The emphasis of the teaching should be ‘bedside notbackside’.

Observe the student: as mentioned earlier, a consistentfinding in studies of clinical teaching has been a lack ofdirect observation of student interactions with patients. Alltoo often the clinical teacher starts with the casepresentation and many never check to see whether thefeatures described are actually present or were elicitedpersonally by the student. Serious deficiencies in clinicalskills are consistently found in interns and residents whichmust be an indictment of the undergraduate clinicalteaching. Only a commitment to the somewhat boring taskof observing the student take the history, perform thephysical examination and explain things to the patient willallow you to identify and correct any deficiencies. This typeof activity is particularly essential with junior students andmust be conducted in a sympathetic and supportive way.

Provide a good teaching environment: the more senior andprestigious you are, the more intimidating you are likely toappear to the students. It is vital that you adopt a friendlyand helpful manner and reduce the natural and inevitableapprehension felt by your students. Not only may they beapprehensive about you, but they will also be apprehen-sive about their impending contact with patients. You canassist this by preparing the patients and by showing to thestudents you understand their fears. Such fears are likely tobe particularly evident with junior students.

IMPROVING THE CLINICAL TUTORIAL

Clinical tutorials are all too often didactic with the emphasisbeing on a disease rather than on the solving of patientproblems. We firmly believe the clinical teacher shouldconcentrate on the latter. The students will inevitably havemany other opportunities to acquire factual information butrelatively little time to grapple with the more difficult task oflearning to apply their knowledge to patient problems. It issad, but true, that in traditional medical schools the studentsare often as much to blame as their teachers byencouraging didactic presentations, particularly whenexaminations are imminent. Surprisingly, clinical teaching

75

in problem-based schools often exhibits the same char-acteristics.

Plan the teaching: once again it is important to establish theaims of the sessions you have been allocated. There maybe fixed topics to cover or you may have a free hand. Ineither case you must be sure in your own mind what youintend to achieve in each session.

Involve the student: make it clear from the beginning thatyou expect most of the talking to be done by the studentsand that all of them are to participate, not just the vocalminority. At the first session, explain what tasks you expectthem to perform in preparation for each tutorial. You may,for example, expect them to prepare cases for discussionor to read up aspects of the literature on a particularsubject.

Provide a good teaching environment: the way in whichyou set up the session is vital for its success, particularlywhen you wish to encourage active participation. Your roleas a facilitator, not the fount of all knowledge, must beemphasised and you must resist the temptation to intervenewith extra information all the time. This is very hard toavoid but if it happens too frequently you will soon find allconversation is channelled in your direction and there willbe no interaction between the students.

As the clinical tutorial is another form of small groupteaching you should read Chapter 4 for further advice.

Concentrate on clinical problem solving: in the last thirtyyears there has been a substantial research effort into howdoctors and students go about solving clinical problems.The findings have major implications for the clinicalteacher, though as yet there is little evidence that this hasbeen widely recognised. The traditional way of teachingstudents is to require them to take a full history, perform acomprehensive examination and only then come up with adifferential diagnosis. The implication has been that clinicalexamination is a routine and sequential process withserious thinking about diagnosis and management beingdeferred until the student is away from the bedside. This isnot the way doctors or students actually operate eventhough they may appear to do so on superficialobservation.

76

Research has shown that experienced clinicians willgenerate diagnostic hypotheses within a minute or so offirst seeing the patient. The bulk of time spent interviewingand examining the patient will then be used to confirm orrefute these hypotheses. This approach to problem solvingis a natural ability and does not have to be taught in its ownright. However, successful clinical problem solving isdependent on previous experience with similar problemsand on effective utilisation of the person's medicalknowledge base relevant to the problem. These findingsprovide one of the arguments for why increasing numbersof medical schools are using problem-based learning asthe keystone of the curriculum. Patient problems are usedto trigger the search for factual information rather thanteaching factual information before exposing students topatient problems (see Chapter 7).

How to teach clinical problem solving: from what we havesaid the aim must be to provide your students with as muchexperience as possible in manipulating their factualknowledge in relation to patient problems. You shouldavoid conducting tutorials in which you or your studentssimply present topics. If, for example, you wish to have atutorial dealing with hypertension then a patient withhypertension should be the focus. The student will then berequired to consider the implications of hypertension inrelation to that particular patient. Figure 5.1 briefly shows aplan for the simplest type of problem solving tutorial.

77

This general approach is used not just for data gatheringbut also for the ordering of investigations (what tests wouldyou order and why?) and treatment (what treatment wouldyou give and why?). Though this may sound ratherstructured and formal, in practice this will not be so. Youwill soon learn to judge the pace, learn how much newinformation is to be given before stopping and so on.However, you may initially find sessions of this type hardgoing if the students are not used to the challenge of thismethod of teaching. Those previously relying on theregurgitation of lists and pages from the books may beparticularly discomforted. They may attempt to avoidanswering or justifying their suggestions but persistencewill pay off.

With sessions of this type it is important to create a non-threatening atmosphere. One way of helping is toparticipate yourself. Let the presenting students bringalong a case or patient whom you do not know. Stillencourage the students to answer first but you can then addyour own thoughts. You may even find this morethreatening than the students but it is important they learnthat infallibility is not an attribute of clinical teachers andthat it is quite normal for even the most experiencedclinician to have to admit indecision and a need to obtainadvice or further information.

ALTERNATIVES TO TRADITIONAL CLINICALTEACHING

We have already provided evidence that traditionalclinical teaching is often inadequate in meeting the aimsof both the medical school and the students. This has ledmany schools to introduce structured courses to teachbasic clinical skills in a less haphazard manner. The skillstaught are often not restricted to interviewing and physicalexamination but include technical skills and clinicalproblem solving. Should you have the opportunity tointroduce or participate in such an approach then the firststep must be to define the objectives of the exercise. Thesemust take into account the seniority of the students, the timeallocated in the curriculum, the facilities, and theavailability of teachers and other resources. There areobviously many ways in which this could be done but wewill restrict ourselves to outlining such a programme

78

which has been run successfully for many years jointly by aDepartment of Medicine and a Department of Surgery. It isone which has retained the support of both staff andstudents.

The course plan for the programme is shown in Figure 5.2.In the left-land column you can see the objectives. Oppositeeach are the teaching activities which are planned to helpthe student achieve the objectives. In the right-handcolumn are the assessment procedures which are alsomatched to the objectives. You can see that a range ofmethods is used. The key to the programme is theattachment of only three students to a preceptor forinstruction on history taking and physical examination.You will note that the problem-orientated medical recordapproach has been adopted and we find this has been avaluable adjunct to our teaching. Whole-group problem-solving sessions include clinical decision making, emer-gency care, data interpretation (clinical chemistry, haema-tology and imaging) and management/therapeutics. Thestudents are also expected to take responsibility for a lot oftheir own learning.

79

As it has been decided that the time of the teaching staffshould be devoted to preceptoring and conducting smallgroup sessions, the revision of the theoretical aspects ofmedicine and surgery is left entirely to the students.Various self-learning materials are also available for thestudents to use in their own time.

Though such a programme is far from perfect, it wasintroduced within a traditional curriculum and with theminimum of resources. The main change was a reallocationof staff time away from didactic activities and into moredirect observation of student performance.

A perusal of the medical educational literature will provideyou with other examples of structured clinical teaching.Increasingly you will find descriptions of the use of clinicalskills laboratories where medical schools have set up fullystaffed and equipped areas devoted to putting groups ofstudents through an intensive training in clinical skills, oftenusing a wide range of simulations. You will also find manyexamples of training students in interpersonal andcommunication skills using simulated patients. All havethe same general approach: to undertake the training ofvarious clinical skills in a structured and supervised way toensure that all students achieve a basic level ofcompetence.

TECHNIQUES FOR TEACHING PARTICULARPRACTICAL AND CLINICAL SKILLS

Many practical and clinical skills can be taught as separateelements. Because there is a wide range of these elements,and as clinical teaching is generally opportunistic, manymedical schools have established programmes to teachbasic skills in a piecemeal fashion. This is normally doneearly in the students’ career, often just prior to their firstclinical attachments. Much of this has come about becauseof new technology (e.g. video recording, computers) andbecause of an awareness of the value of simulation in itsmany forms. This section will introduce you to a variety ofways of teaching basic skills some of which may not be ofimmediate relevance but some of which ought to be inoperation in your medical school because of their provenefficacy.

80

The clinical interview

There are two basic methods with which you should befamiliar; the use of video recordings and the use ofsimulation.

Video recording: any department which has the respon-sibility for teaching aspects of history taking or inter-personal skills should have access to video recordingequipment, preferably of the portable kind that can be setup in ward side rooms, outpatients and other teachingsituations. You should become familiar with the technicaloperation of the equipment.

There are several ways in which video equipment can beused. The simplest is to record examples of interviewingtechniques (good and bad) for demonstration purposes.This may be valuable as a starting point for new students.You may also wish to have an example of a basic generalhistory so the novice student can get an idea of thequestions that are routinely asked. Some medical schoolshave recorded segments of interviews with patients whichshow various emotional reactions (e.g. aggressive beha-viour, grief, manipulation).

The most powerful way of using the video is to record thestudent’s interview with a patient remembering thatinformed consent is essential. This may be initially stressfulbut both student and patient usually forget they are beingrecorded after a few minutes. A time limit of 20-30 minutesis advisable. The student, or a small group of students,meet later with the teacher to review the tape. It is at thisstage that some skill by the teacher is required. Firstly, thesituation must be a supportive one to allow frank and opendiscussion. Secondly, the teacher must have a clear idea ofwhat the students should be learning. (If you have not readformally in the area of the clinical interview, we wouldrecommend the book by Cole and Bird.)

It is not appropriate to play the tape completely throughand then have a discussion. The tape must be stoppedfrequently to discuss points as they arise. Such things asnon-verbal cues, aspects of doctor-patient relationships,avoidance of jargon, adequacy of questions, direction ofthe enquiry, directive versus non-directive questioning,hypothesis generation and many other issues can be

81

identified and discussed, both with the interviewingstudent and with the student’s peers. Such sessions arevaluable both with junior and senior students.

There are clearly several advantages of this approach overdirect observation: the teacher is not committed to bepresent at the actual interview; the teaching can bescheduled at a convenient time; a recorded interview canbe interrupted as often as necessary; and most importantlythe students can review their own performance. The latterby itself often produces a striking impact and a rapidimprovement in competence.

Simulation: the use of simulated (or standardised) patientsis another well-proven and powerful method for teachinginterview skills. However, if does require some expertiseto train the simulated patients and if you wish to pursue thistechnique we strongly recommend that you read the bookby Barrows. The simulated patient offers certain distinctadvantages over the real patient particularly for the novicestudent. The analogy has been drawn with the value offlight deck simulators in pilot training. The advantagesinclude: the ability to schedule interviews at a convenienttime and place; all students can be faced with the samesituation; the interview can be interrupted and anyproblems discussed freely in front of the ‘patient’; thereis no risk of offending or harming the patient (often aconcern of new students); the student can take as much timeas necessary; the same ‘patient’ can be re-interviewed at alater date; and the simulator can be trained to providedirect feedback, particularly in the area of doctor-patientrelationships.

Simulations can also be developed for situations that areusually impossible for students to experience with realpatients. This is particularly so for emotionally chargedareas. With all these advantages it is surprising how long ithas taken for this technique to become widely used. Muchis due to uninformed prejudice. Some is due to a lack ofappreciation as to how easy it is to train simulated patients.This is not an appropriate place to deal with this aspect indepth. However, it is important to remember that it is notessential to use actors. In fact our preference is to avoidactors, who may wish to give a performance rather than bea ‘patient’. The basic technique is to identify a patient with

82

the condition you wish to simulate. A protocol is drawn upof the facts which are relevant to that condition. All otheraspects of this history not relevant to the condition need notbe simulated, i.e. the simulator’s own background can beused or adapted. The simulator then learns the role,including non-verbal cues. The trainer then tests thesimulation by taking the history. Modifications are madeand further practice sessions conducted. It is then ourpractice to try out the simulation, for example, by askingone of the resident staff to conduct an interview. This isobserved and a discussion held with the resident about thepatient. It is readily apparent whether the resident suspectsthe simulator of not being real. It is, in fact, vary rare for awell-trained simulator to be suspected, which is a goodconfidence booster for both the simulator and trainer.

The physical examination

Video recording: video recordings can be used to helpstudents develop physical examination skills in much thesame way as for interviewing skills. However, it is not quiteso effective and direct observation and instruction aregenerally more appropriate. It is very valuable to havevideos which illustrate the recommended way of perform-ing specific components of the examination. These must bereadily available to students together with convenientplayback facilities.

Simulation devices: this is the area where technology hasmade a major contribution. The number of these devices isincreasing rapidly and includes those for cardiac ausculta-tion, breast examination, prostate palpatation, pelvicexamination and laryngeal examination. They are often tobe found in skills laboratories.

Simulated patients: these can also be used in very much thesame way as for history taking, with similar advantages.However, the range of signs that can be simulated islimited, though not as much as you might think. Forexample, Barrows in his classic book showed that a widerange of neurological conditions can be simulated.

In regard to other systems, conditions where pain is themain feature are particularly suitable. Also valuable aresimulations of emergency situations such as perforation of a

83

viscus, myocardial infarction and subarachnoid haemorr-hage. Students can be trained to perform rapid assessmentand acute management for conditions that otherwise wouldbe impossible to programme.

Simulated patient instructors are widely used, particularlyin North America, to teach the pelvic examination. Themain advantage of such an approach is that the instructorgives direct feedback on the students’ performance. Theinstructor is the only one who can say if the student hascorrectly palpated the ovaries and uterus or whetherunnecessary discomfort has been created. The sameapproach has been used to teach rectal examinations inthe male.

Though not simulation, it is perhaps worthwhile remem-bering that self and peer examination is a valuableteaching technique for novice students. The learning ofmany basic manual skills is much better done in this waythan using real patients.

The use of instruments and basic equipment

The student has to become competent with a variety ofinstruments and basic medical equipment other than thestethoscope. These include the opthalmoscope, theauroscope, the proctoscope, the laryngoscope, syringes,infusion apparatus, and endotracheal tubes. For many ofthese, simulation devices are available. In most cases it isinappropriate for the first attempt to use them to be madeon a patient, Students certainly appreciate the chance topractice their skills in a situation where they are not goingto hurt a patient.

Some techniques, of course, are readily practiced on eachother (e.g. examinations of the eyes, ears and throat).

TEACHING PRACTICAL AND LABORATORYCLASSES

Practical and laboratory classes are often regarded as anessential component of science-based courses. However,they are becoming less prevalent in the medicalcurriculum as the pressures to free up time for othercomponents in the course and decreasing resources force

84

a re-appraisal of their value. We have decided not to givedetailed consideration to these important areas of teachingin this book but to refer those with responsibilities of thistype to more specific resources. These are listed at the endof this chapter and include the non-medical version of ourbook entitled “A Handbook for Teachers in Universitiesand Colleges”. However, there is one particular type ofpractical teaching that we will mention, that is the researchproject, which is equally as likely to be managed by clinicalteachers as by basic science teachers.

Research projects

Research projects have always held an important place inscience-based courses. There is now a growing apprecia-tion of their value, albeit in a more limited and lesstechnically demanding form, at various phases of themedical curriculum. Projects should provide the studentwith a real-life experience of research that is quite differentto the more controlled laboratory exercises. They arestrongly motivational to most students because of the highlevel of active participation, the close contact with super-visors and research staff, the lower emphasis on assess-ment and the greater degree of personal responsibility.

Research projects can be undertaken individually, bygroups, or by attachment to a research team in which thestudent accepts responsibility for certain aspects of anestablished project. Whatever the approach, the role of thesupervisor is critical. Should this be your role, the pointsoutlined in Figure 5.3 should be remembered.

85

Supervisors vary markedly in their commitment and skill. Itis recommended that departments using research projectsshould provide guidelines and training for supervisors.Most institutions now run courses and provide suitablehandbooks. Several valuable publications are availablethat could be used or modified for this purpose (seeGuided Reading at the end of this chapter).

Alternative methods

A combination of modern technology and interest in newteaching techniques has provided alternatives to theconventional approaches to practical work. Two that arenow well established are computer-based methods andsimulations.

Computer-based methods

The wide availability of computers and their everyday usein all areas of science and technology makes theirintegration into practical courses almost essential. Earlyin the curriculum, there may be a need to have practicalclasses to teach students how to use computers.

In the laboratory setting, computers may be interfacedwithother equipment or used to enhance or even replaceconventional practical work.

Learning the skills of data analysis and interpretation isoften linked to laboratory teaching. With the computer, it ispossible to provide students with a wide range of data setson which they can practice these skills at varying levels ofcomplexity and sophistication.

Simulations

Simulations are playing an increasingly important role inpractical teaching for many reasons. Some reflect a need toprovide repetitive practice of basic and advanced skillswhich might not otherwise be readily available, be toocomplex (e.g. acquiring of experimental data) or toodangerous (e.g. trainee medical staff performing operativeprocedures). The types of simulations also vary widely,ranging from simple drill and practice (e.g. usingequipment; taking blood samples) through to replacing

86

the need for live animal experiments. The computer is, ofcourse, involved in many of the more complex simulations.The increasing sophistication of graphics and virtual realitywill ensure a burgeoning industry in the instructionalapplication of computer-based simulations.

EVALUATING CLINICAL AND PRACTICALTEACHING

There are few well-developed procedures for evaluatingyour performance as a clinical teacher. The major reasonfor this is that much clinical teaching is done on a one-to-one basis or in very small groups over relatively shortperiods of time. This causes a problem in obtaining validand reliable information about performance. However, thisdeficiency is a matter of concern and research is underwayto develop more useful instruments. You can, of course,adapt the principles and procedures described in thechapter on small group teaching and integrate these withthe checklist displayed at the very beginning of thischapter.

You may have gained the impression that we favour theexclusive use of questionnaires in evaluation. We wish topoint out that this is not the case. Questionnaires are onlyone method which seek data from one source - typicallyyour students. In all evaluation, including clinical andpractical teaching, we would wish to encourage you toexplore other methods and other sources of evaluativeinformation which you will encounter throughout this book.

GUIDED READING

Although there are many good books written on how toperform a medical interview and a physical examination,there seems to be a dearth of recent books on clinicalteaching.

The Physician as Teacher by T. Schwenk and N. Whitman,Williams & Wilkins, Baltimore, 1987, is still a useful guide tothe tasks faced by a medical teacher, including clinicalteaching.

A useful recent resource is a series of articles representingthe output of the Ninth Cambridge Conference on Medical

87

Education which appear in Medical Education (2000), 34,No 10. The topic of this conference was Clinical Teachingand its Assessment.

For additional information on practical and laboratoryteaching we recommend the following:

A Handbook for Teaching and Learning in Universities andColleges by R. Cannon and D. Newble, Kogan Page,London, 2000.

Improving Teaching and Learning in Laboratories by E.Hazel and C. Baillie, HERDSA Gold Guide No 4, 1998available from HERDSA, PO Box 51, Jamieson, ACT, 2614,Australia.

Teaching in Laboratories by D. Boud, J. Dunn and E.Hegarty-Hazel, Open University Press, Guildford, 1989.


S. Cole and J. Bird. The Medical Interview. Mosby YearBook, 1999.

H. Barrows. Simulated Patients (Programmed Patients).Charles C. Thomas, Springfield, 1971.

88

6: PLANNING A COURSE

INTRODUCTION

This chapter aims to assist you when you become involvedin some way in curriculum planning and wish to do so in asystematic manner. Unfortunately, there is no straightfor-ward formula to guide you in this activity. The reasons forthis are as follows. First, curriculum planning is a complexbusiness involving more than purely educational con-siderations. For example, you will find that full accountmust be taken of the political and economic context inwhich you teach. Second, relatively few courses are startedfrom scratch. Much curriculum development is a matter ofrevising and adapting existing courses or materials. Andthird, there are important differences between individuals– especially between individuals working in differentdisciplines – in the ways in which they view a variety ofeducational issues. You may, for instance, see your mainfunction as transmitting appropriate knowledge, skills andattitudes. On the other hand you may perceive your role asbeing primarily concerned with the personal and socialdevelopment of your students as well as with theirintellectual development. In a book of this kind it is notpossible to provide a discussion which can fully take intoaccount these various orientations. However, we believethat you should be aware of these differences and wewould encourage you to read further on the matter to helpdevelop your own particular orientation and your ownapproach to curriculum development.

In our view, the key to good curriculum or course design isto forge educationally sound and logical links betweenplanned intentions (expressed as objectives), coursecontent, teaching and learning methods, and the assess-ment of student learning while taking account of studentcharacteristics. In the past, too many courses started withvague intentions, consisted of teaching which had atenuous relationship to these intentions and employedmethods of assessment which bore little or no relationshipto either. Such courses then placed students in theunfortunate situation of playing a guessing game, withtheir academic future as the stake! This pattern can beimproved by adopting an approach which aligns theintentions with course content, teaching, and the assess-ment.

90

WHO SHOULD BE RESPONSIBLE FOR CURRICULUMPLANNING?

Although we assume you have some responsibility forcourse planning, it is unlikely that this will be a solo affair.You will have additional resources on which to draw whichmay include colleagues in your own and relateddepartments, staff of a university teaching unit, membersof your discipline outside your immediate environment,and students. These people may form a planningcommittee or a panel of advisers. Whatever your situation,experience suggests that some form of consultation withothers is very desirable.

COURSE CONTENT

Content is a broad concept meant to include all aspects ofknowledge, skills and attitudes relevant to the course andto the intellectual experiences of students and theirteachers in a course.

While not always easy to achieve, we feel that coursecontent should be made explicit and that this will then putyou in a better position to make informed and coherentdecisions in your planning. There are several differentcriteria for selecting content that may be more or lessrelevant to your work. These criteria are presented belowfor your consideration.

Academic criteria

These criteria focus attention on theoretical, methodologi-cal and value positions. For example:

Content should be a means of enhancing theintellectual development of students, not an end initself.Content that is solely concerned with technical mattershas a limited place in university education; contentmust also involve moral and ethical considerations.Content should contribute to a deep rather than to asurface view of knowledge.

91

Professional criteria

These criteria recognise that courses, such as medicine,must reflect explicit legal and professional requirementsbefore practice is permitted:

Content must provide the kinds of theoretical andpractical experiences required for registration.Content should include attention to professional ethics.

Psychological criteria

These criteria relate to the application of psychologicalprinciples – especially of learning theory – to teaching:

Content should be carefully integrated to avoidfragmentation and consequential loss of opportunitiesfor students to develop ‘deep’ approaches to learning(see Chapter 1).Content selection must provide opportunities toemphasise and to develop higher-level intellectualskills such as reasoning, problem-solving, criticalthinking and creativity.Content should provide opportunities for the devel-opment of attitudes and values.Content should be selected to assist in the develop-ment of students as independent lifelong learners.

Practical criteria

These criteria concern the feasibility of teaching somethingand may relate to resource considerations:

Content may be derived from one or two major textsbecause of a lack of suitable alternative materials.Content should be influenced by the availability ofteaching resources: library materials, information-technology resources, people, patients, physicalenvironment, etc.

Student criteria

These criteria relate to the characteristics of the studentsyou teach. We consider these criteria to be so importantthat a full section is devoted to them. Student criteria mayaffect the choice of content (and ways of teaching andassessing) in a variety of ways:

92

Content may be selected to reflect the background,needs and interests of all students.Content should be matched to the intellectual andmaturity level of students.Content might take account of the diverse lifeexperiences of students.

How you actually go about selecting content will largely bedetermined by the kind of person you are (especially byyour views about the relative importance of your role as ateacher, the role of students and course content), and thenorms and practices in the discipline you teach.

STUDENTS

Taking account of student characteristics, needs, andinterests is the most difficult part of course planning. Thereason for this is that teachers now face increasinglyheterogeneous groups of students and, at the same time,must take account of legislative requirements to addressspecific issues such as occupational health and equalopportunities. It is no longer enough to state that plannersneed to ‘take account of students’ and then to proceed as ifthey did not exist. Experience shows that students canprovide invaluable assistance in course planning byconsulting them formally and informally.

‘Taking account’ of students is partly your responsibilityand partly your institution’s responsibility. Institutionalresponsibilities – which we would encourage you toinfluence positively – might include:

tutorial assistance in the English language, especiallyfor non-native speakers and international students;bridging courses and foundation courses to assist inthe process of adjustment to higher education.

Your responsibilities are no less onerous. In addition toaccommodating the wide range of personalities, learningstyles, social backgrounds, expectations and academicachievement of normal or direct-entry students fromschool you must also be prepared to teach students fromother backgrounds and with ‘different’ characteristics thanyour own. Five examples of current concern which we willbriefly discuss are: women, mature-age students, studentswith a disability, first-year students and internationalstudents.

93

Women students

The role of women and women students in highereducation has received a lot of attention. In medical schoolsin many countries woman are forming an increasingly highproportion of student intakes. However, as the proportionof women in senior clinical and academic positions remainsa minority the propensity for bias remains. In courseplanning you should ensure:

the elimination of sexist language in course materialsand in teaching.compliance with institutional policies in relation toequal opportunities.

Adult (mature-age) students

Most medical schools aim to recruit a proportion of matureage students, often from diverse backgrounds. There isalso a growing trend towards graduate entry medicalschools (outside North America where this has been thenorm for many years).

This group brings a rich diversity of experiences andissues. Older students usually approach higher educationwith a greater intensity of purpose than their youngerpeers because so much more, in terms of sacrifices andambitions, rests on their study and achievements. Theyalso expect staff to be more flexible and adaptive in theirteaching and assessment methods. These students oftenexperience greater anxiety over assessment arrange-ments. Vagueness on your part, or in the course plan,can only contribute to this concern.

Students with a disability

You will encounter students with physical impairments,who have medical, psychiatric or psychological problems,or who have a learning disability. Most universities havepolicies and support arrangements relating to studentswho have disabilities of these kinds, and we urge you tounderstand the resources that are available to help youwhen teaching and assessing such students.

94

First-year students

The teaching of this group is of particular concern becauseof their need to adjust to the learning environment ofhigher education. Some students will belong to a groupwith specific needs (e.g. mature-age or internationalstudents). The sensitive use of small group work (seeChapter 3) can be a means of dealing with some matters,but not all. The selection of content – taking care to inductstudents into the language and peculiarities of the subjectand to the assessment methods – and above all, the clarityof your expectations can all contribute to a smooth andsuccessful transition.

International students

International students, especially those in their first year ofstudies, require special consideration. These considera-tions relate most closely to matters of your personalpreparation for teaching. Two important aspects are yourown level of cultural awareness and the way in which youteach. Cultural awareness can be developed throughtraining programmes, but a more realistic approach forthe busy teacher is to develop out-of-class contact withrelevant overseas student groups and through reading.The usual principles of good teaching apply as much forthis group as for others but particular care should be givento your use of language – especially your speed,pronunciation and use of unnecessarily complex sentenceconstructions.

As you review these considerations for each group ofstudents you will realise that almost all are worthwhileprinciples for planning and teaching all students andshould therefore be taken into account in routine courseplanning.

In summary, we offer the following general suggestion: beaware of your own attitudes and behaviour, be availableand helpful to all students and, particularly be willing tolearn, to adapt and to adjust. A tall order, we know. Butelsewhere in this book you will find suggestions on ways ofdeveloping these qualities. Of particular relevance isChapter 1 on student learning.

95

As we remarked in our introduction, there is nostraightforward formula to guide you in curriculum andcourse planning. Nowhere is this more evident than in theprocess of linking the many content and student considera-tions we have been discussing to the particulars ofpreparing a course plan. We suggest that you generate asimple checklist of content and student matters to be takeninto account during the next step of planning – writingcourse objectives.

AIMS AND OBJECTIVES

The intentions of the course are usually expressed in theform of aims and objectives. Aims are general statementsof intent. Objectives are rather more specific statements ofwhat students should be able to do as a result of a course ofstudy. We are convinced that clear outcome objectives area fundamental tool in course planning because they enablethe rational choice of content and teaching and learningactivities and are important in planning a valid assessment.Objectives provide a guide to teachers and to students, butshould not be so restrictive as to prevent the spontaneitythat is so essential to the higher education of students. Therelationship between objectives, teaching and learningactivities, and assessment is best set out in a course-planning chart such as that seen in Figure 6.1.

Each defined objective is matched with appropriateteaching and learning activities and with a valid form ofassessment. For instance, in the example, you would notexpect the students to learn to be able to ‘take acomprehensive history at the completion of the course’on the basis of lectures, nor would you expect that thiscould be validly assessed by a paper-and-pencil test. Thecourse designer has provided a relevant teaching andlearning activity and a suitable form of assessment,

WRITING OBJECTIVES

Before you start writing objectives it might help to knowwhat they look like. Here are some examples:

Know the basic terminology of the subjectUnderstand the environmental factors which predis-pose people to lung cancer

96

Obtain a problem-orientated history from a patientPerform a venepunctureDemonstrate a willingness to be critically evaluated bypeers.

In each case, the objective contains a statement whichsuggests the kind of behaviour that students will berequired to demonstrate in order to show that the objectivehas been achieved. Now, if you look at each objectiveagain, you will notice that they suggest rather differentkinds of behaviour. The first two objectives requireinformation of an intellectual kind for their achievementand may be classified as knowledge objectives. The thirdand fourth objectives refer to skills of a practical kind andare thus described as a skill objectives. The fifth objectivesuggests an attitude of mind and is therefore classified asan attitudinal objective.

The three broad divisions – knowledge, skills and attitudes– are often used in grouping objectives but you may comeacross several refinements of each division in the

97

literature. The most common of these refinements is thetaxonomy developed by Bloom and his colleagues. Theycall the three divisions ‘domains’: cognitive (knowledgeand intellectual skills), psychomotor (physical skills) andaffective (feelings and attitudes). These domains have beenfurther subdivided to provide hierarchies of objectives ofincreasing complexity.

Knowledge objectives (the cognitive domain): it is in thisarea that Bloom’s taxonomy has been most widely applied.He proposes six levels:

knowledge;comprehension;application;analysis;synthesis; andevaluation.

The reason for keeping different levels in mind whenwriting objectives is that courses sometimes pay undueattention to one level (usually the recall of information).

Skill objectives (the psychomotor domain): in manymedical courses, teachers need to pay a great deal ofattention to developing skill objectives. Such objectivesmay be improved if the condition under which theperformance is to occur, and the criteria of acceptableperformance, can be indicated. You might find it useful tospecify the expected level of competent performance. Forexample there will be some skills with which one wouldexpect students to show a high degree of competence andothers with which one might only expect familiarity.

Attitudinal objectives (the affective domain): writingobjectives in the affective area is very difficult, whichpossibly explains whey they are so often ignored. This isunfortunate because, implicity or explicitly, there are manyattitudinal qualities we hope to see in our graduatingstudents.

One way of doing so is to attempt to define the startingattitudes of the students and match these with moredesirable attitudes towards which you would hope theywould move. For example, you might start by assuming

98

that the students had a stereotyped attitude. You wouldthen wish to move them away from this towards an attitudewhich demonstrated understanding and acceptance ofother views. The advantage of this method is that itrecognises that not all students will develop the desiredattitude nor will they all necessarily start a course with thesame attitudes. The way to express objectives using thisapproach is to state ‘Awayfrom … (a particular attitude),towards … (a desirable attitude)’.

Where do objectives come from?

Writing objectives is not simply a process of sitting, pen inhand, waiting for inspiration, although original thinking iscertainly encouraged. Objectives will come from a carefulconsideration of the subject matter, what you and yourcolleagues know about the students, and about the subject.This will not be an easy task. You should consider a widerange of sources for objectives. These include:

an analysis of your own and colleagues’ knowledge,skills and attitudes;ways of thinking and problem-solving to be devel-oped;students’ interests, needs and characteristics;subject matter, as reflected in the published literature(especially in suitable textbooks);the needs of patients and the community;the requirements of professional certifying authorities;the objectives of the department or school.

How specific and detailed should objectives be?

This is a question frequently asked. The answer dependson the purposes for which the objectives are to be used. Indesigning a course, the objectives will be more generalthan the objectives for a particular teaching session withinthe course. As objective writing can become tedious,trivial and time-consuming it is best to keep yourobjectives simple, unambiguous and broad enough toconvey clearly your intentions. To illustrate from our ownfield of teaching, the objectives for a six-week clinical skillscourse, conducted for groups of 9-10 students, are shownbelow. Though quite broad, these objectives have proveddetailed enough for course planning purposes and formaking the intentions of the programme clear to students.

99

CHOOSING METHODS AND RELATING OBJECTIVESTO TEACHING AND LEARNING ACTIVITIES

The methods you employ to achieve the objectives shouldnot only allow those objectives to be realised, but will alsoreflect the kind of orientation you have to course planning.If your orientation is primarily the transmission of content, itis likely that your teaching methods will be dominated bylectures, assigned reading of books and electronically

100

based materials, and set problem-solving exercises. If it isto be the intellectual and personal development of yourstudents, small group teaching or individual tutorials ande-mail discussions are likely to play a more important role.

The actual choice of methods will be governed by severalfactors. Among the most important will be:

ensuring that students engage in appropriate learningactivities;your own expertise in using different methods;technical and financial resources to support themethod you wish to use.

Before leaving this subject we should like you to considerone important matter about choice of methods. Courses areoften constructed in ways that reveal a growing complexityof subject matter. For example, early in the first year theremay be an emphasis on basic principles and ideas. In lateryears, subject matter may be very much more complexand demanding. Yet, in our experience, the teachingmethods often used in the later years do not generallydemand higher levels of intellectual performance andpersonal involvement.

The main types of teaching in medical education, such aslecturing, small group teaching and clinical teaching aredealt with in earlier chapters. These are by no means all ofthe methods available. Other possibilities include projectwork, peer teaching and a variety of technology-basedtechniques. In addition, it should be remembered thatstudents undertake many learning activities in the absenceof teaching and there is growing pressure for this tobecome more common. It is reasonable to make explicit inyour objectives areas where you expect the students towork on their own. This particularly applies to knowledgeobjectives which might be achieved just as well indepen-dently in the library or by accessing the Web. It could alsoapply to some skill objectives where students might beexpected to seek out relevant experience by themselvesor in open access skills laboratories.

The way in which this process has been followed throughin the clinical skills course we have already introduced isdemonstrated once again on the course planning chart(Figure 6.3). When planning this course we were awarethat many students needed assistance with their basic

101

102

history-taking and physical examination skills. We thusdecided to put the majority of our staff time into achievingthe first two objectives. The most appropriate teachingmethod was obviously direct observation with feedbackand as this is very time-consuming we opted for apreceptor system where one staff member was respon-sible for only three students throughout the programme.However, opportunity was also provided for students toobtain additional ward practice on their own and with theresident staff. Senior students were also mobilised toprovide further help and instruction in this area. One of theimplications of this decision on staff allocation was to acceptthat the sixth objective (improving their knowledge in thesubjects Medicine and Surgery) would have to beachieved by other methods. This has involved anexpectation that students accept responsibility for doingmuch of this themselves. We have also designed andprepared a variety of self-instructional materials. Otherteaching techniques are incorporated to achieve otherobjectives.

RELATING OBJECTIVES TO ASSESSMENT METHODS

While it is obviously important to match the teaching andlearning activities with the objectives, it is absolutely vitalto match the assessment methods to the objectives and tothe teaching and learning activities. Failure to do so is thereason why many courses fail to live up to expectations. Amismatch of assessment and objectives may lead to seriousdistortions of student learning because, whether we like itor not, what is present in the assessments will drive whatmost students set out to learn.

In designing your course, we believe that it is alsoimportant to distinguish carefully between two types ofassessment. One is primarily designed to give feedback tothe students as they go along (formative assessment). Theother is to assess their abilities for the purposes of decisionmaking or grading (summative assessment). Formativeassessment is a crucial part of the educational process,especially where complex intellectual and practical skillsare to be mastered. Such assessment is notoriouslydeficient in medical schools, particularly in regard toclinical teaching (see Chapter 5).

103

The way in which assessment was designed in the exampleof our clinical skills course is shown in Figure 6.4. As noformal examination is required at the completion of thecourse, the major emphasis of the assessment activities isformative. However, assessment activities of a summativetype are conducted during the final two weeks of theprogramme when aspects of the students’ performance areobserved by preceptors and by other staff members. Youwill note that the assessment of knowledge is left largely tothe students themselves. In other circumstances we mighthave used a written test to assess this component of thecourse.

SEQUENCING AND ORGANISING THE COURSE

It is unlikely that the way in which you have set out yourobjectives, teaching and assessment on the planning chartwill be the best chronological or practical way to presentthe course to students. There are several other things thatmust be done. First, there must be a grouping of relatedobjectives and activities. (In the example we are followingthroughout this chapter, such a grouping occurs forobjectives one to the three which are largely to beachieved by the preceptor sessions.) Secondly, there mustbe sequencing of the teaching activities. There are likely tobe circumstances in your own context that influence you tosequence a course in a particular way, such as semestersor teaching terms. However, there are also a number ofeducational grounds upon which to base the sequencing.These include:

Proceeding from what students know to what they donot know;Proceeding from concrete experiences to abstractreasoning;The logical or historical development of a subject;Prioritising important themes or concepts;Starting from unusual, novel or complex situations andworking backwards towards understanding (e.g. as inproblem-based learning).

As our understanding of how different factors can influencelearning advances, you should give consideration to theways in which you can facilitate deep-learning approachesby your students through the way in which you organise

104

105

and manage the course and the kinds of intellectual andassessment demands you place on them. We suggest thatyou review the relevant sections in Chapter 1 to guide youin this matter.

Finally, you will need to consider the broad organisingprinciples behind your course. Will you, for example, offerit in a traditional way with a set timetable of carefullysequenced learning activities culminating in an end-of-year examination? Or will you design your course moreflexibly around a completely different approach such asproblem-based learning?

OTHER COURSE DESIGN CONSIDERATIONS

Many of the important educational considerations indesigning a course have been addressed, but there areother matters that must be dealt with before a course canbe mounted. These are only briefly described because theway in which they are handled depends very much on theadministrative arrangements of the particular situation inwhich you teach. Having said that, we are not suggesting inany way that your educational plans must be subservient toadministrative considerations. Clearly, in the best of allpossible worlds, the administrative considerations wouldbe entirely subservient to the educational plans but thereality is that there will be a series of trade-offs, witheducational considerations hopefully paramount.

In planning your new course, you will need to take thefollowing into account.

Administrative responsibilities: it will be necessary for oneperson to assume the responsibility of course co-ordination. This job will require the scheduling of teachers,students, teaching activities, assessment time and re-sources.

Allocation of time: many courses are over ambitious andrequire far more time (often on the part of the students) fortheir completion than is reasonable. This is especially trueof parts of a larger course of study. In allocating time, youwill need to consider the total time available and itsbreakdown, and how time is to be spent in the course. It isoften desirable to use blocks of time to deal with aparticular topic, rather than ‘spinning it out’ over a term,semester or year.

106

Allocation of teaching rooms, clinics, laboratories andequipment: courses depend for their success on the carefulallocation of resources. It is important that all competingclaims are settled early so that orderly teaching can takeplace.

Technical and administrative support: whether you teach acourse alone, or as one of a team, you will find a need forsupport of some kind or other. It may be as simple as theservices of someone to prepare course notes andexamination papers, or as complex as requiring, atdifferent times, the assistance of technicians and laboratorystaff. Your needs for support must be considered at theplanning stage.

EVALUATING THE COURSE

Many teachers may find a discussion of course evaluationin a chapter on planning rather odd, perhaps believing thatthis activity is something that takes place after a course hasbeen completed. We believe that this generally should notbe the case. It is our contention that in teaching you shouldcontinually evaluate what you are doing and how thecourse design and plans are working out in practice. In thisway, modification and adjustments can be made in asystematic and informed manner. But what is evaluation?You will often find the terms ‘evaluation’ and ‘assessment’used interchangeably, but evaluation is generally under-stood to refer to the process of obtaining information abouta course (or teaching) for subsequent judgement anddecision-making. This process, properly done, willinvolve you in rather more than handing around a studentquestionnaire during the last lecture. What you do clearlydepends on what you want to find out, but thorough courseplanning and course revision will require informationabout three different aspects of your course. These are thecontext of and inputs into your course; the processes ofteaching, learning, assessment and course administration;and, finally, the outcomes of the course.

Context and input evaluation: this is crucial if mistakes andproblems are not to be attributed, unfairly, to teachers. Inthis type of evaluation you will need to consider the coursein relation to such matters as other related courses, theentering abilities and characteristics of students, theresources and equipment available to teach with, and the

107

overall design and planning arrangement for the course.The major sources of information you can use here will bein the form of course documents, student records, financialstatements and the like.

Process evaluation: this focuses on the conduct of theteaching, learning, assessment and administration. It ishere that the views of students can be sought as they are theonly people who experience the full impact of teaching inthe course. Questionnaires, written statements, interviewsand discussion are techniques that you can consider.

Outcome evaluation: this looks at student attainments at theend of the course. Naturally you will review the results ofassessment and judge whether they meet with the impliedand expressed hopes for the course. Discussion withstudents and observation of aspects of their behaviour willhelp you determine their attitudes to the course you taught.

In all evaluations, whether of a course or of teaching, it ishelpful to keep in mind that there are many sources ofinformation available to you and a variety of methods youcan use. We suggest that you look at Chapter 10 onevaluation for more information about this.

GUIDED READING

For a useful extension of the material in this chapter wesuggest you have a look at S. Toohey’s Designing Coursesfor Higher Education, SRHE and Open University Press,Buckingham, 1999 and J. Biggs’ Teaching for QualityLearning at University: What the Student Does, SRHE andOpen University Press, Buckingham, 1999.

For a clear and systematic guide to curriculum planning werecommend Planning a Professional Curriculum by L.Fisher and C. Levene, University of Calgary Press, 1989. Ifyou wish more detailed guidance on objectives we suggestN. Gronlund, How to Write Instructional Objectives,Prentice Hall Inc, Englewood Cliffs, 1995.

Books and articles referred to in this chapter

B. Bloom et al. Taxonomy of Educational Objectives.Handbook I: Cognitive Domain. McKay, New York, 1956.

108

7: TEACHING IN A PROBLEM-BASED COURSE

INTRODUCTION

Until recently the vast majority of medical teachers wereworking in traditional medical schools and unlikely to bechallenged by alternative teaching methods which were tobe found only in a small minority of radical, and usuallynew, medical schools. However, a dramatic change hasoccurred. Many conventional and well-established medi-cal schools have undertaken curriculum reviews and havedecided to change to ‘problem-based learning’ (PBL).However, experience has shown that when this approachhas been introduced its effectiveness has often beenundermined by a lack of understanding of the purposeand process of PBL. This chapter aims to give you guidanceif faced with teaching in a problem-based course. It will notattempt to debate the rationale behind PBL in any depth norwill it analyse the research on its effectiveness. The GuidedReading will provide a starting point if you wish to pursuethese issues.

WHAT IS PROBLEM-BASED LEARNING?

The traditional way of medical school teaching has been torequire students to undertake sequential courses in thepre-clinical and para-clinical sciences as a prerequisite tocommencing studies in the medical sciences and clinicalpractice. Such courses have been the autonomousresponsibility of academic departments who have jea-lously guarded their curriculum time and their controlover course content and examinations. The degree ofintegration, particularly in the pre-clinical disciplines, hasoften been limited. This structure has formed the basis ofthe curriculum for most medical schools since the Flexnerreport in 1910 but is starting to collapse in the face of theintolerable load of information that each discipline expectsthe student to learn. The veritable explosion of scientificknowledge relevant to medicine, and the increasingspecialisation of clinical practice, has led to unmanageablerequests for the inclusion of more courses and morecontent without agreement or action on what is to beexcluded. Efforts to deal with these issues using strategiessuch as organ systems teaching have made little impact. Analternative approach has become necessary and PBL is onegaining increasing acceptance.

110

In brief, the approach is one in which learning is basedaround problems, usually written clinical cases. Studentswork through these problems, under greater or lesserdegree of guidance from tutors, defining what they do notknow and what they need to know in order to understand(not necessarily just to solve) the problem. The justificationfor this is firmly based in modern psychological theories oflearning which have determined that knowledge isremembered and recalled more effectively if learning isbased in the context in which it is going to be used in thefuture. Thus, if basic science knowledge is structuredaround representations of cases likely to be encountered inmedical practice in the future, it is more likely to beremembered. Problem-based learning is also inherentlyintegrative with the need to understand relevant aspects ofanatomy, physiology, biochemistry, pathology and so onbeing readily apparent in each case.

There is some evidence that students do, in the long term,recall more information in the context of patient problemswhen taught in the PBL way when compared with studentstaught in the disciplinary-based way. What is strikinglyapparent is that students prefer this approach and becomemuch more motivated to learn, a prerequisite to thedesirable deep approach to learning discussed in Chapter1. Other educational objectives believed to be addressedby PBL are, according to Barrows, the development ofeffective clinical reasoning skills and self directed learningskills. However, several recent reviews in the literaturereveal little evidence of major differences betweengraduates from schools with a problem-based curriculumand those from schools with a more traditional curriculum.Whether this is actually true or reflects limitations of theindicators used to evaluate outcomes is unclear. Whatevertrends there are generally seem to favour PBL.

IMPLEMENTATING PROBLEM-BASED LEARNING

Problem-based learning will have different implications ifyou are involved on a curriculum committee than if yourinvolvement is as a tutor to a group of students undertakinga PBL exercise. In the former situation you will be engagedin reviewing the evidence for the effectiveness of PBL, indiscussing the politics and practicalities of making such amajor change to the curriculum, and in conducting or

111

arranging information sessions and workshops for the staffof the medical school in order to gain their support.

Having decided in principle to proceed, your school maychoose one of several implementation models (Figure 7.1).

You may decide to convert the whole curriculum to PBLalong the lines of the innovative schools of which McMasterUniversity in Canada is the exemplar. Alternatively youmay want to follow the path of schools such as New Mexicoor Harvard and commence with two tracks, running thePBL track parallel to the conventional track with theadvantage of gaining experience and undertaking com-parative evaluation. If you wish to be more cautious anotheralternative is to introduce PBL as a component of thecurriculum or into individual courses with or without theexpectation that the whole course will eventually change toPBL.

COURSE DESIGN CONSIDERATIONS

The basic principles of course design are the same as thosein any other course (see Chapter 6). There is no single bestway of approaching this task and it will be important to

112

ascertain which of the many styles of PBL you are expectedto implement. In general, PBL curricula are constructed in amodular format with blocks of several weeks beingcommitted to a common theme (e.g. gastroenterologicalproblems; reproduction). Factors to be taken into accountwill include:

The major purposes of the module

There are generally two major purposes to be achieved ina PBL module. One is the attainment of specific learningobjectives in the form of an integrated knowledge andunderstanding of a defined problem (e.g. diarrhoea). Theother is skill in the process of problem-solving and self-directed learning.

These are combined in various proportions in the GuidedDiscovery and Open Discovery approaches. In the GuidedDiscovery approach the emphasis is on both content andprocess. The course is carefully structured as a series ofmodules containing problems which direct students intolearning the appropriate basic science and clinical content.While it is essential to allow the students to discover thelearning issues from the problem, written guidelines areprovided and tutor prompting occurs to ensure that allcontent areas are considered.

The Open Discovery approach places more emphasis onthe process. The framework of the course and theproblems may be the same as in the Guided Discoveryapproach but the students have much greater responsi-bility for determining what they should learn. Completecoverage of all content aspects is not expected.

The method of instruction

PBL is usually conducted in small groups consisting of 5-10students with a tutor. As a rule, the tutor is there to facilitatethe process rather than to be a provider of contentknowledge. In some schools tutors are deliberately chosento be non-experts particularly where the Open Discoveryapproach is predominant. Small group activities aresupported by independent study for which curriculumtime must be carefully protected. Where tutor resourcesare limited it is possible to conduct PBL in large group

113

settings using student-led groups for discussion or byrelying to a greater degree on independent study. Wehave used this approach successfully in a foundationcourse using PBL for first-year medical students (see later).

The selection of the problems

This is one of the most important considerations in coursedesign. The problems must be of the kind that willultimately be faced by the students after they graduatebut must also be both broad enough and specific enough toengage the students in learning activities which match thecurriculum objectives. They should not be answerable bysimple responses. In general they should be patientproblems or health problems which will require studentsto go through the following process:

Analysis of the problem.Identification of the multidisciplinary knowledge re-quired to understand and solve the problem.Obtaining agreement on the independent learningtasks to be performed.Obtaining agreement on when the learning tasks willbe achieved,Application of the newly acquired knowledge to theinitial problem.Further cycles of the process if necessary.

While these problems will usually be represented inwritten form every chance should be taken to engage thestudents in the same PBL process with real problemsencountered in the health care setting. To this end clinicalteachers should be actively engaged in supporting theprogramme. This can be particularly helpful in the earlierparts of the curriculum where opportunities for clinicalexperience may be more limited. Increasingly, thecomputer is being used to manage and support PBL.

114

Preparing the PBL modules

You may be asked to prepare one or more of thesemodules. Your first task will be to obtain the objectives ifthey have been prepared by an overseeing group. Morelikely you will have to develop them yourself. Suchobjectives should define what it is the students shouldhave achieved when they have completed the module (seeChapter 6). It is important to seek the involvement ofteachers from all the disciplines that are expected tocontribute to the learning outcome of the students.

Once objectives are agreed, case summaries must becarefully prepared, remembering that they should beinteresting and complex enough to engage students in theproblem-solving process. A written guide should bedeveloped for the tutors involved in the module, its contentdepending to some extent on the familiarity they are likelyto have with the problem. An abbreviated example of sucha guide is illustrated in Figure 7.2.

Resource materials for students and tutors should beidentified (e.g. references, weblinks, audio-visual materi-als, computer simulations, static demonstrations and evenlectures). Resource people who could be available forstudents to contact should be approached and times thatmeetings with students could be scheduled should beascertained. An alternative is to use email or an on-linediscussion group.

Assessment

This is one area where there is still considerable debateand development. It is important to involve all disciplines inthe preparation of assessment materials as it is to engagethem in defining the objectives. Only in this way will theybe convinced that their discipline is being adequatelyrepresented. This reassurance is particularly important tothe basic scientists who are often those most threatened bya change to PBL.

The most important focus for assessment in PBL isformative. Constantly challenging the students to evaluatethe success of their learning is a vital role of the tutor.

115

116

Students must also be challenged to develop self-assessment skills. A way of approaching this is outlined inChapter 8 in the section on self-assessment.

Most schools will also wish to have summative assessmentprocedures. It cannot be stated too strongly the potentialdanger this poses. Conventional knowledge-based tests,such as MCQs, should be avoided except perhaps forstudent self-assessment or progress testing. Unfortunately,test methods more suitable for PBL courses are not welldeveloped. The Modified Essay Question may be thesimplest and most flexible format if a written test ofintegrated knowledge is required (see Chapter 8). Inmany schools the tutor is asked to rate students on theirperformance with the emphasis being as much on theirgroup and personal learning skills as on their grasp of thecontent.

Few methods have been specifically designed to evaluateproblem-based self-directed learning skills. The most wellknown of these is the Triple Jump Test developed atMcMaster University. As the name implies this is a threestep procedure. In the first step the student works througha theoretical problem on a one-to-one basis with a tutor,The student is asked to think aloud as the problem isassessed and learning needs determined. The second stepconsists of a fixed period of time (2-3 hours) during whichthe student may seek out relevant information. The finalstep involves a return to the tutor where the newinformation is used to re-analyse the problem and hope-fully come to some conclusions. The tutor evaluates theefficiency and effectiveness of the student’s problem-solving and self-directed learning skills. These arecompared with the student’s self-assessment of his or herperformance. This method has obvious merit as aformativeevaluationbut, as there is little information aboutits validity and reliability, so its value for summativepurposes has yet to be established.

To obtain more information on assessment in PBL courseswe suggest you look at the book by Boud and Feletti andthe review article by Nendaz and Tekian. You will also findit very helpful to contact PBL schools and see whatstrategies and methods they are using.

117

TUTORING

The first task you are likely to have to perform in a PBLcourse is that of tutor. Your role will generally be one offacilitator rather then expert, a role you may initially findrather difficult. The sessions will usually be conducted insmall groups so Chapter 3 may also be helpful.

In brief, your main task is to help your students develop theskills to work effectively as a group member. To do this thestudent must understand the purpose of their PBL activitiesand not see the session simply as one of solving theproblem. They are usually expected to learn a lot of ‘stuff’.The process skills they will need to do this effectivelyinclude group skills, information literacy (locating, retriev-ing, evaluating and using information of all kinds),negotiating, interviewing and presenting. So while youmay appear to be teaching your subject less, you shouldhave the pleasure of observing students learn theexpected content in a much more interesting and excitingway.

Modules are designed to be completed in a fixed period oftime. For the purpose of this chapter the time allocated to amodule will be assumed to be one week. A possible modelof a week’s activities is seen in Figure 7.3.

118

At the first session you will introduce the problem. Usuallythis will be a written case-history (Figure 7.4) but could besupported by a videotape of part of the history or even by avisit to a real patient. One possible way of conducting thissession is outlined in Figure 7.5.

Students are then engaged in formulating questions aboutthe problem (e.g. What might be the cause of thediarrhoea? Why did it resolve then recur? Why was thestool offensive? What was the mechanism for thediarrhoea? How should it be investigated? etc.). To assistthe process you should be provided with additionalinformation about the case (physical examination findings;investigations) to feed into the discussion at the initialsession or later in the week. You should also have a list of

119

resources which are relevant to the problem such asbooks, articles, videotapes, web-based materials andexperts available for consultation. There might even be alecture or two for the students to attend, The expectedlevel of tutor intervention will depend to some extent onwhether the approach in your school is Guided or OpenDiscovery.

When agreement has been reached on the learning tasksto be performed, arrangements are made to meet againduring the week to review progress and pool information.You may or may not attend such meetings. Students willdetermine whether further information is needed and, if so,additional learning tasks will be assigned.

At the end of the week, progress with the problem isreviewed. Remaining difficulties are resolved. At such atime some expertise relating to the problem may be ofvalue. However, complete resolution of the problem israrely possible nor is it to be seen as the aim. Studentsshould become aware that there is always more to belearned.

If the institution is using expert tutors it is unlikely that youwill spend many sessions with one group of students. Youmay only operate as a tutor for a few weeks a year. On theother hand, if non-expert tutors are the policy then you maybe the facilitator to one group for an extended period oftime. In many ways this is likely to be more rewardingalbeit more time consuming.

STAFF RESOURCES FOR PROBLEM-BASEDLEARNING

One of the major concerns for medical schools contemplat-ing changing to PBL is that of staff resources. It is widelyperceived that PBL is dependent on small groups and thatthis will demand more staff or a considerable extra timecommitment for existing staff. This may well be true if oneis trying to emulate the McMaster model of PBL. However, itis possible for compromises to be made. For instance, ifnon-expert tutors are acceptable for some components ofthe course it may be possible to use staff (e.g. laboratory-scientists, clinicians) who previously have not had a majorteaching role.

120

Another alternative is to undertake the normally tutor-ledsessions in large group settings. We have some experi-ence of this in a first-year foundation course in which weaim to introduce students to the process of PBL. Clinicalproblems (e.g. diarrhoea in a traveller; chest pain in asquash player) are introduced to a whole class of studentsin a lecture theatre session. Students are asked to arrangethemselves informally into groups of 4-5. The process ofproblem analysis is undertaken by these groups, aprocedure that produces a rewarding hum of activity inthe lecture theatre. The teacher then gathers together theideas in an interactive manner from representative of thegroups. Agreement is reached on the information thestudents require to proceed with the problem. If theexercise is to be completed in one session this can then beprovided by the teacher or an invited colleague who is the‘expert’. On the other hand the full process of PBL can becontinued with students departing to report back at asubsequent session having completed their independentlearning tasks.

This approach requires skill on the part of the teacher.However, it is one we have found to be very successful andhighly rewarding. Student feedback has been verysupportive. In a sense it is an extension to the additionaltechniques referred to in Chapter 2 on Large GroupTeaching.

Whatever approach is used the need for effective staffdevelopment is paramount if PBL is to be successfullyimplemented. Unfortunately may schools fail in this regardwith inevitable disappointment to staff and students.Particularly helpful are workshops in which staff takedifferent roles in a PBL group exercise and observing areal student PBL group in action.

GUIDED READING

There are several books and many articles which providethe necessary background to the philosophy, justificationand practice of PBL.

Some of the seminal work in the area was conducted byBarrows and two of his books are recommended. Theseare H. Barrows and R. Tamblyn, Problem-Based Learning:

121

An Approach to Medical Education, Springer, New York,1980 and H. Barrows, How to Design a Problem-BasedCurriculum for Preclinical Years, Springer, New York,1985.

A more recent book which provides examples of PBL frommany other professions is The Challenge of Problem-BasedLearning by D. Boud and G. Feletti (eds), Kogan Page,London, 1998. This contains a valuable section onassessment.

A useful practical guide to implementing PBL is ProblemBased Learning in Medicine by T. David et al., RoyalSociety of Medicine Books, London, 1999.

A source of written examples of clinical problems suitablefor pre-clinical and clinical courses is Clinical Problem-Based Learning: A Workbook for Integrating Basic andClinical Science by R. Waterman et al., University of NewMexico Press, Albuquerque, 1988. It is also recommendedthat examples are sought from other PBL schools, some ofwhich are available on the Web.


H. Barrows (1986). A taxonomy of problem-based learningmethods. Medical Education, 20, 481-6.

A. Kaufman et al. (1989). The New Mexico experiment:educational innovation and institutional change. AcademicMedicine, 64, 285-94.

G. Feletti and S. Carver (eds.) (1989). The new pathway togeneral medical education at Harvard University. Teach-ing and Learning in Medicine, 1, 42-6.

Additional references

The following is a selection of references which will be agood starting point for someone wishing to become moreinformed about the research evidence for PBL.

M. Albanese (2000). Problem-based learning: why curri-cula are likely to show little effect on knowledge andclinical skills. Medical Education, 34, 729-738.

122

J. Colliver (2000). Effectiveness of PBL curricula. AcademicMedicine, 75, 259-266.

R. Foley, A. Poison and J. Vance (1997). Review of theliterature on PBL in the clinical setting. Teaching andLearning in Medicine, 9, 4-9.

G. Norman and H. Schmidt (2000). Effectiveness of PBLcurricula: theory practice and paper darts. MedicalEducation, 34, 721-728.

123


8: ASSESSING THE STUDENTS

INTRODUCTION

Being involved in student assessment is among the mostcritical of all tasks facing the teacher. Generally, teacherstake such involvement quite seriously but, sadly, thequality of many assessment and examination proceduresleaves much to be desired. The aim of this chapter,therefore, will be to help you to ensure that the assessmentswith which you are involved do what they are supposed todo in a fair and accurate way. We will provide somebackground information about the purposes of assessmentand the basic principles of education measurement. Wewill then detail the forms of assessment with which youshould be familiar in order that you can select anappropriate method.

THE PURPOSE OF ASSESSMENT

When faced with developing an assessment you must bequite clear about its purpose. This may appear to be statingthe obvious but try asking your colleagues and studentswhat they think is the purpose of the assessment with whichyou are concerned. We are certain that there will be aconsiderable diversity of opinion.

Purposes of assessment may be described as follows:

Judging mastery of essential skills and knowledge.Rank ordering students.Measuring improvement over time.Diagnosing student difficulties.Providing feedback for students.Evaluating the effectiveness of the course.Motivating students to study.Setting standards.Quality control for the public.

Though it may be possible for one assessment method toachieve more than one of these purposes, all too oftenassessments are used for inappropriate purposes andconsequently fail to provide valid and reliable data.

It must never be forgotten how powerfully an assessmentaffects students, particularly if it is one on which their futuremay depend. This influence may be positive or negativeand even harmful. For many students, passing the

126

examination at the end of the course is their primarymotivation. Should this examination not be valid, and thusnot truly reflect the content and objectives of the course,then the potential for serious distortions in learning and formaking errors of judgement about students is evident. Anexample from our own experience may illustrate this point.A revision of the final-year medical curriculum inadver-tently led to the multiple-choice test component of the end-of-year assessment having considerably more weight thanthe clinical component. Students were observed to bespending excessive amounts of time studying the theore-tical aspects of the course in preference to practising theirclinical skills, the latter being the main aim of thecurriculum revision. A subsequent modification of theassessment scheme, giving equal weighting to an assess-ment of clinical competence, corrected this unsatisfactorystate of affairs.

It is our view that assessments on which decisions about thestudents’ future are to be made (summative assessment)should be kept separate from those which are for thebenefit of the students in terms of guiding their furtherstudy (formative assessment) as they have profoundlydifferent effects on student behaviour.

Summative assessment

In dealing with summative assessment, every effort mustbe made to ensure that all assessments are fair and basedon appropriate criteria. Students should be fully informedof these criteria, on the assessment methods to beemployed and on the weightings to be given to eachcomponent. Such information should be given to studentswhen a course begins. This is important because it issurprising how often information obtained from othersources, such as past students or even from the institutionitself, can be inaccurate, misleading or misinterpreted bythe students. The best way of avoiding this is to publishdetails of the course plan, including the assessmentscheme. Examples of past papers can be provided andwe have found an open forum on the assessment schemeearly in the course to be both popular and valuable.

127

Formative assessment

Formative assessments are generally organised moreinformally and undertaken on a continuous basis. Suchassessments must be free of threat, as the aim is to get thestudents to reveal their strengths and weaknesses ratherthan to disguise them – the opposite to the students’approach to summative assessments. Opportunities toobtain feedback on knowledge or performance are alwaysappreciated by students and can lead to positive feelingsabout the department and the staff concerned.

WHAT YOU SHOULD KNOW ABOUT EDUCATIONALMEASUREMENT

Whatever the purpose of the assessment, the method usedshould satisfy the following requirements:

1234

Validity: does it measure what it is supposed tomeasure?Reliability: does it produce consistent results?

Practicality: Is it practical in terms of time andresources?

Positive impact on learning.

Our intention in raising these requirements is to encourageyou to apply the same critical interest in the quality ofeducational assessment as you undoubtedly apply to thequality of the methods used in your research or patientcare. This section will provide you with some basicinformation about the aspects of educational measurementwith which we think you should be familiar.

Validity

Content validity is the first priority of any assessment. It is ameasure of the degree to which the assessment contains arepresentative sample of the material taught in the course.A numerical value cannot be placed against it and it mustbe judged according to the objectives of the assessment.Therefore, in approaching any assessment the firstquestion you must ask is: what are the objectives of thecourse?

Unfortunately, such objectives are not always available.Should you be in this situation, with no written objectives

128

for the assessment you have to design, then you have noalternative but to develop them. This is not such a difficulttask as you might imagine because, as far as the assessmentis concerned, the objectives are embodied in the coursecontent. A look at the teaching programme, lecture andtutorial topics, and discussions with teaching staff shouldallow you to identify and categorise the key features of thecourse. What you are, in fact, attempting to do is toconstruct a course plan in reverse and you may find ithelpful at this point to consult Chapter 6 on course planningwhere this process is discussed in greater detail.

The objectives of the course are the blueprint againstwhich you can evaluate the content validity. For the contentvalidity to be high, the assessment must sample thestudents’ abilities on each objective. As these objectivesare likely to cover a wide range of knowledge, skills andattitudes, it will immediately become apparent that nosingle test method is likely to provide a valid assessment.For instance, a multiple-choice test will hardly be likely toprovide valid information about clinical skills. Someobjectives, particularly those in the attitudinal area willbe hard to assess validly in examination settings andalternative forms of assessment may have to be sought.

Other forms of validity exist but generally speaking youwill not be in a position to evaluate them so they will not bediscussed further. If you are interested, you should consultthe Guided Reading at the end of the chapter.

Reliability

The reliability of any assessment is a measure of theconsistency and precision with which it tests what it issupposed to test. Though its importance is initially less vitalthan validity, you should remember that an unreliableassessment cannot be valid. The degree of reliabilityvaries with the assessment format itself, the quality of itsadministration and the marking.

Theoretically, a reliable assessment should produce thesame result if administered to the same student on twoseparate occasions. Various methods are available whichprovide statistical indices of reliability and you should seekexpert advice on these. A common approach used in

129

multiple-choice tests is to produce a measure of internalconsistency.

Another key component in determining the reliability of anassessment is the consistency of the marking. The absenceof consistency is a major reason for the unacceptable levelsof reliability in most forms of direct assessment, in clinicalexaminations and in written tests of the essay type.However, methods are available to help you minimise thisproblem and these will be discussed later in this chapter.

It has become widely recognised in recent years that aneven more important contributor to unreliability isinadequate content or performance sampling. This is aparticular problem for assessments of clinical competencewhich need to be much longer than is usually the case ifhigh levels of reliability (generalisability) are to beachieved. You must explore this issue further if you areinvolved in developing a ‘high stakes’ clinical examinationsuch as ones to be used for certification purposes.

Improving validity and reliability of assessments

Validity can be improved by:

carefully matching an assessment with the learningobjectives, content and teaching methods;increasing the sample of objectives and content areasincluded in any assessment;producing an examination blueprint (see Figure 8.1);using test methods that are appropriate for theobjectives specified;employing a range of test methods;ensuring adequate security and supervision to avoidcheating in examinations;improving the reliability of the assessment.

Reliability can be improved by:

ensuring that questions are clear and suitable for thelevel of the students;checking to make sure time limits are realistic;writing instructions that are simple, clear, andunambiguous;developing a marking scheme of high quality (e.g.

130

explicit and agreed criteria, checking of marks,several skilled markers);keeping choices within a test paper to a minimum;when using less reliable methods increasing thenumber of questions, observations or examinationtime.

Practicality

An assessment scheme must be practical. You may decideto use a scheme that is potentially highly valid and reliablebut find that it is not practical to do so in yourcircumstances. Some questions you might consider hereare:

131

Do I have the skills to administer, mark and grade theassessment?Can I interpret the results accurately?Will the assessment scheme demand too much time?Does the scheme require special resources (e.g.labour, materials or equipment) and are these readilyavailable to me?If appropriate, can I combine less reliable methodswith more reliable methods?

Obviously, there will be other considerations of a practicalnature that are peculiar to your own circumstances and thatyou will have to consider before implementing anyparticular scheme.

Norm-referenced versus criterion-referencedassessment

Before we finish dealing with some of the basic principlesof educational measurement, we wish to introduce thedifference between norm- and criterion-referencedassessment. You are likely to be familiar with norm-referenced assessment, as this reflects the traditionalapproach to testing. Any assessment which uses the resultsof all the students to determine the standard is of this type.In such tests the pass level is often set by predeterminingthe proportion of students given each grade or ‘grading onthe curve’ as it is often called. This is also sometimes calleda relative standard setting procedure.

This traditional approach is one that we urge you to moveaway from. Some assumptions made are not appropriate toassessing learning in universities, and the approach can beshown to lack an educational justification. For example, dowe seriously set out in our teaching to ensure that, nomatter how well students achieve the objectives, becauseof our grading on the curve policy, some will fail? Surely,our task is to help all students to achieve the objectives andreduce gaps between them rather than getting a ‘spread ofscores’. We refer you to Biggs to understand this conceptmore fully.

In medicine we are often more concerned to establish thatthe students achieve some minimal standard of compe-tence. In this case, the criterion-referenced approach is

132

more appropriate. Such an approach necessitates thedetermination of an absolute standard before administer-ing the assessment, rather than waiting to see the overallresults before doing so. Though this can be difficult toimplement, we have found that attempting to do so is apowerful way of improving the validity of the assessment.Everyone concerned is forced to consider each item in theassessment and ask themselves if it is relevant and set at theappropriate level of difficulty. Our own experiences withsuch an approach used to test clinical competence in thefinal year of the medical degree have been very revealingand rewarding.

The issue of standard setting is one which is achievinggrowing attention as the focus of assessment moves fromnorm-referenced to criterion-referenced or competencybased testing. Established procedures are available forsetting absolute standards in objective-type tests but areless well developed for clinical examinations. This is toocomplex and too difficult an area to discuss in this book butreading the article by Norcini is strongly recommended ifyou have responsibility for a ‘high stakes’ examination.

Positive impact on learning

It is clear that how and what students learn is influencedmore by our assessment practices than by any other factorin the curriculum. This influence is exerted at two levels. Ata policy level, an over-emphasis on formal examinationsand the implicit threat that this may carry will have anegative impact. At a methodological level, an emphasis onobjective tests, such as true/false and multiple-choice, willalmost certainly encourage and reward the use of surfacelearning strategies by students rather than approaches thatdemand higher-level intellectual processes such asreasoning and analysis. On the other hand, there areseveral assessment practices that can encourage andreward the kinds of learning that are more highly valuedtoday. These approaches include direct assessments ofperformance, learning portfolios, research projects, selfand peer assessment, and regular and constructivefeedback on learning.

133

If we appreciate this influence, then we have a solution tothe problem. In the criterion-referenced approach de-scribed above, the objectives are embedded in theassessment tasks, so if students focus on assessment, theywill be learning what the objectives say they should belearning. This is a positive solution to the common problemof the negative impact of assessment.

ASSESSMENT METHODS

In planning your assessment, it is necessary to be aware ofthe variety of methods available to you. It is impossible tobe comprehensive for reasons of space so we will restrictourselves to some common methods. We will also includeinformation about some innovative approaches developedrecently, which may be of interest. We do this deliberatelyin an attempt to encourage you to become subversive!With your new-found knowledge of assessment you willsoon be involved in situations where it is obvious thatinappropriate methods are being used. This may be due toa combination of tradition, ignorance and prejudice. Thefirst two you may be able to influence by rational argumentbased on the type of information we provide in this book.The last is a more difficult problem with which to deal.

TYPES OF ASSESSMENT

123456789

Essay

Short-answer

Structured

Objective tests

Direct observation

Oral

Structured clinical/practical assessment

Self-assessment

Learning portfolio

1. ESSAY

We suggest caution in the use of the essay, except insituations where its unique attributes are required. Theessay is the only means we have to assess the students’ability to compose an answer and present it in effective

134

prose. It can also indirectly measure attitudes, values andopinions. There are other reasons for retaining essays forsome purposes. Of particular importance in highereducation seems to be the assumption that the productionof written language and the expression of thought arescholarly activities of considerable worth and that essaysencourage students to develop more desirable studyhabits.

Though they are relatively easy to set, essays are time-consuming to mark. The widespread use of multiple-choice tests and the advent of computer scoring has liftedthe marking burden from many academics, few of whomwould wish to take it up again. Excluding such selfishreasons, there are other grounds for being concernedabout using essays. The most important is the potential forunreliable marking. Several studies have shown significantdifferences between the marks allocated by differenceexaminers and even by the same examiner re-marking thesame papers at a later date.

Essay questions tend to be of two kinds. The first is theextended response. An example is seen in Figure 8.2.

In the extended response question the student’s factualknowledge and ability to provide and organise ideas, tosubstantiate them and to present them in coherent Englishare tested. The extended essay is useful for testingknowledge objectives at the higher levels such as analysisand evaluation.

Another type of essay question is the restricted response,an example of which is shown in Figure 8.3. The restrictedresponse form sets boundaries on the answer requiredand on its organisation.

135

This type of essay is best for testing lower-level knowledgeobjectives. An advantage of the more restricted format isthat it can decrease the scoring problems (and hence bemore reliable).

If you intend to set and mark essay questions in anexamination, then we suggest that you keep in mind thepoints in Figure 8.4.

For essays, or other written assignments required during acourse of study, you should take steps to improve thequality of feedback to students. One way is to use astructured assignment attachment. Not only can such anattachment provide very useful individual feedback, butused early in a course with a model answer, it can showstudents the standards you expect from them, and also helpyou in awarding marks.

2. SHORT-ANSWER AND SIMPLE COMPUTATIONQUESTIONS

Short-answer tests have been surprisingly little used inrecent years, yet another casualty of the multiple-choiceboom. However, we have found them increasingly usefulas our concerns about the limitations of the objective typetests have become more apparent.

Though easy to mark, it is essential that markers areprovided with a well constructed marking key, especiallyif more than one correct answer is possible, or if severalprocesses are involved in answering the question (SeeFigure 8.5).

Obviously more short-answer questions than essays can befitted into a fixed time period. If one of the purposes of theassessment is to cover a wide content area, then short-answer questions have distinct advantages. Much of thesame may be said about multiple-choice questions butshort-answer questions have the advantage of avoidingcueing and requiring students to supply an answer, ratherthan to select or to guess from a fixed number of options.The major limitation of the short-answer test is that it is notsuitable for testing complex learning outcomes.

If you wish to employ short-answer questions you shouldtake account of the point in Figure 8.6.

136

137

138

3. STRUCTURED TESTS

Broadly speaking there are two types of structured testswhich are used in medical education – patient managementproblems (PMPs) and modified essay questions (MEQs).Both claim to assess problem-solving skills.

Patient management problems

PMPs aim to represent, with varying degrees of fidelity, aninteraction with a patient incorporating various amounts ofdata gathering, diagnosis and management. Writtenformats, using many ingenious technical devices, werequite widely used at one stage particularly in postgraduateeducation. However, difficulties with developing validscoring systems have led to a marked reduction in theiruse and it seems unlikely that many medical schoolteachers will be called on to develop them.

On the other hand the use of computer-based PMPs is arapidly expanding field particularly in North Americawhere they are increasingly being used in nationalcertifying examinations. The levels of expertise requiredto construct these sophisticated simulations are beyondthose expected of the average medical teacher so will notbe considered further in this book. However, we stronglyrecommend looking at examples of such PMPs which areavailable on the Web, and consider incorporating theminto teaching programmes. Good quality versions will bepopular with your students.

Modified essay questions

This test was initially developed for the Royal College ofGeneral Practitioners in London. The MEQ is now beingused with increasing frequency. The term essay is perhapsa little misleading because the format more closelyresembles a series of short-answer questions than anessay.

The student is provided with a limited amount of patientdata and then asked to write a brief answer to a question.Such questions may relate to history taking, examinationfindings, diagnosis, investigations and so on. Following oneor more initial questions, further information may beprovided and additional questions posed.

139

A certain amount of skill is required when preparing anMEQ to avoid giving the answers to previous questions andto avoid the student being repeatedly penalised for thesame error. Scoring is by comparison of the student’sanswer with model answers. The precautions to be taken inmarking are the same as those described for markingshort-answer questions.

MEQs are very popular with students, particularly whenused in formative assessment. Marked papers can bereturned or the students can be given the model answerand mark their own. The value of this exercise is enhancedwhen a tutorial is held to discuss disputes with the modelanswer. Their increasing use in summative assessment isprobably due to the fact that when skilfully prepared it ispossible to test a range of disciplinary aspects within theone question, a major advantage as examinations becomemore integrated.

An example of an MEQ is seen in Figure 8.7.

4. OBJECTIVE TESTS

This generic term is used in education to include a varietyof test formats in which the marking of the answers isobjective. Some classifications include short-answer ques-tions in this category. The term multiple-choice test issometimes used synonymously with the term objective test.However, we encourage you to use the more general term‘objective tests’ as it allows us to include a wide variety oftest types, only one of which can be accurately describedas ‘multiple-choice’. Other commonly used examples ofobjective tests are the true-false and matching types.

The characteristics of such tests are the high reliability ofthe scoring, the rapidity of scoring and the economy of stafftime in this task, and the ability to test large content areas.They lend themselves to the development of banks ofquestions, thus further reducing the time of examinationpreparation in the long-term. These advantages havesometimes led to an over-reliance on objective tests anda failure to be critical in their use.

While it is relatively easy to write objective items tomeasure the recall of bits of factual knowledge, consider-

140

able skill is required to write items that measure higher-level intellectual skills.

While it is possible you will not have to set and mark otherforms of written test, it is almost certain that you will have toparticipate in some way in writing or administeringobjective tests.

Choosing the type of question

You must find out or decide which type of item you will beusing. Objective items can be classified into three groups:true-false, multiple-choice and matching. We wouldsuggest you avoid the more complex matching types

141

which, in some examinations, often seem to behave morelike tests of reading ability, rather then tests of the coursecontent! For a variety of technical reasons, experts favourmultiple-choice over true-false and other types of objectiveitems.

True-false questions

Examples of true-false questions are shown in Figures 8.8and 8.9.

The simple type will obviously cause you the leastproblems in construction and scoring. The more complexmultiple type (also know as the cluster type) is verypopular because it allows a series of questions to be askedrelating to a single stem or topic. Each question may bemarked as a separate question. However, the questionsmay also be considered as a group with full marks givenonly if all the questions are correct and part-marks given ifvarying proportions of the questions are correct. Researchhas shown that the ranking of students is unaltered by themarking scheme used, so simplicity should be the guidingprinciple.

142

If you intend to use true-false questions you should takeparticular note of the points listed in Figure 8.10.

Multiple-choice questions

An example of a simple multiple-choice question (MCQ) isshown in Figure 8.11.

The MCQ illustrated is made up of a stem (‘In a 40 year old’)and five alternative answers. Of these alternatives one iscorrect and the others are known as ‘distractors’.

One advantage of the MCQ over the true-false question is areduction in the influence of guessing. Obviously, in asimple true-false question there is a 50 per cent chance ofguessing the correct answer. In a one from five MCQ thereis only a 20 per cent chance of doing so if all the distractors

143

are working effectively. Unfortunately it is hard to achievethis ideal and exam-wise students may easily be able toeliminate one or two distractors and thus reduce thenumber of options from which they have to guess.Information about the effectiveness of the distractors isusually available after the examination if it has beencomputer-marked. Some advocate the use of correctionformulas for guessing but this does not – on balance -appear to be worth the effort and may add an additionalstudent-related bias to the results.

If you intend to use multiple-choice questions you shouldtake particular note of the points in Figure 8.12.

144

Context-dependent questions

Having mastered the basic principles of setting goodobjective items, you may wish to become more adventur-ous. It is possible to develop questions with a morecomplex stem which may require a degree of analysisbefore the answer is chosen. Such items are sometimesknown as context-dependent multiple-choice questions.

One or more multiple-choice questions are based onstimulus material which may be presented in the form of aclinical scenario, a diagram, a graph, a table of data, astatement from a text or research report, a photograph andso on. This approach is useful if one wishes to attempt to testthe student’s ability at a higher intellectual level thansimple recognition and recall of factual information.

Extended-matching questions

The technical limitations of conventional objective-typeitems for use in medical examinations has stimulated asearch for alternative forms which retain the technicaladvantages of computer scoring. Many such efforts haveachieved little more than increasing complexity andconfusion for students. However, the extended matchingquestion (EMQ) is becoming increasingly popular. Themain technical advantage is the reduced impact of cueingby increasing the number of distractors. Other advantagesinclude ease of construction and flexibility as they workequally well for basic science as for clinical areas.However, they are particularly well suited for testingdiagnostic and management skills. An example of an EMQis given in Figure 8.13.

The EMQ is typically made up of four parts: a theme ofrelated concepts; a list of options; a lead-in statement todirect students; and two or more item sterns.

The item shown includes two stems that illustrate how thisEMQ might test at different levels. The first stem requiresproblem solving in order to determine a diagnosis; thesecond stem tests only recall. More stems could, of course,be added to this example to increase the content coverageof the test item and the range of levels tested. In somerespects, EMQs share similarities with the context-dependent MCQ we described earlier.

145

Putting together an objective test

This is the point where many tests come to grief. It is notenough simply to select 100 questions from the item bankor from among those recently prepared by yourcolleagues. The selection must be done with great careand must be based on the objectives of the course. Ablueprint, or table of test specifications, should beprepared which identifies the key topics of the coursewhich must be tested. The number of questions to beallocated to each topic should then be determinedaccording to its relative importance. Once this is donethe job becomes easier. Sort out the items into the topicsand select those which cover as many areas within the topicas possible. It is advisable to have a small working group atthis stage to check the quality of the questions and to avoidyour personal bias in the selection process. You may findthat there are some topics for which there is an inadequatenumber or variety of questions. You should thencommission the writing of additional items from

146

appropriate colleagues or, if time is short, your committeemay have to undertake this task. This process of blue-printing will establish the content validity of the test.

The questions should now be put in order. It is lessconfusing to students if the items for each topic are kepttogether. Check to see that the correct answers arerandomly distributed throughout the paper and if not,reorder accordingly. Organise for the paper to be word-processed, with suitable instructions about the formatrequired and the need for security. At the same time makesure that the ‘Instructions to Students’ section at thebeginning of the paper is clear and accurate. Check andrecheck the copy as errors are almost invariablydiscovered during the examination, a cause of muchconsternation. Finally, have the paper printed and arrangefor secure storage until the time of the examination.

Scoring and analysing an objective test

The main advantage of the objective type tests is therapidity with which scoring can be done. This requiressome attention to the manner in which the students are toanswer the questions. It is usually inappropriate to have thestudents mark their answers on the paper itself. Whenlarge numbers are involved a separate structured sheetshould be used. Where facilities are available it isconvenient to use answer sheets that can be directlyscored by computer or for responses to be entereddirectly into a computer by students. However, a hand-marking answer sheet can easily be prepared. An overlayis produced by cutting out the positions of the correctresponses. This can then be placed over the student’sanswer sheet and the correct responses are easily andrapidly counted. Before doing so ensure that the studenthas not marked more than one answer correct!

In most major medical examinations a computer will beused to score and analyse objective-type examinations.You must therefore be familiar with the process and beable to interpret the subsequent results. The computerprogramme will generally provide statistical data aboutthe examination including a reliability coefficient forinternal consistency, a mean and standard deviation forthe class and analyses of individual items. Should you be

147

the person responsible for the examination you will need toknow how to interpret this information in order to processthe examination results and to help improve subsequentexaminations. If you are not familiar with these aspects westrongly suggest you seek expert advice or consult one ofthe books on educational measurement listed at the end ofthe chapter.

5. DIRECT OBSERVATION

Direct observation of the student performing a technical oran interpersonal skill in the real, simulated or examinationsetting would appear to be the most valid way of assessingsuch skills. Unfortunately, the reliability of these observa-tions is likely to be seriously low. This is particularly so inthe complex interpersonal area where no alternative formof assessment is available. Nevertheless, in professionalcourses it is essential to continue to make assessments ofthe student’s performance, not least to indicate to thestudent your commitment to these vital skills. In doing so,you would be well advised to use the informationpredominantly for feedback rather than for importantdecision-making.

Various ways have been suggested by which theselimitations might be minimised. One it to improve themethod of scoring and another is to improve theperformance of the observer. The former involves thedesign of checklists and ratings forms.

Checklists

A checklist is basically a two-point rating scale. Evidencesuggests that the reliability of a checklist decreases whenthere are more than four points on the scale. The assessorhas to decide whether each component on the list ispresent/absent; adequate/inadequate; satisfactory/unsatis-factory. Only if each component is very clearly defined andreadily observable can a checklist be reliable. They areparticularly useful for assessing technical skills.

An example is shown in Figure 8.14.

148

Rating forms

Rating forms come in many styles. The essential feature isthat the observer is required to make a judgement along ascale which may be continuous or intermittent. They arewidely used to assess behaviour or performance becauseno other methods are usually available, but the subjectivityof the assessment is an unavoidable problem. Because ofthis, multiple independent ratings of the same studentundertaking the same activity are essential if any sort ofjustice is to be done. The examples in Figure 8.15 showseveral alternative structures for rating the same ability.They are derived from published formats used to obtaininformation about ward performance of trainee doctors.The component skill being assessed is ‘Obtaining the database’ and only one sub-component (obtaining informationfrom the patient) is illustrated.

149

Format 3 is the one we would recommend for two reasons.The first is that there is an attempt to provide descriptiveanchor points which may be helpful in clarifying for theobserver what standards should be applied. The second isa more pragmatic one. In a study we undertook, it was theformat most frequently preferred by experienced clinicalraters.

Improving the performance of the observer

It has often been claimed that training of raters will improvereliability. This seems to make sense but what evidencethere is shows that training makes remarkably littledifference! A study of our own suggested that a betterapproach might be to select raters who are inherentlymore consistent than others. Common sense dictates thatobservers should be adequately briefed on the ratingsform and that they should not be asked to rate aspects of thestudent’s performance that they have not observed.

150

6. ORAL

The oral or vice-voce examination has for centuries beenthe predominant method, and sometimes the only method,used for the clinical assessment of medical students. Thetraditional oral, which gives considerable freedom to theexaminer to vary the questions asked from student tostudent and to exercise personal bias, has consistentlybeen shown to be very unreliable. One major study oftraditional clinical examinations showed that the correla-tion between different examiners was overall no greaterthan would have occurred by chance!

Without doubt, face-to-face interaction between studentand examiner provides a unique opportunity to testinteractive skills which cannot be assessed in any otherway. However, these skills are not usually the focus ofattention and several studies have shown that the majorityof questions in oral examinations require little more thanthe recall of isolated fragments of information, somethingmore easily and more reliably assessed by more objectivetests.

We would recommend that reliance on traditional oralexamination be considerably reduced as it is now usuallypossible to incorporate many of the activities currentlyassessed in such examinations into the objective-structuredapproach discussed in the next section.

Should you wish to retain oral examinations then certainsteps should be undertaken to minimise the likelyproblems, as outlined in Figure 8.16.

7. STRUCTURED CLINICAL/PRACTICALASSESSMENT

In recent years there has been a search for newapproaches to assessment. One of the most interesting ofthese developments has been the ‘objective structuredclinical examination’ (OSCE) first described by Harden andhis colleagues in Dundee and subsequently developed byourselves and others as an integral part of our examinationprocedures. This approach to the assessment of clinicaland practical skills has now been taken up by most medicalschools and many professional bodies.

151

The structured examination is not really a test method in itsown right. Rather it is an administrative structure into whicha variety of test methods can be incorporated. The aim is totest a wide range of skills in an objective fashion.

The students proceed through a series of ‘stations’ andundertake a variety of tasks. Marking sheets and checklistsare prepared beforehand to improve the reliability ofscoring. All students are thus examined on the samecontent and marked on the same criteria by the sameexaminers. As in any form of assessment, the definition ofthe content to be tested, the construction of a blueprint andthe preparation of good test items is essential if a highdegree of validity is to be obtained. If reliability is a prioritythe examination will have to be a long one – probably 20-30stations.

152

Figure 8.17 shows the outline of a structured clinicalexamination. The range of tasks to be tested is shown in theleft-hand column. In the middle column is the type ofassessment used which in this case includes directobservations of performance on real patients andsimulators and the use of short-answer questions. Theright hand column lists the time allocated to each stationwhich in this instance is usually five minutes. However, thetime allocated varies widely between different examina-tions used for different purposes. The advantage of shortstations is that more tasks can be assessed in a given timegiving cost-benefit advantages in terms of reliability.

Figure 8.18 provides an example of a structured markingsheet for a physical examination station.

Should you wish to consider introducing such an approachyou should read the articles given in the references.Observation of a well run OSCE and advice from anexperienced OSCE organiser will pay enormous divi-dends.

8. SELF-ASSESSMENT

By ‘self-assessment’ we mean an assessment system whichinvolves the students in establishing the criteria andstandards they will apply to their work and then in makingjudgements about the degree to which they have been met.We believe that the skill of being able to make realisticevaluations of the quality of one’s work is an attribute thatevery graduate should have. Yet, in conventional courses,few opportunities are provided for self-assessment skills tobe learnt and developed.

The introduction of self-assessment practices into existingcourses have been shown to be feasible and desirable.Whether marks generated in this way should counttowards a final grade is an undecided issue. Workreported in the literature suggests that so long as theassessment scheme is well designed and students gradethemselves on achievement (and not effort), they willgenerate marks which are reasonably consistent with staffmarks. Thus, there is little doubt that self-assessment, usedprimarily to improve the students’ understanding of theirown ability and performance, is worthwhile educationally

153

154

155

and encourages openness and honesty about the assess-ment.

If you wish to embark on a trial scheme you must first setabout the task of establishing criteria and standards. Thiscan be done at a series of small group meetings attendedby staff and students. Both must agree on the criteria to beapplied to the students’ work, To help focus on this task youmight have students reflect on questions such as:

How would you distinguish good from inadequatework?What would characterise a good assignment in thiscourse?

Once criteria have been specified, students use them tojudge their own performance. Marks are awarded withreference to each criterion and a statement justifying themark should be included. An alternative is to contrast theirown mark with one given to them by a peer. The teachermay also mark a random sample to establish controls and todiscourage cheating or self-delusion. We urge you to givethis approach to assessment very serious considerationindeed. In our view, it is among the most educationallypromising ideas in recent years, and we suggest you studythe book by Boud listed at the end of this chapter.

9. THE LEARNING PORTFOLIO

All assessment methods require that students presentevidence of their learning, yet in most cases (with thesesand project work being notable exceptions) it is theteacher who controls the character of that evidence.Requiring students to respond to objective tests, writeessays and participate in clinical examinations for example,does this.

If we really believe in student-centred learning then wemust work hard to ensure that our assessment practicesreflect, encourage and reward this belief. In Chapter 1, wenoted that assessment in student-centred learning needs tobe more flexible with greater emphasis on studentresponsibility. The learning portfolio is one way ofreinforcing student-centred learning. The portfolio clearlyhas validity as an assessment method in this situation, but itsreliability for summative purposes has yet to be deter-

156

mined. This should not, however, discourage you fromexperimenting with learning portfolios with your students.

A learning portfolio is a collection of evidence presentedby students to demonstrate what learning has taken place.In the portfolio, the student assembles, presents, explains,and evaluates his or her learning in relation to theobjectives of the course and his or her own purposes andgoals. Used for many years in disciplines like the fine artsand architecture, portfolios are now being used morewidely and are being strongly advocated as an approachto the revalidation (re-certification) of practising doctors inthe UK. They also have particular relevance in problem-based learning.

A learning portfolio might have several parts, such as:

An introductory statement defining what the objectivesare and what the student hopes to accomplish.A presentation of items of evidence to show whatlearning has taken place.An explanation of why items are chosen and pre-sented, evidence of the application of learning to someissue or real-life situation, and an evaluation of learningoutcomes.

A danger of using portfolios is that students might do toomuch, and some of their material might be less thanrelevant! So it is important to provide structure and tosuggest sample items, the number of different items andtheir approximate size. Obviously, the items in a portfoliowill reflect your particular discipline. Some ideas are listedin Figure 8.19.

Implementing learning portfolios as an assessment methodrequires careful preparation and planning on your part. Inhis discussion of portfolios, Biggs recommends that most ofthe following should be considered, and we suggest youlook at his book for more information on these matters:

Clarify for students, perhaps in the course objectives,what the evidence for good learning might be;Specify the requirements for a learning portfolio interms of:

157

the structure of the portfolio;

a list of sample items;

the number of items required (Biggs suggest thatfour is the maximum in a semester-length unit);

the size of items and the overall portfolio;

any required items.

Decide how the portfolio will be assessed. It is suggestedthat global assessment of the whole portfolio is preferred toanalytical marking to ensure that the broader purposes ofstudents reporting and evaluating their learning arepreserved and not broken down into discrete elements.

ASSESSING STUDENTS WITH A DISABILITY

Institutions have implemented many policies and practicesto assist students with a disability. Unfortunately, con-sideration of their special needs is not always extended tothe assessment of their learning. It is good practice for staffin departments to review and share alternative assessmentarrangements on a regular basis as such arrangement arelikely to be specific to both the kind of disability and to thenature of the discipline.

158

You are not expected to lower academic standards toaccommodate these students but rather to provide themwith a reasonable and fair opportunity to demonstrate theirlearning. Thus, you may need to make adjustments toassessment tasks once you understand how the particulardisability affects performance. Space does not allow us togo into all the possible options here, but the followingsuggestions listed in Figure 8.20 for specific disabilities willgive your students more equal opportunity in your course.Common strategies will be to simply follow goodassessment practices we have described elsewhere andto be flexible in your insistence on assignment deadlinesand in the time allowed in formal examinations.

ASSESSING STUDENTS AS GROUPS

With the increasing use of group and team-based learning,such as in problem-based learning, there is the relatedchallenge of assessing the outcomes of group learning inways that are fair to individuals but which recognise theparticular dynamics and realities of such learning. Moredetailed descriptions of this assessment approach aregiven in Miller et al. and in Brown and Glasner, but thefollowing ideas will be helpful when assessing groupassignments.

159

Preparatory matters are important. Remember to keepgroup size down (greater than six members is too large);help students to work as effective group members; formgroups randomly and change membership at least eachsemester; and ensure all students understand the assess-ment mechanisms you will use to encourage the diligentand forewarn the lazy.

Marking group submissions can be a way of assessingmore students but taking up less time on your part. Whenallocating marks, the following strategies will be helpful:

Give all members of a group the same mark where itwas an objective to learn that group effectiveness is theoutcome of the contribution of all.Give the group a mark to distribute as they determine.For example, if the group report was given a mark of60 per cent and there were 4 members, give the group240 (4 x 60) to divide up. This will be best managed ifyou have forewarned the group and assisted them withwritten criteria at the onset as to how they will allocatemarks. An alternative is to have members draw up acontract to undertake certain group responsibilities orcomponents. Components may be marked separately,or students may be given the task of assessingcontributions themselves.Enhance the reliability of this form of assessment byconducting short supplementary interviews withstudents (e.g. What was your contribution? How didyou reach this conclusion?) and build project-relatedquestions into any final examination.

USING TECHNOLOGY IN ASSESSMENT

Computer technologies can be used to support assessmentand we suggest you explore the facilities that are likely tobe available to you in your own institution. There areseveral ways in which technology can be used. Theseinclude:

As a management tool to store, distribute and analysedata and materials. An assessment system should beintegrated with larger systems for curriculum manage-ment such as processing of student data and delivery ofcourse materials.

160

As a tool in the assessment process such as for themarking and scoring of tests. Answers from objective-type tests can be read by an optical mark reader andresults processed by computer. However, moreelaborate tools are now available to assess studentswork directly. Software can be purchased that enablesyou to prepare, present and score tests and assign-ments. You should check to see if your institution has alicence for some of these software products.As a resource for student learning and assessment.Basically, this involves students using technology toprepare and present work for assessment. Somesimple examples include students preparing essaysusing a word processor or completing tasks using aspreadsheet application and submitting their work viae-mail. E-mail can also be used to provide amechanism for the all-important feedback processfrom the teacher or from other students if collaborativegroup work or peer assessment is being used.

We recognise that information technology and telecommu-nications can be helpful and positive tools or resources forassessment. But we also have serious reservations aboutthe way technology is being used as a tool in theassessment process. This is because the technology is sowell suited for the administration and scoring of objective-type tests of the multiple-choice or true/false kind. We areseeing something of a resurgence of this kind ofassessment in higher education with all of the well-knownnegative influences this may have on learning when itemsare poorly constructed or test only recall. All we can dohere is urge caution, use good-quality test items, and toalways ensure that students receive helpful feedback ontheir learning.

FEEDBACK TO STUDENTS

Major purposes of assessing student learning are todiagnose difficulties and to provide students with feed-back. Several approaches to doing this have already beenidentified in this chapter and some of the methodsdescribed readily lend themselves to providing opportu-nities for feedback. To be specific:

161

use structured written feedback on essays;provide immediate feedback on technical, interperso-nal, or oral skills as an outcome of direct observations,orals or practical assessments; anduse self-assessment which includes feedback as part ofthe process.

Some guidelines for giving feedback include the following:

keep the time short between what students do and thefeedback;balance the positive with the negative;indicate how the student can improve in specific ways;encourage students to evaluate themselves and givefeedback to each other; andmake the criteria clear when setting work and relatefeedback to the criteria.

REPORTING THE RESULTS OF ASSESSMENT

In many major examinations you will be required to reportthe results as a final mark or grade based on a number ofdifferent assessment methods. What usually happens is thatmarks from these different assessments are simply addedor averaged and the final mark or grade awarded, Simplethough this approach may be, it can introduce seriousdistortions. Factors contributing to this problem may bedifferent distributions of marks in each subtest; varyingnumbers of questions; differing levels of difficulty; and afailure to appropriately weight each component.

The answer is to convert each raw subscore to astandardised score. This is not the place to do more thanalert you to the need to do so and refer you to a text oneducational measurement or to advise you to enlist the aidof an educational statistician, who can usually be found bycontacting the teaching unit in your institution.

GUIDED READING

There are many useful general texts on educationalmeasurement. An example is N. Gronlund’s and R. Linn’sMeasurement and Evaluation in Teaching, Merril Press,Bellevue, Washington, 2000. Most will have useful discus-sions of broad assessment considerations such as objec-tives, planning, reliability, validity and scoring, and also

162

will provide a wide range of examples of test items that youcould use as models for your own tests.

Assessment Matters in Higher Education by S. Brown andA. Glasner, SRHE and Open University Press, Buckingham,1999 is another general overview text that we recommendbecause of the many examples relevant to highereducation.

We can also recommend Student Assessment in HigherEducation by A. Miller, B. Imrie and K. Cox, Kogan Page,London, 1998, Assessing Student Learning in HigherEducation by G. Brown and others, Routledge, London,1997 and Chapter 9 in Teaching for Quality Learning atUniversity by J. Biggs, SRHE and Open University Press,Buckingham, 1999.

A useful adjunct to this chapter is Assessing ClinicalCompetence at the Undergraduate Level by D. Newble;ASME Medical Education Booklet No 25, ASME, Edinburgh,2000.


D. Boud (1995). Enhancing Learning Through Self Assess-ment, Kogan Page, London.

S. Case and D. Swanson (1996). Constructing Written TestQuestions for the Basic and Clinical Sciences, 2nd edition,National Board of Medical Examiners, Philadelphia. It isavailable online at: http://www.nbme.org/new.version/item/htm.)

R. Harden and F. Gleeson (1979). Assessment of medicalcompetence using an objective structured clinical exam-ination (OSCE) Medical Education, 13, 41-54.

J. Norcini (1999). Standards and reliability: when rules ofthumb don’t apply. Academic Medicine, 74, 1088-1090.

163


9: PREPARING LEARNING ANDTEACHING MATERIALS AND USINGTECHNOLOGY

INTRODUCTION

This chapter has proved to be challenging to write. Thereasons for this are not hard to understand – there is nowsuch rapid development in the application of computersand information communication technologies in highereducation that much of what is said today is out-of-datetomorrow! On the other hand, it is fair to judge that as onelooks around our campuses, for much of the time mostteaching can still be described as ‘traditional’with studentgroups of varying sizes meeting with a teacher for a setperiod of instruction. While we would not wish to see all oftraditional teaching preserved for its own sake, it isnevertheless the case that it is in these settings, as well asin more contemporary approaches to learning andteaching, that we find a continuing need for assistance withsuch fundamental issues as using an overhead projectorproperly and preparing well-designed handouts.

We have addressed the challenge in three ways. First,from the previous edition we have updated material onsome of the more basic technologies and retained a focus inthis chapter on materials and technologies rather than onapproaches to teaching. Teaching approaches are pre-sented elsewhere in this book, for example, in the chapterson small groups and problem-based learning. Second, wehave provided introductory ideas on using informationtechnologies, and finally, we have distilled some principlesof good practice that we believe apply to the use of alltechnologies and that provide a benchmark against whichto evaluate what you are doing with your students.

BASIC PRINCIPLES IN PREPARING LEARNING ANDTEACHING MATERIALS

In your teaching career you will use quite a wide range ofteaching materials and technologies. How you mightproduce and use them is the focus of this chapter. Thefundamental criterion for judging the effectiveness of yourteaching material is its audibility and/or visibility. If thatseems too obvious to warrant mention, have a look at someof the materials used by others: overheads and slides withexcessive amounts of tiny detail that cannot be read on thescreen; complicated Web pages that look like art showsand take for ever to load on your computer; and fadedhandouts that cannot be read. Exaggeration? It does

166

happen! When it does, it seriously interferes with theeffectiveness of learning. Attention to the way in which thematerial is produced and how it is used in teaching willeliminate many of these problems.

Whether you are preparing a simple handout or multi-media materials, there are some basic principles that canbe incorporated into your design and preparation that willenhance the quality and effectiveness of the material.

Relevance

Materials should be relevant to the purpose for which theywere created and to the students’ level of understanding ofthe topic. Complex handouts distributed at the end of alecture and never referred to by the teacher are classicoffenders of this principle.

Linkage

An introduction is usually required to establish the purposeof the material and to link it with what it is reasonable toexpect students to know already.

Simplicity

Simplicity in the use of language and design, the avoidanceof needless qualifications and the use of suitable abstrac-tions of complex situations can be positive aids tounderstanding. For example, a simple line diagram maybe more helpful in an explanation than a full-colourphotograph or a complicated computer graphic.

Emphasis

Emphatic ‘signs’ can be incorporated into all teachingmaterials to stress important ideas, to indicate a change inthe development of an argument, or to identify newmaterial. Examples of emphasis include: headings andunderlining in print; the use of colour and movement inWeb pages; and statements such as ‘this is a major factor’or ‘to summarise these issues’ on a sound recording of alecture.

167

Consistency in the use of pattern and style

Students acquire a ‘feel’ for the particular style you use topresent material. Needless changing of style is only goingto confuse them. In the increasingly corporatised world ofmodern education, many institutions have now adopted ahouse style. You should check for its application to thepreparation of your teaching materials.

TYPES OF LEARNING AND TEACHING MATERIALSAND AIDS

With these principles in mind, the preparation and use ofseveral common types of teaching materials and aids willnow be described. These are:

1234567

The overhead projectorThe 35 mm slide projectorThe video projectorThe whiteboard and blackboardVideo and filmPrinted materialsMaterial on the World Wide Web

This list is by no means exhaustive. In keeping with thegeneral thrust of this book, the intention is to get youstarted and to help you develop some confidence in thebasic aspects of your teaching work.

1. THE OVERHEAD PROJECTOR

This valuable visual aid can project a wide range oftransparency materials and silhouettes of opaque objectson to a screen positioned behind the teacher. Because itcan project both written and diagrammatic information, itreduces your need to engage in detailed descriptions andincreases the opportunities for discussion with students. Italso allows you to indicate material on the transparencywithout turning your back to the audience, an advantageover using a slide projector or a whiteboard.

The full benefit of the overhead projector will not berealised in your teaching unless you give careful attentionto three things: the preparation of the transparency, theway the projector is set up in a room or lecture theatre, andthe way you actually use it. We shall now turn to aconsideration of each of these matters.

168

Transparency preparation

Figure 9.1 shows what a transparency looks like. In its basicform, it consists of an acetate sheet mounted on to acardboard frame. Additional sheets of acetate on the sameframe are known as overlays. Overlays are particularlyhelpful to build up an idea as a presentation develops.Often, however, teachers dispense with the cardboardframe.

The following methods of making transparencies areavailable.

Felt pens: felt pens containing water-soluble or permanentink are available for making transparencies. Information isprinted or drawn directly on to an acetate sheet. Asuggested procedure is to place a piece of ruled paperunderneath the acetate as a guide and write on to the sheet.Another sheet of clean paper placed under your hand willprevent smudges from appearing on the acetate. Letteringshould be no smaller than 5 mm in height and preferablylarger. Use black, blue, brown or green pens for lettering,avoiding red, orange and yellow which are difficult to readat a distance.

Photocopying: plain-paper photocopiers will accept acet-ate sheets, enabling the production of transparencies at thetouch of a button. It is essential that the type of transparencysheet selected is suitable for use with the copier and thatthe original material is large and clear. Lettering, forexample, must be at least 5 mm in height. Avoid thetemptation to make overheads directly from books or fromtyped materials. In general, transparencies made in thisway will be invisible to most of the class and useless, unless

169

students are each given an exact copy of the transparencyas a handout. If you wish to use such material you shouldfirst make an enlargement.

Presentation packages: software presentation packages(such as PowerPoint) give you a means of producingmaster sheets for high quality transparencies. They alsoenable you to make and project transparencies electro-nically. If you are not already familiar with one of thesepackages, we recommend that you enrol in a short courseto learn how to use it well. You will discover that makingoverheads is only the beginning! You will also be able toproduce handouts, notes, and multi-media presentationsincorporating pictures, movies and sound as well as textand diagrams of many kinds. Your finished work can bepresented as slides and overheads using video projectorsor standard equipment, via the Internet, and on desktopcomputers. The time required to learn to use a packagewill pay handsome dividends and open up a range ofpossibilities for you to make high quality materials.

When making overhead transparencies using this kind ofsoftware, remember the principle of simplicity – avoid therisk of overpowering your students with complex type-faces, distracting background designs and inappropriatecolours. Learn some simple concepts of presentationdesign as well as how to master the technology.

Other uses of the overhead projector

There are other, less orthodox ways in which you can usethe overhead projector. Silhouettes of cardboard cut-outsor solid objects can be projected on to the screen. Thesemay be co-ordinated with a prepared transparency.Transparent or translucent materials such as liquids in testtubes or biological specimens mounted on, or contained in,clear containers can also be prepared.

The guidelines listed in Figure 9.2 should be kept in mindwhen preparing an overhead transparency.

170

Setting up and using the overhead projector

In some situations you will have flexibility in setting up theprojector and screen. It is usual to place the projector sothat it is adjacent to the lectern or table from which you areworking. Ensure that the projected image is square on thescreen and free from angular and colour distortions.Angular distortions in the vertical axis can sometimes beovercome by tilting the top of the screen forward. Colourdistortions, such as red or blue in the corners of theprojected image, can usually be remedied by making anadjustment to the lamp. A control for this is often inside theprojector. It is important to turn the electricity off at thepower point before the adjustment is attempted.

Whenever a projector is moved, or before a presentationis commenced, the focus and position of the image must bechecked. Once this is done, it is usually unnecessary to lookat the screen again, particularly if you use a pen or pointerdirectly on the transparency. This enables you to maintaineye contact with students. If you wish to mask out part of thetransparency, place a sheet of paper between the film andthe glass stage of the projector. The weight of thetransparency should prevent the paper from moving orfalling away.

Remember to allow students plenty of time to read whatyou have projected. Many teachers find this difficult to do.One way is to read carefully the transparency to yourselfword for word. As well, make sure that anything you haveto say complements the transparency. Do not expect

171

students to listen to you and to look at something on thescreen that is only vaguely related to what is being said. Itis advisable to have the lamp on only when a transparencyis being used in your teaching otherwise the projectedimage or the large area of white light will distract thestudents’ attention.

2. THE 35 MM SLIDE PROJECTOR

Much of what has been said about the overhead projectorapplies to slide projection. However, you will recognizethat there are important differences between the two andthat one of these is that full-colour images can be used inslides. This may be an advantage but with some material itmay also be a disadvantage, unless the students’ level ofunderstanding is sufficient to enable them to see what isrelevant and pertinent in the material you are using. Slideinterpretation can be aided by including in the photographan appropriate reference point or a scale.

Slide preparation

The major error in slide making is to assume that legibilityin one medium, such as a table in a book or a journal,ensures slide legibility. Slides made from printed materialsfrequently contain too much detail and fine line work toenable them to be projected satisfactorily. This means thatyou may need to have artwork redrawn and new letteringadded. Check any slides in your possession for legibility. Auseful rule of thumb is that a slide which can be readwithout a magnifier is generally satisfactory. A bettermethod is to go with a colleague to a large lecture theatre,project your slides and check to see if all details are legibleand understandable at the rear of the auditorium.

When making slides, avoid the temptation to put all thedetails into the slide. If it is important for students to have allthe details, provide these in a handout so that they can referto it and keep it for reference. This ensures that they haveaccurate information on hand.

Photographers will advise you on the different processesavailable to produce your slides. These processes willusually include simple black-on-white slides, colour slidesand diazo slides (white against blue, green or red

172

backgrounds, the blue being preferred for clarity). Yourinstitution may also be able to help you produce computer-generated colour slides of high quality. Another attractiveand inexpensive way to prepare slides is to obtainnegatives (white-on-black) and colour the white sectionsin by hand using coloured marking pens. The possibilityexists for using separate colours to highlight differentpoints on the slide. Whatever you choose, try to achieve adegree of consistency by sticking to one type of slide.Guidelines for the preparation of effective slides are givenin Figure 9.3.

Setting up and using the slide projector

Slide projection equipment is normally part of the standardfixtures in a lecture theatre or seminar room so thequestion of setting up does not usually arise. If it does,locate the projector and screen with care to give the bestview to students and so that it is convenient for you tooperate the projector and room lights with a minimum offuss. A remote control device will be an invaluable aid.

Slide projection

Before loading your slides into a carousel, carefully planthe sequence of their use. If your teaching is to beinterspersed with slides, consider using black slides toseparate your material and to avoid having to keep turningthe projector on and off or leaving an inappropriate slideon view. Black slides are simply pieces of opaque filmmounted in a slide frame to block off light to the screen andcan be easily made from exposed film. If you plan to use thesame slide on more than one occasion during a

173

presentation, arrange to have duplicates made to save youand your students the agony of having to search back andforth through a slide series.

It is essential to have your slides marked or ‘spotted’ forprojection (see Figure 9.4). As a check, the slides should beupside down and emulsion side (i.e. the dull side) towardsthe screen. When showing your slides, it is rarelynecessary to turn off all the lights. Remember that studentsmay wish to take notes and so you should plan to leavesome lights on or to dim the main lights. Further advice onusing slides is given in Chapter 4 on presenting a paper ata conference.

3. THE VIDEO PROJECTOR

This exciting device enables you to project a variety ofmaterials from a computer onto a screen for large andsmall group viewing. These materials include videos,broadcast television, slides and overheads, multi-mediapresentations, computer output, and Internet displays.When it is professionally set-up, supported, and used, thevideo projector is an outstanding presentation tool.Regrettably, this is often not the case. In our view, thecurrent situation with video projectors is one that must beapproached with caution as it is a good example of theembarrassing immaturity of much educational technology.If you doubt this judgement, have a close look at thesystems currently in use with cords and cables every-where, the need for backup computers, incompatiblesoftware and systems, the risk of system crashes, and so on!It is probably unwise to rely on a video projector systemunless you are very familiar with its use and even then bewell prepared with back-up resources.

174

Preparation

Your preparation involves four distinct matters: prepara-tion of your material, familiarisation with the use of theequipment, arrangement of the teaching room, andproduction of back-up resources.

The preparation of your material is covered elsewhere inthis chapter, keeping in mind the simple rule that whatevermaterial is used, it must be clearly visible and audible! Weurge you to consider producing back-up resources andalternative teaching strategies in case something should gowrong. For example, if you intend to be teaching in anunfamiliar environment or place, take overhead transpar-encies.

Equipment preparation can be broken down into under-standing and preparing of the computer hardware andsoftware, the operation of the projector itself, and the waythe projector and computer are linked together. These arematters that need to be addressed well before any use ofequipment is undertaken before an audience. To believeyou can sort matters out in front of an audience is to invitedisaster. If you cannot get tuition or expert assistance, taketime to study equipment manuals and try out theprocedures well in advance of any teaching or presenta-tion commitment.

As with all projection equipment, you will need to giveconsideration to siting your video projector in relation tothe room. In particular, review the position and focus ofimages on the screen, the level of illumination in the area ofthe screen, and the position of equipment and where youwill be speaking in relation to the audience.

4. THE WHITEBOARD AND BLACKBOARD

The whiteboard is a ubiquitous presentation tool found inmany meeting rooms as well as in classrooms these days.

The principles of board use and preparation are outlinedbelow. Do take care to use the correct pens with awhiteboard as some can ruin its surface. Also take carewhen cleaning a whiteboard. A dry cloth is often adequatebut sometimes you may need to use water, detergent orperhaps methylated spirits. Never use an abrasive cleaner

175

as it will scratch the surface and do irreparable damage tothe board.

The colour of the pens you use is important. Black, darkblue and green are best. Avoid yellow, red and lightcolours, as these can be difficult to read from a distance.

The blackboard (which these days may be green) is still acommonly used visual aid and the one that you may usefrequently, unless you rely exclusively on the overhead orvideo projector.

Few teachers give much thought to the material that theyput on the board or to the way they use it. This is a pity. Theresults of the work are often ugly and indecipherable.Well-planned and well-used board work is a delight to seeand is a valuable ally in presenting information accuratelyand clearly to your students.

Preparation

It is important to think ahead about your use of the boardand make suitable notations in your teaching notes. Planyour use of the board by dividing the available space into anumber of sections. Each section is then used for a specificpurpose such as references, diagrams, a summary of thestructure of the lecture, and so on.

Using a board

Some guidelines for using a board are given in Figure 9.5.

5. VIDEO AND FILM

Video gives you the opportunity to experiment with novelapproaches to producing teaching materials, particularlynow that relatively cheap cameras are available as well aspresentation packages which enable you to integrate yourvideo in a multi-media presentation. However, you shouldalso become familiar with the range of suitable commer-cially or Web-based materials before embarking on acareer as a producer. You will find that several subjectareas are well catered for in this regard.

176

Although the uses of video and film are similar, video doesoffer you several additional advantages such as ease ofproduction and relative cheapness. These have tended tomake this medium more popular and flexible than film.

Using video and film in teaching

As with many teaching aids their uses are restricted onlyby your imagination and by the resources at your disposal.Some of the potential uses of video and film are describedbelow.

177

As introductory material: video and film can be used at thestart of a course of study to stimulate interest, to provide anoverview and to form a basis for further learning andteaching. For example, a film on the effects of cigarettesmoking could be used as an introduction for a study oflung cancer.

As a major source of information: a constant flow of newideas, techniques and procedures are a fact of life in mostdisciplines. Video and film can be used to disseminate thisnew information to your students or to professionalmeetings with which you may be involved. A furtheradvantage of these media is that they can provide theviewer with vicarious experience where this might bedifficult or dangerous to obtain at first hand.

As a means of modelling: this use is similar to the previoususe, but you may find it helpful to produce material whichdemonstrates a technique or procedure in a clear step-by-step manner that students can watch and emulate at theirown pace. An example might be a demonstration of how toconduct a clinical interview with a patient.

As a stimulus for discussion: short open-ended sections ofvideo or film can be made to stimulate discussion amongstudents. These are known as ‘trigger films’. Studentsrespond to the material as it is presented and both thestimulus and their responses are then discussed. This wehave found to be valuable for starting discussion aboutattitudes dealing with emotional situations. Sometimes it ispossible to locate suitable stimulus material in old films thatwould otherwise have no use.

As a means of distribution and relay: carefully placedvideo-cameras can be used to distribute pictures to aseparate viewing room or even to relay them to remotelocations. An obvious example of this is their use inoperating theatres to enable a large number of students towitness an operation.

As an information storage system: video has a role to playin storing information for later teaching or for research use.For example, a recording can be made (with permission)by a student of a patient interview which can be reviewedlater with a tutor at a convenient time and place.

178

As a means of assembling visual and audio information: inthe past, video and film have been used to assemble avariety of information into one ‘package’. Film clips, stills,models, interviews, recorded sounds and graphics can berecorded, assembled and edited to make a teachingprogramme. Today, presentation packages provide anaccessible tool to achieve the same kind of outcome.

As a magnification medium: many teachers find that videois a handy tool to ‘blow-up’ the action or to display picturesof a demonstration. These can, of course, be recorded ifneeded for subsequent use.

These examples of video and film use are by no meansexhaustive nor are they mutually exclusive in theirapplication. For example, in teaching anatomy, video isused to magnify materials, to distribute and display this in alarge laboratory (thus ensuring that all students are seeingthe same thing) and sometimes to record the information asa resource for independent learning.

It is becoming less likely that you will be called on toproduce videos for use in teaching. However, if you do, werecommend that you review the guidelines on makingeducational videos contained in earlier editions of thisbook.

6. PRINTED MATERIAL

Books, journals, handouts and study guides carry a verylarge part of the instructional burden in teaching and willcontinue to do so in both paper and electronic media. Yet,often, surprisingly little thought is given by universityteachers to the design and use of these important materials.

Design

We strongly recommend Hartley’s book as a reference tohave beside you. Care needs to be taken in designing andpreparing printed materials. Over-organisation of the textdoes not help the reader and may actually interfere withlearning. You will find it helpful to standardise on layouts.For example, in a paper-based system, you may wish toinstitute a system of coloured papers for different kinds ofmaterial you prepare for students (e.g. white for lecture

179

notes, green for bibliographies, yellow for exercises). Thebasic principles for layout and design of printed materialsare outlined in Figure 9.6.

The variety of fonts available in personal computersoftware makes it necessary to select with care. Have alook at the typographical layouts in better-quality news-papers and journals for ideas that you can put into practice.

Using printed material

Handouts can serve a number of useful purposes in yourteaching, but this medium is frequently misused becausethe material is often simply distributed to students and thenquickly forgotten. Remember that you can producehandouts with the presentation software to support a formallecture and that you can distribute them via the Internet.

Handouts can be used by students as a note-taking guide toa lecture. Supplementary information, or perhaps a copy ofa paper you think is important, can also be given in ahandout.

How you use the handout in your teaching is a crucialmatter, We suggest that your students’ attention bedirected to the handout by discussing a particular

180

definition, reading through a brief list of points withstudents, or asking them to fill in some part of it withadditional information. If your students have to use thehandout in the teaching session, it is likely that they willremember it and not simply file it away to be forgotten.

Prescribed reading

Prescribed reading of textbooks and journals is anothermatter that warrants your careful attention. Some teachersswamp their students with lists of books and articles to beread and give little thought to how students might managethe task. If you want the students to undertake somereading, then consider the following points:

What are students expected to achieve by undertakingthe reading? (Make this purpose clearly known to thestudents.)How will the reading be followed up in subsequentteaching?Will the recommended reading be readily available inlibraries, through bookshops, or on the Web?How can the reading be usefully organised? (Arrangethe material in a logical fashion, indicate why an itemhas been listed and what is especially important aboutit.)

7. PUBLISHING MATERIAL ON THE WORLDWIDE WEB

Preparing and publishing your own material on the Webshould not be too difficult a task, particularly if you canenlist the assistance of locally available expertise to get youthrough the main technical issues. If you cannot locate suchdirect assistance then we suggest you either visit your localbookshop or computer retailer for the most recent booksand software on the topic or seek advice on a current Website for on-line help,

As the ‘Internet revolution’ matures the tools you need arebecoming simpler. At the time of writing, the followinggeneral directions were valid. All you will need is acomputer with a connection to the Internet and a ‘webbrowser’, which is the software that allows you to navigatethrough the different sites on the Web.

181

The basic steps are:

Prepare your material. Your word processor may havean automatic ‘Save As . . . HTML’ option under the Filemenu. If so, your work is now ready for placement onthe Web. This is the easiest option for preparing yourmaterial. The drawbacks are that some formatting maybe lost and some punctuation marks altered. You alsomiss out on using more versatile Web building toolsthat can offer greater options (images, page colour,links to other sites etc) and thereby influence the wayyou may wish to present your material.Another option for preparing your work for the Web isto access free on-line Web page builders availablefrom sites such as Geocities (www.geocities.com) orThe Globe (www.theglobe.com). Or you can purchaseone of several entry-level web page design softwarepackages such as Adobe Pagemill or NetscapeComposer. These packages come with helpful paper-based guides and they will enable you to save yourwork in a format that is ready for publication on theWeb.Once your material is ready for the Web you need toplace it on a ‘server’, that is a computer that isdedicated to the task of allowing viewers to accessyour page from anywhere, at anytime. For teachinguses, server space will usually be available from yourinstitution. Space is also available through ISPs (InternetService Providers). Alternatively, you may wish to takeadvantage of free server space offered through manysites on the Web. You will pay a price, though, usuallyin the form of a banner advertisement that displayswhen someone views your page!

You’re done! Details on all the points above are availableon the Web itself. If you are using or plan to use NetscapeComposer, Netscape provides an excellent step-by-stepguide (< http://home.mcom.com/browsers/createsites/index.html>).

As with all teaching preparation, you need a clear idea ofwhat you are trying to achieve and for whom you arepreparing the material. Assuming the material is for yourstudents you could provide them with a diversity ofresources to assist them with their learning, such as links

182

to helpful learning resources, assignments and generalfeedback, reading material, examples of exemplarystudent work, and so on. Alternatively, you may beplanning to teach interactively via the World Wide Web.In both of these cases, we urge you to review the currentlyavailable literature on the topic, some of which is identifiedin the Guided Reading section.

USING TECHNOLOGY IN LEARNING ANDTEACHING

New technologies are having a significant impact onlearning and teaching in higher education and willcontinue to do so. As we have already seen in this chapterand elsewhere in the book, computer and communicationtechnologies can enhance a wide range of traditionalteaching activities from the production and distribution ofmaterials to the ways in which learners and teachersinteract with each other. But these are examples of the waysin which technology replicates traditional teaching.

It is now clear that the forces of change are combining tomove us to different ways of learning and teaching wherewe will see more of the following developments:

students becoming more active and independent intheir learningstudents working collaboratively with each otherrather than competitivelyteachers becoming more designers and managers oflearning resources, and guides for their studentsrather than dispensers and controllers of informationrapidly changing curriculum content reflecting free-dom to access a diverse range of ever-expandingresources for learningmore effective assessment with a growing emphasis onassessment for learning.

How can you respond to these new and challengingdemands and where can you learn more? Of course wehope that the material in this book will assist you with thebasics of learning, teaching and assessment issues. But howcan you learn more about the technologies (if these arenew to you) or how can you keep abreast of developments?These matters are well beyond the scope of this book andso we hope the following Guided Reading will be helpful.

183

GUIDED READING

For those new to technology, and in particular thetechnologies applied in higher education, we recommendtwo complementary books. The first is by A. Warren et al.,Technology in Teaching and Learning: An IntroductoryGuide, Kogan Page, 1998. The second is by P. Maier et al.,Using Technology in Teaching and Learning, Kogan Page,1999. The first book takes the reader through thefundamentals of using computers and the second bookexplores ways in which computers can be used to supportthe teaching of large groups, to deliver learning resourcesto students, and for communication between students.

If you want to go further, and explore more of using theInternet in your teaching, we suggest I. Forsyth, Teachingand Learning Materials and the Internet, Kogan Page, 2000.

To maintain your currency in the uses of technologybeyond the material in these books we urge you to monitorthe literature in books, journals and especially in theelectronic resources of the kind available on the WorldWide Web.

There is rapidly growing number of books, as well asresources on the Web, that can assist you with some of theeducational issues of using technology in education. Wesuggest the following three books:

A. Inglis, P. Ling and V. Joosten, Delivering Digitally:Managing the Transition to the Knowledge Media.Kogan Page, London, 1999.R. Phillips, The Developer’s Handbook to InteractiveMultimedia, A Practical Guide for Educational Applica-tions. Kogan Page, 1997.D. Brooks, Web Teaching. Plenum Publishing Corp,New York and London, 1997.

If you are concerned to evaluate materials and educationaltechnologies we suggest M. Tessmer, Planning andConducting Formative Evaluations. Kogan Page, London,1993. This is an interesting mixture of useful guidance onplanning evaluations, evaluating materials, and the wholenotion of formative evaluation. Hartley’s book (see below)is also helpful on evaluating materials.

184

Books referred to in this chapter:

J. Hartley, Designing Instructional Text (3rd edition), KoganPage, London, 1994, is highly recommended for preparingtext-based materials (books, manuals, handouts, computer-generated or stored text). We used Hartley whenpreparing this book.

185


10: THE EVALUATION OFLEARNING AND TEACHING

INTRODUCTION

There is an increasing demand that university teachersprovide evidence of their teaching effectiveness forpurposes of appointment, tenure and promotion. As aresult we have included this new chapter which we hopewill help you in three important ways.

Provide you with information and resources that willassist you to evaluate your teaching and the learning ofyour students.Guide you in ways that will assist you to make good useof the information you create through your evaluativeactivities.Arm you with ideas on how to improve the practice ofevaluation in your institution.

The third is especially important and is often overlooked.In our experience, as many difficulties in evaluation arecreated by the implementation of poor policies andpractices as by the processes of collecting and presentingevaluative information. One poor practice is an obsessionwith quantification. This has led to an over-emphasis onthose things that can be most easily counted, such asstudent ratings of a teacher's behaviour, and an under-emphasis on those areas of academic work less easy toquantify such as learning processes or advising students.

There is another important factor here as well. Just as theways we go about assessing our students will directlyinfluence their learning behaviours, so too will the waysinstitutions evaluate their teachers drive teacher beha-viour. For example, the strong emphasis on research inmany universities is, in part, a direct consequence of theway we evaluate and reward this academic activity bypromoting people on the basis of their research output andgrant income generation. This may compete with activitiesrequired to perform teaching duties at a high level.

Before we address these matters in more detail, we want tooutline the context in which we are presenting ideas to youand to clarify some important concepts.

188

THE CONTEXT OF EVALUATION

Evaluation and learning

Evaluation is an important part of the process of learning. Itis to do with finding out from our students about the qualityof their learning and obtaining information about theeffectiveness of our teaching. How can we do this? Weshall be suggesting some ideas about this after we havebriefly reviewed another important side of evaluation -accountability.

Evaluation and accountability

One of the most dramatic shifts in higher education practicein the past decade has been the move towards account-ability. By ‘accountability’ we mean a demand to provideclear evidence of what is being done in higher educationand of the outcomes of learning and teaching. Thisevidence is then used in a variety of ways, one of whichof major interest to you is decisions about academicpromotions and contracts.

At the national level, governments are generally underpressure to account for the way public funds are used andso exert a corresponding pressure on institutions toimprove their effectiveness. Owners and trustees exertsimilar pressures in the non-government sector of highereducation. Institutions are thus required to gather dataabout learning and teaching and present it as evidence fortheir claims of effectiveness and quality.

Institutions, in turn, have exerted accountability demandsonto faculties, teaching departments and individuals. Theycommonly require that courses be evaluated on a regularbasis and that teachers evaluate their teaching and use theinformation obtained for both the improvement of teachingand of courses.

Evaluation: some definitions and principles

Evaluation is a process of obtaining information forjudgement and decision making about programmes,courses and teachers. Assessment, a term which issometimes used interchangeably with evaluation, is aboutobtaining information for judgement and decision making

189

about students and their learning. However, the results ofan assessment of student learning is a very important partof evaluation.

We are sure you will be familiar with some of the wayscommonly used to gather information – questionnaires andinterviews for teaching evaluations, and assignments andexaminations to assess student learning. Strategies forjudgement and decision-making are less well-developed,however, and we will look at these later in the chapter.

You should keep in mind two broad intentions behindevaluation. The first is formative evaluation. This isintended to assist in change, development and improve-ment of teaching. The second is summative evaluation. Thisis used to make decisions such as whether to promote orre-appoint a teacher.

Whatever the intentions of an evaluation, you will find ituseful to keep in mind that there are several sources ofevaluative information and methods you can use to get thisinformation. These matters are summarised in Figure 10.1and Figure 10.2

190

Different methods are available to gather information fromthese sources. For example, if you are particularlyinterested in the students’ experience, you may decide touse several different methods including diaries, ques-tionnaires and focus groups. Some of these methods arelisted below in Figure 10.2.

Space does not allow us to explore all of these sources andmethods, which of course can be used in a wide range ofcombinations. For more help, we recommend you consultsomeone in your institution’s teaching unit or review thereferences provided at the end of the chapter.

However, in deciding among the options in the table, youneed to be aware that two fundamental characteristics ofevaluation are validity and reliability. Other importantcharacteristics are the practicality of an evaluation and, ofcourse, its acceptability to all those involved.

Validity refers to the truthfulness and appropriateness ofinformation provided as evidence of learning andteaching. For example, high levels of student achievementmay not be a valid indicator of teaching competencebecause of the problem of identifying the relativecontribution of teacher, student effort, library resources,students’ peers, and so on. On the other hand it is a validindicator of learning. Students can provide valid feedbackon the availability of resources and teacher behavior, forexample, because they observe and experience thesethings as part of their course.

191

Reliability refers to the extent to which informationprovided is dependable and consistent. For example,information about a teacher based on the results of just onestudent survey will not be as reliable as informationderived from several surveys conducted over severalyears and from a representative range of classes taught.The reliability of an evaluation may also be enhanced ifdifferent, but valid methods, are used in combination.

Now, armed with this background knowledge, how mightyou go about evaluating your teaching? There is oneimportant preliminary matter to consider - planning.

Planning evaluation as part of your teaching

Among the things you need to think about are thefollowing:

Determine what are your institution’s formal require-ments for evaluation. For example, is there arequirement that you should gather student feedbackon a regular basis for curriculum development or forpromotion or tenure?Make contact with the staff of your teaching unit whowill at least be able to advise you if not provide directevaluation services to you.Draw up your own plan of evaluation. As we havealready suggested, evaluation is an important elementof good teaching and so it is something you should bedoing all the time.

Matters you will need to consider in your plan are how andwhen you will evaluate your teaching, how you willevaluate student learning and what you will do with theinformation you gather. You must have some plan to use theinformation in ways to improve or develop learning andyour teaching otherwise there is little point in doing it at all.One way of using information will be to incorporate it intoan on-going record of your work known as a TeachingPortfolio. We explain this below. Another important way ofusing information is to ensure you give your studentsfeedback on the outcomes of your evaluations and what it isplanned to do with the results. Practical ways in which youcan communicate this kind of information are through yourpersonal or departmental Web page, or by postinginformation on the student notice board.

192

THE EVALUATION OF LEARNING AND TEACHING

Evaluating learning

As with all evaluation, the process of evaluating learningconsists of two major elements: gathering information andthen making judgements based on that information. Whatsources of information are there about learning? Thereliability of your evaluation of student learning will beenhanced by your judicious use of more than one validmeasure of learning. So, what is available? The first, andmajor source of information about student learning, will bethe results of your programme of student assessment - theexamination results; assignments and projects; reports;clinical notes, and other products; and observations ofstudent learning.

In our experience a great deal of useful information fromstudents is overlooked. For example, it is necessary to gobeyond the scores and grades from tests, and to askquestions about learning, for example:

What are common errors - and how can I addressthese in my teaching?In what areas have students shown particularstrengths, weaknesses, interests and why might thisbe so?What misconceptions are evident in students’ work -and how can I address these in my teaching?What levels of intellectual achievement are revealed instudents’ work: for example, do they simply appear torote learn or is there evidence of analysis and criticalthinking?What feedback will I give students and how will I dothis?

This process of questioning and thinking about one’steaching is a fertile way of developing an understanding ofstudent learning so that you can modify teaching andprovide additional assistance to students if this is indicated.

A second way of evaluating learning is through well-designed and administered evaluations of teaching. Thereis a considerable body of research evidence that shows thisto be a valid approach provided that the questions askabout factors related to learning and not something else!

193

Information about how students are progressing inachieving the goals of your teaching can be found inquestionnaire items such as:

I have understood the concepts presented in thesubject.I have a positive attitude to this subject.My ability to work independently has been increased.My ability to think critically/solve problems/performclinical tasks/etc. has increased.

Apart from formal questionnaire surveys, you will find thatevaluation by brainstorming an issue with your students(such as ‘are the course objectives being achieved?’) canbe a very powerful learning experience for all involved!

The third way you can obtain information about theprocesses of learning is by using tools such as CATs(Classroom Assessment Techniques); two examples ofwhich we have already mentioned in Chapter 2.

Another CAT has been developed to help teachersevaluate student reaction to exams and tests and toimprove these as effective learning and assessmentdevices. Called the Exam Evaluation, the procedure forusing this evaluation tool is:

Focus on a type of test you are going to use more thanonce.Develop questions that you would like to ask ofstudents. You might also consider asking studentswhat questions they would like to be asked.Choose the questions and decide whether you will askthese at the end of a test or as part of a follow-upevaluation. Examples of questions are ‘Did you learnmore from one type of test than another?’ ‘What is itabout the test that accounts for this?’ ‘Was the test a fairassessment of your learning?’ ‘What are the reasons foryour response?’

In their book Classroom Assessment Techniques, Angeloand Cross describe 50 different CATs so an exploration ofthese is well worth the effort. CATs not only give you usefulinformation about learning, but research has shown howthey can have positive instructional benefits such asincreasing students’ perceptions of the quality of their

194

learning and the level of activity by stimulating classparticipation. One of the most important elements of goodteaching is feedback and CATs can help in this. Of course,it is essential you give your students feedback on theoutcomes of these exercises too! Remember one of our keyprinciples is that ‘Reliable and valid assessment of learningand giving helpful feedback on students' work is adistinguishing characteristic of good teaching.’

Evaluating learning in these ways will yield a rich variety ofinformation. But it may be lacking in one important respectand that is, it may not indicate ways in which you canimprove. To do that, you have to ask your students (andmaybe your colleagues too). There are a number of waysyou can do this that range from a frank discussion with anindividual student, or a group, to sophisticated studentrating systems.

Evaluating your teaching

As we have seen from Figure 10.2, questionnaires are onlyone way you can gather information about your teaching.Depending on what you wish to evaluate and why, we thinkyou will find it necessary to use more than one technique.For example, several of our colleagues use focus groupswhere issues are explored in depth with small groups.They also arrange to visit each other's classes forobservation and feedback. Increasingly, some are alsousing e-mailed questions and feedback from students as ameans of evaluation.

Examples of evaluation tools are given in various parts ofthis book. We also suggest, if you do not have ready accessto an evaluation service on your campus, that you consultone of the many books on the subject. Centra givesexamples of several questionnaires in his book and wethink you will find these helpful.

If you decide to use questionnaires we suggest givingcareful consideration to the following:

That the evaluation questions asked of studentsaccurately reflect the learning and teaching they haveexperienced; that is, they are ‘valid’.That students are able to answer questions based ontheir direct experience of teaching (for example, a first

195

year student will generally not be in a position to judgewhether a teacher’s subject knowledge is up-to-date,whereas a graduate student or colleague may well beable to make such a judgement).That when an individual is being evaluated forsummative purposes that only responses to questionsabout matters that the individual can reasonably beheld accountable for are used and reported on.

In recent years there has been considerable innovation inmedical teaching and the evolution of methods, such asproblem-based learning and computer-assisted instruc-tion, where teams of teachers are more commonlyresponsible for development and implementation. At thesame time there has been an expansion of teaching at thegraduate level where individual supervision of researchprojects is common. These different approaches toteaching demand different approaches to evaluation andthe points we have made above apply to these as well. Ifyou find yourself teaching in one or more of these situationswe think you need to be fully aware of the general issueswe have presented here and to also seek help andguidance from the more specialised literature (some ofwhich we comment on in the Guided Reading section) orfrom staff in your teaching unit.

Recommendations for using evaluations for develop-mental purposes (formative evaluation)

The outcomes of all your evaluations should be included inyour Teaching Portfolio. An important part of any portfoliois a section in which you complete a critical self-review, or‘reflection’, about your teaching. We say more about thisbelow.

As well as having a plan of action, the other majorrecommendation we can make is to discuss the outcomesof an evaluation with someone, perhaps an adviser orconsultant on teaching, or a trusted colleague or mentor. Inaddition, a frank discussion with your students will yieldtwo positive benefits. First, it gives them feedback on theoutcomes of the evaluation and about your plan to act on theinformation they have given you. Secondly, you will beable to clarify some areas and seek suggestions from themfor changes.

196

Recommendations for using evaluations for adminis-trative purposes (summative evaluation)

In this rather longer part we summarise some of thecurrently available ideas from research about usingevaluations for administrative purposes such as promotionor tenure. Because you will find that there is on-goingdebate and dispute about some of these recommendations,we have provided some references so that you can reviewthe findings and more detailed reasoning behind them.

It is more appropriate to document and describe yourachievements in teaching and to relate these to yourspecified duties and expectations than it is to pursuesome quantitative criterion, such as a ‘magic number’derived from evaluative data (e.g. a score of 6 on a 7-point scale). Several sources of valid evaluativeinformation are preferred over any single source.For example, a portfolio approach which combinespeer evaluation, student evaluation, and appropriateevidence from other sources is recommended (Magin,1998).If questionnaires are used, short rating forms withfewer global items are recommended rather thanforms containing a large number of heterogeneousitems. Examples of global items should relate to ateacher's responsibilities – ‘the stated aims wereachieved’, ‘accessible to students’, and ‘feedbackprovided promptly’ or an overall rating of effective-ness rather than to stylistic questions (‘I liked thepresentation techniques’ or ‘enthusiastic’) (Scriven,1981; Magin, 1998).Whatever kinds of statistical scores are used in areport on teaching, that only broad categories of finaljudgement be made (such as ‘outstanding’, ‘adequate’,and ‘unacceptable’; or ‘promote’ and ‘do not promote’(McKeachie, 1997).The circumstances under which evaluations arecollected and analysed are appropriate and rigorouslyapplied. Standardised procedures should be used forlegal and ethical reasons (d’Apollonia and Abrami,1997).Evaluations should be collected before final examina-tions (d’Apollonia and Abrami, 1997).You should avoid comparing teachers by usingnumerical means or medians. Comparisons of ratings

197

from different groups, even within a specific disci-pline, is a very dubious exercise because of a myriadof variables such as between-class differences amongstudents; teacher’s gender; different goals, learningand teaching methods, and assessment arrangements;different course content; and varying degrees ofdifficulty. Just as important as the mean or median, isthe spread of scores - the standard deviation – whichraises issues of how much variation in perception ofexperience exists within groups. Comparisons are notnecessary for personnel decision making. What isnecessary is evidence to allocate the teacher to one oftwo or three broad categories, such as ‘promote’ or ‘donot promote’ and this can be achieved by looking at theoverall distribution of student ratings on key questionssuch as teacher effectiveness (McKeachie, 1997).

A key observation made in McKeachie’s paper (p. 1218) canbe a guiding principle in using student evaluations forsummative purposes. He declares: “ . . . the basic problemis not with the ratings but rather with the lack ofsophistication of those using them for personnel pur-poses.”

BRINGING IT ALL TOGETHER: A FRAMEWORKFOR THE EVALUATION OF YOUR TEACHING

By now you will be aware of the complexity of evaluationand we hope that the following framework will help youthink about the ways in which you evaluate your teachingand also other aspects of your academic work. Forexample, our focus is on the documentation necessary toassist in a summative evaluation. However, this focus canincorporate both formative evaluation of teaching and theevaluation of learning because the process of thoughtfuldocumentation is likely to stimulate thinking about yourpractices which may, in turn, lead to improvements.

In a major publication on evaluation by Glassick et al., 1997,a broad framework is proposed which is intended to applyto all forms of academic scholarship, including teaching.Applied to teaching and learning, the main elements of theframework are:

198

establishing standards of teaching as scholarly workdocumenting teachingensuring a trustworthy system of evaluation.

Establishing standards

Teaching evaluation can be guided by the followingqualitative standards. You will recognise most of these asthey reflect the qualities of good teaching we keepreminding you about!

123456

Clear goalsAdequate preparationAppropriate methodsSignificant resultsEffective presentationSeeking feedback

1. Clear goals

Having clear goals is one of the elements of theorganisation of effective teaching identified by theresearch, but having clear goals is of little value if theycannot, realistically, be met. Consider the following:

Have you stated the goals of your teaching in terms ofexpected learning outcomes?Have you defined goals that are clear, realistic andachievable?

2. Adequate preparation

Preparation involves the identification and bringingtogether of resources and materials to support teaching.The following questions may help you here:

Do you demonstrate an understanding of existingscholarship in your field?Do you bring the necessary skills and understandingabout learning and teaching to your work as a teacher?Do you obtain and organise the resources necessary toachieve your goals?

199

3. Appropriate methods

Methods and procedures appropriate to the goals andyour context have to be chosen, applied effectively, andmodified and adapted as teaching proceeds. Methodsmake a great difference in teaching, from the methodsused to organise and present a syllabus, to learning andteaching methods, and the assessment arrangements. Thefollowing questions are relevant here:

Do you use methods appropriate to the goals?Do you apply these methods effectively?Do you modify your methods in response to changingcircumstances and feedback?

4. Significant results

Teaching will ultimately be judged by its results as well asthe quality and integrity of its processes. We often askabout learning outcomes in our questionnaires to students;whether the teacher stimulated their interest, whether theirability to work independently has been increased, and theextent of their understanding. The significance of theoutcomes of teaching might be judged by asking questionlike:

Did your students achieve your stated goals?Does your scholarship in teaching make a difference toyour students and to your field of work?Does your work open up new areas for developmentor enquiry?

5. Effective presentation

The importance of presentation in teaching, whether it ispresentation in a traditional lecture setting or thepresentation of learning materials using an electronictechnology, is of major significance. Teaching scholarshipshould also lead to presentation to colleagues, throughpublication and conferences, as well as presentation tostudents. The following questions are relevant here:

Do you use presentation methods appropriate to yourgoals and teaching context?Do you use appropriate forums to present yourteaching scholarship?Do you present with appropriate clarity and integrity?

200

6. Seeking feedback

The final standard involves the teacher’s continuinglearning about his or her work by actively seeking outfeedback from others so that the whole process of teachingcan be monitored and improved. Ask yourself:

Do you regularly evaluate your own work?Do you bring to this evaluation a broad range ofevidence?Do you critically integrate evaluation, reflect on it, anduse it to improve the quality of your teaching?

Documenting teaching: the Teaching Portfolio

The second part of the framework is documenting yourteaching. The purpose of documentation is to provide theevidence that can be used to judge the extent to which thestandards have been achieved. Our purpose here is tofocus on how you might record your academic achieve-ments. A valuable tool in working towards this purpose isthe teaching portfolio or teaching dossier.

Your portfolio might have three main parts:

A statement of responsibilities defining what you wererequired to do, or had hoped to accomplish. This is thebasis against which your work will be judged.A statement of your philosophy of teaching andlearning, preferably in relation to your institution’sgoals and plans. This might also include a very briefbiographical note outlining the breadth and depth ofyour scholarly work as a teacher.The summarised evidence of your achievements andeffectiveness as a teacher. Figure 10.3 gives anextensive list of items that might be appropriate. Thissummarised evidence can be organised using thestandards described above.

There are several reasons why we advocate the prepara-tion and maintenance of a portfolio. These reasons are:

Self-evaluation: the summary you prepare provides aninvaluable record of your teaching which assists you toreflect on your teaching over a particular period (say asemester or year) and to make improvements shouldthese be necessary.

201

Evidence for new appointments, tenure, annual reviewor promotion: institutional appointments and promo-tions committees increasingly demand evidence ofteaching accomplishments to assist them in theirdecision making. This reflects a worldwide emphasison better quality teaching. So it is very wise to havemaintained a portfolio for at least the past two or threeyears. A portfolio will also be very useful in yourannual review.Evidence in cases where your teaching is challenged:in these days of increasing accountability and staffappraisal there may be occasions where the quality ofyour work is challenged. Documentary evidencemaintained by you in your portfolio may proveinvaluable in defending your case.Fostering discussion and review of teaching: keeping aportfolio and encouraging others to do so will help tocreate an environment where discussion of teachingbecomes the norm rather than an unusual practice inyour department.

Do remember that a teaching portfolio is a summary ofyour major teaching activities and accomplishments; it is animportant adjunct to your curriculum vitae (CV). It is notintended that it include all the material listed in Figure 10.3.Accordingly, you should initially be comprehensive inyour collection of information for your portfolio and thensummarise the material when it is to be used for someexternal audience.

Creating, assembling and using your portfolio

The most important things to remember are to keepevidence of your teaching activities and to file away a copyof relevant materials, letters received, articles published,evaluations conducted, and so on, and that your portfolio isa summary. The evidence is the basis from which yourportfolio is constructed and the source from which anystatements you make in your portfolio can be verified. It issuggested that your portfolio might end up being betweensix and twelve pages long.

202

To compile your portfolio we recommend the followingsteps:

Clarify the purposes to which the portfolio will be putand choose appropriate criteria to describe andevaluate your teaching.Keep files of back-up material to follow the structureused in the portfolio. Remember, these materials arenot part of the portfolio, but are evidence if required.Prepare brief statements of explanation against each ofthe criteria selected. You should also add your ownbrief evaluation of the item and the steps you havetaken to modify your teaching in light of (say) feedbackreceived.Incorporate the portfolio into your CV.Finally, remember to constantly review your portfolioand keep it up to date. It is surprising how easy it is toforget the diverse teaching activities we undertake andthe feedback we receive. Remember too that yourportfolio is an important tool for learning about yourteaching. Use it for this purpose also!

Ensuring a trustworthy system of evaluation

The final part of the framework for an evaluation system isensuring that the system in place is ‘trustworthy’. Wecannot go into much detail about this here, but the followingquotation from Scholarship Assessed summarizes what wehave in mind:

“. . . successful evaluation would be a process with cleargoals for institutional and individual performance andadequate preparation for evaluators and candidates.Appropriate methods would be used and significantresults would advance the institution and individualtowards their goals. The process would be effectivelypresented and discussed as openly as possible in publicforums. Finally, reflective critique would keep evaluationflexible and open to improvement over time.” (p.51).

We do not expect that you will be able to change yourinstitution’s policies and practices overnight. But, bytalking about the characteristics of good evaluation withthe right people, you will have an influence upon bringingabout useful improvements for the advancement oflearning and teaching.

203

204

205

GUIDED READING

There is a wealth of material on evaluation and you maycare to check your library’s holdings for recent books andguides. A good overview of evaluation, the issues,methods, and resources is J. Centra, Reflective FacultyEvaluation, Jossey-Bass, San Franciso, 1993. This book alsoexplores evaluating research and service and so comple-ments Scholarship Assessed which is noted below.

On portfolios, the most straightforward advice is containedin the original Canadian work on this subject by B. Shore etal., The Teaching Dossier, Guide to its Preparation andUse, Canadian Association of University Teachers, 1991.Another useful introduction to portfolios, which alsoconsiders their relationship to scholarship, is The TeachingPortfolio: Capturing the Scholarship of Teaching by R.Edgerton, P. Hutchings and K. Quinlan, American Associa-tion for Higher Education, Washington, D.C., 1991.

If you are concerned to evaluate materials and educationaltechnologies we suggest M. Tessmer, Planning andConducting Formative Evaluations, Kogan Page, London,1993. This is an interesting mixture of useful guidance onplanning evaluations, evaluating materials, and the wholenotion of formative evaluation. Hartley’s book is alsohelpful on evaluating materials; Designing InstructionalText, Kogan Page, London, 1994.


T. Angelo and K. Cross. Classroom Assessment Techni-ques, Jossey-Bass, San Francisco, 1993.

S. d’Apollonia and P. Abrami (1997). Navigating studentratings of instruction, American Psychologist, November,1198-1208.

C. Glassick et al. Scholarship Assessed, Evaluation of theProfessoriate, Jossey-Bass, San Francisco, 1997.

W. McKeachie (1997). Student ratings, the validity of use,American Psychologist, November, 1218-1225.

D. Magin (1998). Rewarding good teaching: A matter ofdemonstrated proficiency or documented achievement?

206

The International Journal of Academic Development, 3, 2,124-135.

M. Scriven. Summative teacher evaluation. In J. Millman(ed), Handbook of Teacher Evaluation, pp. 244-271. Sage,Beverly Hills, 1981.

207


APPENDIX:WHERE TO FIND OUT MORE ABOUTMEDICAL EDUCATION

As a result of your involvement in teaching you mayeventually wish to pursue in greater depth an interest inmedical education. This section of the book will identifyvarious resources which might be helpful.

BOOKS AND JOURNALS

We have already provided selected readings at the end ofeach chapter. There are some other texts which may be ofmore general interest and which cover a wider range thanthe selected readings.

On the principles of good teaching in higher education wesuggest Teaching for Quality Learning at University by J.Biggs, SRHE and Open University Press, Buckingham, 1999and Learning to Teach in Higher Education by P. Ramsden,Routledge, 1992.

An excellent introduction to the important concept of lifelong learning which covers a wide range of relatedteaching issues is Lifelong Learning in Higher Educationby C. Knapper and A. Cropley, KoganPage, London, 2000.

For those wishing to delve more deeply into aspects ofmedical education research there is now a major textentitled An International Handbook for Research in MedicalEducation by J. Norman, C. Van der Vleuten and D.Newble (eds), KluwerAcademic, Dordrecht, 2001.

Articles relating to medical education appear regularly inmost of the major general medical journals. Many of theseare now searchable on-line. There are also severaljournals specifically concerned with publishing researchand review articles in the field of medical education.

Medical EducationThis is the official journal of the Association for the Study ofMedical Education (ASME), which is the organisationcatering for individuals interested in medical educationin the United Kingdom. It should be readily available inyour medical school library.

The Association also produces an excellent series ofbooklets dealing with various aspects of medical educationand has a series on medical education research. These arealso published in the journal.

210

Academic Medicine (previously The Journal of MedicalEducation)This is a publication of the Association of American MedicalColleges. As well as containing articles relating toteaching, this journal also deals with the broader issuesof the organisation of medical education as it relates to theUnited States. It also publishes the Proceedings of theAnnual Conference on Research in Medical Educationwhich is the world’s premier medical education researchmeeting.

Medical TeacherThis journal is now published in collaboration with theAssociation for Medical Education in Europe. In the past ithas not been primarily a research journal. Rather, it hasbeen a journal containing review articles and descriptionsof educational activities by medical teachers from aroundthe world. While this focus remains it is increasinglypublishing quality research articles. It remains an excellentsource of ideas and information.

Teaching and Learning in MedicineAn American medical education journal gaining areputation as a major international journal. It often containshigh quality research based reviews.

Advances in Health Sciences EducationA relatively new addition to the list of medical educationjournals. While published in Europe it has a prestigiousinternational editorial board and is a source of high qualityeducational research and review articles.

World Health Organisation

The WHO has been very active for many years in the fieldof medical education. It frequently commissions experts towrite review documents. A list of such publications shouldbe available in your library. If not, you could write direct toDistribution and Sales, CH-1211 Geneva 27, Switzerland orsearch the WHO Website.

211

TRAINING OPPORTUNITIES

Many tertiary institutions have individuals or units whosejob is to provide assistance to teachers. We would stronglyrecommend you make full use of these resources.Alternatively, you could elect to attend courses run byother organisations in your area or even overseas. TheWHO supports regional teacher training centres whichoffer short courses for medical teachers and even degreecourses in health personnel education.

Many universities now have medical education depart-ments or units which run courses of various types and somehave well-developed degree or fellowship programmesfor local and overseas postgraduate students.

OVERSEAS TRAVEL

Generally speaking, people concerned with medicaleducation are friendly and helpful. Should you betravelling, do not hesitate to write to any individual orinstitution you may wish to visit and observe at first handeducational activities or facilities which have interested youduring your reading. If you come from a traditionalmedical school, you should attempt to visit one of the wellknown less conventional schools such as McMasterUniversity in Canada; Maastricht University in the Nether-lands; Ben Gurion University of the Negev, in Israel; orNewcastle University in Australia. Many other institutionsare now experimenting with alternative curricula soopportunities to experience different ways of teachingmedical students should be found reasonably close to yourhome institution.

ORGANISATIONS

As your interest in medical education grows you may wishto join an association or attend one of the national orinternational conferences which are held each year. Thefollowing is a list of some of the most well know English-speaking organisations which conduct major annualconferences.

212

Association for Medical Education in Europe (AMEE)

This is an umbrella organisation for national medicaleducational associations in Europe. It is also well supportedby the WHO. In addition it provides a regular forum for themeeting of medical school deans from Europe andsometimes from other parts of the world.

Further information can be obtained from the Secretary,AMEE, Tay Park House, 484 Perth Road, Dundee, DD2 1LR,Scotland or on <www.dundee.ac.uk/MedEd/AMEE>

Association for the Study of Medical Education (ASME)

This is also a European organisation with a largely Britishmembership but with a significant number of membersfrom other countries. It caters for individuals with aninterest in medical education and provides a forum forcommunication of ideas and information. It organisesconferences and workshops and produces several pub-lications including the well know ASME booklets series.

Further details can be obtained from the Secretary, ASME,12 Queen Street, Edinburgh, EH1 1JE or on< www.asme.org.uk >

Association of American Medical Colleges

This is another umbrella organisation covering medicalschools in America. It holds an annual meeting and inconjunction with this is held the Research in MedicalEducation Conference. This meeting provides the majorannual gathering of workers in the field of medicaleducation in the United States and Canada.

Further information can be obtained from the AAMC, 2450N Street N.W., Washington, DC 20037, USA or on< www.aamc.org >

ANZAME: The Association for Health ProfessionalEducation

This is an association covering Australasia which holds anannual conference and also supports state and regionalgroups which meet on a regular basis. It publishes ajournal Focus on Health Professional Education.

Further information can be obtained on< www.anzame.unsw.edu.au >

213

International Ottawa Conferences on Medical Educa-tion

These are held every 2 years and have become the mostpopular international conferences on research and devel-opment in medical education. They take place in differentcountries each time. Information about future conferencescan be found in the News Section of the journal MedicalTeacher.

214

INDEX

abstractideas 23preparation 61

academic argument, construction 19academic courses, high achievements 6academic performance 12accountability 189active participation

clinical teaching 74–5large group classes 27–8research projects 85small group teaching 28, 41

analysing 42anxiety 4

‘feelingfraudulent’ 26relaxation exercises 26scientific paper, presentation 56students

disability, with 159 (fig.)mature age, assessment arrangements 94

teaching large groups 25–6ANZAME: the Association for Health

Professional Education 213art departments 4assessing the students 126–163

assessment methods 134–58direct observation 148–51essay 134–6learning portfolio 156–8objective test 140–8oral examination 151self-assessment 153–6short-answer questions 136–8structured clinical/practical assessment151–3structured test 139–40

disabilities, with 158–9adjustments 159assessment support strategies 159 (fig.)

educational measurement 128–34criterion-referenced assessment 132–3learning, positive impact 133–4norm-referenced assessment 132–3practicality 131–2reliability 129–31validity 128–31

feedback to students 161–2formative assessment 102groups 159–60guided reading 162–3

interactive skills, testing 151purpose 126–8

formative assessment 128summative assessment 127

results reporting 162small group teaching 43, 52–3standard setting 133summative assessment 102technology used 160–1

Association of American Medical Colleges 213Association for Medical Education in Europe

(AMEE) 213Association for the Study of Medical Education

(ASME) 210, 213attention

during note-taking 30during slide presentation 32

attitude, development 40audio recording 18, 46audio-visual aids 20, 24–5, 64

conference sessions 67handling 26

audio-visual materials, problems with 33

bank of questions 140–1behavioural skills 16blackboards 31, 33, 175–6

guidelines for use 177 (fig.)blueprint, clinical competence examination

131 (fig.)brainstorming 29–30

large group teaching 29–30small group teaching 48

bridging course 93buzz groups, small group teaching 28, 46–7

cartoons 20case discussion/presentation 44 (fig.), 45, 75case study 22checklists 148–9Classroom Assessment Technique (CATs) 22,

30, 194–5, 206clinical examination structured (outline) 151–3,

154 (fig.)marking sheet, checklist 155 (fig.)

clinical skillsalternatives to traditional clinical teaching

78–80clinical interview 81–3

simulation 82–3

215

video recording 81–2clinical teaching situations 74–5laboratories 80objective structured clinical examination

(OSCE) 151–2physical examination 83–4structured courses 78, 79 (fig.)teaching by example 74teaching environment 75techniques and teaching 80–1

programmes 80clinical teacher

desirable attributes 72–3environment for tutorials 75importance of example 74importance of involving students 74–5

clinical teachingalternatives to 78–80effective clinical teacher, attributes 72–3evaluation 87formative assessment 102good teaching environment 75improvement 73–5

setting good example 74student, involvement 74–5teaching plan 73–4

clinical tutorials 75–8good teaching environment 76problem solving 76–8student, involvement 76teaching, planning 76

clinical tutorials see also small group teachingcollaborative working 9competence, minimal standard 132–3comprehension learner 4computers 50

aided learning, impact and outcomes 6–7answer sheets, interpretation 147assessing the student 161assisted instruction 196based methods 86generated colour slides 173laboratory setting 86medical examinations 147objective-type examination 147presentation systems see Power Pointsimulation 87small group teaching 50

conferenceaudience, organisers responsibility 68making a presentation 56–69poster 64–6

definition 64lay out 65 (fig.)preparations 65–6

session

chairing 66–9contact with speakers 66–7discussion periods 67finishing off 67, 69responsibilities to organisers 66–7timing, importance 67venue 66

speakersapprehensive 67confident under-prepared 68responsibilities to 67–8well organised 67

consistency of marketing 130content validity, of assessment 128–9context characteristics 3context-dependent questions 145course

assessmentarrangements 17content 10–11objectives 10–11procedures 10–11

attendance 40co-ordinator 17medical, science based 10objectives 9planning chart, assessment 97 (fig.)sequential 110structured, basic clinical skills 79 (fig.)

course see also course, planningcourse design, curricula structuring 8course, planning 90–108

administrative responsibilities 106advisers, panel 91aims 96, 97 (fig.)

objectives 96, 97 (fig.)allocation of facilities 107allocation of time 106chart, example 97 (fig.), 103 (fig.)content 91–3

academic criteria 91practical criteria 92professional criteria 92psychological criteria 92student criteria 92–3

evaluation 107–8context and input 107–8outcome 108process 108

guided reading 108objectives and assessment 96, 97 (fig.), 100,

103–4 (figs)organisation 105–6relating objectives to:

assessment methods 102–5teaching/learning activities 100–2

216

responsibility for 91sequencing courses 105–6students 93–6

adults (mature-age) 94disability, with 94first year 95international 95–6women 94

technical support 107writing objectives 96–100

attitudinalobjectives 97–8knowledge objectives 97–8, 101skill objectives 97–8specificity of objectives 99

criterion-referenced assessment 132–3critical thinking 9criticising 42curriculum

fragmentation 8medical 8structuring 8

data selection, scientific paper 57decision making 42diagrams 30, 46direct observation 148–51

checklists 148–9rating forms 149–50reliability 148

distance learning 16‘divergent’ thinking 30doctor-patient relationship 81‘Dr Fox effect’ 23

Education Resources Information Centre(ERIC) 204 (fig.)

educational measurement 128–34reliability 129–31validity 128–9, 131

electronic mail (E-mail) 50evaluation, means of 195impact on learning 7

English, as second language 93essays 11, 134–6

extended response 135marking 135, 137 (fig.)

re-marking 135unreliable 135

restricted response 135–6evaluation

accountability 189definitions 189–92framework for 198–207

documenting teaching 201–3feedback, seeking 201goals 199

methods 200preparations 199presentation 200–1results 200standards 199

information, sources 190 (fig.)learning 189–92planning, as part of teaching 192principles 189–92recommendation for using for:

administrative purposes 197–8developmental purposes 196

reliability 192techniques, methods 191 (fig.)trustworthy system 203validity 191

exam evaluation 194examination 10 (fig.)extended-matching questions 145–6

face-to-face contact, small group teaching 41film see videosFlexner report (1910) 110formative assessment 128formativeevaluation 190, 196, 198foundation courses 93

Guided Discovery 113

handouts 18, 166–7, 180–2suggested headings, small group teaching

43 (fig.)take-away 64when to give 31

histograms 57history taking 81, 102

illustrations 59, 64information

gathering 191 (fig.)technology 161

instruments, their use 84internal consistency 147internal consistency, of assessment 130International Ottawa Conferences on Medical

Education 214international students see non-traditional

studentsInternet 18, 117, 204

class notes, posting 18revolution 181web browser 181

Internet service providers 181interpersonal skills 40

video recording 81

217

interviewsinformation, gathering 191 (fig.)techniques, video recording 81

jokes 20journals 19

publications 60references 18

knowledgebased tests 117interpretation 4

laboratory classes, teaching 84–7data analysis 86

large group teaching 16–37active learning strategies 27–30‘are there any questions?’ 28brain storming 29–30concluding thought 36context 16–17evaluation 35–6guided reading 37non-traditional students 17–18one-to-one discussion 28–9preparations 19–23

content, identification 19–20plan, finalising21–3purpose of teaching 19

problem-solving activities 29quality, improvement 35–6session, presentation 23–6

anxiety when teaching 25–6finishing 25format 24rehearsal and check 25starting 23–4

small group activity 28social contacts 18teaching materials and technology 31–3technology, use 18things going wrong 33–5

‘lateral’thinking 30learning

active, large lecture classes 27–30case-based 8environment, improvement 5, 8–11evaluation 193–5information, gathering 193more effectively 7–12

research evidence 7objectives, modelling 18outcomes 5

deep approach learning 5portfolios 8problem-based see problem based

learningquality, modern technology used 7skills, improvement 12–13strategies 12student-centred 10 (fig.)students’ 2–13

assessment 2experiences, previous 3factors influencing 3failure, fear of 4model 3 (fig.)

technologyimpact of 7new 6–7used 183–4

lecture rooms 10 (fig.)lectures see teachinglibrary 10 (fig.), 101

materials 17tapes 18

literature, impact on learning process andoutcomes 7

manual skills, self examination 84mature-age student course, planning 94mean and standard deviation 147medical education

book and journals 210–11organisations 212–14overseas travel 212training opportunities 212WHO 211

medical equipment, use 84medical schools 5, 22

lectures 16problem-based learning 110, 112 (fig.)programmes, clinical skills 80traditional 110traditional teaching, sequential course 110women students 94

medical students, traditional curriculaeundertaking 5

memorisationfactual material 9mechanical rote learning 6process leading to understanding 6understanding with 4versatile learner 4

‘minute paper’ 30modified essay questions 117, 139–40, 141

(fig.)motivation 4

extrinsic, conventional teaching 10 (fig.)intrinsic, student 9 (fig.)

centred learning 10 (fig.)strategic approach to learning 4–5

218

motivation and learning 111multiple choice questions 11, 27–8, 117, 140

problems141–2setting procedures 143–4 (figs)vs. short answer questions 136

‘netiquette’ 50non-traditional students 5–6

different cultural backgrounds 5–6large group teaching 17–18older students 6problems 5

norm-referenced assessment 132–3note taking 30

objective structured clinical examination(OSCE) 151–2

objective testsassessment 140–8

extended-matching questions 141multiple choice questions 141, 143–4putting together 146–7question type choice 141–2scoring and analysing 147–8scoring, reliability 140–1true-false questions 140–3

objectivesassessment 128–9attitudinal 97–8educational 40how detailed? 99knowledge 97–8skill 97–8writingobjectives96–9

observation, direct 102observation of students

technical/interpersonal skills 148–51on the ward 75

observation of students performance 148‘on-line’ teaching 50one-to-one discussion 28–9

small group teaching 46operation learner 4oral examination

problems 151procedure 152 (fig.)reliability 151

overhead projector 31–2, 168–72guidelines 171 (fig.)other uses 170setting up and using 171–2software presentation packages 170transparency preparations 168–9

overheads 18, 33, 59, 166lettering, instructions 59transparency 28

overseas student see students, internationaloverseas travel, medical education 212

papers and scientific meetingspreparation 56–61

arrangement 57–8data, collection and selection 57polishing, writing it out and rehearsal 60–1presentation aids 59

presentation 56para-clinical sciences 110pass level, setting 132patient management problems 139

computer based 139scoring systems 139

peerassessment 11teaching 8

photocopying, for overhead projector 169–70physical examination 83–4, 102

simulated patients 83–4simulation devices 83video recording 83

planningcourses 91–3teaching/clinical tutorials 73–4

plenary sessions, conducting 49portfolio

learning 156–8content 158 (fig.)

teaching 201–3poster session 56Power Point 32, 170practical classes

evaluation 87teaching 84–7

practical skills see clinical skillspre-clinical disciplines 110principles 23printed material 179–81

design 179–80guidelines 180 (fig.)

prescribed reading 181using 180–1

‘pro and con grid’ 30problem solving 42problem-based learning 40, 110–123, 196

case-based problem 119 (fig.)course design 112–17

assessment 115–17guided discovery 113instruction method 113–14modules, major purposes 113modules, preparations 115open discovery 113

219

problem selection 114definition 110–11educational objectives 111guided reading 121–2implementation 111–12

models for change 112 (fig.)introductory session, procedure 119 (fig.)modules 118–20

week’s activity 118 (fig.)staff resources 120–1

staff development 121tutor guide to aims 111, 116 (fig.)

problem-solving 40activities 9, 29

proceedings 61projector (35 mm) 32publishing material, World Wide Web 181–3

questionnairesclinical and practical teaching, evaluation 87evaluation, tools 195

teaching 35–6information, gathering 191 (fig.)small group teaching, evaluation 51, 54 (fig.)

questions during conferences, handling 62

reasoning 40relative standard, setting procedure 132reliability, of assessment 129–31reliability coefficient 147Research in Medical Education Conference

213research projects 8, 11, 85–6

supervisors, tasks 85 (fig.)rote learning 6

science department 4science-based courses 4scientific paper 56–61

arrangement 57conclusions 58data selection 57drafting 60editing 60introduction 57–8methods and results 58preparations 56–7

day of the presentation, checklist 62, 63(fig.)

presentation 56presentation aids 59purpose statement 58questions, handling 62read the paper or not 60rehearsal 60–1

self examination 84

self-assessment 11assessing the students 153–6discs 116skills 117

self-organising skills 12short-answer questions 136–8, 140

with answer key 138 (fig.)setting and marking procedure 138 (fig.)

simulated patients 82–3simulated patients, instructors 84simulation

clinical interviews 82–3devices 83practical teaching 86–7

slide projector (35 mm) 172–4slide

making, guidelines 173 (fig.)preparation 172–3projection 173–4‘spotting’ procedure 174 (fig.)

slides 18, 28, 46, 59, 166diazo 172–3making, guidelines 173 (fig.)photocopied enlargements as back up 33position, correct 60Power Point 32preparation 172projection 172results, scientific paper 59‘spotting’ procedure 174 (fig.)

small group teaching 16, 40–54alternative discussion techniques 46–9

brain storming 48buzz groups 47–8one-to-one discussion 47plenary sessions 49role playing 48–9

characteristics 41–2active participation 41face-to-face contact 41purposeful activity 42

evaluation 50–1good group, criteria for 44 (fig.)guided reading 53importance 40managing 42–4

maintenance 43–4tasks 43

stimulus materials 46structure 44–6technology 50when things go wrong 51–3

snowballing group discussion 45–6‘stimulus’ material 46structured tests 139–40

220

student-centred learningactive role 10 (fig.)flexibility 10 (fig.)responsibilities 10 (fig.)

studentsassessment, influence on learning

behaviour 188attention and learning during teaching 23–4

rest period 24 (fig.)characteristics 3course, planning 93–6

disability, with 94–5first year 95international 95–6mature-age 94needs and interests 93–6women 94

disruptive behaviour, disciplinarymeasures 35

explanation to answers to questions 10from Asia 6hearing-impaired 18independence 9 (fig.)international cultural awareness 95notice board, information 192older 6–7

compared to younger 7 (fig.)large group teaching 16–17

overseas 5passive, lecturing 16sight-impaired 18things go wrong with presentation,

challenges 34women 94younger vs. mature, compared to 6–7

students’ learning 2–13active

large lecture classes 27–30strategies 16

acute management, simulated patientsassessment 83–4

approaches 3–5deep 4strategic 4–5surface 3–4teachers’ 3

competition between 10 (fig.)difficulties arising 2failure to learn the content 11how students learn? 2–5

studies 2language of instruction, not the first

language 18model 3 (fig.)motivation 4non-traditional students see non-traditional

studentsoutcomes 5

teachers’ approach to teaching 11–12perceptions 3

emotional aspects 4self-directed 9traditional teaching vs. student-centred

learning, distinctions between 8, 10 (fig.)understanding, teachers’ role in 11

study skills 12summative assessment 117, 127summative evaluation 190, 197–8

teachersapproach to teaching 3, 11authority, traditional 18behaviour, student rating 188, 192career development 201–2confrontationwithstudents 34covering the subject 11good, teaching techniques 7–8leadership role, small group teaching 42–3learning environment, creation 8promotion 188students’ identification, specific study skill

counselling 12students’ learning, assessment 2, 126students’ understanding, assisting 11‘teach the teachers’ course 72time in helping students 10what and how much students learn 2

teachingacademic-argument 22approaches 5

deductive 23inductive 23modification 11–12

assessment methods 3assessment procedures 10–11conventional 8, 10 (fig.)

students’ role 10 (fig.)didactic 9do’s and don’ts for teachers 26effectiveness 188evaluation 188–92, 195–6expert feedback 25face-to-face vs. ‘on-line’ 50first contact with students 23large group see large group teachingpassive students, active learning strategies

16personal preparations 26plan presentation, transparency 23–4plans, content-oriented 21poor 5presentations of ideas, sequence 22

221

problem-focused21–2session, purpose of 17starting the session 23–4student-centred learning 8–9traditional 8unexplained difficulties, strategies to deal

with 33using technology 183–4

teaching materials and aids 166–85basic principles 166–8types 168

overhead projector 168–72printed material 179–81slide projector (35 mm) 172–4video projector 174–5videos and film 176–9white/black board 175–6

Teaching Portfolio 192, 197, 201–3contents 204–5 (fig.)

telecommunications 161term objective test 140testing

norm-referenced assessment 132pass level 132

theories 23training opportunities, medical education 212Triple Jump Test 117tutorials 40

groups 41tutoring

problem-based learning 117–20Triple Jump Test 117

undergraduate clinical teaching 75understanding

rote learning 6strategic approach learning 4–5superficial level 4teaching activities 9

validity, educational measurement 130versatile learner 4versatile learning 4vice-voce examination 151video 22–3, 32–3, 46, 64, 176–9

history taking 81impact on learning 7recording

interview 81–2patient’s consent 81physical examination 83

teaching technique, viewing 25, 177–9video projector 174–5

equipment preparations 175visual aids 35

scientific paper, presentation 59–60visual aids see also overheads; slide

projector; video

ward-based teaching 72–5web page, departmental or personal 192web-based material 46whiteboards 31, 33, 175–6

guidelines for use 177 (fig.)World Health Organisation 211World Wide Web 101

publishing materials 181–3

X-rays 46

222