Embed Size (px)
Citation preview
EDUCATIONAL INNOVATION IN AN UNDERGRADUATE MEDICAL COURSE
Implementation of a Blended e-Learning, Team-Based Learning Model
by
LINDSAY K. DAVIDSON
A thesis submitted to the Faculty of Education
in conformity with the requirements for
the degree of Master of Education
Queen’s University
Kingston, Ontario, Canada
January, 2009
Copyright © Lindsay K. Davidson, 2009
ii
ABSTRACT
Medical education has been the subject of ongoing reform since the second part
of the 18th century (Papa & Harasym, 1999). Most recently, medical education has been
redefined to include a broad set of competencies over and above traditional expertise. In
an attempt to facilitate this approach, different instructional models have been proposed.
Most of these seek to foster learner engagement and active participation and promote
life-long learning. Nevertheless, there is no consensus amongst medical educators about
the optimal way to teach future physicians.
Despite the efforts of both researchers and local champions, instructional
innovations frequently fail. Fullan (2001) ascribes this to faulty assumptions on the part
of planners as well as to the inherent complexity of the organizations involved, further
stating that effective change requires some degree of reculturing. This study examines
the process of educational change in an undergraduate medical course over a three-year
period. Formerly taught exclusively by large class lectures, the course was redesigned to
include a blend of e-learning and Team-Based Learning (TBL). The process of change is
described and viewed in parallel from the perspectives of both student and teacher while
uncovering contextual and process elements that contributed to the outcome.
iii
Shifting student attitudes to teaching and learning were identified over time,
suggesting that these evolve in parallel to faculty experience implementing a new
teaching strategy. Van Melle (2005) has suggested that acceptance of educational
innovation is dependent on the environment and organizational context. The results of
this study highlight the importance of these factors in the successful introduction of a
new instructional paradigm as well as the value of longitudinal evaluation of
instructional changes in order to better understand their transformational potential.
iv
ACKNOWLEDGEMENTS
This has been a long journey. Long enough to have two children and to have seen
several cats come and go. Long enough to plant a seed and actually watch it sprout and
grow into something satisfying. But roads are lonely if you go alone, and I have been
fortunate enough to have had many companions, colleagues and friends assist me along
the way. My senior colleagues, Dr. Peter Brown and Dr. David Pichora suggested to me,
when I was a new faculty member that “Education” was a country that I might like to
visit. For that advice and for their support and that of Dr. Dale Mercer, I am grateful. Dr.
David Holland was the person who introduced me to TBL and who won me over with
his enthusiasm. His early support and encouragement of this project was instrumental in
its success.
Dr. Lyn Shulha was one of my first teachers when I began this degree and I have
been fortunate to have her as my thesis supervisor. Her cheerful and wise approach has
given me direction and led me down paths I might not have taken alone. Dr. Denise
Stockley and the Centre for Teaching and Learning have been co-conspirators all along.
Their knowledge, advice and enthusiasm for teaching and learning have helped me grow
as both a teacher and a student.
v
I am fortunate to be surrounded by others with a passion for to improve
education. Dr. Elaine Van Melle, and her predecessor Dr. Ros Woodhouse, are both
responsible for aiding, inspiring and nurturing this project. The tireless group of faculty
who have worked to create, maintain, evaluate and renew the curriculum in the School of
Medicine have heard and lived this story in bits and pieces and have always provided
helping hands and support when the going got tough. The wizards at MedTech, Matt
Simpson and Amy Allcock, have both given me amazing tools to work with and also
listened patiently to my rants and ideas. Sheila Pinchin has brought knowledge and
experience from the world of “real” teaching and is an endless source of encouragement.
I’m grateful to Heather Conway for helping me enter, collate and organize the
huge amount of data generated over the past three years. Thank you also to King Luu
who lent his time, statistical expertise and advice as I worked to make sense of it all in
the past few months.
Most importantly, I’d like to thank my family. Evelyn and Douglas, my parents,
you taught me to work hard but you also taught me the importance of a loving home.
Andrew, James and Eileen, I’ve stolen many hours away from you in the basement,
plotting, conducting, analyzing, writing and re-writing this thesis. The three of you are
my roots, and my anchor, and my heart.
vi
TABLE OF CONTENTS
ABSTRACT....................................................................................................................... ii
ACKNOWLEDGEMENTS ............................................................................................ iv
TABLE OF CONTENTS ................................................................................................ vi
LIST OF TABLES ........................................................................................................... ix
CHAPTER ONE: INTRODUCTION............................................................................. 1
BACKGROUND.................................................................................................................. 1 MEDICAL EDUCATION IN THE LAST 100 YEARS............................................................... 2 MEDICAL SCHOOL CURRICULAR STRUCTURE .................................................................. 4 THE RESEARCH SETTING.................................................................................................. 7 RESEARCH QUESTIONS .................................................................................................... 8 ORGANIZATION OF THESIS ............................................................................................... 8
CHAPTER TWO: LITERATURE REVIEW.............................................................. 10
OVERVIEW OF CHAPTER ................................................................................................ 10 LEARNING TO TEACH IN THE IVORY TOWER .................................................................. 10 TEAM-BASED LEARNING................................................................................................ 15 BLENDED LEARNING...................................................................................................... 17
CHAPTER THREE: METHODS OF DATA COLLECTION AND ANALYSIS ... 22
OVERVIEW OF THE CHAPTER.......................................................................................... 22 Research Questions................................................................................................... 22
METHODS OF DATA COLLECTION .................................................................................. 24 Question #1 ............................................................................................................... 24 Question #2 ............................................................................................................... 24 Question #3 ............................................................................................................... 28
METHODS OF DATA ANALYSIS ...................................................................................... 29
CHAPTER FOUR: RESULTS ...................................................................................... 33
OVERVIEW OF THE CHAPTER.......................................................................................... 33 DEMOGRAPHICS ............................................................................................................. 34 QUESTION #1 ................................................................................................................. 34
Introduction to Course.............................................................................................. 34
vii
Changes in Course Content over Time ..................................................................... 35 Classroom Time Weightings by Instructional Method.............................................. 36 Physical Space .......................................................................................................... 37 Course Website ......................................................................................................... 38 Methods of Student Assessment ................................................................................ 41
QUESTION #2 ................................................................................................................. 45 Historical Course Evaluation Data .......................................................................... 45 Student Survey Results .............................................................................................. 49
CHAPTER FIVE: THE TEACHER’S TALE ............................................................. 64
PROLOGUE ..................................................................................................................... 64 BEGINNING TO TEACH.................................................................................................... 65
“Is Anybody Learning Anything?” ........................................................................... 66 Early Attempts with Different Instructional Methods............................................... 69 Discovering TBL ....................................................................................................... 70
2005: YEAR ONE............................................................................................................ 73 2006: YEAR TWO........................................................................................................... 78 2007: YEAR THREE........................................................................................................ 85
CHAPTER SIX: DISCUSSION..................................................................................... 89
GOING TO SEA................................................................................................................ 89 EDUCATIONAL INNOVATION AND CHANGE .................................................................... 90 THE EVOLUTION OF THE MUSCULOSKELETAL COURSE.................................................. 91 STUDENT RESPONSE TO THE CHANGING COURSE .......................................................... 93 CRITICAL ELEMENTS THAT INFLUENCED THE COURSE OUTCOME.................................. 96
External Factors ....................................................................................................... 96 Technology................................................................................................................ 98 The Teacher .............................................................................................................. 99
CHARTING THE CORRECT COURSE............................................................................... 101
REFERENCES.............................................................................................................. 105
APPENDIX A: DOCUMENTS RELEVANT TO THE MSK COURSE ................ 112
APPENDIX B: STUDENT COURSE EVALUATION SURVEY ITEMS (2005-2007) ............................................................................................................................... 114
APPENDIX C: RESEARCH SURVEY...................................................................... 118
viii
APPENDIX D: STUDENT RANKING OF INSTRUCTIONAL METHODS BY EDUCATIONAL OBJECTIVES ................................................................................ 123
APPENDIX E: SAMPLE APPLICATION EXERCISE (USED IN 2006) .............. 135
ix
LIST OF TABLES
Table 1: The five ages of a lecturer .................................................................................. 14 Table 2. Seven Principles for Effective Online Teaching................................................. 21 Table 3. Methods of Data Collection and Analysis .......................................................... 23 Table 4. Common Course Evaluation Survey Items from 2005 and 2007 ....................... 31 Table 5. Student Demographics ........................................................................................ 34 Table 6. Content of Introductory Sessions........................................................................ 36 Table 7. Time Allotted to Different Instructional Methods (including Pain teaching) .... 39 Table 8. Time Allotted to Different Instructional Methods (excluding Pain teaching).... 40 Table 9. Course Resource Material by Year ..................................................................... 42 Table 10. Assessment of Student Achievement and Grade Weightings 2005-2007 ....... 43 Table 11. Comparison of Common Course Survey Items from 2005 and 2007............... 48 Table 12. Summary of Findings and Themes from Course Evaluation Surveys.............. 51 Table 13. Principal Components Analysis of Statements about Instructional
Methodologies........................................................................................................... 54 Table 14. Comparison of Didactic and Active Learning Scores over Time..................... 55 Table 15. Major Themes Related to Online Learning ...................................................... 60 Table 16. Major Themes Related to Team-Based Learning ............................................. 62
1
CHAPTER ONE: INTRODUCTION
Background
The last century has seen an unprecedented amount of change in both medicine
and medical education (Cooke, Irby, Sullivan, & Ludmerer, 2006). In addition to an
explosion in medical knowledge and technology, the healthcare environment has become
influenced by “patient consumerism, government regulations, financial imperatives,
medical information on the internet, [and] litigation” (Frank, 2004, p.27) as well as by
changes in physicians’ work expectations and the emergence of new types of healthcare
delivery (Dent, & Harden, 2005). In response to these changes, regulatory groups such as
the Royal College of Physicians and Surgeons of Canada (RCPSC) and the Canadian
College of Family Practice (CCFP) have developed frameworks that make explicit a set
of competencies required for the modern physician. For example, in Canada, the RCPSC
has articulated six physician roles, in addition to that of medical expert, that underlie the
assessment of postgraduate specialist medical trainees (Frank, 2004).
Medical schools are now mandated by accrediting bodies to teach their graduates
a broad skill set including communication skills, the medical consequences of common
societal problems, an understanding of diversity, medical ethics, and human values
(Liaison Committee on Medical Education, 2005). Information literacy (Murphy,
2
Ingram, & Howard, 1998) practice management, and problem solving skills (General
Medical Council, 2003) are also regarded as essential to the training of future physicians.
Driven partly by these new standards and partly by of conceptions of cognition and
learning (Shuell, 1986), the delivery of undergraduate medical education has also
undergone a transformation in the past three decades.
Medical Education in the Last 100 Years
Curricular change is a recurring theme in medical education. Papa and Harasym
(1999) describe five curricular models that have emerged in North American medical
education since the late 17th century: (1) apprenticeship, (2) those based on disciplines,
(3) organ-systems, (4) clinical problems, and most recently, (5) clinical presentations
(Woloschuk, Harasym, Mandin, & Jones, 2000). Medical education has undergone
significant changes since the publication of Abraham Flexner’s report in 1910. Flexner,
an educator, was hired by the Carnegie Foundation for the Advancement of Teaching to
study and make recommendations about the education of physicians in North America,
addressing concerns that medical education of the day lacked both standardization and
rigour (Beck, 2004). The reforms that followed led to the regulation and standardization
of medical education in North America with a redefinition of the curriculum to consist of
two years of laboratory sciences followed by two years of hospital-based clinical
3
rotations. Flexner’s work was instrumental in developing standards for medical
education, which had previously been lacking. The Flexnerian model emphasized the
scientific basis of medicine over the haphazard clinical apprenticeship model that had
dominated previously. It was rapidly adopted and dominated North American medical
education for much of the 20th century.
Since the 1990s, many medical schools have redesigned their curricula to
encourage the development of active, self-directed and life-long learning strategies in
students (Kelly, Haidet, Schneider, Searle, Seidel, & Richards, 2005). Problem-based
learning (PBL), which was pioneered at McMaster University (Schmidt, 1983), is an
instructional technique that has been widely adopted for this purpose. During PBL,
students learn in small groups by exploring fictional cases that are constructed to
simulate real patient situations. These integrate basic and clinical sciences and require
students to identify social, psychological, and ethical issues that intersect with the
medical conditions being studied. Ideally, students develop learning objectives, research,
collaborate and interact with each other as they negotiate each case from start to finish.
Although there are exceptions, most PBL groups include a facilitator who may be
an expert or non-expert faculty member or even a senior student. This raises a major
problem that has dogged the implementation of PBL: because of the low student-
4
facilitator ratio required for this technique (generally one facilitator for every 6-12
students), human resources can be a limiting factor.
Medical School Curricular Structure
The School of Medicine (SoM) described in this research offers a four-year
program leading to a doctor of medicine (MD) degree. The average student entering the
medical program has over four years of prior experience as a university student; some
enter after graduate degrees. Students are not required to adopt a particular course of
study prior to acceptance into medical school; therefore, the students come from a
variety of academic disciplines, bringing with them a heterogeneity of knowledge and
experience.
Until the early 1990s, the curriculum reflected Flexner’s original
recommendations. Instruction in the pre-clinical years consisted largely of didactic,
teacher-focused instruction. The curriculum was revised in the early 1990s, in part due to
changing standards set by the Liaison Committee on Medical Education (LCME, the
North American accrediting authority for institutions granting the M.D. degree).
Departmental courses were combined and reformatted to create courses based on body
organ systems. Weekly, small-group clinical skills and PBL sessions were introduced
into the pre-clinical curriculum. Horizontal courses were created and interwoven
5
throughout the three pre-clinical years, emphasizing clinical skills and the role(s) of the
physician within society.
The curriculum is currently divided into three main sections, which are termed
“Phases”. Phase I is a 4-month block in which students are taught a wide variety of basic
scientific facts and concepts foundational to the study and practice of clinical medicine.
In Phase II, students are introduced to the normal and abnormal functioning of the body,
with the teaching units organized by organ system (for example: cardiovascular,
musculoskeletal, and renal). Phase II begins in January of first year and continues until
December of third year. This portion of the curriculum is divided into three instructional
blocks and one independent research component.
PBL is introduced in Phase II with weekly small-group meetings. Phases I and II
together comprise the pre-clinical component of the MD program. Phase III is the
clinical component of the students’ training. The student participates in a supervised
fashion in the assessment, management and follow-up of patients in a variety of venues
including physicians’ offices, hospital clinics, in-patient wards, emergency rooms and
community settings. Students are expected to be able to function as a member of a
clinical team including physicians, nurses and other health care professionals.
6
The curricular reform initiated in the early 1990s has been effective in temporally
linking related basic and clinical science content using a systems-based approach;
however, fostering learner engagement and active participation has proven to be an
ongoing challenge. Both clinical skills and PBL courses require a significant number of
small group facilitators. There is perennial difficulty recruiting and retaining an adequate
number of faculty members to meet the needs of both courses. Increasing enrollment
(30%) over the past 5 years has compounded this problem.
Despite longstanding concerns in the educational literature about the utility of the
lecture (Hurst, 2004), most of the SoM’s pre-clinical curricular content continues to be
delivered using large-class didactic methods. With the exception of clinical skills and the
research component, the objectives of all of the courses in each of the pre-clinical
instructional blocks are examined by a combination of multiple-choice and short answer
examinations administered at the end of each block. Students discover early on in their
course of study that the bulk of the exams require the recall of lecture material. This
causes many students to focus on knowledge components (the so-called “medical expert”
competency) at the expense of the content and competencies emphasized during non-
didactic teaching sessions.
7
The Research Setting
Currently taught in the winter term of the first medical year, the musculoskeletal
(MSK) course spans a 4-week period within a 19-week term. The course involves
collaboration between faculty from the Departments of Medicine (Rheumatology and
Infectious Disease), Pathology, Pharmacology, Physical Medicine and Rehabilitation,
Radiology, and Surgery (Orthopaedic and Plastic Surgery). Up to 25 faculty members
have historically been involved in the delivery of classroom lectures, often leading to
poor coordination of the material presented to students.
Since taking on the role of musculoskeletal course director, I have used
technology to enhance the course. While I initially used the University’s learning
management system (WebCT © Blackboard Inc) to distribute electronic versions of
lectures, over time, additional instructional possibilities became available. Early
experience in the use of technology, combined with a growing personal interest in active
learning techniques, has developed into a new instructional format for the course. Since
2005, a blended instructional design has emerged using a combination of self-directed e-
learning and classroom team-based learning (TBL). Authentic clinical cases are used as
the basis for much of the course content, in both virtual and face-to-face settings. The
8
process of change has been emergent and iterative, guided by student feedback and
informed by educational literature.
Research Questions
The purpose of this research was to develop a better understanding of the
implementation a non-traditional instructional model over a 3-year period. The larger
goal is to inform those who may be considering the introduction of similar initiatives in
other courses and programs of study. To fulfill this purpose I address three specific
questions:
1. What was the evolution in the design and implementation of the musculoskeletal
(MSK) course between 2005 and 2007?
2. How did each cohort of students respond as the course design evolved over time?
3. What were the critical elements that influenced (both positively and negatively)
the course outcome in each iteration?
Organization of Thesis
This chapter has set the stage by situating the work within the larger context of
medical education as it has evolved over the past century. The local context has been
outlined, and three research questions have been articulated to frame the subsequent
methodology, analysis and discussion.
9
Chapter Two consists of a review of the literature pertaining to key topics and
themes that explore the rationale underlying this instructional change. Chapter Three
outlines the methodologies involved in each stage of the research, from data collection to
analysis. Findings are presented in Chapters Four and Five, organized by each of the
three research questions.
In Chapter Four, both quantitative and qualitative data are used to answer the
original queries. This is followed by a reflective narrative (Chapter Five) detailing my
early experience as a faculty member in the School of Medicine, told from the
perspectives of a former medical student and developing teacher. The factors that
motivated me to explore new forms of instruction are described along with significant
milestones along the road to curricular change. These findings and analysis are discussed
in Chapter Six; linkages are made to the literature presented previously as well as that
related to complexity and change. Key elements that influenced this instructional
innovation are outlined. Future research directions are considered.
10
CHAPTER TWO: LITERATURE REVIEW
Overview of Chapter
The chapter begins with a consideration of the process by which university professors
learn to become teachers. A case is made for the consideration of active learning in the
design of instruction. TBL is described with particular reference to implementation in the
field of medical education. Blended learning is defined and best practices are outlined.
Learning to Teach in the Ivory Tower
Harpaz outlines four pictures that represent a traditional conceptualization of
schooling where “learning is listening; teaching is telling; knowledge is an object; and to
be educated is to know valuable content” (2005, p. 137). This model of learning
continues to inform instructional practices and curricular design at many medical schools
(Newman, 2002) and is intuitively accepted by many students, teachers, and decision
makers. When students are taught using traditional didactic methods, poor information
transfer and knowledge retention have been documented. Wieman (2007) suggests that
this may be related to the limitations in human short-term memory that have been
identified by cognitive psychologists, further noting that when teaching includes more
engaging methods, students’ achieve a better understanding of fundamental concepts.
While newer theories of learning have been used to inform medical educational
11
planning, the deeply embedded pictures of traditional learning have proven difficult to
change amongst front-line teachers and learners. After reviewing North American
medical curriculum reform since 1765, Papa and Harasym (1999) stated, “medical
educators must call upon and utilize the literature, research methods, and theoretical
perspectives of cognitive science if future curricular reform efforts are to move forward
efficiently and effectively.” (p. 154).
It is unfortunately the norm that fledgling professors lack specific training and
expertise as teachers. “Doctoral students who intend to become teacher-scholars
typically have little opportunity for systematic professional development as teachers;
many complete their doctoral studies without ever having taught a class, taken an
education course, or had any sort of organized opportunity to develop their teaching
skills” (Trautmann, 2008, p. 42). Austin, Sorcinelli and McDaniels have identified that
“teaching is a primary source of anxiety among new professors, many of who begin their
first academic positions with little or no preparation in teaching” (2007, p. 65).
New faculty members are generally hired for their content expertise and research
skills rather than their instructional repertoire; the former has been equated with
competence in the latter (McKeachie, 2007). Traditional transmissive instructional
methods (lectures and laboratories) have been used for generations and the presumption
12
is that new teachers will emulate what they themselves have experienced as students.
However, Trautmann (2008) suggests that these conventions are being broken down by
an improved understanding of how individuals learn as well as by an increasing diversity
of students and the emergence of educational technologies that facilitate less traditional
paradigms of learning.
Kugel (1993) has outlined five stages, which describe the development of a
professor as teacher (see Table 1). Early on, higher education teachers are concerned
with their own performance. As they gain confidence in the classroom, focus shifts to the
development of improved subject matter content. Dissatisfaction with student learning
prompts some to recognize the diversity of student learning approaches and other
qualities and causes them to adopt new educational techniques. While this may
accommodate a wider audience, the emphasis is still on teacher performance rather than
student engagement. A modern image of learning shifts focus away from the teacher and
centres instead on the learner. Kugel’s final two stages of teacher development recognize
the student as an active participant, aspiring toward the goal of educational
independence. The effective teacher, by this standard, is successful when she has been
rendered unnecessary.
13
Ten Cate, Snell, Mann, & Vermunt (2004) have described a two-part model that
can be used to reflect upon and plan teaching. These authors promote evaluation of the
amount of guidance that students require in each educational setting, reconceptualizing
the teacher’s role as providing just enough direction to promote the development of
“constructive friction between learning and teaching” (p. 221). This implies a paradigm
shift where the teacher’s role is to balance external guidance carefully with student self-
regulation, providing just enough structure within the learning experience to enable the
student to actively participate in the acquisition of new knowledge and skills appropriate
for their developmental level.
This type of participatory or active learning employs “a variety of cognitive
processes and verbal and nonverbal learner behaviours” (Kelly et al, 2005, p.112).
Talking, listening, writing, reading and reflecting occur simultaneously and
synergistically to increase assimilation and retention of knowledge. In addition to the
specific instructional elements listed previously, active learning also involves the use of
specific learning strategies and teaching resources that set the stage for student
engagement (Meyers & Jones, 1993).
Teachers bring a variety of personal experiences and biases that may shape their
approach. Five teaching perspectives have been suggested: information transmission,
14
Table 1. The five ages of a lecturer adapted from Kugel (1993) Stage Focus Characteristics Transitions 1 Self Teacher concerned with own
survival in the classroom. Once survival assured, focus shifts to subject matter.
2 Subject Teacher enthusiastic; works to extend own knowledge and share with students.
Beginning of awareness that student learning may not be optimal.
3 Student Teacher becomes aware of heterogeneity of students – adopts a wider variety of instructional approaches.
Teacher begins to understand the limits of focusing only on teaching approaches.
4 Student learning Teacher introduces student activities and opportunities for learning.
As students engage with their work their level of responsibility also increases.
5 Student as an independent learner
End point: the student knows how to learn for him/herself.
15
apprenticeship, development, nurturing, and social reform (Pratt, Collins,
Selinger, 2001). The related Teaching Perspectives Inventory (TPI) allows teachers to
articulate their beliefs, values and assumptions about teaching using this model. This can
serve to aid reflection and the study of instructional strategies.
There is a call for teachers of adults to be critically reflective in their practice of teaching. For several years now professions have pushed for their members to reflect critically on the underlying assumptions and values that give direction and justification to their work. For many teachers this is not an easy task. What is it that one should reflect upon? How are the underlying values and assumptions to be identified? In other words, the objects of critical reflection are not self-evident. Indeed, it is something of a new twist to look not only at the world, but at the very lenses through which we view the world (Pratt, et al., 2001. p. 8.)
Team-based Learning
Originally described by Dr. Larry Michaelsen, a professor of business at the
University of Oklahoma, TBL has been described as “a set of instructional principles
designed to foster the effectiveness of small groups working independently in large
classes with high learner:faculty ratios” (Kelly et al., 2005, p. 113). Many authors have
recogized that learning is an intrinsically social activity (Bruner, 1966; Chickering &
Gamson, 1987). When students work together in small groups, they are required to
communicate with and learn from each other, becoming actively engaged with the
material while being exposed to perspectives other than their own and as a result learning
more effectively (Fink, 2004; National Leadership Council, 2007). Rather than being
16
simply an adjunct to an existing lecture-based course, TBL is a specific teaching
strategy. It requires a fundamental shift in course structure by making small group
activities the primary classroom focus (Fink, 2004).
An essential element of this method is the transformation of student groups into
so-called high performance learning teams; this is achieved by careful design of learning
activities and assessment strategies to promote collaboration and accountability.
When the groups are properly formed, remain intact long enough to become cohesive teams, and are repeatedly given challenging tasks with prompt and clear feedback, then students learn the content, they learn how to use the content, they learn about themselves and about how to interact with others on major tasks, and they learn how to keep on learning after the course is over.” (Fink, 2004, p. 8).
This has particular appeal to medical educators seeking to embed the
development of essential competencies such as communication, collaboration,
management and professionalism into their instructional design.
TBL combines individual out-of-class preparation with in-class assessment and
group application exercises. Inter- and intra-group peer discussion is integral to the
learning that occurs. In medical school courses, classroom exercises usually take the
form of patient cases. There is a long tradition of case-based learning in medicine:
postgraduate medical training is built around the care of real patients both informally (as
part of clinical care) and formally (as “teaching rounds” sessions). The last 20 years have
17
seen a surge of interest in the inclusion of simulated case-based learning in
undergraduate medical education. In addition to promoting motivation, learning
activities designed around case-studies “promote independent study directed toward
issues that would constitute a realistic discussion in medicine” (Shanley, 2007, p. 483).
As they unravel, inquire about and make sense of case-based problems, students
are ideally engaged both emotionally and cognitively with the course content. This
engagement is intended to promote the processing and integration of ideas, creating
linkages with prior knowledge and facilitating subsequent use of course content and
concepts (Knapper, 2007) and, ideally, elaboration. Elaboration is a term used to
describe a type of knowledge organization that has been observed in successful
diagnosticians. These individuals have successfully organized their knowledge into
networks of relationship in memory and are able to relate these to novel problems, which
facilitates reasoning (Bordage, 1994).
Blended Learning
Blended learning has been defined as the combination of face-to-face and
electronic learning opportunities while reducing classroom contact hours (Dziuban,
Hartman, Moskal, 2004). “Electronic learning or more commonly, e-learning is an all-
encompassing term to describe learning supported by the use of information and
18
communications technology” (Academy of Medical Royal Colleges, 2007, p. 5). This
may take a variety of forms ranging from electronic posting of course materials to more
interactive models using self-directed modules, formative assessment, email, discussion
boards, wikis and other collaborative tools. The common theme is one of engagement,
with students using online resources to interact with course content and gauge their
knowledge and progress (Twigg, 2003). An important distinction is made between
blended learning and Web-enhanced courses that do not reduce classroom time. Inherent
in blended learning is a fundamental redesign of the instructional model, shifting from
lecture to student-centred instruction, increasing all forms of interaction and
incorporating formative and summative assessment (Twigg, 2003; Dziuban et al, 2004).
The Learning Technology Consortium Study examined the effect of the
introduction of Web-enhanced teaching in 7 universities in the United States (Wingard,
2001). Faculty goals for Web enhancements were categorized as either pragmatic or
instructional; it was noted that these two categories were not necessarily mutually
exclusive. Many faculty members reported changing goals over time “towards increased
awareness of instructional benefits” (p. 30). In many studies, the implementation of a
blended course was coexistent with a pedagogical redesign (active vs. passive learning)
and it is not always clear how to separate this out from the overall intervention (Riffell &
19
Sibley, 2003). Graham, Cagiltay, Lim, Craner, & Duffy, (2001) used Chickering and
Gamson’s (1987) “Seven Principles for Good Practice in Undergraduate Education” to
evaluate four online university courses. Their observations, which are summarized in
Table 2, provide a framework that can be used to evaluate other online courses and
modules.
In a recent review of the effective use of educational technology in medical
education, the AAMC Institute for Improving Medical Education highlighted the concept
“that use of technology be linked to what we know about learning” (2007). Harden
(2008), endorses this, commenting that “too often our students are subjected, in the guise
of e-learning, to an amateurish forrago that is neither an inspiring or rewarding learning
experience”.
The importance of attention to pedagogical principles and instructional design as
technology is supported by the results of the Pew Charitable Foundation Program in
Course Redesign (Twigg, 2003). Thirty institutions across the United States received
funding to support the redesign of existing college and university courses to include
technological tools. All of the institutions studied were able to reduce cost significantly
by implementing technologically enhanced instruction. Student satisfaction with
instructional methods and attitudes towards course content were improved, as were
20
course-completion rates. While different models of instruction were adopted at each
school, six common characteristics were identified across all of the redesign projects.
These included a significant increase in active and learner-centred activities that were
facilitated by technology.
Allen, Walls & Reilly (2008) describe the successful implementation of a
blended learning design (online modules combined with didactic lectures) in an
undergraduate medical anatomy course; however, these authors noted both opportunities
and pitfalls with the introduction of online modules. Motteram (2006) identified that
teachers must consider the time available for students to complete out-of-class online
tasks when designing a blended curriculum, noting that “if tasks are relevant and are set
up well, then they can help learners develop their knowledge and skills”(p. 29).
Combining face-to-face learning opportunities, especially those that encourage a
dynamic interplay between learner and teacher, with e-learning may help to obviate
some of the potential pitfalls of purely online instruction (Academy of Royal Medical
Colleges, 2007).
21
Table 2. Seven Principles for Effective Online Teaching Adapted from Graham, Cagiltay, Lim, Craner, & Duffy, (2001). Chickering and Gamson’s Principles Implications for online courses 1) Good practice encourages student-faculty contact
Instructors should provide clear guidelines for interaction with students.
2) Good practice encourages cooperation among students
Well-designed discussion assignments facilitate meaningful cooperation among students.
3) Good practice encourages active learning
Students should present course projects. This may be done asynchronously.
4) Good practice gives prompt feedback Instructors need to provide two types of feedback: information feedback and acknowledgment feedback. Peer feedback (to asynchronously posted projects, as in #3) can be used to enhance learning.
5) Good practice emphasizes time on task Online courses need deadlines. 6) Good practice communicates high expectations
Challenging tasks, sample cases, and praise for quality work communicate high expectations.
7) Good practice respects diverse talents and ways of learning
Allowing students to choose project topics incorporates diverse views into online courses.
22
CHAPTER THREE: METHODS OF DATA COLLECTION AND ANALYSIS
Overview of the Chapter
This chapter begins by restating the three research questions. For each question,
the methods of data collection are outlined followed by a summary of analytic methods
applied to the data. These are referenced back to the research questions and summarized
in Table 3.
Research Questions
As outlined in chapter 1, the purpose of this research is to develop a better
understanding of the implementation of a blended e-learning TBL instructional model in
an undergraduate medical course over a 3-year period. The larger goal is to inform those
who may be considering similar changes in their own instruction. To fulfill this purpose I
have collected data around three specific questions:
1. What was the evolution in the design and implementation of the musculoskeletal
(MSK) course between 2005 and 2007?
2. How did each cohort of students respond as the course design evolved over time?
3. What were the critical elements that influenced (both positively and negatively)
the course outcome in each iteration?
23
Table 3. Methods of Data Collection and Analysis Question What was the evolution in the design and
implementation of the musculoskeletal (MSK) course between 2005 and 2007?
How did each cohort of students respond as the course design evolved over time?
What were the critical elements that influenced (both positively and negatively) the course outcome in each iteration.
Method of data collection
Document review Document review Student research questionnaire
Document review Student research questionnaire Autobiographical narrative
Source of data
Course schedules Syllabi Administrative emails (staff/faculty) MSK course reports
Historical course evaluation data (2005 and 2007) All students enrolled in MSK course in 2005, 2006 and 2007
Historical course evaluation narrative data (2005, 2006, 2007) All students enrolled in MSK course in 2005, 2006 and 2007 Course director
Data to be collected
Structure of course as delivered in 2005, 2006 and 2007
Student attitudes towards instructional methods Student ratings of course effectiveness and suitability of curricular elements
Major themes that influence student opinion of each iteration of the course (course evaluation data and research questionnaire) Major themes identified by the course director in each iteration of the course (autobiographical narrative)
Analysis
Development of course description and chronology Identification of significant changes over time
Descriptive statistics and graphs (SPSS and excel) characterizes the responses of each cohort of students ANOVA (SPSS) compares the responses between cohorts and identify any significant differences
Grounded theoretical approach (Atlas.ti) used to identify themes expressed by both students and the course director
24
Methods of Data Collection
Question #1
The first question was addressed by reviewing documents relevant to the MSK
course. During the study period relevant documents were identified and catalogued
chronologically and by type (Appendix A). These included course schedules, syllabi,
reports, emails relating to administrative matters and other administrative documents.
These documents were used to develop a picture of the evolution of the course design
during the study period, highlighting significant changes in both the course and the
context within which it was implemented.
Question #2
This question was addressed using a combination of historical and original data
generated for this study.
Historical course evaluation data
Course evaluation processes are variable within the School of Medicine. While
many course directors rely on a standard set of questions administered by the
Undergraduate Medical Education office, others have designed evaluation processes
specific to their own courses. The musculoskeletal course is evaluated annually on the
last day of the course using a customized student survey. Completion of the survey is
25
voluntary and anonymous. Results are collated and circulated back to the course teachers
as well as to educational administrators within the School of Medicine. Use of these data
for this study was approved by the Health Sciences Research Ethics Board (REB).
The majority of survey items are drawn from the standardized University Survey
of Student Assessment of Teaching selection list (available at the institution used in this
study); however, the survey is modified annually as specific issues are identified from
previous surveys. For example, only 25% of students surveyed in 2005 identified that the
course web page was helpful and informative. Therefore, in 2006, four more specific
items were used to explore the theme of website utility (“The course website was easy to
navigate”; “The course website modules helped me to learn and better understand the
course material”; “Web-based quizzes and interactive cases helped me to learn and better
understand the course material”; “The online references linked to from the course
website were helpful and informative).
Survey items were completed using a standard 5-point Likert scale (McMillan &
Schumacher, 1997) ranging from 1: Strongly Disagree to 5: Strongly Agree. The MSK
survey does not include questions designed to evaluate specific faculty members;
however students may include narrative comments, which sometimes identify faculty
names. Any comments naming specific teachers have been removed from analysis in
26
order to preserve the privacy of individual faculty members. Survey items used between
2005 and 2007 are tabulated in Appendix B.
Student Research Questionnaire
For the purpose of this research, a three-part questionnaire was developed to
investigate student conceptions of different instructional methods: TBL, online learning
and lectures (Appendix C). The Health Sciences Research Ethics Board (REB) approved
the survey and research protocol. The survey was administered during the last 10
minutes of a MSK review session held in late April, just prior to the end-of-term
examination period. An information letter explaining the research project was circulated
electronically to the students prior to the review session. Completion of the survey was
voluntary and anonymous.
The first part of the survey included 21 items concerning attitudes towards
working with peers, the value of working in groups, online study and lectures.
Respondents were also asked to self-rate their mastery of the six global course
objectives. Items that explored attitudes towards different instructional models were
analyzed separately from those related to objectives. Each item was scored using a 5-
point Likert scale ranging from 1: Strongly Disagree to 5: Strongly Agree.
27
Nine of the survey items were taken from the Value of Teams Survey, which has
been used previously to evaluate student attitudes towards TBL (Levine, O’Boyle,
Haidet, Lynn, Stone, & Wolf, 2004). The original Value of Teams Survey consists of 13
items, 9 of which purport to measure two dimensions, working with peers (4 items) and
the value of group work (5 items). Eight of these items were included in the current
study’s survey. Omitted were the one negatively phrased item (“It is a waste of my time
to work in groups”) and three of the original survey’s distracters. While these items were
no doubt essential in enabling the authors to establish arguments for the instrument’s
reliability and validity, a decision was made not to include them in this student research
questionnaire based on the need to keep the survey a reasonable length.
Additional Likert-scale items were developed by this author to investigate student
attitudes towards online study and lectures. These were constructed to mirror the
phrasing of the group work questions (“an effective way to learn” and “an effective way
to practice what I have learned”). Because of informal student comments regarding their
preference for the efficiency of lectures, another two questions were added exploring
perceptions of efficiency in lecture-based and online learning.
The second part of the survey listed 11 items representing a set of idealized
educational objectives for the course from my perspective as teacher. These described
28
features of the instructional experience from the student point of view (“allows me
control of my learning”, “allows me to apply what I’m learning”, “helps me integrate my
learning”, “helps me judge the extent of my own learning”) as well as desired outcomes
or products of the experience (“a practitioner’s perspective”, “brings medicine to life”,
“helps me to become more reflective”, “strengthens my communication skills”).
This list was developed largely based upon my experiences and evolution as a
medical teacher; however, it was also informed by the needs expressed by students in
formal and informal course evaluation feedback over the years (“organizes a large
amount of material”, “guides me through difficult topics”). For each objective, students
were asked to rank the three instructional methods included in the course in terms of
their suitability to achieve the stated goal (from 1=most suited to 3=least suited).
Question #3
The historical course evaluation data provided insight into the effects of course
innovation as experienced by the students as well as their perceptions of the teaching and
learning activities included in the course. As such, these data contribute to answering the
third research question. In addition, the final part of the student research survey included
two open-ended questions. Students were asked to complete statements describing the
most useful features of online study and TBL from their perspective. While there are
29
drawbacks to relying exclusively on student feedback data as the only method of
evaluation of teaching, “students are uniquely placed to provide feedback on education
delivery and the overall education experience - after all, they are at the point of use and
are essentially the customer base. They are in the ideal position to evaluate learning and
teaching and provide constructive opinion” (Quality Assurance Agency for Higher
Education).
In considering the transformation of the MSK course, it is important to include
the teacher’s voice, which in this case is autobiographical. Given that the results of the
study may inform other instructors who may be considering the adoption of e-learning
and TBL, an exploration of my own motivations, experiences, and reactions over time
provides a context within which to interpret the findings that are described. To this end, a
narrative is included in Chapter Five, intertwined with the other more traditional
elements of this thesis.
Methods of Data Analysis
Multiple methods of analysis were used because of the variety of the data. The
initial document review allowed the development of a chronology of changes in course
structure. Course evaluation data from the study period was analyzed using SPSS a
statistical software package. Eight questions were common to all 3 years of course
30
surveys (Table 4). Descriptive statistics for these items were tabulated by year to identify
positive or negatives trends in student opinion. These were compared using an analysis
of variance order to identify any significant differences between the groups.
Research survey items for all 3 years relating to student impressions of different
instructional methods (group work, online learning and lectures) as well as their self-
perceptions of mastery of the course objectives were also analyzed using SPSS.
Descriptive statistics for each question were stratified by year, as with the course survey,
to identify trends. Survey items were designed to be grouped into three dimensions
(similar to the Baylor Value of Teams survey), each reflecting student attitudes towards
different instructional techniques. The validity of these groupings was explored using a
principal component analysis, which is described in detail in Chapter Four.
Student rankings of the suitability of each instructional method in achieving each
of the 11 educational objectives were tabulated and graphed using Excel software. Some
students only recorded their highest or lowest ranking for individual objectives; therefore
only complete datasets (where a ranking was attached to all three instructional methods)
were analyzed. While this approach necessitated the exclusion of some datasets (and
31
Table 4. Common Course Evaluation Survey Items from 2005 and 2007 1. Overall this is an excellent course 2. I learned a great deal from this course 3. The course was helpful in developing new skills 4. I felt that this course challenged me intellectually 5. My interest has been stimulated by this course 6. The course covered the right amount of material 7. The workload in this course was reasonable and appropriate 8. The course material was presented at a satisfactory level of difficulty
risks skewing the data), it was felt to be justified as the majority of responses were
complete.
Graphs were developed for each educational objective, stratified by year
(Appendix D). These outlined the number of students who had ranked each educational
technique from 1 (most appropriate) to 3 (least appropriate). Changes in the rank order of
the different methods were compared over time to identify evolving student perceptions
of teaching and learning.
Narrative comments from both the course evaluation and student research
surveys were coded to reflect the survey (CES or SR), the year (05, 06, or 07) and the
individual (a non-identifying number used to differentiate answers from different
students). These were analyzed using a grounded theory methodology (Patton, 1990).
The comments were coded and grouped, uncovering themes emergent from the
32
data. Data from each cohort of students were analyzed separately, identifying trends over
time. Finally, my experiences as a teacher were outlined in narrative form utilizing a
reflective approach as advocated by Richardson, who noted that “experiences are
connected to other experiences and are evaluated in relation to the larger whole” (1997,
p. 30).
33
CHAPTER FOUR: RESULTS
Overview of the Chapter
The chapter begins with a description of student demographics during the study
period. Using data identified from the document review, a picture emerges of the
changing shape of the course over the 3 years, providing an answer to the first research
question.
The second research question is addressed by comparing the attitudes and stated
preferences of each cohort of students, using data from historical course evaluations as
well as that from the student research survey. The latter is analyzed in three parts:
attitudes towards different instructional modalities, self-perception of mastery of course
objectives and student perceptions of the advantages of TBL and e-learning. Shifts in
student attitudes over time are identified.
Narrative data from both students and teacher (Chapter Five) are presented to
help unravel the third research question. These data are used to uncover facilitators and
detractors that contributed to the shape and pattern of educational change in the MSK
course between 2005 and 2007.
34
Demographics
Student enrollment, sex breakdown, and province of origin are summarized in
Table 5. The composition of each of the three classes was similar other than an increase
in the proportion of females over time (see Table 5). One of the students enrolled in the
course in the first year was originally a member of the previous class who returned
following a leave of absence. (This student is not included in Table 5).
Table 5. Student Demographics
Question #1
Introduction to Course
An hour-long introductory session was held on the first day of the course in all 3
years (Table 6). In all years, standard items such as course objectives and evaluation
structure were outlined. In the second and third years there was increased emphasis on an
explanation of the instructional format. Students completed a sample readiness
Research study year 2005 2006 2007 Class size 100 100 100 Male:Female 52:48 45:55 46:54 Province of origin: Ontario Alberta/BC Quebec Other/USA
76 14 2 8
70 22 0 8
78 16 1 5
35
assessment test based upon a short reading available in class to illustrate this component
of TBL. In 2006 and 2007, a tour of the course website was included outlining expected
preparation and evaluation milestones. In the third year, student teams were created
during this introductory session by having students line up and sort themselves based on
teacher-determined criteria.
Changes in Course Content over Time
While the bulk of the course content was relatively stable over time, in the third
year, there was one significant change. In 2005 and 2006, several sessions were devoted
to teaching ‘Pain’ topics relevant to the MSK system. This was done collaboratively with
teachers from the Palliative care course that is taught in parallel to the MSK course. In
2007, all of the Pain teaching was consolidated within the Palliative care course,
resulting in a reduction of MSK course instructional time by 480 minutes in comparison
to the previous year. It should be noted that these differences are largely a form of
curricular bookkeeping, important to the administrators of the undergraduate program.
From the student point of view, the curriculum is seen as a whole; therefore, this change
was largely invisible to the learners involved.
36
Classroom Time Weightings by Instructional Method
Total classroom time scheduled appears to decrease over the study period;
however, if the analysis is done eliminating ‘Pain’ teaching in the first 2 years, there is
little change in total classroom time for each year. In all years, didactic methods
predominated; however, over the period of the study there was an increase in both the
absolute and the relative amount of time scheduled for TBL and a parallel reduction of
time scheduled for didactic teaching (Table 7). The differences between 2005 and 2007
are less pronounced if the Pain sessions are removed from analysis (Table 8); however,
even with this adjustment, there is a 15% reduction in didactic teaching over this period.
Table 6. Content of Introductory Sessions
2005 2006 2007 Course aims Evaluation Personal and global relevance Team activities
Course aims Evaluation Overview of content Linkages to Phases I and III Description TBL Sample RAT (Global relevance) Website tour
Course aims Evaluation Overview of content Linkage to Phase I and III Group formation Description TBL Sample RAT (Global relevance) Website tour
37
Physical Space
In the first year of the study, all whole-class sessions (lecture and TBL) were
scheduled into a fixed seating, tiered auditorium in the basement. This auditorium, which
seats 107, is used for most of the teaching in Phase IIA and had been booked 12 months
in advance, consistent with the University’s usual booking practices. In the second year,
most of the TBL sessions were relocated to a former hospital gymnasium. Digital
projection and amplification equipment was brought in for each session in the
gymnasium. During these sessions students were seated at round tables to facilitate team
activities. Some lecture sessions were also scheduled in the gym in order to minimize
student travel between classes. One TBL session was re-located to a larger auditorium
because of double-booking which prevented use of the gym.
During the third year, all of the lecture sessions were scheduled in the fixed
auditorium. Student feedback from the previous year indicated that space suitable for
TBL was suboptimal for lectures. At the suggestion of one of the Meds 2009 students, a
large classroom (and sometimes research lab) on the top floor of one of the engineering
buildings was booked for most of the TBL sessions. This room was equipped with
reconfigurable tables and chairs with a capacity of approximately 100. As with the
gymnasium, digital projection equipment was not available. Two other smaller teaching
38
spaces were used for TBL in the third year. Because these rooms did not accommodate
the entire class, these TBL sessions were repeated more than once.
Course Website
The course website was redeveloped each year of the study as new information
technology resources became available. The technological platforms used, site content
and other resources are summarized in Table 9.
In 2005, WebCT was used to post administrative data (such as contact
information), course content (online module, presentations, references, videos and links),
formative evaluation (self-administered quizzes), and to allow asynchronous discussion
between students, faculty and the course director. The course schedule and objectives
were also posted on an additional website, the electronic portal used by all courses in the
pre-clinical portion of the undergraduate medical program.
In 2006, course content was migrated to a new website created using qWeb, a
cross-platform, open-source content management system developed by the medical
education technology unit of the Queen’s School of Medicine (MedTech). The website
included a learning map consisting of schedule information, objectives, and a detailed
syllabus identifying the required advance preparation for each classroom session. Simple
online modules were created using hyperlinked web pages to guide students through
39
content material formerly delivered in lecture format. Interactive formative assessment
questions and exercises were added to the site using third-party freeware: Hot
PotatoesTM (Half-Baked Software Inc) and Eclipse Crossword (©Green Eclipse).
One drawback to qWeb was a lack of specific functionality such as a discussion forum,
gradable quizzes and delayed file release. A stripped-down version of the course WebCT
site was maintained in order to provide asynchronous discussion as well as a formative
multiple-choice exam. Students continued to access their schedule using OCR; this was
also used to provide delayed release of notes from some classroom sessions.
Student feedback was helpful in streamlining and simplifying the course website
for the 2007 version of the course. WebCT was abandoned completely because an online
forum, newly implemented by MedTech, was now available, providing a tool permitting
asynchronous online collaboration and discussion. Course content modules were
reshaped using a new technological platform, Thinking Cap® Studio (Agile Software),
an online e-learning authoring system. These modules were able to incorporate multiple-
choice and short-answer questions, both with automated feedback capabilities.
40
Table 7. Time Allotted to Different Instructional Methods (including Pain teaching) Instructional time expressed in minutes (% scheduled instructional time)
Table 8. Time Allotted to Different Instructional Methods (excluding Pain teaching) Instructional time expressed in minutes (% scheduled instructional time)
Year 2005 2006 2007 Total didactic • Lectures – single teacher
• Symposia – team taught
• Presentations – student taught
1980 (75%) • 1110
• 720
• 150
1650 (64%) • 840
• 810
1350 (60%) • 720
• 630
TBL 660 (25%) 930 (36%) 900 (40%)
Total classroom time 2640 2580 2250
Year 2005 2006 2007 Total didactic – minutes • Lectures – single teacher
• Symposia – team taught
1590 (71%) • 870
• 720
1170 (56%) • 840
• 810
1350 (60%) • 720
• 630 TBL 660 (29%) 930 (44%) 900 (40%)
Total classroom time 2250 2100 2250
41
Methods of Student Assessment
Over the course of the study period, there were major changes to the evaluation
structure of the MSK course. These are outlined in Table 10. In the first year, 75% of a
student’s final grade was based upon end-of-term multiple-choice and short-answer
examinations questions. These questions were administered as part of comprehensive
examinations including content from all courses taken during the winter term of first
year.
During the second and third year of the study, the examination weightings
dropped significantly to 50% (2006) and 40% (2007). There was a corresponding rise in
the proportion of the grade determined by TBL activities, which was 13% in 2005,
37.5% in 2006 and 40% in 2007. Peer evaluation was used in all 3 years of the study.
Students assigned marks to their team members based on their contribution to team
functioning and learning. These peer grades were pooled to develop a composite peer
evaluation score for each student. In the first 2 years of the study, the peer evaluation
score was applied as a multiplier to the group marks, raising or lowering the team portion
of student grades. In the third year of the study, the peer grade was only applied if it
raised the team grade. While students with negative peer scores did not lose any grades
42
Table 9. Course Resource Material by Year
2005 2006 2007 Website WebCT:
Cases Contact information Discussion board Links (references, web sites) Module (1) Objectives Presentation/notes Quizzes References Schedule Videos Online Course Resources: Schedule Objectives
WebCT: Discussion board Formative exam qWeb: Contact information Modules (12) Quizzes (formative) Learning map Objectives Presentations/notes References Video clips Online Course Resources: Schedule Delayed-release notes Presentations/notes
qWeb: Contact information Modules (12) with embedded formative quizzes Learning map Presentations/notes Link to MedTech discussion forum Question of the week References Video clips Online Course Resources: Schedule Objectives
Other Chapters from 3 textbooks Outline notes (120 pages)
Chapters from 2 textbooks Chapters from 2 textbooks
43
in 2007, the score and grouped anonymous peer comments were fed back to each
individual at the end of the course.
In addition to TBL-related grades (both individual and group) and end-of-term
exams, students submitted an individual assignment in all 3 years. This case-based task
replaced previous didactic teaching about osteoporosis, requiring students to use the
Information Literacy skills acquired in the fall term to research this disease and its
management. While the specifics of this assignment changed over the course of the
study, the essence of the task remained the same. The grade assigned to this task was
higher in the second and third years than in the first.
In the first and third years of the study, the course evaluation scheme included
group assignments requiring students to work together outside of scheduled classroom
TBL time. While the weighing of this was the same (5%) in both years, the 2005 project
was quite substantial (a research-type poster and classroom presentation about a pain-
related topic) in contrast to that in 2007 (development of a patient information handout
about osteoporosis). Students were encouraged to collaborate asynchronously using the
discussion board for the latter task.
44
Table 10. Assessment of Student Achievement and Grade Weightings 2005-2007
2005 2006 2007 Individual grade Individual Tests 7% 12.5% 10% Individual assignment 5% 12.5% 10% Participation in online discussion N/A N/A 5% Expanded clinical skills attendance 2% N/A N/A Group grade Group Readiness Assessment Tests 6% 12.5% 30% Group Application Exercises N/A 12.5% N/A Group assignment (out-of-class) 5% N/A 5% Peer evaluation grade could modify team grade
Up/down Up/down Up
Summary of evaluation weighting In-course work • Individual grade
• Group grade
25% • 14%
• 11%
50% • 25%
• 25%
60% • 25%
• 35% End-of-term examinations • Multiple Choice Question exam
• Short Answer exam
75% • 25%
• 50%
50% • 25%
• 25%
40% • 20%
• 20%
45
Question #2
Historical Course Evaluation Data
Quantitative Data
A comparison of end-of-course student feedback surveys from 2005 and 2007
revealed a statistically significant improvement of student ratings of the course on all
eight items common to both years (Table 11). Quantitative data from the 2006 course
evaluation survey were not used due to administrative irregularities at the time of survey
administration leading to some duplication of data. This did not affect the narrative
portion of the survey, therefore all 3 years’ qualitative responses were analyzed (see the
next section).
In 2005, the mean response for the statement “overall, this is an excellent course”
was 3.35, which represents a neutral (“neither agree nor disagree”) rating. Two years
later, the mean for this statement rose to 4.34, representing a point in between “agree”
and “strongly agree.” There is a parallel reduction in standard deviation associated with
this question from 1.027 (2005) to 0.621 (2007). While the larger response rate in 2007
(90%) in comparison to 2005 (52%) may be partially responsible for this change, this
may also suggest greater clustering of opinion around the more positive answer in the
most recent survey.
46
Of the eight items compared, all but one in the 2005 survey averaged a score of
less than 4 (agree). In contrast, in 2007, all but one item averaged a score greater than 4.
The one item that averaged between 3 and 4 in both years was the statement “the
workload in this course was reasonable and appropriate.”
Analysis of Narrative Responses
During the period studied, several themes emerged from student narrative
comments on the end-of-course surveys. These are summarized in Table 12. The tone of
the narrative changed over the study period. In 2005, feedback was generally negative.
Students felt that TBL was an ineffective instructional method and had concerns about
the organization of the course. The website was felt to be confusing and students had
difficulty identifying how to prepare for each classroom session. A definite preference
for more traditional types of instruction was expressed.
The classroom used for TBL was identified as an obstacle to success because of
space limitations. While students recognized that a large amount of effort had gone into
the course design, they were particularly harsh in their evaluation of me as course
director. For example, “Dr. Davidson is by far the worst instructor I have ever had at the
university level” (CES0511). The following year was notable for a positive shift in
student opinion of the course. TBL was now described in mostly positive terms, as were
47
the web-based modules. Students were appreciative of the work that had gone into the
course and were much more positive towards me, personally. “I appreciate that you care
so much about our learning – so far, you’re the most involved and conscientious
professor we’ve had – Thanks!” (CES0630)
The biggest set of student complaints revolved around the use of multiple
websites (WebCT, the MSK qWeb site and OCR) and resultant difficulty accessing
printable course notes. Students were particularly frustrated when course notes were
posted using a delayed release following the relevant classroom session. This group of
students expressed concerns about the classrooms used during the course: the gym had
poor acoustics, poor visibility and uncomfortable chairs. The gym had been used as a
venue for half-day teaching blocks, including some lectures and symposia, and this was
felt to be particularly problematic for these types of teaching.
Student opinion of the course was enthusiastically positive in 2007. “I have never
learned so much in a semester before! I think this course changed the way I approach
learning” (CES0748). The course was felt to be well organized with clearly stated
expectations. While students noted an increased workload, they indicated that there was
an educational “pay off.” For example, MSK was described as a “very good course,
more work but better in the long run, team learning was good” (CES0728). The online
48
Table 11. Comparison of Common Course Survey Items from 2005 and 2007 Study
year n= Mean Standard
Deviation t Sig (2-
tailed) 1. Overall this is an excellent course
2005 2007
52 90
3.35 4.34
1.027 0.621
-6.370 .000
2. I learned a great deal from this course
2005 2007
52 90
4.00 4.34
0.840 0.640
-3.467 .001
3. The course was helpful in developing new skills
2005 2007
52 90
3.60 4.12
0.846 0.684
-3.819 .000
4. I felt that this course challenged me intellectually
2005 2007
52 89
3.69 4.33
0.829 0.636
-5.092 .000
5. My interest has been stimulated by this course
2005 2007
52 88
3.63 4.06
1.067 0.902
-2.499 .014
6. The course covered the right amount of material
2005 2007
52 90
3.58 4.11
0.936 0.678
-3.605 .001
7. The workload in this course was reasonable and appropriate
2005 2007
52 90
3.10 3.68
1.089 0.992
-3.246 .001
8. The course material was presented at a satisfactory level of difficulty
2005 2007
52 89
3.60 4.24
0.891 0.523
-4.723 .000
49
modules were popular for preparation, for self-assessment and for review. TBL was also
seen as a positive experience that enhanced learning. Students commented that these
aspects of the course enhanced their learning and allowed them to integrate and
consolidate knowledge. In contrast, online asynchronous discussion, which focused
around the “Question of the Week” activity, was unpopular and generally felt to be
unhelpful. There were many comments similar to this one: the “discussion board seemed
like a waste of time to me.” (CES0759).
While students continued to express frustration when notes were posted
following a session rather than before, there were few concerns raised about the course
website. There were several comments about TBL classrooms: the engineering
classroom was unpopular because of poor visibility from the back of the room and
because it felt cramped, but many fewer concerns were raised than the previous year. A
number of students simply offered that they did not like moving between classrooms.
Student Survey Results
Response Rate
In 2005, 61 students (60%) of the class completed the research survey
administered as part of this project. There were a similar number of respondents 66
(66%) in 2006 and 85 (85%) in 2007. Although administered 3 months later than the
50
course survey, patterns of response are similar, with 51% of students completing the
course evaluation survey in the first year and 90% of students completing it in the third
year.
Analysis of the survey is divided into four sections: comparison of attitudes to
instructional methodologies (items 1-15), student perception of achievement of course
objectives (items 16-21), ranking of the suitability of instructional modalities to achieve
educational objectives (items 22-32), and the most useful features of the active
instructional modalities used in the MSK course (items 33-34). In developing the student
research survey, the original intention was to group items by each of the three
instructional methods and generate scores that could be compared across the three
cohorts.
In order to determine the validity of this approach, a principal component
analysis was performed including the 14 items related to instructional methods. (Item 2
was taken from the Value of Teams survey where it was considered a distracter,
therefore it was not analyzed.) This identified two sets of items that grouped strongly
together (Table 13). The first component included all questions linked to TBL and online
learning; the second included questions linked to lectures. Because of this, items were
grouped to create two scores (rather than three) differentiating between student attitudes
51
Table 12. Summary of Findings and Themes from Course Evaluation Surveys Theme 2005 2006 2007 Online modules “Internet computer teaching aids
do NOT substitute for effective lecturing”(CES0501); “I don’t mind the online quizzes. I found that I did more reading and understood the subject, however I then found it very difficult to sit through a full lecture on the same topic.” (CES0508)
“I thought the modules were a good learning tool” (CES0603); “Online modules were very helpful for consolidating information” (CES0617); “will be great references over the next few years”(CES0630)
“I really liked the modules, interactive, lots of pictures, clinical examples and kept me on top of homework” (CES0724); “I really liked the online modules, but everything online could be easier to navigate.” (CES0754)
Classroom “group based learning in a lecture hall does NOT work” (CES0501)
“Burr Gym is NOT a suitable classroom (poor air/too cold/hard to hear)!!!” (CES0605); “Patio chairs are not comfortable when sitting for hours” (CES0620); “Burr gym is not a great classroom – we understand its purpose in TL sessions, but please make sure all lectures are in B139” (CES0655)
“Jackson Hall was awful, especially for those at the back” (CES0751)
Organization “overall organization was a mess” (CES0502); “with so many instructors it must, at times be difficult for everything to run smoothly” (CES0518)
“ORGANIZATION IS NEXT TO GODLINESS. LEARN HOW PLEASE.” (CES0615)
“I think the MSK course was well organized” (CES0709); “Very well organized. I hope the rest of phase II courses are as well put together” (CES0708)
52
Theme 2005 2006 2007 Team-based learning
“Group work: WE ARE NO LONGER IN ELEMENTARY SCHOOL! The group quizzes are useless – we all look up answers we didn’t know before we write and then waste valuable time in a pointless exercise” (CES0514); “in class quizzes were a waste of time” (CES05004); “I personally felt a lot of pressure to “collaborate” on the evaluation in order to ensure all group members received the same mark in the end” (CES0518)
“Team learning works well. I think that I learned a lot more than in a lecture” (CES0604); “I really liked the group learning method (I am dreading returning to lectures next week!)” (CES0657)
“I loved the in-class group work and well define[d] studying before class” (CES0717); “The team based learning was one of the most rewarding learning experiences I have had in university” (CES0746); “I enjoyed the team learning – helped me retain the material better” (CES0706)
Website “website is a pain to use; message board is awful, quizzes were good, resources in too many locations” (CES0515)
“Website was confusing” (CES0605); “Too many websites. It was very difficult to find what I needed and I often ended up coming to class without the notes I needed” (CES0604); “It would be more useful to combine all 3 sites into one.” (CES0618)
“Although the MSK website was well organized, it would be easier to just put the information on the MedTech [Online Course Resources] since we still have to use MedTech for other courses for information. Getting as much information as possible in one place would be great”(CES0753)
53
to active and didactic learning methods. Survey items were combined into two scores
representing active learning (items 1 and 3-12) and didactic learning (items 13-15). All
items were weighted equally to develop the two scores. A one-way analysis of variance
(ANOVA) was then performed which demonstrated a significant difference in each of
the two scores compared across the 3 different years of the study. (Some data sets were
incomplete, therefore the response rates reported for the grouped data are slightly lower
than those noted for the survey as a whole). As illustrated in Table 14, there was a
deterioration of the scores for didactic learning over time (from midway between
“neutral” and “agree” to just below “neutral”) with an improvement in the active
learning scores (from midway between “neutral” and “agree” to “agree”).
Educational Objectives
Analysis of student rankings of the different instructional methods with respect to
the 11 educational objectives that I had defined revealed significant changes over time.
These results are summarized graphically in Appendix D. In 2005, three-quarters of the
students surveyed identified lectures as the optimal method to organize material and
guide student learning (Graph D1); lectures were also the top ranked method to give
students a practitioner’s perspective (Graph D2) and to guide students through difficult
54
Table 13. Principal Components Analysis of Statements about Instructional Methodologies
Component STATEMENTS 1 2 1. The ability to collaborate with my peers will be necessary if I am to be successful as a student.
0.668
2. I have a positive attitude about working with my peers.
Omitted Omitted
3. The ability to work with my peers is a valuable skill. 0.689 4. Collaborating with my peers will help me be a better student.
0.781
5. Solving problems in a group is an effective way to practice what I have learned.
0.807
6. Solving problems in a group is an effective way to learn.
0.718
7. Working in teams in class is productive and efficient. 0.679 8. Group decisions are often better than individual decisions.
0.636
9. Solving problems in groups leads to better decisions than solving problems alone.
0.709
10. Online study modules are an efficient way to learn. 0.682 11. Online study modules are an effective way to learn. 0.696 12. Online study modules are an effective way to practice what I have learned.
0.543
13. Lectures are an efficient way to learn. 0.882 14. Lectures are an effective way to learn. 0.882 15. Lectures are an effective way to practice what I have learned.
0.719
55
Table 14. Comparison of Didactic and Active Learning Scores over Time Year N Mean Standard deviation Mean
(normalized out of 5) Didactic Learning
2005 2006 2007
61 65 84
10.24 9.92 8.98
2.28 2.43 2.18
3.41 3.31 2.99
Active Learning
2005 2006 2007
60 66 84
37.37 41.68 44.21
6.51 6.68 7.90
3.40 3.79 4.01
df F Significance Didactic Learning
Between Groups Within Groups Total
2 207 209
6.182 0.002
Active Learning
Between Groups Within Groups Total
2 207 209
16.084 0.000
56
topics (Graph D6). Online study methods were felt to be superior in encouraging
reflection and integration (Graphs D4 and D9) and in allowing students to control, apply
and assess their learning (Graphs D5, D8 and D10). The provision of feedback was also
associated primarily with online study (Graph D7). TBL was ranked last in the categories
of organization, guidance, reflection and control of learning, but it was the most
commonly ranked method to strengthen communication and collaboration (Graph D11).
Of note in 2005 was the absence of predominance of any of the three instructional
methods in bringing medicine to life for the students (Graph D3).
In 2006, student preference shifted significantly towards TBL and away from
lecture-based learning and online study. Online study was identified as the optimal
method to organize material by just over half of the students (Graph D1); this
instructional method was also felt to be the most appropriate to allow students to control
their learning (Graph D5). Opinion regarding the method best to guide students through
difficult topics was evenly split between lectures and online study (Graph D6).
Similarly, TBL and lectures were identified equally as useful methods to provide a
practitioner’s perspective (Graph D2), and online study and TBL were both identified as
important techniques to help students judge the extent of their own learning (Graph
D10).
57
This group of students identified TBL as the top choice for seven categories
including: the provision of a practitioner’s perspective (Graph D2), bringing medicine to
life (Graph D3), reflection (Graph D4), the provision of feedback (Graph D7),
application of knowledge (Graph D8), integration (Graph D9), collaboration and
communication (Graph D11).
The cohort of students surveyed in 2007 demonstrated a clear preference for TBL
and loss of favour for lectures. TBL was ranked first in seven categories (bringing
medicine to life, helping reflectiveness, feedback, application of knowledge, integration,
judgment of extent of learning and communication/collaboration) and lectures were
ranked last in eight categories. Online study was ranked first in the categories of
organization (Graph D1), control of learning (Graph D5) and guidance (Graph D6).
Course objectives
There was no significant difference noted in student self-ratings of mastery of
five course objectives related to MSK conditions. In all 3 years, the mean rating was in
the neutral (“neither agree nor disagree”) category. The sixth course objective was
related to understanding the principles of effective group work. The average rating for
this question improved over time from 3.69 (in 2005) to 4.09 (in 2007), which was
statistically significant (F=7.774, 0.001).
58
TBL and Online Modules: Student Perspectives
The major themes that emerged from the data are outlined in Tables 15 and 16.
Student comments from all 3 years indicated that online modules enhanced their
learning. The modules allowed and motivated students to prepare for class, to practice
what they had learned and to self-assess their understanding based upon feedback.
Students felt that the modules were useful to help them to identify gaps in their
knowledge.
Over the course of the study, an increasing number of comments cited the focus
and organization of the modules as strengths. An important theme was that of student
autonomy: the modules were self-paced and could be used flexibly and selectively to
improve understanding or review content based upon identified gaps in knowledge.
Students also commented on the value of the modules as a resource that they would
return to.
The most common theme identified in the first year from the narrative question
dealing with TBL was that it was an unhelpful instructional method. While there were
some indications of continued unpopularity in the second and third years, these were
minor in comparison to the feelings expressed by students in 2005. The strongest themes
emerging from the student responses in the second and third years were the value of
59
learning with peers, the ability to practice recently acquired knowledge, enhancement of
knowledge and self-assessment.
60
Table 15. Major Themes Related to Online Learning Enhancement of learning
2005: “I really liked this, it forced you to read around and really understand a topic in order to answer the online quiz” (RS05142) “Test myself, apply concepts. Cement vague knowledge from lecture” (RS05137) 2006: “Focus on what is important – learn details better” (RS06150); “It summarized the information in one place” (RS06157); “prepare for lecture using the most succinct resources – most important stuff was easy to find” (RS06181); “I could easily go back to things I didn’t understand & integrate it” (RS06165); “solidify my knowledge” (RS06209) 2007: “learn concepts thoroughly” (RS07227); “learn the material in an organized way” (RS07228); “consolidate background knowledge”; (RS07240) “Recognize key points + get info effectively without having to search through many references” (RS07261)
Ability to self-pace learning
2005: “Learn at my own pace”(RS05103); “Use my time as I want” (RS05112) 2006: “Review information at my own speed” (RS06154); “go through topics in my own time” (RS06188) 2007: “Go at my own pace to fully understand the learning objectives” (RS07224) “go through well organized info at my own pace – refer back to when needed” (RS07226) “Set the schedule of my own learning and navigate through the learning objective[s]” (RS07283)
61
Table 15 (continued). Major Themes Related to Online Learning Self-assessment 2005:
“Excellent way to assess how much I have learned” (RS0594) “Evaluate whether I had learnt the material to the current depth, and identify areas that I need to work on” (RS05125)“learn with immediate feedback” (RS05109) “Find my weaknesses” (RS05146); 2006 “focus on areas in which I needed the most practice” (RS06192) 2007: “fill in gaps in my knowledge” (RS07255); “review and test myself” (RS07232); “Get quick feedback for my learning” (RS07270)
Organization 2005: none 2006: “understand the basics and details on MSK topics, which were clearly organized” (RS06160) 2007: “organize + partition information in a manageable form” (RS07266) [the online quizzes] “were more efficient + condensed than a 3 hr lecture” (RS07277) “piece and parcel a large amount of information in a purely factual way. It’s efficient.” (RS07286)
62
Table 16. Major Themes Related to Team-Based Learning Peer learning 2005:
“Collaborate, look at things from a different perspective” (RS05109); “interact with future colleagues” (RS05093); “Gain knowledge only from what others knew - knowledge was incomplete, so not my preferred way of learning” (RS05131) 2006: “Learn from others” (RS06150) “Broaden my understanding of material thru exposure to group members experiences/reasoning” (RS06165) “Show others how much I know” (RS06193) 2007: “Build knowledge thru interactions w colleagues” (RS07216) “get help from others” (RS07220) “consider other viewpoints” (RS07232)
Enhancement of learning
2005: “Excellent way to go over material that was presented. Can use it to teach if more organized.” (RS05094) “Do nothing that lectures didn’t already provide” (RS05139) 2006: “consolidate my learning” (RS06162) “apply my knowledge – see if I could explain the knowledge to others (good test of understanding)” (RS06172) “Talk about more different cases which helps solidify the basics” (RS06211) 2007: “apply + reinforce concepts” (RS07227) ; “expand my knowledge base” (RS07232) “challenge the way I thought about the material and consider other approaches” (RS07249) “come prepared to meet the expectations of my team and with and approach to the material” (RS07283)
63
Table 16 (continued). Major Themes Related to Team-Based Learning Practice what I’ve learned
2005: “use the knowledge I gained from online modules and apply it to patient cases” (RS05138) 2006 “practice, apply, interact” (RS06178) 2007: “put knowledge to work, problem solve” (RS07216) “apply concepts to real-life problems” (RS07230) “Think about things + put it into practice” (RS07252) “Practice what I’d studied and learn how other people study (like memory tricks). This was the best way to work on clinical scenarios” (RS07287)
Not useful 2005: “Waste time in class” (RS05239) “I don’t learn effectively in small groups” (RS0595) “Become annoyed and waste my time” (RS05119) 2006: “I absolutely hated it. The volume of information was large enough without having to worry about a competitive team environment” (RS06131) 2007: “Didn’t do anything. Could’ve been made into online modules. It let the “know-it-all” personalities to shine, but anyone who is reserved or shy was pretty much shut out of discussion.” (RS07238)
64
CHAPTER FIVE: THE TEACHER’S TALE
Prologue
In 1994, shortly after joining the School of Medicine as an assistant professor, I
sat in my office across from the Chair of the Division of Orthopaedics. During the course
of the conversation he mentioned that I might be interested in taking on the role of
undergraduate educational coordinator. I started to say that I would consider this and
then I stopped … clearly, from his expression, it was a rhetorical question. The job was
mine.
At that time I was embarking on a career as a pediatric orthopaedic surgeon. My
expertise lay in the area of broken bones, infants with clubfeet and intoeing toddlers. My
skills involved diagnosing childhood musculoskeletal problems, straightening bent
limbs, applying casts, and wielding power tools (safely) in the operating room. Along the
way, largely by osmosis, I had learned how to behave like a physician. This meant
knowing how to work well with others, developing the ability to communicate with a
wide variety of people, learning to research and solve patient problems that I had never
previously encountered, juggling many competing demands, advocating for my patients
needs and (perhaps most importantly) learning, always learning. But … no one had ever
taught me how to be a teacher.
65
Beginning to Teach
And so I began to teach. For the first few years, my prime educational focus
involved developing lecture content and resource material. In the mid-1990s, this
involved the creation of 35 mm slides using PowerPoint with the help of the Medical
Photography unit. The inclusion of pictorial content (photographs, x-rays) was
particularly challenging, as digital imaging was not readily available. Thus the process of
creating a lecture required a significant investment of time and money. Slides were not
reproduced for students (as they are today) and so I also created narrative summaries of
key course content to simplify note taking in class.
By the end of the 1990s, I had developed a series of undergraduate lectures and I
had become comfortable speaking to an auditorium full of students. I now had 5 years of
experience as an independently practicing pediatric orthopaedic surgeon and could “ad
lib” and readily bring examples from my clinical life to the classroom. Students seemed
to enjoy this approach, voicing appreciation for the practical, case-based sessions that I
gave as well as for the provision of electronic and handout versions of the classroom
material. In 1997, I received a teaching award from the Aesculapian (Medical Student)
Society recognizing my efforts. During this time, I had become also an active member of
the MSK course committee, and had led an initiative to begin streamlining the course
66
objectives, which lacked cohesion and included areas of obvious content overlap
between sessions.
“Is Anybody Learning Anything?”
In August 2001, I was named MSK course chair. Suddenly, I was responsible for
more than my own lectures. My job was to manage the course as a cohesive entity and I
began to realize how difficult this was, given the fact that 25 teachers were contributing
to the whole. But the pivotal experience came during a lecture on children’s fractures
that I had given many times in the past. About three quarters of the way through the
session, a brave student put her hand up and asked me to explain the word “reduction”.
In the context that I was teaching that day, this meant the restoration of normal anatomy
to a fracture or dislocation, a basic concept necessary to understanding all of the cases
that I had included in my presentation. How much of my 2-hour lecture had this student
failed to grasp if she had not known such a fundamental term? How many other students
were in the same boat? If this was happening in my session, what was happening in all
the other classes? Was anyone learning anything? Looking up from the podium towards
the rows and rows of quiet, note-taking students, I had no way of knowing.
I attended a large, traditional medical school in another city, another century. I
spent 3 years sitting in lecture halls and attending laboratory sessions. Much of what I
67
learned then has slipped away from me. Who were the good teachers? Not the ones that
were simply memorable, such as the teacher who accompanied his lecture on the guitar
or the dermatologist of questionable taste who showed us pictures of risqué tattoos, but
the teachers who helped me learn. There was an elderly Histology technician who held
incredibly useful tutorials at lunch where he related images of cellular structure to
everyday items; the bladder’s transitional epithelium became footballs and suddenly I
could remember what it was. There was our first year anatomy professor, who could
bring the human body to life on the blackboard in coloured chalk and who managed to
learn most of our names over the course of the year; gently, but pointedly he asked
individual students questions (even those who sat in the back row, like me). There was a
pathologist who had harnessed and reshaped an unruly second-year course by limiting
the number of teachers and imposing organization and structure to the teaching that he
coordinated. What made these teachers different?
When I took over as MSK course chair, the course consisted of a series of whole-
class lectures. In the months preceding the course that year, I used course management
software (WebCT) to archive copies of the lecturer’s PowerPoint presentations. The use
of a course website was quite novel within the School of Medicine at this time. In
addition to providing a central location for course notes, this was a useful exercise for me
68
as it gave me the opportunity to become familiar with the content being delivered by
other teachers.
Previously, paper-based handouts had been collated, packaged and sold in the
Campus Bookstore. I knew that many teachers revised their session a few days prior to
their scheduled lecture. My use of technology was an attempt to avoid the mass printing
of notes, which were commonly out-of-sync with what students saw in the classroom.
Students appreciated the availability of presentations online, but they persisted in
wanting hard copies and arranged as a class to have the notes printed in bulk. This led to
significant dissatisfaction when inevitable last-minute teacher revisions resulted in
differences between the projected and the printed slides. Minor differences were the
source of passionate complaints.
The course came and went and I survived, but I remained troubled about the way
it had unfolded, in particular about what I interpreted to be a generally passive approach
to learning assumed by both students and teachers. The students were like Indiana Jones
seeking treasure. The basement classroom had somehow become the “Temple of
Knowledge,” each bulleted point conveying “The Truth.” Not quite 20 years previously,
as a first-year medical student myself, I had exhibited the same sort pragmatism. Faced
with a content-heavy, lecture-based curriculum, my classmates and I adopted similar
69
tactics focusing on classroom notes to learn what was necessary to pass multiple-choice
examinations. By 2002 I had been an independently practicing physician for 8 years and
had learned that real-life patients seldom presented with characteristic features and
classic findings. Real-life patient problems were just that: problems, requiring a degree
of effort and creativity to find a solution. And so began my first real efforts in course
redesign. I was driven to action by a growing concern that students were relying far too
heavily on their teachers’ interpretations of the subject matter. Motivated more by a
sense that this was wrong rather than by any specific pedagogical knowledge, I began to
imagine a course that would require student activity and engagement.
Early Attempts with Different Instructional Methods
The focus of this work is the evolution of the MSK course between the years of
2005 and 2007 during the implementation of TBL. Prior to that time, I spent several
years developing web-based curricular materials (self-study modules, cases with
reflective questions, formative quizzes and other exercises such as crossword puzzles).
This was an attempt both to manage the course effectively and to deliver the content to
the students using a variety of methods, hoping to engage them with the subject matter. It
was evident that students were accessing the course material available online: from
January to May 2004, the course website received over 2700 hits. Parallel attempts to
70
introduce active learning exercises into the classroom (lecture bingo, muddy points,
student-led case presentations) were met with significant resistance. In the course
feedback survey, students clearly indicated a preference for the didactic format. The
novel format of the course was particularly problematic as it was scheduled immediately
prior to high stakes end-of-term exams. Students felt pressure, not just from the MSK
course preparation and assignments, but also from the weight of content from the whole
term. Clearly, any future innovation in the course would be challenging in this time slot.
Discovering TBL
In reflecting on the 2004 implementation of the course, there remained “an
emphasis on detailed content over conceptual knowledge, problem-solving and
diagnostic clinical skills,” further noting that the evaluation system tended to “reinforce
this and direct student focus and learning” (Davidson, 2004). Despite this, several
lecturers noted that students “appeared to arrive better prepared for lecture than in
previous years” (Davidson, 2004). My intuition suggested that, despite the apparent
“failure” of the new strategies that had been introduced, active learning was something to
be pursued while correcting obvious problems that had arisen in this iteration of the
course. I presented my concerns and ideas to the Phase coordinator who agree to move
the 2005 version of the MSK course to the start of term rather than the end.
71
I was introduced to TBL at a School of Medicine Education Journal Club in
2004. While initially skeptical of another educational fad, I was inspired to learn more
about this instructional strategy by the enthusiasm of the presenter (who happened to be
the Dean of Undergraduate Medical Education). Shortly thereafter, I read a book
describing the method in detail (Michaelsen, Knight, & Fink, 2004). TBL seemed like a
feasible solution to the problems that I faced, designing a course that would require (and
hopefully inspire) student engagement and activity. With a little planning, it seemed that
I could introduce some TBL components to the course while preserving some lectures,
symposia and clinical skills sessions. As it appeared from my readings that one teacher
in the existing lecture hall could accomplish this, there would be no need to recruit
additional teachers, a perennial problem with small-group learning.
Much of the groundwork had already been done. The course objectives had been
revised in 2002, informed by a quality assurance project that had correlated published
objectives with the material presented in the classroom (Davidson, 2003). They were
clear and well written and had been recently linked to an external standard (the Medical
Council of Canada Examination Objectives). The course website included an electronic
library of all teachers’ presentations and was organized into four distinct themes (injury,
arthritis, pain and pediatric topics). There were existing resources including readings
72
from textbooks, articles, physical examination videos and self-study modules online. My
intention was to continue using WebCT to deliver related material (cases, quizzes, online
modules and presentations) to help students learn, practice and eventually master the
course content. It did not seem feasible (or wise) to transform the whole course to this
format in the first year; therefore, what resulted was a mix of lectures, symposia (often
just several lectures back-to-back) and TBL. Individual and group assignments were also
used to encourage students to work and learn beyond the boundaries of the lecture hall.
Over the course of the months preceding the 2005 course, I developed TBL
content material (readiness assessment tests and group exercises). My initial focus
involved the sessions that I taught personally, anticipating that other teachers in the
course might be skeptical of the efficacy of the new approach. The content development
took more time than I had anticipated. While creating the Readiness Assessment Tests
(RATs) was no different than a standard multiple-choice test, design of the group
application exercises was more complicated. I found the development of case-based
exercises using Michaelsen’s 3S framework (same problem, simultaneous reporting,
single answer) to be particularly difficult. As the first to introduce TBL within the
School of Medicine, I had no local experience to tap into. While I had read and re-read
73
Michaelsen’s book, translating his method into specific tasks related to the course
material was another matter.
2005: Year One
Shortly after New Year’s, 2005, I arrived in the basement classroom, computer
and a briefcase full of papers in hand. I carefully marked out classroom locations for
each of the 13 groups. As students arrived in class, there was some confusion and
dissatisfaction relating to the imposed groupings and the seating plan. I heard later from
a colleague that students had expressed dismay to her that they had been told where to
sit. As the month wore on, certain groups became fragmented with members spread
across the classroom. Over the course of the next 4 weeks, I experienced a sort of TBL
“trial by fire.” The logistics were daunting, exams needed to be constructed to match
special answer sheets, photocopied, marked by hand and the data entered electronically.
Every day, it seemed, I was just keeping my head above water, making sure that I had
the materials available for the next session.
Students arrived prepared for the TBL sessions and appeared engaged by the
case-based material. The group tests and some of the group exercises appeared to fall flat
as they were easily answered using the course notes, which I had allowed students to
consult. It was therefore somewhat difficult to generate the type of inter-group
74
discussion described by TBL advocates. Nevertheless, there were many student
questions and a generally higher level of classroom discussion than I was used to when I
taught using a standard lecture.
Early on, it was obvious that the classroom space and configuration were
suboptimal. Just large enough for the class size, there was little space and fixed seating
that worked to undermine group communication, dynamics and functioning. I had a
difficult time circulating around the classroom during the session and the groups in the
centre of the room got little attention. Some students responded by moving out to the
adjacent cafeteria space during team application exercises.
Several of the TBL sessions were team taught with colleagues who had formerly
lectured. For each, the format was similar, and students would first engage in group
activities based upon structured cases and questions. My colleague and I would then
facilitate discussion of the answers and towards the end of the class (the original teacher
for the topic) would provide a short “debriefing” presentation. The latter was difficult to
gauge and tended to morph into a mini-lecture rather than being a commentary focused
on specific areas the students had found difficult to grasp. On several occasions, classes
ran over time.
75
Towards the end of the course, students were asked to evaluate their team
members and assign a peer evaluation grade. I had scheduled class time for this, not
realizing that students would immediately set about negotiating with one another. There
was tremendous opposition to this process, which required students to differentiate (even
by just a little) between high and low performing members of the team. One student
approached me several days after the peer evaluation exercise, reporting that she had felt
intimidated by her colleagues and had been pressured into assigning a favourable score
when she actually felt that one of them had underperformed as a team member.
During the implementation of the course, I received a lot of student feedback,
both formally and informally. In addition to the annual course survey, students handed in
anonymous comment cards at the end of several sessions. I received over 150 emails
from approximately half of the class outlining problems with WebCT, questions about
online and in-class quizzes, clarification of course content and administrative details
such as scheduling, assignments and course notes. The Centre for Teaching and Learning
(CTL) had agreed to participate in an evaluation of the course, to give me an “arms
length” opinion on the implementation of the new teaching strategies. One of the
educational developers at the CTL observed four classes (two TBL and two lecture) and
was able to comment on apparent student activity and engagement. In addition, she and
76
the Director of the CTL conducted focus groups with 11 students to explore their
reaction to the changes in more detail.
Both the focus groups and the course survey results were disappointing. While
students recognized the effort that had gone into the MSK course, they were harsh in
their assessment of the results and of me in particular. Students felt that the course was
disorganized, that expectations were unclear and that the website was a confusing
nightmare. They felt overburdened by the course assignments and requirements for
preparation in advance of class and expressed a clear desire for a return to lecturing. I
was “by far the worst instructor,” both “deluded” and “inconsiderate.” TBL was “a
pointless exercise” and a “waste of time.” I was particularly fascinated by an anonymous
comment card following a session team taught with one of my surgical colleagues (he
arrived dressed in ‘greens’, looking more the part than I did that day) who provided a
“mini-lecture” debrief following the team application exercise. “It is incredibly rude and
inconsiderate for you to have busy surgeons to come in to lecture, then have them wait
while we do quizzes… Guest surgeons should speak first, and be allowed to get back to
work. Stop wasting our time and theirs!”
The consultants from the CTL were more tactful in their recommendations.
While they recognized the value of pursuing instructional approaches that promoted
77
active learning, they also identified significant areas for improvement. In addition to
seeking out a new venue to teach and streamlining the website, they advised an improved
introduction to the course laying out the rationale for the instructional design, a re-
evaluation of assessment strategies and the inclusion of specific teaching about group
processes and skills.
How can one little course eat up so much of my time? February 2005 … only 11
months to work further towards “getting it right.” Despite the negative feedback I had
received, I could not face returning to 4 hours of lecture a day. Students were not
generally happy, but they were no longer asking me to explain basic vocabulary halfway
through class. There was more discussion, and discussion of problems with more
substance, in the classroom. There was much discussion outside the classroom, by email,
on the discussion board, in person. And then there were the students who stopped me as I
was leaving the classroom to tell me how much they enjoyed learning by doing, how it
forced them to keep up with the material and just how much they felt they had learned.
I had not been convinced of the efficacy of my previous (pre-TBL) attempts to
introduce active learning into the classroom. By the end of the first year of this method, I
recognized an increased level of engagement in the class as a whole and had clearly
struck a positive chord with a few individual students. The hornets’ nest of discussion
78
that my intervention had stirred forced me to reflect on what had unfolded and critically
assess what I could do to improve the course in 2006. I was not surprised by the negative
feedback, as it had also seemed to me that some parts of the course had not worked. This
reflection (and the negative feelings that came, inevitably, as I read the student feedback)
spurred me to read more about TBL and seek guidance from colleagues and local
educational experts. Guidance…. and support.
2006: Year Two
Clearly, some changes were needed: a better classroom (more room, table
seating), revision of the website and ongoing development of the TBL cases. I sought
support from my Department Head and the Dean of Undergraduate Education and
received funding to redevelop the course website using a new platform (qWeb)
developed by the MedTech unit. This allowed me to hire an assistant, a mechanical
engineer who had developed a corporate online training program working with the
Continuing Education group from the Faculty of Education. Together, we developed a
new course website based upon existing lecture content. We created 12 online modules
using a series of linked web pages. These incorporated core content, case examples,
images and linked references. The website development tool that we had available
lacked important functionality, providing little beyond the capability to build and link
79
static web pages. A number of crosswords, short ungraded quizzes, and other interactive
exercises were developed using freely available software. The course WebCT site was
pared down and maintained to provide a course discussion board. Any documents that
required delayed-release were posted to the main student portal (Online Course
Resources or OCR), which the students accessed regularly for all other courses.
In June of 2005 I attended the 4th annual conference on Team-Based Learning in
Medical and Health Sciences Education at Wright State University. This 3-day event was
an opportunity to attend workshops to increase my knowledge and skills related to TBL
and to network with other teachers who had adopted the approach. Both the formal and
informal learning opportunities at the conference were excellent and I arrived back home
with a better understanding of what had gone wrong and what to do next. I found it
particularly motivating to hear stories of other teacher’s explorations of this method and
the results that they had observed in student performance.
As the second year of TBL approached, a completely new course website had
been developed. We created a course “learning map” which outlined both the schedule as
well as any required preparation, in-class quizzes or assignment due dates; students had
clearly described a need for this sort of consolidated syllabus in their evaluation of the
previous year’s course. Course content was now presented as online modules
80
incorporating text, images and case examples. Each module included formative
evaluation questions that incorporated formative feedback. I made minor revisions to the
readiness assessment tests and major revisions to the group exercises. I had initially
avoided using multiple-choice questions in these exercises; however, attendance at the
TBL conference had helped me to understand that this was the simplest way of enabling
students to report back with succinct, comparable answers which could lead to inter-
group discussion. The secret was the construction of case scenarios with some degree of
ambiguity, usually reflected in the possible answers. Questions that incorporated at least
two credible answers could lead to a rich inter-group discussion as student teams debated
their respective choices.
I had hoped to use the former hospital cafeteria for my TBL sessions; it was a
large open space with configurable tables and a window looking out over the lake.
However, at the last minute I learned that it was unavailable as teaching space. The
newly appointed pre-clerkship curricular administrative coordinator was instrumental in
relocating the sessions to our second choice, an ancient hospital gymnasium. The
provision of dedicated administrative support was welcome, allowing me to concentrate
on curricular development rather than day-to-day course operation. In early January, I
arrived at the gym, laden down with technology (computer, projector, audience response
81
system) and curricular materials (team folders, tests, answer sheets). As the CTL
consultants had suggested, the introductory session included specific information about
TBL, including the process and underlying rationale. At the TBL conference I learned
that I had made some mistakes in my original implementation of this technique,
particularly by allowing students to complete the readiness assessment tests “open
book.” The first readiness assessment process went well. Students were interacting better
when seated together at round tables and during the group test discussion was animated.
As groups uncovered their (correct) answers, many tables “whooped” with joy.
While the gymnasium provided a large, reconfigurable space that was superior to
the previous year’s lecture hall, it was a challenge to set up equipment for computer slide
projection. There were other issues that became quickly apparent: acoustics were poor,
the lighting was erratic and the room became hot and stuffy when occupied by 100
students. Students seated with their backs to the screen found it uncomfortable to
reposition themselves whenever slides were shown. This became a particular problem
during didactic presentations that followed TBL sessions; these had been scheduled in
the gym to minimize students moving between classrooms.
The most noticeable improvements, from my point of view, were the team
application exercises (Appendix E). Dissatisfied with the previous year’s experience, I
82
had created a new set of progressively revealed cases interspersed with multiple-choice
questions to stimulate inter- and intra-group discussion. Students quickly learned to bring
notes, textbooks and wirelessly enabled laptops to the session (fortunately, the hospital
had recently added wireless internet access to public areas) and as I walked around the
room students were engaged in looking up answers, reviewing course material and
(perhaps best of all) explaining concepts to one another. This year, I was careful to
structure the post-application exercise debrief as just that, a short summary of key points,
complete with handout (posted electronically); no more “lecture-in-disguise”.
In 2006, one of the rheumatologists agreed to experiment with TBL in his
teaching. Over the summer, we had created three online modules designed to introduce
students to the fundamental concepts of arthritis. I now designed two TBL sessions that
would extend this into the classroom. The rheumatologist consulted on development of
both the online and the classroom material and he and I team-taught both sessions, which
were well received. I think that his conversion to the method became complete when he
won the course teaching award that year! In addition to the arthritis teaching, several
other TBL sessions were team-taught. This enhanced the teaching experience for me. In
addition to giving the students another individual’s perspective on the material (often
from another discipline than my own), the presence of another teacher allowed me to
83
pause and listen more than I had previously. When there were two of us, we often stood
across the classroom from one another and were able to identify when students at the
back could not hear or if other problems arose. More than anything, team teaching was
fun.
Another key strategy was the simplification of the grading process. Gone were
the graded online quizzes and the clinical skills attendance mark. The group assignment
had been eliminated and, staying true to Michaelsen’s original description of TBL, all
group work was done in classroom time. I continued to use the individual assignment in
which students researched osteoporosis based upon a fictional patient case. This created
nice linkages to the Information Literacy teaching from the previous term and had been
well received by students the previous year. The evaluation scheme had been adjusted to
credit the students for the work done during the course (50% of the course grade, up
from 25% the year before), with half of the in-class grade reflecting TBL work. In
addition to streamlining evaluation for the students, this change simplified the
administrative tracking associated with the course.
In 2006, I conducted the peer evaluation process electronically. Students
completed evaluations of their team members in private without coercion. Despite the
fact that team scores were uniformly high, students continued to object to the possibility
84
of having their grade reduced by the peer evaluation process and this remained an
unpopular feature of the course. As the course came to a close, I awaited the results of
the student course evaluation survey with trepidation. It seemed to me that the teaching
had gone well, as students arrived prepared and seemed engaged with the material. Other
teachers noted this as well, citing excellent classroom discussion during their sessions. It
became apparent that students were using the learning map to guide their preparation, not
just for the TBL sessions but for more traditional sessions as well. This ‘halo’ effect
seemed to change the dynamic in didactic as well as TBL sessions. Despite the
skepticism of the second-year class (which was well known on the student grapevine),
the 2006 first-year class gave TBL good reviews. Criticism was focused upon the locale
(the gym was better than the lecture hall for TBL, but it was still fundamentally
uncomfortable for extended periods of time) and the multiple websites (qWeb course
site, WebCT as well as the main medical student online portal, Online Course Resources
or OCR) containing course-related material. The online modules that were created were
generally well received, but our attempt to compensate for their reduced functionality by
using the other platforms had back-fired and created another focus of student
controversy.
85
2007: Year Three
Another year behind me, with another 11 months to plan for next year’s course.
Again, the search for a more suitable classroom, this time strictly to be used for TBL
sessions with all lectures in an auditorium. Although I had sworn not to spend another
summer developing course content, the student concerns about the course’s electronic
backbone were legitimate and something had to be done. The School of Medicine had
recently acquired a site license for Thinking Cap® Studio, an online authoring tool used
to create e-learning modules with interactive assessment capability. With the help of a
web developer and a summer student, I translated our previous work to this new format
(Appendix F). Each module had a standard format and navigation system, interactivity
and formative self-assessment components. The course website was simplified and the
use of WebCT was eliminated. Discussion groups were added by linking the course
website to the School of Medicine online forums which appeared seamless to the
students, even though it represented a different website. Course objectives were
published on OCR as was the schedule, however all downloadable course references,
notes and presentations were linked to from the learning map.
The course introductory session was similar to the previous year: the
development of linkage to past and future components of the curriculum, an overview of
86
the website and of TBL as well as a sample RAT. I decided to create the teams in class
as I had seen done at the Dayton TBL conference. Although I had initially been skeptical
about the feasibility of this approach (students stood up and sorted themselves by criteria
that I had defined; in this case, previous experience with MSK, years of university study
and place of birth), it worked well and created a sense of energy and interaction in the
classroom. At the suggestion of one of the 2006 students, we booked all TBL sessions
into a large top-floor classroom/laboratory space in one of the Mechanical Engineering
buildings. As with the gym, a drawback to this room was the lack of digital projection
equipment.
Thus, January 2007 found me making my way across a snowy parking lot with
computer, projector and a large carrier bag full of team folders and course materials.
Despite a few glitches with the room (it was locked on the first day, there were barely
enough chairs, and a huge ramp from an engineering research study occupied the centre
of the classroom during the first session), it was by far the best space for TBL that I had
encountered to date.
My general observations about the students were similar to the previous year.
Attendance was excellent, as was preparation and class discussion. The “halo” effect was
again noted in non-TBL sessions. Students remained lukewarm to online discussion tasks
87
that were used to promote out-of-class peer interaction around the course material.
Although this was an unpopular feature of the course, there were over 1000 posts to the
discussion forum over the month, reflecting the presence of a small participation grade
based on online activity. At the end of the course, students were again asked to evaluate
the performance of their team members. Unlike previous years, the peer evaluation grade
was used to raise the score of high-performing students, but students identified as lower
performers did not lose marks. All students received their peer grade as well as
aggregated, anonymous comments from all team members. This basically eliminated all
serious opposition to peer evaluation.
It had taken me 3 years, but by the end of January 2007 I felt confident in the
design and implementation of the MSK course. Student evaluation of the course was
extremely positive but, perhaps more importantly, several other MSK teachers
commented that the students were coming to class prepared and able to engage with the
course content, even during traditional didactic sessions.
Two other courses (Palliative Care and Hematology) that were scheduled in the
same term as MSK had recently begun to explore similar instructional techniques. When
the students arrived at their Hematology TBL session towards the end of the term that
year, they spontaneously re-formed their MSK teams.
88
At times it was challenging to persist in the face of some of the obstacles that
arose as well as negative student feedback. However there were also breakthroughs
where suddenly the innovations that I had been struggling to implement came more
sharply into focus: experiencing application exercises as a learner during the TBL
conference; the first time I created teams in class; witnessing a swell of energy amongst
the students; reading student posts in the discussion forum that attested to their
engagement with the material outside the classroom.
My experience watching students move from simply occupying space in a lecture
hall to making learning happen by interacting with each other has changed the way I
think about teaching. This experience has shown me that the students I teach will rise to
the expectations that are set out, and that teaching to a room of individuals who have
already begun the groundwork to learn is altogether a richer experience than lecturing to
those who are encountering concepts for the first time. Although, at the outset, my
intention was to shake the students out of their apparent passivity in the lecture hall, I
think that the biggest change that I have witnessed is the change in my own thoughts
about learning and how best to facilitate it.
89
CHAPTER SIX: DISCUSSION
Going to Sea
In 1937, Arthur Ransome published We Didn’t Mean to Go to Sea. Reflecting on
my experiences described in the previous chapter recalls the plot of this novel: “While on
vacation in East Anglia, four children, whose previous sailing experience is limited to
dinghies, accidentally drift out to the North Sea after the rising tide causes their cutter to
drag anchor” (Fantastic fiction, 2008). They “decide that it is safer to hoist the sails and
go farther out to sea rather than stay near the shore” (Wikipedia, 2008). In some ways,
the voyage that I have been on as a teacher – outlined in this thesis – feels something like
that fictional experience. My original goal was to move out of the fog (student passivity)
that seemed to characterize the didactic course that I had inherited. However, in moving
away from this known shore, the waters I was sailing were uncharted, treacherous, and to
some extent unintentional.
When I began to make changes in the instructional design of the MSK course, I
had no conception of myself as an innovator; in fact I was scarcely acquainted with the
term in the context used by Rogers (2003). And yet it seems that is what I am. Three
years into the experience it appears that a positive change has been achieved – the
students and I have reached more settled and altogether remarkable waters. Perhaps most
90
compelling is the knowledge that others have begun to adopt similar strategies within my
School. This chapter revisits the three original research questions and uses the data
gathered to make sense of what happened.
Educational Innovation and Change
The heart of this work is a reflection on educational innovation. Innovation of
any type is a difficult process (Rogers, 2003) and new educational strategies may fail at
the implementation stage for a variety of reasons (Silverthorn, Thorn, & Svinicki, 2006).
At the outset of this project, I did not aim to innovate (or to be labeled an innovator) –
rather my purpose was to shift student attitudes away from unquestioning acceptance of
the black-and-white lecture content. By the time I embarked upon this project, I had
enough experience to know the importance of coping with the shades of gray of real
practice. I did not anticipate the nonlinear messiness that would ensue as I disrupted the
status quo, nor did I understand that success would hinge on a process of slow learning,
in context, over time, and that it was I, the teacher, who would do the learning (Fullan,
2001).
While it is intuitively obvious to anyone who has tried to implement a new
curriculum that “content and pedagogy interact in complex ways” (Chickering &
Gamson, 1987), only recently has complexity science been formally applied to
91
education. Complexity embraces the simultaneity of dyads traditionally considered
distinct: knower and knowledge, research and education, even teacher and student. This
approach to inquiry encourages the simultaneously examination of educational practice
in combination with contextual factors in order to uncover what may be tangled in the
intersection of the two (Van Melle, 2005; Davis & Sumara, 2006). The use of narrative
data (my experiences, thoughts and feelings) and qualitative data (students’ reactions to
the course and their experiences in it) help to flesh out shifting attitudes and beliefs about
instructional methods and practices that are captured by the survey data.
The setting into which change was introduced was a critical element in this story;
student expectations and reactions were driven by their experiences in the rest of the
curriculum, stories from upper classes, the School’s IT structure and even the physical
space used to teach. Each of these factors likely contributed to intentional and
unintentional adaptations to the course as did the feedback received from each successive
class of students.
The Evolution of the Musculoskeletal Course
The first question addressed by this study was to quantify and describe the
evolution in the design and implementation of the MSK course between 2005 and 2007.
Substantial changes have been demonstrated in the design and implementation of the
92
musculoskeletal course between 2005 and 2007. There was a net decrease in direct
instructional time as well as a significant increase in the classroom time allotted for TBL,
from 25% in 2005 to 40% in 2007. Most importantly, there was a net gain in student
activities associated with active learning: reading, reflecting and talking/listening
(Meyers & Jones, 1993).
The transformation of the course was a gradual process, initially involving
sessions where I had direct control as primary teacher with subsequent expansion as
other instructors began to accept a different approach. This allowed for the staged
introduction of this instructional technique as I sought out further faculty development
opportunities to learn more about TBL and e-learning (by reading, attending conferences
and networking with other teachers) and gained experience over time.
There was also an evolution in the e-learning component of the course over the
three-year period. While this reflected my learning curve in developing and using
instructional technology, it also relates the growth of IT resources available within the
School of Medicine over time. Much of the negative feedback received from students
during the second year of the study revolved around the use of three different web-
platforms to host course materials. This was a poor compromise made in an attempt to
provide enhanced functionality of online resources. In retrospect, this was probably a
93
mistake, as I had not anticipated the significant level of student frustration that this
strategy would generate. A larger issue that emerges from the data is my personal lack of
knowledge and an institutional lack of resources toward the development of robust
models of e-learning. As Harden comments, “e-learning implementation has to be
monitored and managed by staff with the necessary education, technology and content
expertise” (2008, p.1).
As the proportion of active learning sessions increased, there was a
corresponding shift in the student assessment strategy de-emphasizing the importance of
the final examination (from 75% of the course grade in 2005 to 40% in 2007). Students
were rewarded for both individual and group effort with (often immediate) feedback both
in and out of class. The combination of graded and ungraded formative assessment
served to motivate students to change their behaviour and prepare prior to classroom
sessions, something that I noted, as did other teachers in the course.
Student Response to the Changing Course
The second question, posed at the outset, was to describe the response of each
cohort of students to the instructional changes in the MSK course. Over the course of the
research period, there were no changes in the admission policies or procedures of the
School of Medicine. The demographics of the three cohorts of students who participated
94
in the study were similar, although there was an increase in the proportion of female
students from the first to the third year.
In other studies of undergraduate medical students, there has been a positive
correlation between female gender and the introduction of integrated curricular units
(Greenfield, Brown, Dawlatly, Reynolds, Roberts, & Dawlatly, 2006), therefore this may
account for some of the positive shift noted in this study. It is, however, unlikely that all
of the differences observed in student responses and attitudes stem from inherent
dissimilarity in the three groups of students, given the small scale of the demographic
differences between the three cohorts of students.
The response rates to both the course evaluation and the research surveys were
significantly higher in the third year of the study. Anecdotal reports from participants
suggest that students disenchanted with the course were less inclined to complete either
survey (particularly in the first year). If this is correct, then one could hypothesize that
the 2005 course evaluation survey actually overestimates the number of positive
responses as students with negative opinions (of the course and the instructional
methods) were less likely to be included in the results analyzed.
Student approval of the course, as measured by items taken from the standard
course evaluation survey as well as by narrative comments, improved significantly
95
between the first and third year of the study. Had student feedback and opinion in 2005
been the only measure of success, active learning initiatives would have been abandoned.
In additional to strongly positive ratings on seven of the eight items in 2007, six items
demonstrated a reduction in the associated standard deviations suggesting a tighter
consensus amongst respondents. One item, “the workload in this course was reasonable
and appropriate,” received a rating below 4 in 2007. This is consistent with literature that
identifies an increased workload for students with the introduction of active learning
instructional techniques (McAlpine, 2004).
The research survey results provide data allowing the comparison of student
attitudes towards didactic and active learning techniques. As the MSK course evolved
over time, there was a statistically significant decrease in student approval ratings of
didactic techniques and a parallel increase in ratings of active techniques. Students in the
third cohort, who were exposed to the most mature version of the course, revealed a shift
of preference away from the lecture as the optimal instructional technique with strong
support for TBL and online modules. Students in particular valued the opportunity to
control, assess and pace their own learning with online modules and to practice
knowledge and learn from peers with TBL. These changes mirrored the growth of my
expertise in and comfort with the altered course design.
96
Critical Elements that Influenced the Course Outcome
The final question central to this work was to identify the critical elements that
influenced the course outcome as the instructional model matured over time. It is clear
from the data presented above that not only did the MSK course evolve over the study
period, but also there was a significant difference in the attitudes of students exposed to
different iterations of the course. It is also evident that, at times, the ship sailed on
stormy waters with significant and sometimes vitriolic student feedback.
The success of the course is judged from two perspectives: student feedback and
student behaviour, the latter observed from the teacher’s vantage. In addition to the
improvements in student opinion of the course, even in the first year of implementation,
there was increased student preparation and interaction noted. Analysis of the data
collected identifies key elements that appear to have influenced the course outcome.
These can be expressed in three distinct groupings: external factors, technology and the
teacher.
External Factors
External support is often a critical factor in enabling educational transformation;
a lack of institutional support may cause well-planned instructional interventions to fail
97
(Silverthorn, Thorn & Svinicki, 2006). Thompson and colleagues (2007) identified the
importance of buy-in from both faculty and administrators as well as the availability of
resources including expertise in the method, faculty preparation time for the
development of TBL materials, appropriate classroom space and administrative support.
Without buy-in from my Department head and Associate Dean, I would have had
difficulty persisting in course redesign past the first year. Administrative support is a
crucial element in fostering educational innovation (Silverthorn, Thorn & Svinicki,
2006).
In addition to moral support, local administrative endorsement extended to the
provision of resources that facilitated the work of developing curricular materials (the
course website) and the course administration (the introduction of dedicated
administrative curricular coordinators). The success of this project as well as the
difficulties of implementing this type of teaching in existing physical space has been
recognized at a senior management level. Currently, our institution is planning the
construction of a medical school building that would include classrooms suitable for
TBL and other non-traditional instructional formats.
The expectations of the LCME accreditation surveyors were aligned with the
philosophy underlying the course redesign (decreased didactic teaching, formative
98
assessment opportunities, as well as opportunities for students to work both
independently and collaboratively). This created an incentive for institutional support of
this project, another important factor in promoting the success of an innovation (Rogers,
2003).
Technology
Technology has been both a facilitator as well as a detractor of the MSK course
redesign. Faculty who introduce web enhancements to traditional courses usually do so
for pragmatic reasons, however the instructional potential of this method quickly
becomes apparent (Wingard, 2004). This was my experience the year that I became
director of the MSK course. In addition to increasing my contact with students because
of the use of email, technology allowed me to capture and collate the didactic
presentations that originally defined the course. By 2005, I had begun to use web pages
to deliver pre-packaged content as well as formative assessment. This became a critical
enabler for TBL as it allowed student preparation prior to class, thus allowing the
transformation of classroom interactions.
Early on, the use of technology also contributed to some student dissatisfaction.
This echoes the experience of other educators (Allen, Walls, & Reilly, 2008) who have
embarked upon similar projects. Van Melle noted that “the ease with (which) the
99
technology can be used for learning is an important factor to consider” (2000, p. 87). The
dissonance between the available University IT resources and an evolving, homegrown
School of Medicine learning management system (LMS) created a dilemma for me.
While the former included greater interactive functionality, the latter was more widely
accepted by the students. My attempts to combine both systems – prizing functionality
over ease of use - led to student frustration and general unhappiness.
The Teacher
My efforts in course redesign were stimulated primarily by inner motivation. I
was driven to action by dissatisfaction with the level and quality of student interaction
that I had experienced as a teacher. My own experiences as a student helped mold the
directions that I chose to explore: the changes that I have been responsible for echo the
good experiences that I recall from my own education.
I suppose that I could characterize myself as a risk-taker (choosing to enter a
male-dominated specialty, Orthopaedic Surgery, in the mid-1980s attests to that) and
therefore willing to venture into unknown instructional territory. The characteristics of a
sense of moral purpose, risk-taking and “venturesomeness” have been associated with
those likely to innovate and engender change (Fullan, 2001; Rogers, 2003; Searle,
Haidet, Kelly, Schneider, Seidel, & Richards, 2003).
100
I found it particularly useful to develop expertise in TBL by attending a
conference devoted to the use of this method in Health Sciences Education in 2005. In
addition to hands-on sessions, this opportunity allowed me to connect with an
international community of educators interested in sharing their experiences. My
understanding of both the technical and pedagogical principles of e-learning also grew
over the course of the project, often in direct response to problems and challenges that
arose from student feedback about the website and online modules.
There were times when it was difficult to persist in the face of strongly negative
feedback. However, I was sustained by what I was witnessing in the classroom.
Although I had no way of measuring it, every day that I stood in the classroom with 100
students, the interactions were subtly different. There was more noise, more questions,
more peer interaction and, generally, a higher level of discussion than I had ever seen
before.
Searle and colleagues (2003) reviewed the initial implementation of TBL in
medical education in 10 institutions. These authors noted an observable facilitation of
both teaching and learning which motivated many teachers to continue. While the
personal feedback that I received face-to-face was also encouraging, it was truly the
101
transformation of the group experience that prevented me from returning to an easier,
more traditional dispensation of knowledge from the front of the room.
Charting the Correct Course
“The extent to which there is a ‘fit’ represents the extent to which students are
receptive to learning within a given instructional environment. Theoretically fit is
optimized when an equilibrium or balance exists between the students and the teaching
context” (Van Melle, 2000, p. 2). During the early phase of this study, the new
instructional model was something of a square peg trying to fit into a round hole. As
students constructed their knowledge of musculoskeletal medicine, I as a teacher did the
same learning about TBL and online instruction, while also learning how to blend the
two and what was actually feasible in the context of my course, school and university.
This chapter began with the analogy of a small sailboat in the fog. A truer
representation of the situation pictures me (the course director) steering an ocean liner
along a turbulent new course, with little ability to shift direction quickly. Palmer states
that “teachers possess the power to create conditions that can help students learn a great
deal” (1998, p.6). The experience outlined in this thesis reminds us that creating the
conditions for effective change requires both adaptation and adaptability – a combination
of stability and transformation (Van Melle, 2005). These processes take time and require
102
consideration of contextual complexity and the shifting of beliefs, along with the more
obviously necessary development of curricular materials and expertise in new
pedagogical approaches (Fullan, 2001).
This thesis outlines the successful transformation of one 4-week course within a
4-year professional educational program. Critics may argue with this claim of success, as
there has been no objective measurement of superiority of the transformed course.
Roberts, Lawson, Newble, Shelf, & Chan, (2005) have advocated a similar instructional
approach combining e-learning with large class integrated learning activities as a more
resource-efficient alternative to PBL. These authors demonstrated no significant
difference in student outcomes (compared with traditional PBL). Levine and colleagues
(2004) noted an improvement in standardized test scores as well as a higher student
perception of engagement following the introduction of TBL.
Ten Cate (2001) makes the point that changes in student behaviour stimulated by
instructional change may be more significant than change (or lack of change) in test
performance. Narrative data from both students and teacher in this study suggest that
improved student preparation, interaction and increased “time-on-task” have resulted
from the blending of e-learning and TBL.
103
There have been obvious practical and pragmatic lessons learned from
examination of the transformation of this course. This experience also provides insight
into a process of instructional change developed via an iterative process, informed by
pedagogical expertise, student feedback and teacher reflection. As the experience of the
MSK course has been shared at both administrative (Undergraduate Medical Education
Committee) and pedagogical forums (Health Science Education Rounds; Faculty
Development sessions), several other teachers have experimented with variants of TBL
in the past 2 years. Wider dissemination of similar innovations will require institutional
leadership. Central to this will be the provision of a sense of shared purpose and concrete
support in addition to the development of policies supporting the use of instructional
techniques that promote student engagement with learning (Chickering & Gamson,
1987).
The exposition of this journey traveled has been an opportunity to analyze both
the transformation of a traditionally taught course and my own transformation as a
teacher. Future research directions include the exploration of similar voyages initiated by
innovative teachers as well as inquiry into the changes that occur in the behaviour and
attitudes of the students who travel with them.
104
Other teachers interested in the experiences described herein may avoid some of
the pitfalls that I have described in the preceding pages. But no two voyages are truly
identical and, like the students they teach, teachers too must practice, experience and
reflect to truly learn. They should not be afraid to adapt and shape these ideas to their
own particular circumstance. They will undoubtedly make new and creative errors and
missteps and grow stronger and wiser in the process. Perhaps most importantly, the
ability to innovate is not only a reflection of an individual’s characteristics (as has been
suggested), but rather a mindset and process that can be facilitated and fostered by the
culture of an institution.
105
REFERENCES
Academy of Medical Royal Colleges (2007). The development of e-learning for doctors. Retrieved November 3, 2008, from http://www.aomrc.org.uk/news.aspx AAMC Institute for Improving Medical Education (2007). Effective use of educational technology in medical education colloquium on educational technology: Recommendations and guidelines for medical educators. Retrieved November 3, 2008, from http://www.aamc.org/meded/iime/expertpanels.htm Allen, E. B., Walls, R. T., & Reilly, F. D. (2008). Effects of interactive instructional techniques in a web-based peripheral nervous system component for human anatomy. Medical Teacher, 30(1), 40-47. Austin, A. E., Sorcinelli, M. D., & McDaniels, M. (2007). Understanding new faculty: Background, aspirations, challenges, and growth. In The scholarship of teaching and learning in higher education: An evidence-based perspective. 39-89.
Springer Netherlands. Retrieved November 16, 2008, from http://www.springerlink.com.proxy.queensu.ca/content/w4j072/ Beck, A. H. (2004). The Flexner report and the standardization of American medical education. JAMA: The Journal of the American Medical Association, 291(17), 2139-2140. Bordage, G. (1994). Elaborated knowledge: A key to successful diagnostic thinking. Academic Medicine, 69(11), 883-885. Bruner, J. S. (1966). Toward a theory of instruction. Cambridge, MA.: Belknap Press of Harvard University. Chickering, A., & Gamson, Z. (1987). Seven principles of good practice in undergraduate education. AAHE Bulletin, 39, 3-7. Cooke, M., Irby, D. M., Sullivan, W., & Ludmerer, K. M. (2006). American medical education 100 years after the Flexner report. The New England Journal of Medicine, 355(13), 1339-1344.
106
Davidson, L. (2004). The Musculoskeletal Course: An overview, analysis and future directions (unpublished internal report). Davidson, L. (2003). Variety show syndrome: Making a diagnosis. Medical Education, 37(11), 1034. Davis, B., & Sumara, D. J. (2006). Complexity and education: Inquiries into learning, teaching, and research. Mahwah, NJ: Lawrence Erlbaum Associates. Dent, J. A., & Harden, R. M. (2001). A practical guide for medical teachers. Edinburgh ; NY: Churchill Livingstone. Dziuban, C. D., Hartman, J. L., Moskal, P. D. (2004). Blended Learning. Educause Research Bulletin, 7, 2-12. Fantastic fiction website (2008). Retrieved November 7, 2008, from http://www.fantasticfiction.co.uk/r/arthur-ransome/we-didnt-mean-to- go-to-sea.htm Fink, L.D. (2004) Beyond small groups: Harnessing the extraordinary power of learning teams. In L. K. Michaelsen, A. B. Knight, L. D., Fink (Eds.), Team-based learning: A transformative use of small groups in college teaching, pp. 3-26. Sterling, VA: Stylus Frank, J. R. (2004). The CanMEDS project: The Royal College moves medical education into the 21st century. Royal College Outlook, 1, 27-29. Fullan, M. (2001). The new meaning of educational change, 3rd edition. Teachers College Press, NY. General Medical Council (2003) Tomorrow’s doctors. Retrieved July 6, 2008, from http://www.gmc-uk.org/education/undergraduate/undergraduate_policy/ tomorrows_doctors.asp
107
Graham, C., Cagiltay, K., Lim, B., Craner, J., & Duffy, T. M. (2001). Seven Principles of Effective Teaching: A Practical Lens for Evaluating Online Courses. The Technology Source Archives. University of North Carolina. Retrieved November 7, 2008, from http://www.technologysource.org/article/seven_principles_of_effective_teaching/ Greenfield, S. M., Brown, R., Dawlatly, S. L., Reynolds, J. A., Roberts, S., & Dawlatly, R. J. (2007). Gender differences among medical students in attitudes to learning about complementary and alternative medicine. Complement Ther Med, 14, 207- 212. Harden, R. M. (2008). E-learning-caged bird or soaring eagle? Medical Teacher, 30(1), 1-4. Harpaz, Y. (2005). Teaching and learning in a community of thinking. Journal of Curriculum and Supervision, 20(2), 136-157. Hurst, J. W. (2004). The overlecturing and underteaching of clinical medicine. Archives of Internal Medicine, 164(15), 1605-1608. Kelly, P. A., Haidet, P., Schneider, V., Searle, N., Seidel, C. L., & Richards, B. F. (2005). A comparison of in-class learner engagement across lecture, problem- based learning, and team learning using the STROBE classroom observation tool. Teaching and Learning in Medicine, 17(2), 112-118. Knapper, C. (2007). Engaging students through active learning. Teaching and Learning in Higher Education, 45, 1. Kugel, P. (1993). How professors develop as teachers. Studies in Higher Education, 18(3), 315. Levine, R. E., O’Boyle, M., Haidet, P., Lynn, D. J., Stone, M. M., & Wolf, D. V. (2004). Transforming a clinical clerkship with team learning. Teaching and Learning in Medicine, 16(3), 270-275.
108
Liaison Committee on Medical Education (2005) Functions and Structure of a Medical School. LCME Secretariat, Washington, DC. McAlpine, L. (2004). Designing learning as well as teaching. Active Learning in Higher Education, 5(2), 119-134. McKeachie, W. J. (2007). Good teaching makes a difference – and we know what it is. In The Scholarship of teaching and learning in higher education: An evidence- based perspective. Springer Netherlands. 457-484. Retrieved November 16, 2008, from http://www.springerlink.com.proxy.queensu.ca/content/w4j072/ Meyer, C., & Jones, T. B. (1993) Promoting active learning: Strategies for the college
classroom. San Francisco, CA: Jossey-Bass Publishers. Michaelsen, L. K., Knight, A. B., & Fink, L. D. (2004). Team-based learning : A transformative use of small groups in college teaching. Sterling, VA: Stylus Pub. Motteram, G. (2006). ‘Blended’ education and the transformation of teachers: A long- term case study in postgraduate UK higher education. British Journal of Educational Technology, 37(1), 17-30. Murphy, J., Ingram, D., & Howard, W. (1998). The Internet ward round. In, B. Jolly, & L. Rees, (Eds.), Medical education in the millennium, 104-128. Oxford Medical Publications. National Leadership Council for Liberal Education & America’s Promise (2007). College learning for the new global century. Association of American Colleges and Universities. Newman, P., & Peile, E. (2002). Valuing learners’ experience and supporting further growth: Educational models to help experienced adult learners in medicine. BMJ,
325(7357), 200-202. Palmer, P. (1997). The courage to teach: Exploring the inner landscape of a teacher’s life. San Francisco: Jossey-Bass.
109
Papa, F. J., & Harasym, P. H. (1999). Medical curriculum reform in North America, 1765 to the present: A cognitive science perspective. Academic Medicine, 74(2),
154-164. Pratt, D.D., Collins, J.B., Selinger, S.J. (2001) Development and use of the teaching perspectives inventory (TPI). Presented at the American Educational Research Association Annual Meeting, Seattle, WA. Retrieved November 7, 2008, from http://www.one45.com/teachingperspectives/PDF/development1.pdf Quality Assurance Agency for Higher Education. Retrieved June 29, 2008 from
http://www.enhancementthemes.ac.uk/student_evaluation/whyfeedback.html.
Richardson, L. (1997). Fields of Play. Rutgers University Press.
Riffell, S.,Sibley, D. H. (2003). Learning online: Student perceptions of a hybrid learning format. Journal of college science teaching, 32, 394-399. Roberts, C., Lawson, M., Newble, D., Shelf, A., & Chan, P. (2005). The introduction of large class problem-based learning into an undergraduate medical curriculum: an evaluation. Medical Teacher, 27(6), 527-533. Rogers, E. M. (2003). Diffusion of innovations (5th ed.). Free Press. New York. Schmidt, H. G. (1983) Problem-based Learning: rationale and description. Medical Education, 17, 11-16. Searle, N. S., Haidet, P., Kelly, P. A., Schneider, V. F., Seidel, C. L., & Richards, B. F. (2003). Team learning in medical education: Initial experiences at ten institutions. Academic Medicine, 78, S55-8. Shanley P. F. (2007). Leaving the “empty glass” of problem-based learning behind: new assumptions and a revised model for case study in preclinical medical education. Academic Medicine, 82(5). 479-485.
110
Shuell, T. J., (1986). Cognitive conceptions of learning. Review of Educational Research, 56(4), 411-436.
Silverthorn, D. U., Thorn, P. M., & Svinicki, M. D. (2006). It’s difficult to change the way we teach: Lessons from the integrative themes in physiology curriculum module project. Advances in Physiology Education, 30(1), 204-214. Ten Cate, O., Snell, L., Mann, K., & Vermunt, J. (2004). Orienting teaching toward the learning process. Academic Medicine, 79(3), 219-228. Ten Cate, O. (2001). What happens to the student? The neglected variable in educational outcome research. Advances in Health Sciences Education, 6, 81-88. Thompson, B. M., Schneider, V. F., Haidet, P., Perkowski, L. C., & Richards, B. F.
(2007). Factors influencing implementation of team-based learning in health sciences education. Academic Medicine, 10, S53-S56.
Thompson, B. M., Schneider, V. F., Haidet, P., Levine, R. E., McMahon, K. K., Perkowski, L. C. (2007). Team-based learning at ten medical schools: Two years later. Medical Education, 41(3), 250-257. Trautmann, N. M. (2008). Learning to teach: Alternatives to trial by fire. Change, 40(3,
3), 40-46. Twigg, C. A. (2003). Improving learning and reducing costs: New models for online learning. Educause Review, 38(5), 28. Van Melle, E. P. (2000). Evaluating the use of a CD-ROM to foster learning for understanding using a case study approach. Unpublished Masters of Education
Thesis. Queen’s University. Van Melle, E., Tomalty, L. (2000). Using computer-teaching to foster learning for understanding. Microbiology Education, May, 7-13. Van Melle, E. P. (2005). Exploring the diffusion of innovation in teaching practices in higher education. The case of information and communication technology. Unpublished doctoral dissertation, Queen’s University.
111
Wieman, C. (2007). Why not try a scientific approach to science education? Change, 39(5), 9-15. Wikipedia website (2008). Retrieved November 1, 2008, from http://en.wikipedia.org/wiki/We_Didn't_Mean_To_Go_To_Sea. Wingard, R. G. (2004). Classroom teaching changes in web-enhanced courses: A multi- institutional study. EDUCAUSE Quarterly, 27(1), 26-35. Woloschuk, W., Harasym, P., Mandin, H., & Jones, A. (2000). Use of scheme-based problem solving: An evaluation of the implementation and utilization of schemes in a clinical presentation curriculum. Medical Education, 34(6), 437-442.
112
APPENDIX A: Documents Relevant to the MSK Course
Date Item Author/source Schedules and Syllabi 2005 MSK Course Schedule http://meds.queensu.ca/courses
(School of Medicine’s online portal) 2005 Phase IIA MSK Course (ppt) Dr. Lindsay Davidson 2005 MSK Course Evaluation Scheme Dr. Lindsay Davidson 2006 MSK Course Schedule http://meds.queensu.ca/courses
(School of Medicine’s online portal) 2006 MSK Course Syllabus/Learning Map Dr. Lindsay Davidson 2007 MSK Course Schedule http://meds.queensu.ca/courses
(School of Medicine’s online portal) 2007 Overview of Phase IIA
Musculoskeletal course (ppt) Dr. Lindsay Davidson
2007 Phase IIA MSK Course Feedback Dr. Lindsay Davidson 2006 Introduction to MSK presentation Dr. Lindsay Davidson
113
Date Item Author/source Correspondences and Reports 99/09/14 Memo “Phase IIA Musculoskeletal teaching –
2000” Dr. Lindsay Davidson
2002 Template for Course Review (pre-accreditation)
Dr. Lindsay Davidson
03/05/03 Email “RE: FYI, official email to follow in a little while”
Dr. Lindsay Davidson
03/09/09 Excerpt from Teaching Dossier: Course Revision and Curriculum Development
Dr. Lindsay Davidson
04/04/26 Email “RE: nomination” Dr. Lindsay Davidson 04/05/28 The Musculoskeletal Course: an overview,
analysis and future directions Dr. Lindsay Davidson
05/02/10
Report on 3rd party MSK course evaluation Dr. Joy Mighty and Heather Sparling, Queen’s Centre for Teaching and Learning
05/02/10 Summary of Lecture and TBL Observations Heather Sparling, Queen’s Centre for Teaching and Learning
05/02/10
Letter Dr. Joy Mighty and Heather Sparling, Queen’s Centre for Teaching and Learning
05/02/10 Emails “RE: Student complaints” Dr. David Holland/Dr.Lindsay Davidson
05/02/15 Email “student feedback from MSK” Dr. David Holland, Assistant Dean Undergraduate Medicine
Course evaluation surveys 2005, 2007
Part 1: Likert-scale questions MSK course students (2005, 2007)
2005, 2006, 2007
Part 2: Free-text response questions MSK course students (2005, 2006, 2007)
114
APPENDIX B: Student Course Evaluation Survey Items (2005-2007)
Survey Items Source 2005 2006 2007 Overall, this is an excellent course. USAT –
mandatory
Y Y Y
I learned a great deal from this course. USAT – mandatory
Y Y Y
This course was helpful in developing new skills.
USAT Y Y Y
I felt that this course challenged me intellectually.
USAT Y Y Y
My interest in the subject has been stimulated by this course.
USAT Y Y Y
This course included components that addressed my personal learning style
Instructor Y Y
It was clear to me what was expected of me in this course
USAT Y
The objectives of the course were adequately explained.
USAT Y Y
This course covered the right amount of material.
USAT Y Y Y
The workload in this course was reasonable and appropriate.
USAT Y Y Y
The course material was presented at a satisfactory level of difficulty.
USAT Y Y Y
The course was well organized USAT Y The course was organized in a logical sequence. Similar to
USAT Y Y
I received timely and useful communications from the course coordinator when necessary.
Instructor Y Y
115
Survey Items Source 2005 2006 2007
Team teaching (i.e. multiple “expert” lecturers co-teaching a session) was effectively used in this course.
Similar to USAT
Y Y
Instruction was well coordinated among the teachers who taught separate but related sessions (e.g. the different components of arthritis).
Similar to USAT
Y Y
I participated in more in class discussion in this course than in other courses
USAT
Team discussion of the readiness assessment tests helped me to learn and better understand the course material.
Instructor Y
The use of the audience response units (“clickers”) helped me to learn and better understand the course material.
Instructor Y
The team learning exercises in class helped me to learn and better understand the course material.
Instructor Y Y
The lectures helped me to learn and better understand the course material.
Instructor Y Y
The symposia effectively used cases, patients and expert panels to help me learn and better understand the course content
Instructor Y
The online discussion board helped me to learn and better understand the course material
Instructor Y
I only used the online discussion board because it was required.
Instructor Y
The inclusion of patients/parents in symposia and other sessions was valuable. (wording slightly different 2006/2007)
Instructor Y Y
The course web page was helpful and informative
USAT Y Y
116
Survey Items Source 2005 2006 2007
The course website was easy to navigate. Instructor Y Y
The course learning map/outline provided the information that I needed to prepare for the class sessions.
Instructor Y Y
The course website modules helped me to learn and better understand the course material.
Instructor Y Y
Web-based quizzes and interactive cases helped me learn and better understand the course material
Instructor Y
The on-line references linked to from the course website were helpful and informative
Instructor Y
The individual assignment helped me to learn and better understand the course material.
Instructor Y Y
The goals and objectives of the expanded clinical skills sessions were clear.
Instructor Y Y
The patient partner expanded clinical skills sessions were helpful in improving my ability to perform a “screening” musculoskeletal examination.
Instructor Y Y
The radiology expanded clinical skills session was helpful in improving my ability to understand simple radiographs and recognize different types of imaging.
Instructor Y
The videos provided were useful resources that helped me learn how to perform focused upper and lower extremity musculoskeletal examination
Instructor Y Y
Henry’s Clinical Surgery (required text) was worth the money I paid for it
USAT Y
117
Survey Items Source 2005 2006 2007
Cecil’s Essentials of Medicine (required text) was worth the money I paid for it
USAT Y
I found the required chapters in Cecil’s Essentials of Medicine Essentials of Surgical Specialties to be valuable resources that aided my understanding of course material
Similar to USAT
Y
I found the required chapter in Essentials of Surgical Specialties to be a valuable resource that aided my understanding of course material
Similar to USAT
Y
I used an alternate text as my reference Instructor Y
The course was organized to develop linkages between the different MSK themes (injury, arthritis, pediatrics) as well as between the horizontal Medicine in Society components (Epidemiology, Geriatrics)
Instructor Y Y
118
APPENDIX C: Research Survey
Dear students: I hope that you will complete this survey, which will serve two purposes. The information gathered from your class as well as the MEDS 2009 class will be used in the further development of the MSK Course. Data collected will also be included in my M.Ed. thesis. In particular, the survey asks you to describe your personal learning style (after Kolb’s description), and capture your perceptions of working with peers, working in teams, working individually using electronic learning aids as well as your subjective impression of the growth of your MSK content-specific knowledge after completion of the course. The survey is voluntary and anonymous and has been approved by the Faculty of Health Sciences Research Ethics Board. It will take you between 5 and 10 minutes to complete. Students who choose to complete the survey will return it to the class curriculum representative and receive a ballot. There will be a random draw for a $50 Campus Bookstore gift certificate once the surveys are submitted. If you have any questions regarding the survey, please feel free to contact me ([email protected], 544-9626), Assistant Dean of Undergraduate Education, Dr. David Holland ([email protected]) or Chair of the Faculty of Health Sciences Research Ethics Board Dr. Albert Clark ([email protected]). Thanks, in advance, for your participation in the survey, Dr. Lindsay Davidson MSK Course Chair Assistant Professor Department of Surgery Queen’s University
119
Part 1: Please circle the number beside the phrase that best describes the extent to which you agree with the following statements:
Strongly Disagree
Disagree Neither Agree nor Disagree
Agree Strongly Agree
1. The ability to collaborate with my peers will be necessary if I am to be successful as a student.
1 2 3 4 5
2. I have a positive attitude about working with my peers.
1 2 3 4 5
3. The ability to work with my peers is a valuable skill.
1 2 3 4 5
4. Collaborating with my peers will help me be a better student.
1 2 3 4 5
5. Solving problems in a group is an effective way to practice what I have learned.
1 2 3 4 5
6. Solving problems in a group is an effective way to learn.
1 2 3 4 5
7. Working in teams in class is productive and efficient.
1 2 3 4 5
8. Group decisions are often better than individual decisions.
1 2 3 4 5
9. Solving problems in groups leads to better decisions than solving problems alone.
1 2 3 4 5
10. Online study modules 1 2 3 4 5
120
are an efficient way to learn. 11. Online study modules are an effective way to learn.
1 2 3 4 5
12. Online study modules are an effective way to practice what I have learned.
1 2 3 4 5
13. Lectures are an efficient way to learn.
1 2 3 4 5
14. Lectures are an effective way to learn.
1 2 3 4 5
14. Lectures are an effective way to practice what I have learned.
1 2 3 4 5
15. I feel that I could evaluate a patient with a musculoskeletal injury or disease, and propose a differential diagnosis.
1 2 3 4 5
16. I feel that I can select appropriate investigations for a patient with a musculoskeletal injury or disease and understand the relevance of the results.
1 2 3 4 5
17. I feel that I can construct an initial management plan for a patient presenting with musculoskeletal injury or disease.
1 2 3 4 5
18. I feel that I can identify preventative measures relevant to musculoskeletal injury and disease.
1 2 3 4 5
19. I feel that I can anticipate, recognize and
1 2 3 4 5
121
treat complications related to musculoskeletal injury or disease or to the common therapies used in their treatment. 20. I have an understanding of the principles of effective group work.
1 2 3 4 5
122
Part 2: For each of the following educational objectives, rank order the three listed educational modalities in terms of their suitability to achieve the stated goal. Rank the modality “1” if it is the most suited to achieve the objective and “3” if it is the least. If you feel that none of the described modalities achieve this goal, tick the fourth box. Educational Objective Lecture Team
based learning
Online study module
None are helpful
21. Organizes a large amount of material into a manageable learning experience.
22. Gives me a practitioner’s perspective. 23. Brings medicine to life. 24. Helps me become more reflective. 25. Allows me control of my learning. 26. Guides me through difficult topics. 27. Provides me with appropriate and useful feedback.
28. Allows me to apply what I’m learning. 29. Helps me integrate my learning. 30. Helps me judge the extent of my own learning.
31. Strengthens my communication and collaborative skills.
Part 3: Complete the following statements describing, in your own words, the most useful feature(s) of each type of educational modality from your perspective as a medical student. 32. Online study helped me to: 33. Team based learning helped me to:
123
APPENDIX D: Student Ranking of Instructional Methods by Educational Objectives
124
Graph D1: Organizes a large amount of material into a manageable learning experience (% responses)
74.7
19.3
6.0
40.7
39.0
20.3
21.1
54.4
24.6
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0
Most
Some contribution
Least
Lecture
2007 2006 2005
2.4
15.7
81.9
6.8
20.3
72.9
3.5
22.8
73.7
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0
Most
Some contribution
Least
TBL
2007 2006 2005
26.5
62.7
10.8
54.2
40.7
5.1
75.4
22.8
1.8
0.0 20.0 40.0 60.0 80.0
Most
Some contribution
Least
Online module
2007 2006 2005
125
Graph D2: Gives me a practitioner's perspective (% responses)
65.4
25.0
9.6
47.4
22.8
29.8
44.3
21.5
34.2
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0
Most
Some contribution
Least
Lecture
2007 2006 2005
23.1
42.3
34.6
49.1
21.1
29.8
44.3
27.8
27.8
0.0 10.0 20.0 30.0 40.0 50.0 60.0
Most
Some contribution
Least
TBL
2007 2006 2005
13.5
36.5
50.0
3.5
57.9
38.6
11.4
50.6
38.0
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0
Most
Some contribution
Least
Online module
2007 2006 2005
126
Graph D3: Brings medicine to life (% responses)
32.4
45.9
21.6
26.0
52.0
22.0
11.8
33.8
54.4
0.0 10.0 20.0 30.0 40.0 50.0 60.0
Most
Some contribution
Least
Lecture
2007 2006 2005
30.8
46.2
23.1
70.0
14.0
16.0
79.4
14.7
5.9
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0
Most
Some contribution
Least
TBL
2007 2006 2005
4.5
31.8
63.6
6.0
32.0
62.0
8.8
51.5
39.7
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0
Most
Some contribution
Least
Online module
2007 2006 2005
127
Graph D4: Helps me become more reflective (% responses)
31.4
31.4
37.3
22.9
31.3
45.8
4.1
23.0
73.0
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0
Most
Some contribution
Least
Lecture
2007 2006 2005
31.4
31.4
37.3
45.8
27.1
27.1
67.6
17.6
14.9
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0
Most
Some contribution
Least
TBL
2007 2006 2005
39.2
35.3
25.5
33.3
39.6
27.1
28.4
59.5
12.2
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0
Most
Some contribution
Least
Online module
2007 2006 2005
128
Graph D5: Allows me control of my learning (% responses)
34.5
34.5
30.9
20.8
47.2
32.1
10.1
35.4
54.4
0.0 10.0 20.0 30.0 40.0 50.0 60.0
Most
Some contribution
Least
Lecture
2007 2006 2005
7.3
30.9
61.8
5.7
35.8
58.5
7.6
48.1
44.3
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0
Most
Some contribution
Least
TBL
2007 2006 2005
63.6
30.9
5.5
71.7
20.8
7.5
82.3
16.5
1.3
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0
Most
Some contribution
Least
Online module
2007 2006 2005
129
Graph D6: Guides me through difficult topics (% responses)
65.5
25.5
9.1
46.4
37.5
16.1
28.4
42.0
29.6
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0
Most
Some contribution
Least
Lecture
2007 2006 2005
7.3
27.3
65.5
12.5
25.0
62.5
8.6
28.4
63.0
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0
Most
Some contribution
Least
TBL
2007 2006 2005
36.4
43.6
20.0
44.6
35.7
19.6
65.4
28.4
6.2
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0
Most
Some contribution
Least
Online module
2007 2006 2005
130
Graph D7: Provides me with appropriate and useful feedback (% responses)
25.5
27.3
47.3
7.0
30.2
62.8
3.9
11.7
84.4
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0
Most
Some contribution
Least
Lecture
2007 2006 2005
27.3
38.2
34.5
55.8
34.9
9.3
53.2
39.0
7.8
0.0 10.0 20.0 30.0 40.0 50.0 60.0
Most
Some contribution
Least
TBL
2007 2006 2005
54.5
30.9
14.5
37.2
34.9
27.9
42.9
49.4
7.8
0.0 10.0 20.0 30.0 40.0 50.0 60.0
Most
Some contribution
Least
Online module
2007 2006 2005
131
Graph D8: Allows me to apply what I'm learning (% responses)
13.0
18.5
68.5
8.3
16.7
75.0
2.6
3.8
93.6
0.0 20.0 40.0 60.0 80.0 100.0
Most
Some contribution
Least
Lecture
2007 2006 2005
29.6
48.1
22.2
64.6
25.0
10.4
80.8
19.2
0.0
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0
Most
Some contribution
Least
TBL
2007 2006 2005
64.8
27.8
7.4
27.1
58.3
14.6
17.9
76.9
5.1
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0
Most
Some contribution
Least
Online module
2007 2006 2005
132
Graph D9: Helps me integrate my learning (% responses)
22.2
29.6
48.1
12.0
22.0
66.0
6.6
5.3
88.2
0.0 20.0 40.0 60.0 80.0 100.0
Most
Some contribution
Least
Lecture
2007 2006 2005
25.9
38.9
35.2
63.3
22.4
14.3
78.9
17.1
3.9
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0
Most
Some contribution
Least
TBL
2007 2006 2005
53.7
33.3
13.0
28.3
50.0
21.7
15.8
76.3
7.9
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0
Most
Some contribution
Least
Online module
2007 2006 2005
133
Graph D10: Helps me judge the extent of my own learning (% responses)
17.9
19.6
62.5
15.7
23.5
60.8
1.3
6.6
92.1
0.0 20.0 40.0 60.0 80.0 100.0
Most
Some contribution
Least
Lecture
2007 2006 2005
19.6
57.1
23.2
43.1
39.2
17.6
60.0
36.0
4.0
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0
Most
Some contribution
Least
TBL
2007 2006 2005
58.9
28.6
12.5
43.1
37.3
19.6
38.2
57.9
3.9
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0
Most
Some contribution
Least
Online module
2007 2006 2005
134
Graph D11: Strengthens my communication and collaborative skills (% responses)
17.0
53.2
29.8
2.4
61.0
36.6
0.0
47.9
52.1
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0
Most
Some contribution
Least
Lecture
2007 2006 2005
78.7
12.8
8.5
90.2
2.4
7.3
100.0
0.0
0.0
0.0 20.0 40.0 60.0 80.0 100.0 120.0
Most
Some contribution
Least
TBL
2007 2006 2005
4.3
31.9
63.8
7.3
31.7
61.0
1.4
52.1
46.5
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0
Most
Some contribution
Least
Online module
2007 2006 2005
135
APPENDIX E: Sample Application Exercise (used in 2006)
136
MSK Application exercises
1. Jason X., a 27=year old male inmate is brought to the Emergency room for assessment. He reports that he banged his right hand against his metal bunk three days ago and injured his left fifth knuckle. He presented to the institution infirmary complaining of increasing hand pain and was sent to the hospital for an x-ray and orthopaedic consultation. Upon arrival to the Emergency room, his vital signs are:
HR 85/mn BP 130/70 RR 20/mn T 37.9 o C Mr. X is very reluctant for you to examine his right hand and is requesting morphine for the pain. You convince him to allow you to examine his hand and note the following: (photo of a hand with swelling in the area of the metacarpal head) Redness and swelling in the area of the metacarpal head. Extreme tenderness upon palpation of the distal half of the fifth metacarpal. The patient has no active motion of the fifth metacarpal joint. Passive motion is excruciatingly painful. Part 1: Based on this information, choose the ONE diagnosis that you are MOST CONCERNED about from the following list: a) Fractured fifth metacarpal b) Dislocated fifth metacarpophalangeal joint c) Osteogenesis imperfecta d) Septic arthritis e) Narcotics addiction
Two reasons why you chose this answer: Part 2:
137
The radiology technologist arrives with the x-ray that she had taken just prior to your arrival. Which of the following best describes the positive findings on this x-ray? a) Fractured fifth metacarpal with dorsal angulation b) Fractured fifth metacarpal with volar tilt c) Fractured fifth metacarpal with dorsal displacement d) Both a) and b) e) Both a) and c)
Part 3:
The nurse arrives with the cast cart, assuming that you would like to apply a cast. Just prior to applying the cast padding, you notice some yellow fluid draining from a 3mm laceration over his metacarpal head. Your next step would be to:
a) Apply a steri-strip to the laceration and proceed with closed reduction and
casting of his hand. b) Take a culture from the laceration, proceed with closed reduction and
casting of his hand and start antibiotics if culture result is positive. c) Omit the closed reduction for the time being and apply a dressing and a
removable splint to his hand and start broad spectrum antibiotics. d) Take a culture from the laceration, leave the wound open and start broad
spectrum antibiotics pending culture results. e) Order that the patient remain fasting and contact the operating room to
book emergency drainage of his joint. Reason why you chose this answer:
138
Amber B., an 11 year old girl is brought to the Emergency room by ambulance following a pedestrian-car accident. She was crossing Highway 2 near the library in Picton when she was hit by a car traveling at approximately 80 km/hr. She was found moaning by the side of the road and transported to hospital complaining of right leg pain. Part 1: Rank the following aspects of her emergency room evaluation in the order that you feel they should be performed (a indicates that this is the first evaluation, e is the last).
1. Examination of eyes 2. Auscultation of chest 3. Complete blood count 4. Neurovascular examination of foot 5. Blood pressure
Explain why you chose a: Explain why you chose e: Part 2: Miss. B. is fully assessed by the trauma team. A detailed examination reveals the following findings: HR 90/mn BP 100/65 RR 20/mn T 36.2 o C Her trachea is central, her chest is clear to auscultation and her heart sounds are normal. She is awake, alert and oriented to place, person and time but does not recall the accident. Her abdomen is soft and non tender. There is no tenderness or deformity of either upper limb, the left lower limb, pelvis or spine. She is noted to have a shorted right thigh with an extensive contusion along the medial surface and external rotation of the foot. Her skin is intact and her foot appears is pink and warm with a palpable dorsalis pedis pulse. She can dorsiflex and plantarflex her ankle and has intact sensation to light touch over the entire leg and foot. Please review the accompanying x-ray. (xray of broken femur in a child)
139
6. Assuming that this is the only injury, what treatment would you recommend.
a) Application of skeletal traction b) Closed reduction and casting c) Closed reduction and external fixation d) Closed reduction and intramedullary rod e) Open reduction and plate fixation
140
APPENDIX F: Sample Online Module (used in 2007)
Bone Morphology and Fracture Healing
Objectives
Upon completion of this module, the student will be able to:
1. Describe the primary functions of bone 2. Describe the structure of bone at the macroscopic and microscopic level 3. Identify the function of different types of bone cells 4. Describe fracture healing as a sequence of stages 5. Predict anomalies in fracture healing based on information about the patient, the fracture
and other clinical modifiers
Reference
SEER online module (review of bone morphology, formation and skeletal structure) - individual webpages from this site will be referenced in the module.
Related websites and image collections
University of Ottawa Histology website
American Society for Bone and Mineral Research - Bone Curriculum
School of Anatomy and Human Biology - The University of Western Australia - Blue Histology webpage Start by reviewing the following two webpages: 1) Functions of the Skeletal System and 2) Structure of Bone Tissue. Based on your understanding of this material, summarize the key functions of the skeletal system: (Answer A1 on answer sheet)
Describe a pathological situation where one of the mechanical functions of the skeleton is disturbed. What long term deficit may result? (Answer A2 on answer sheet)
At the macroscopic level, bone is organized into cortical and cancellous bone. The metaphyseal and epiphyseal regions consist of cancellous bone. In these areas, bony trabeculae are arranged for maximum strength and have capacity to realign in the direction of an applied stress. This is an illustration of Wolff's law: "bone is deposited and resorbed in accordance with the stresses placed upon it". The diaphyseal areas of the long, tubular bones (femur, tibia, humerus, etc.) are made of cortical bone. These have a thin endosteal membrane lining the inner surfaces. All bones (with the exception of those portions that are intra-articular) are covered with periosteum on their outer surface. Periosteum (image Copyright Lutz Slomianka 1998-2004) is thickest in young children and may provide a degree of stabilization in certain pediatric fractures. Biochemically, the skeleton contains 65% inorganic elements (minerals) and 35% organic bone matrix. Three types of bone cells, osteoblasts, osteocytes and osteoclasts, function together to regulate mineral homeostasis in response to endocrinological and mechanical signals.
Review histologic specimens of the two types of bone tissue found in the skeleton by clicking on the links provided: cortical (a.k.a. compact - image Copyright Lutz Slomianka 1998-2004) and cancellous (a.k.a. spongy or trabecular) bone.
Bone healing is influenced by the potential and activity of the cells that line the surface of the bone (osteoblasts and osteoclasts). Based on this fact alone, the greater the surface are of bone involved in the healing process, the more rapid the healing is likely to be.
Which type of bone tissue will heal the fastest? (Answer A3 on answer sheet)
a) Cortical bone
b) Cancellous bone Bone Cells There are three types of cell in bone. Their functions are outlined below: Osteoblasts Osteoblasts line the inner (endosteal) and outer (periosteal) surfaces of the bone. Osteoblasts synthesize organic bone matrix (osteoid) and collagen. Osteoblasts also synthesize the enzyme alkaline phosphatase and initiate the process of mineralization. Osteoblasts represent the final stage of differentaition of pluripotential stem cell in the bone marrow. They typically live about three months before flattening to become metabolically inactive bone lining cells. About 10-15% of osteoblasts become osteocytes. Osteocytes The most abundant cell in bone is the osteocyte. The cells known as osteocytes are formed from metabolically inactive osteoblasts which have become entombed in a newly formed bone matrix during active bone formation. Osteocytes no longer secrete osteoid; however they do maintain their sensitivity to PTH and vitamin D and participate in calcium regulation. Osteocytes are interconnected by tiny channels called canaliculi.
Osteoclasts
Osteoclasts are multi-nucleated giant cells that resorb bone by enzymatic degradation. These cells are characterized by a ruffled border and lie in bone resorption pits called Howship's lacunae. Osteoclasts are related to hematopoietic cell lines (monocytes, macrophages). Osteoclasts are seen in increased numbers in diseases with increased bone turnover.
How many types of bone cells can you identify in this histological slide? What biological process does it illustrate?
from the PEIR pathology library © Copyright UAB and the UAB Research Foundation, 1999-2000. All rights reserved.
This is a high magnification view of bone remodeling during fracture healing; osteoclasts, osteocytes and osteoblasts are shown.
adapted from from the PEIR pathology library © Copyright UAB and the UAB Research Foundation, 1999-2000. All rights reserved.
Bone is a composite material, consisting of inorganic and organic components as well as the cellular elements discussed previously.
Inorganic matrix
The inorganic matrix of bone consists of a calcium hydroxyapatite, which is in a crystalline structure. This serves to:
1. Mineralize the osteoid produced by osteoblasts, 2. Provide strength and hardness to the bone, 3. House the body's mineral reserves, including:
o 99% of the body's calcium, o 85% of its phosphorous, o 65% of its sodium and magnesium
Organic matrix
Thirty-five percent (35%) of bone is organic. The organic matrix is composed of the bone cells that you have just learned about, and of protein. Ninety percent (90%) of bone protein is Type I collagen which has a triple helical structure, and ten percent (10%) consists of noncollagenous
proteins. Abnormalities of collagen formation may occur and lead to bone fragility. Watch a flash movie depicting the normal sequence of collagen synthesis on the American Society for Bone and Mineral Research Bone Curriculum webpage.
This newborn male is observed to have shortened, excessively bowed limbs. He cries with even gentle manipulation of his arms and legs. Xrays reveal multiple fractures in various stages of healing. What abnormality of bone could result in this clinical picture? (Answer A4)
Complete this gap-fill exercise to test your knowledge about bone cells.
This type of bone is found in abundance in the metaphyseal regions of long bones, contains irregularly arranged trabeculae and has an ability to remodel secondary to mechanical stresses.
Choose the correct answer: (Answer A5)
a) Cancellous bone
b) Cortical bone
Which of the following statements about bone matrix is CORRECT? (Answer A6)
a) Most of the collagen found in bone is Type II collagen
b) Calcium carbonate crystals act to mineralize osteid
c) The inorganic matrix of bone houses the body's mineral reserves
d) Osteoid is produced by osteocytes
e) The organic bone matrix is arranged in a crystalline structure
Before we discuss fracture healing, it is important to understand some basic principles about bone growth. Review this related webpage from the Association for Bone and Mineral Research, with a particular focus upon the growth of long bones, flat bones and bone remodeling. Make sure that you watch the movies that outline these three important processes.
Which of the following statements best describes the growth of long bones? (Answer A7)
a) FIbroblasts transform into osteoblasts and create woven bone which is subsequently reorganized into lamellar bone
b) Microscopic cracks are created by activity. These stimulate first bone resorption and subsequently bone formation.
c) Cartilage is transformed as the chondrocytes hypertrophy and eventually die; there is vascular ingrowth and mineralization resulting in bone formation.
Which type of bone growth is similar to fracture healing? If you are not sure of the answer, review the reference cited above. (Answer A8)
*Bone is a unique tissue as it is able to reform itself and does not heal with scar as other tissues do.
Stages of fracture healing
A fracture occurs when the continuity of a bone is broken and local blood supply is interrupted. If the overlying soft tissues are also injured, fracture healing may be delayed or disrupted, particularly in anatomic regions with decreased vascular networks such as the tibial diaphysis. Bone is unique in its ability to regenerate itself. Healing occurs via reactivation of embryologic processes resulting in the formation of bone not scar. Fracture healing can be described in three conceptual stages. An understanding of the timing and mechanisms associated with each stage is important in planning fracture treatment. The stages are:
Inflammation
This begins immediately after bone injury with the formation of a local hematoma or fibrin clot. There is local cell death where vessel disruption has resulted in ischemia - usually at the very ends of the fractured bone. Over the course of the next few days, this area becomes infiltrated by inflammatory cells and is characterized by local swelling and warmth. The inflammatory cells release lysosomal enzymes and other mediators that attract pluripotent cells to the area; they also act to remove necrotic tissue. Fibroblasts, mesenchymal cells and osteoprogenitor cells appear and may transform nearby tissues. The fracture is tender and may be grossly mobile to physical examination at this stage. Inflammation is at its peak 48 hours after a fracture.
Repair
The reparative phase begins a few days after the injury with the arrival of mesenchymal cells able to differentiate into fibroblasts, chondroblasts and osteoblasts. The repair phase persists for several months; it can be divided into two distinct phases: soft and hard callus formation.
1. "Soft callus" formation lasts for approximately six weeks from the time of injury. During this preliminary stage of repair, pain and swelling subside and bony fragments become united by fibrous and cartilagenous tissue. Woven bone is formed. While this creates some stability, the fracture may still angulate at this stage if not held with stable external support, such as a cast or external fixator, or internal support provided by plates, screws or intramedullary devices.
2. "Hard callus" formation - During this second stage of repair, woven bone is transformed into lamellar bone. This takes approximately three months.
Remodeling
Remodeling is the process by which bone is removed in tiny increments and then replaced by new bone. After a fracture, remodeling may continue for months or even years. The adult human skeleton continuously replaces itself at rate of 10-18% per year. The rate of remodeling is accelerated in children and during fracture repair. In addition to being an essential part of fracture healing, remodeling plays an important role in calcium homeostasis. During the remodeling phase the woven bone is converted to lamellar bone and the medullary canal is reconstituted. During this phase, bone responds to loading characteristics according to Wolff's law. Some angular deformity may correct during this stage in children with sufficient growth remaining (up to 5o per year of growth remaining).
Factors affecting bone healing
Various local and systemic factors affect the duration and effectiveness of the healing process. Abnormalities in any of these areas may lead to abnormally slow healing (delayed union) or failure to heal (non union). These include:
Systemic factors
1. Age: Children heal more quickly than adults; healing potential is decreased with advancing age
2. Nutrition: Poor nutrition and/or vitamin deficiency adversely affects healing 3. General health: Chronic illness depresses healing response (diabetes, anemia, systemic
infection) 4. Generalized atherosclerosis: Decreases healing 5. Hormonal factors: Growth hormone enhances healing; corticosteroids depress healing 6. Drugs: Non steroidal anti-inflammatory drugs (e.g. ibuprofen) depress healing 7. Smoking: Decreases healing
Local factors
1. Degree of local trauma/bone loss: A comminuted fracture with more soft tissue injury is slower to heal
2. Area of bone affected: Metaphyseal fractures heal faster than diaphyseal 3. Abnormal bone (infection, tumour, irradiated): Slower to heal 4. Degree of immobilization of fracture: Motion at site delays healing
Disruption of vascular supply: Delays healing
Abnormal Healing in Bone
Although, in most cases, fracture healing proceeds uneventfully, there are certain situations when the outcome is not normal. Several patterns of abnormal healing are described in this section.
Fracture non-union
Healing is described as 'delayed' if union is not seen within the expected time after initial treatment. At the six month mark, an unhealed fracture is termed a "non- union". There are two types of non-union: atrophic, where little callus has formed and hypertrophic, where there is obvious callus but continued instability. Atrophic non-unions respond to bone grafting. This involves transplantation of the patient's own healthy bone, often from the iliac crest, to the non-union site. An alternative is the implantation of donor bone or an artificial bone substitute. Hypertrophic non-unions often result from increased motion at the fractures site. These improve with by surgical stabilization of the non-union. If a non-union persists and remains mobile, the fibrous tissue at the fracture site may undergo transformation into synovial cells forming a "false joint" or pseudarthrosis.
Stress fractures
Stress fractures are the result of an imbalance of bone formation and bone healing, usually in young healthy individuals involved in repetitive physical activity. Common sites include the tibia, metatarsal and femoral neck. The diagnosis is often challenging as plain radiographs may be negative or may reveal calcification only in the late stages of healing. Bone scans and/or Magnetic Resonance Imaging scans are frequently diagnostic. Treatment with immobilization is usually successful.
The red arrow points to a stress fracture in this patient's calcaneus
Reactive bone formation
Reactive bone formation isn't true "healing" but instead a response of the bone to an underlying abnormality. Review this link from the University of Washington School of Medicine to learn more about the different patterns of reactive bone formation. "Sunburst" or "onion skin" patterns are growing rapidly and so are classified as "aggressive", while more solid patterns of periosteal reaction (such as are seen in children's fractures) indicate slower growths that are likely benign.
Benign periosteal reaction after fracture
This is an example of benign periosteal reaction seen as a normal part of fracture healing in a child four weeks after a supracondylar humeral fracture
Sunburst pattern of periosteal reaction
This child has an agressive humeral osteosarcoma. The sunburst reactive bone formation shown in this image is growing so rapidly that there is not enough time for a layer of bone to form over the periosteum. This is indicative of an aggressive growth and suggests, albeit not conclusively, the presence of a malignancy. In this case, the sunburst pattern is caused by an osteosarcoma
Review your understanding of normal and abnormal fracture healing
Check that you recall the normal sequence of fracture healing by unscrambling this mixed-up sentence. (Answer A9)
A 65-year old man slips and falls on ice resulting in a closed fracture of his tibia. The fracture line is short and oblique involving the middle third of the diaphyseal portion of the bone. He has a history of heart disease (triple-vessel bypass two years ago) and diabetes. He is a non-smoker with no prior history of fractures. Identify factors from the described scenario which may cause this patient to have a slow rate of fracture healing.
The patient is treated with cast immobilization. The xray seen above illustrates his fracture eight months after the injury. How would you describe his injury now? (Answer A10)
a) Healed fracture
b) Delayed union
c) Hypertrophic non-union
d) Atrophic non-union
You have now completed the Bone Morphology and Fracture Healing module!
