The Introduction of Digital Game-Based Learning in a Surgical Clerkship: A Pilot Study

Medical Science Educator © IAMSE 2013 Volume 23(3) 329

MEDICAL SCIENCE EDUCATOR The Journal of the International Association of Medical Science Educators Med Sci Educ 2013; 23(3): 329-335

ORIGINAL RESEARCH

The Introduction of Digital Game-Based Learning in a Surgical Clerkship: A Pilot Study Anuradha Subramanian & Shahrzad Vafa Baylor College of Medicine, Houston, TX, USA Introduction At the Baylor College of Medicine, the National Board of Medical Examiners’ (NBME) surgery subject exam is administered at the end of the Core Surgery Clerkship. Topics tested on the exam include neurosurgery, trauma, pediatric surgery, gynecology, and otolaryngology, just to name a few. However, not all students get to experience all of these specialties during their time on the Core Surgery Clerkship. Although efforts are being made to improve on this, changes in the overall structure of curriculum can take time. In 2011, the director of the surgery clerkship decided to take advantage of gaming technology and implemented a pilot study in order to provide an opportunity for students to review surgical subspecialty topics that do not get emphasized during the clerkship. The theory of gaming as a formal teaching tool was introduced nearly 80 years ago.1 Since then, modern technology has opened new facets of learning that could have never been imagined even a decade ago. Games can simulate real life experiences than more traditional educational methods and have been shown to provide an excellent link between theory and practice.2 Numerous applications of simulation and games have been a staple of engineering education for many years.3 The inclusion of such approaches in higher education programs, like medical education, has been seen as frivolous and based on entertainment rather than learning.4 However, many in medical education have shown that active learning tools such as digital games can be complementary, e-teaching/learning resources

that can greatly expand and enrich the curriculum. In 2000, Fukuchi et all successfully used an Interactive Board Game to teach a multidisciplinary approach to cancer treatment during a surgical clerkship.1 Mann et al, in 2002, developed an interactive game-based tool to teach students about surgical management algorithms in breast disease.2 In 2004, Ogershock and Cottrell and Eckert et al revealed that gaming can stimulate learning pediatrics and immunology, respectively.5,6 Also, DaRosa et al, in 2006, demonstrated an innovative game on viral hepatitis which was well-received by students.7 Multiple advantages of using games as a learning tool have even been described in gerontology, geriatrics, and aging-related courses.8 However, creating interactive and innovative games are expensive and time consuming and before implementing many modules at our institution with our limited resources and budget, it would be useful to do a pilot study to learn if they provide additional value, will be accepted and used by students, and ultimately impact grades for the clerkship courses. Therefore, the purpose of this study was to determine if digital game-based learning (DGBL) modules with a supplementary reading list would be utilized and perceived as effective learning tools to increase students’ National Board of Medical Examiners (NBME) surgery subject exam scores. These modules would provide self-paced additional instruction and allow introduction of various surgical subspecialty topics to students who did not rotate on those surgical services.

Corresponding author: Shahrzad Vafa, EdD, Baylor College of Medicine, Houston, TX, USA. Tel: +1 - 713-798-7836, Fax: +1 - 713-798-7571, Email: [email protected]


Methods After approval by the Baylor College of Medicine’s Institutional Review Board, the following items were created: learning modules using Raptivity (San Jose, CA) and recommended reading list from Sabiston’s Textbook of Surgery. Raptivity is a software which allows users to create learning interactions such as games, simulations, brainteasers, and interactive diagrams (Figures 1-3). The questions and exercises for the modules were derived from high yield topics noted in boards review books. The modules engaged the students in questions and provided them immediate feedback after they submitted their answers. The learning modules and recommended reading list were then placed on Blackboard (Washington DC), a secure course management system. At the orientation for the Baylor College of Medicine Core Surgery Clerkship, 91 students were introduced to the modules and reading list. Several screenshots of the modules were demonstrated during the orientation (Figure 1-3). It was explained to the students that the modules would provide them with immediate feedback of correct and incorrect answers. The reading list was given to the students during orientation as well. Students were encouraged to utilize the reading list, especially regarding topics they missed on the modules. After their NBME exam, 84% (n = 76) of the students completed an anonymous survey consisting of 7 Likert scale questions and one open-ended comment section to qualitatively evaluate the parameters of the modules, the experience of the module-user interface, and the perceived effectiveness of the modules and the reading list (Figure 4). Mean scores on the NBME exams were compared to those scores of students who took the standardized exam one year prior (n = 88) to the implementation of the learning modules and recommended reading list. Mean scores of the responses to the survey questions were analyzed using Microsoft Excel. Results are presented as mean ± standard deviation. Student t-tests were performed for continuous variables. Statistical significance was defined as p ≤ 0.05 for all measures. Results Survey Results The students that responded to the survey were evenly distributed between males (52%) and females (48%). Their average age was 25.2 ± 1.8 years. According to the survey results, on a scale from 1-4 (1 being the most positive), students rated

the modules as an effective learning tool 2.3 ± 0.6. They were motivated to utilize this resource 2.3 ± 1.0. The students on average spent 2-6 hours, once a week, completing the module and utilizing the reading list. Students reported that the modules and reading list had some influence on their knowledge of surgical diseases and management and in preparation for the NBME surgery exam. In regards to recommending similar learning modules to other Core Clerkship Directors, the responses were positive. The open-ended question at the end of the survey provided additional data. From students that responded to the open-ended questions, 15 (24%) students enjoyed the modules and even stated that they wished there were more available. One student commented that “the games were a very good tool for preparing for questions similar to NBME exam questions.” On the other hand, 20 (26%) students sensed that there were too many study options for their clerkship and for the NBME surgery exam and felt that they did not have enough time to utilize both the recommended reading list and the learning modules. “I felt we had lots of resources and did not spend time on these,” remarked another student. Five students (7%) felt that Sabiston’s Textbook of Surgery was too challenging to comprehend; however, four (5%) felt that the sections recommended to them were beneficial for the NBME surgery exam. Twenty-six students (24%) declared that they did not even know about the reading list or modules even though they were introduced at orientation, and reminders were sent periodically throughout the rotation. A student wrote, “Sorry I didn’t know about the modules at all – I never heard about them.” NBME Surgery Subject Exam Scores All students (100%) completed the NBME Surgery Subject Exam. The average score of the study group was 77.1 ± 9.5. The control group’s average score was 78.7 ± 9.5. The statistical analysis revealed no significance difference (p = 0.13). Discussion Digital game-based learning (DGBL) provides the novel opportunity for the learners to contextualize information and to study the consequences of their choices, leading to trial and error exploratory learning. The unique combination of enjoyment, motivation, interactivity, immediate feedback, and engagement/flow offered by DGBL provides a sound platform on which learning becomes an anticipated fun event instead of a tedious boring chore. Skills essential for medical education, including problem


solving, trial and error learning, and parallel attention are all core elements of many popular digital games.7 We are aware that this current generation of medical students command being taught via innovative technologies, like simulation, You Tube, and Apps on their PDA. The lecture format is no longer accepted by students as the way for educators to teach and for them to learn. Although there is noted success with DGBL in medical education, this pilot study revealed different results.1-6 Our hypothesis was that web-based self-paced DGBL modules would facilitate the learning about surgical topics and improve national exam scores. However, it seems that was not the case. Based on our findings, we realized that just providing the modules will not guarantee students to utilize them. Although, students reported that the modules and reading list had some influence on their knowledge of surgical diseases and rated them high; however, because of advancements in technology, students have abundance of information available to them including free Apps for their mobile devices, many online interactive resources, e-books, and many other alternative methods for studying. Students seem to choose the study methods and resources which they perceive as having the most added value and which have withstood the test of time, like boards review, question, and case review books. Students feel that they need more time, more communication, and more guidance on the value of the modules. We were also surprised by the number of students who stated that they were unaware of the provided modules and reading list even though it had been visually presented at their orientation and even when reminders were sent to them throughout the clerkship. This made us believe that there was an associated decreased perceived value of these learning tools to many students. These findings provided us with some insight on the future implementation of DGBL modules. We learned that even if we create high quality and useful modules, students have to find extreme value in choosing these modules over all the other options available to them. More innovative structured learning material that is integrated into lectures and curriculum has the promise to be more effective. The researchers, in the future, propose the creation of new learning modules which the students will complete before and after each of fifteen mandatory lectures for the clerkship. Students then will be held accountable to complete them. These modules will educate the students before the lecture and allow for self-

assessment after the lecture, providing a perception of value for the modules. Notes on Contributors ANURADHA SUBRAMANIAN, MD, is an Assistant Professor of Surgery, Michael E. DeBakey VA Medical Center and a Director, Medical Student Education, Baylor College of Medicine (BCM), Houston, Texas, USA. SHAHRZAD VAFA, EdD, is an Assistant Professor, Department of Family and Community Medicine and an Assistant Director, Education Resource Center, Baylor College of Medicine (BCM), Houston, Texas, USA. References 1. Fukuchi SG, Offutt LA, Sachs J, Mann BD.

2000. Teaching a multidisciplinary approach to cancer treatment during surgical clerkship via an interactive board game. Am J Surg. 000;179:337–340.

2. Mann BD, Eidelson BM, Fukuchi SG et al. The development of an interactive game-based tool for learning surgical management algorithms via computer. Am J Surg. 2002 ;183(3):305-.8.

3. Kanthan R, Senger JL. The impact of specially designed digital games-based learning in undergraduate pathology and medical education. Arch Pathol Lab Med. 2011;135(1):135-42.

4. Royse MA, Newton SE. How gaming is used as an innovative strategy for nursing education. Nurs Educ Perspect. 2007;28(5):263–267

5. Ogershok PR, Cottrell S. The pediatric board game. Med Teach. 2004;26(6):514-7.

6. Eckert GU, Da Rosa AC, Busnello RG, et al. Learning from panel boards. T-lymphocyte and B-lymphocyte self-tolerance game. Med Teach. 2004;26(6):521-4.

7. Da Rosa AC, Moreno FL, Mezzomo KM, Scroferneker ML. Viral hepatitis. An alternate teaching method. Educ Health. 2006;19(1):14-21.

8. Schmall V, Grabinski CJ, Bowman S. Use of games as a learner centered strategy in gerontology, geriatrics, and aging related courses. Gerontol Geriatr Educ. 2008;29(3):225–233.


APPENDIX

Figure 1

Figure 2


Figure 3


Figure 4 BAYLOR COLLEGE OF MEDICINE DEPARTMENT OF SURGERY Please complete this survey which will assist us in rating the effectiveness of these modules as learning tools. If you fill out this survey, you are consenting or agreeing to take part in this study. We will take all steps legally possible to keep this information confidential. Your participation in this study is strictly voluntary, and you may withdraw from the study or stop participating at any time. If you do not complete this survey, it will not affect your grade in any way. The survey will remain confidential and will take only 2-5 minutes to complete. If you have any questions about this survey or the study, please contact Dr. A. Subramanian, Clerkship Director ([email protected]) at Baylor College of Medicine. If you have additional questions about your rights as a research subject, contact the Institutional Review Board for Human Subject Research for Baylor College of Medicine & Affiliated Hospitals at (713) 798-6970. Thank you for your time. PLEASE SEE SURVEY ON BACK SURVEY Age _____ Gender (circle) M F 1. How would you evaluate the modules as a learning tool? □ Excellent □ Good □ Fair □ Poor 2. Did you feel motivated to utilize this resource? □ Very much □ Quite a bit □ Some □ Very little 3. How much time did you spend total completing the learning modules? □ > 6 hours □ 4-6 hours □ 2-4 hours □ 1-2 hours □ < 1 hour □ None 4. How often did you spend completing the learning modules? □ > 3 times/week □ 2-3 times/week □ Once a week □ None 5. To what extent has this activity contributed to your knowledge of surgical diseases and management? □ Very much □ Quite a bit □ Some □ Very little


6. Do you feel these learning modules helped prepare you for the NBME surgery exam? □ Very much □ Quite a bit □ Some □ Very little 7. Would you recommend similar learning modules to other clerkship directors? □ Very much □ Quite a bit □ Some □ Very little 8. Comments on game interactions, ideas for improvement, and recommended readings in Sabiston:

Documents

The Introduction of Digital Game-Based Learning in a Surgical Clerkship: A Pilot Study