Effectiveness and Feasibility of Utilizing E4D Technology ... · Abstract: Recent efforts have been directed towards utilizing CAD/CAM technology in the education of future dentists

1416 Journal of Dental Education ■ Volume 78, Number 10

Use of Technology in Dental Education

Effectiveness and Feasibility of Utilizing E4D Technology as a Teaching Tool in a Preclinical Dental Education EnvironmentRichard S. Callan, D.M.D., Ed.S.; Christie L. Palladino, M.D., M.Sc.; Alan R. Furness, D.M.D.; Emily L. Bundy, D.M.D.; Brittany L. Ange, M.S.Abstract: Recent efforts have been directed towards utilizing CAD/CAM technology in the education of future dentists. The purpose of this pilot study was to investigate the feasibility of implementing CAD/CAM technology in instruction on preparing a tooth for restoration. Students at one dental school were assigned access to CAD/CAM technology vs. traditional preparation methods in a randomized, crossover design. In a convenience sample of a second-year class, seventy-six of the seventy-nine students volunteered to participate, for a response rate of 96 percent. Two analyses were performed on this pilot data: a primary effectiveness analysis comparing students’ competency exam scores by intervention group (intention-to-treat analysis) and a sec-ondary efficacy analysis comparing competency exam scores among students who reported using CAD/CAM versus those who did not. The effectiveness analysis showed no difference in outcomes by intervention group assignment. While student survey results indicated interest in utilizing the technology, the actual utilization rate was much less than one might anticipate, yield-ing a sample size that limited statistical power. The secondary analysis demonstrated higher mean competency exam scores for students reporting use of CAD/CAM compared to those who did not use the technology, but these results did not reach statistical significance (p=0.075). Prior research has investigated the efficacy of CAD/CAM in a controlled educational trial, but this study adds to the literature by investigating student use of CAD/CAM in a real-world, self-study fashion. Further studies should inves-tigate ways in which to increase student utilization of CAD/CAM and whether or not increased utilization, with a larger sample size, would yield significant outcomes.

Dr. Callan is Associate Professor and Chair, Department of General Dentistry, Georgia Regents University College of Dental Medicine; Dr. Palladino is Assistant Professor, Educational Innovation Institute, Georgia Regents University Medical College of Georgia; Dr. Furness is Instructor, Department of Oral Rehabilitation, Georgia Regents University College of Dental Medicine; Dr. Bundy is Instructor, Department of Oral Rehabilitation, Georgia Regents University College of Dental Medicine; and Ms. Ange is Research Statistician, Department of Academic Affairs, Georgia Regents University Medical College of Georgia. Direct correspondence and requests for reprints to Dr. Richard S. Callan, College of Dental Medicine, Georgia Regents University, Room GC 3080, 1430 John Wesley Gilbert Drive, Augusta, GA 30912-1290; 706-721-3881; [email protected].

Keywords: dental education, preclinical education, educational technology, CAD/CAM, crown preparation

Submitted for publication 3/20/14; accepted 5/31/14

Dental education continues to experience increasing resource demands,1,2 but these challenges are not unique to dentistry. For

example, unintended consequences of the enormous growth in the biomedical research and clinical care activities of medical school faculty members have resulted in a decline in faculty time and rewards for the educational mission of medical schools.3 Nursing educators also lament a shortage of faculty, insufficient clinical education sites, and budget cuts as barriers to future growth.4 Virtually every higher education institution is searching for more effective and efficient ways to educate students, including digital technologies.

Many dental schools have reported the results of utilizing various technologies in the education of

their students. The Virtual Reality Dental Simulator (Dent Sim, DenX Ltd., Philadelphia, PA, USA) was studied by Gottlieb et al.,5 Buchanan,6 and others to determine its effectiveness in relation to time, ef-ficiency, and faculty perceptions of dental student performance. The Haptic Virtual Reality Simulator (IDEA Dental, Las Vegas, NV, USA) was studied by Ben Gal et al.7 as a means of training dental manual dexterity. Callan et al.8 and Renne et al.9 evaluated the E4D Compare (Richardson, TX, USA) as a means of utilizing computer-assisted design/computer-assisted manufacture (CAD/CAM) technology for student assessment. More recently, Gadbury-Amyot et al.10 reported on the use of tablet technology (iPad, Apple Inc., Cupertino, CA, USA) to enhance preclinical lab-oratory learning with “on-time” instructional videos.

October 2014 ■ Journal of Dental Education 1417

may vary depending on the experience and skill level of the operator,12 the timing and frequency of the feedback,13 and the presence or not of instructor support.14,15

In an earlier article,8 we described effective-ness as the ability of a technology, technique, or material to produce a desired effect. We went on

Using CAD/CAM technology, a digital model of an ideal preparation can be superimposed over a digital model of a student’s attempted preparation (Figure 1, Figure 2), thus giving the student the op-portunity to visually compare the difference between the two. Although many students prefer self-directed learning,11 the actual benefit of virtual feedback

Figure 1. Pre-prepped tooth, ideal preparation, and student’s attempt at preparation

Figure 2. Overlay: ideal preparation over student’s preparation


the final training session to the start of practicing for the first competency examination. This would, by design, indicate an additional two-week washout period for Group B.

The study was conducted during the students’ introductory fixed prosthodontics course, tradition-ally taken during the fall term of their second year. During this course, three competency examinations (tooth #30 Full Gold Crown preparation) were ad-ministered to the class over a six-week period, with approximately two weeks’ practice time allotted between each of the exams. The grading of the com-petency exams was performed by the faculty of the fixed prosthodontics course, as per our normal pro-tocol, with individual faculty members responsible for evaluating specific aspects of each preparation. These faculty members undergo calibration every year prior to the beginning of the course. The same faculty members graded all three competency exams, with the identical individuals grading their respective aspect of the preparations.

Utilizing a crossover study design (Figure 3), Group A was permitted to use the technology for a two-week period leading up to the first competency exam, while Group B was not. For the second compe-tency exam, the roles were reversed: Group B being permitted to use the technology, while Group A was not. For the third competency exam, both Group A and Group B were permitted to use the technology. Use of the technology was voluntary, and the overall time of utilization was recorded for each individual between each competency exam. Each student was provided a time log and was responsible for docu-menting his or her own time of utilization. The E4D Lab Works were placed in our Simulation Labora-tory, and faculty involved in the study supervised the utilization. The faculty members were instructed not to assist the students in use of the technology or interpretation of what they were looking at, but to address any technical problems that might arise.

Post-competency exam surveys, identical to the post-training surveys, were given to the group utilizing the technology for the two weeks prior to their respective practical exam: Group A for the first competency exam, and Group B for the second competency exam. In the two weeks prior to the third competency exam, Group A and Group B were permitted free access to the technology. Although “on call” for any technical difficulties, there was no faculty supervision during this time period. The students were again instructed to record their time spent utilizing the technology.

to define feasibility as the barriers that prohibit the utilization of a particular technology, technique, or material. The aim of this pilot study was to assess the effectiveness of a portion of our current efforts regarding utilization of CAD/CAM technology in the preclinical education of our students. In addition, we offer evidence to suggest parameters by which one can increase the probability of success in utilization of this technology.

MethodsThis study received Institutional Review Board

approval at Georgia Regents University. The study utilized a convenience sample of a sophomore class at Georgia Regents University College of Dental Medicine. The students were divided into two groups, Group A and Group B. Using a stratified randomized approach, the two groups were selected according to their mean competency examination scores from previous operative technique courses, resulting in two groups with similar levels of competence. The CAD/CAM system used in this study was the E4D Lab Works from D4D Technologies (Richardson, TX, USA). The E4D system enables one to capture digital images of an ideal tooth preparation and a student’s attempt at the ideal tooth preparation. The images are subsequently aligned to effectively level the plane of occlusion; then, the aligned images are overlaid, with the student’s preparation over the ideal, to reveal visual differences between the two preparations.

None of the students involved in this study had any previous experience with the E4D system. Prior to the beginning of the study, every student received a minimum of two hours of training on use of the E4D technology. This hands-on training consisted of students’ acquiring a digital image of their own dentiform (Kilgore International, Inc., Coldwater, MI, USA) tooth preparation, followed by the superimposing of a pre-downloaded image of an ideal preparation over their preparation. Following this training, a number of students (n=13) voluntarily completed a post-training survey designed to quan-tify their perception as to the ease of learning and implementing the CAD/CAM process. This survey also included four open-ended items that provided opportunity for participants to indicate what aspects of the technology they most liked and least liked and why they would or would not choose to use it in preparation for learning basic dental clinical skills. There was a period of approximately one month from


vestigated the effectiveness of CAD/CAM, using a repeated measures analysis of variance (ANOVA) to determine if the change in competency exam scores was dependent on intervention group (intention-to-treat analysis). This analysis investigated outcomes by group assignment, regardless of whether students actually used the technology or not. In this manner, it was a test of the real-world effectiveness of offering CAD/CAM as a self-study tool for competency exam preparation after instruction in CAD/CAM technol-ogy. We also performed a secondary efficacy analysis using a repeated measures ANOVA to determine if the change in competency exam scores was depen-dent on whether or not one actually used the E4D tool, regardless of intervention group. Descriptive statistics were used to analyze responses to the post-training, post-competency exam, and final surveys. The primary investigator, using a thematic analysis,

Immediately following the third competency exam, all participants were asked to complete the final survey. This survey contained four closed-ended items asking students to quantify their perception of the benefits of utilizing this technology with regards to designated areas of the particular preparation (the occlusal reduction, the proximal reduction, the facial-lingual reduction, and the margins & draw). The survey also included two questions, asking 1) would you use this technology in your presentation for the next practical exam if it were made available to you? and 2) what suggestions would you have to help increase the value of this technology to your preclinical education?

To determine the effectiveness of the E4D tool, quantitative and qualitative analyses were conducted. The quantitative analysis investigated two aspects of CAD/CAM use. First, our primary analysis in-

Figure 3. Crossover study design


that the technology was most beneficial when applied to the facial-lingual reduction and least beneficial when applied to the margin & draw and proximal surfaces (Table 4). The actual utilization rates are shown in Table 5.

Regarding the open-ended items, students com-pleting the post-training survey commented on the technology’s user-friendliness and ease of flow, while at the same time lamenting the difficulty in obtaining a digital scan of their preparations. Although a flow-chart indicating the mode of operation was placed beside each station, students also commented on the need for written instructions. The groups overwhelm-ingly (with one abstention) indicated they would use the technology in preparation for subsequent compe-tency exams if given the opportunity, citing the ability to visualize the differences between the preparations and to self-assess as their primary reasons. The one abstention was concerned about the practicality of the utilization for this purpose.

Students completing the post-competency exam survey also commented on the ability to accu-rately visualize the differences in their preparations when compared to the ideal. They reported that they liked the convenience of not having to “hunt down” a faculty member for their critique and the elimination of the subjectivity of the assessment. The students reported least liking the process of navigat-ing through the various steps of the technology and trying to obtain an accurate digital impression of their preparations. Some students stated it was “hard to learn” and “time-consuming.” Again, when asked if they would use this technology in preparation for their next practical exam, there was only one dis-senting response. The majority of positive responses indicated an appreciation for the opportunity to make necessary adjustments to their preparations based on the ability to clearly see where they were deficient. The negative responses were in relation to the time invested in utilizing the technology. Some students commented that the time would be better spent in actual practice doing the preparations rather than evaluating each practice preparation in this manner.

In responses to the final survey, the students overwhelmingly indicated that they would use this technology if it were made available to them (twenty-two yes, seven no, four maybe). Suggestions for increasing the value of the technology included the following: “more training,” “time-consuming,” “written instructions,” “help onsite,” “had technical problems,” “need more time,” “make it mandatory,” and “use it more.”

qualitatively analyzed open-ended comments and, in an iterative fashion, developed codes, applied these codes to the data, compared and contrasted codes, and repeated this process until final thematic patterns were identified.

ResultsSeventy-six of the seventy-nine students in

the second-year class volunteered to participate, for a response rate of 96 percent, and the students were divided into two groups. Group A consisted of seventeen females and twenty-one males with an age range of twenty-two to forty-nine years and an average age of twenty-six years. Group B consisted of seventeen females and twenty males with an age range of twenty-two to forty-two years. The average age of Group B was also twenty-six years. The mean score in the previous preclinical course for Group A was 84±2.56 and for Group B was 84±2.38.

We compared the competency exam scores from each practical examination (1, 2, and 3) be-tween the groups (Group A and Group B) (Table 1). The results of a repeated measures ANOVA showed that the mean scores did change across time, F(2, 73)=16.75, p<0.001, and that the change in mean score was not dependent on intervention group, F(2, 73)=2.21, p=0.117.

In addition, we compared the competency exam scores for each practical examination (1, 2, and 3) be-tween the students who utilized the technology prior to the examination and those who did not, regardless of their group designation (Table 2). Only sixteen, seven, and four students reported use of the technol-ogy before competency exams 1, 2, and 3, respec-tively. The results of a repeated measures ANOVA found that the mean competency exam scores did change across time, F(2, 73)=10.48, p<0.001. The mean competency exam score was higher for those reporting use of CAD/CAM, although this was only significant at Time 1, t(74)=2.07, p=0.042. Overall, the change in mean competency exam score was not dependent on whether or not the individual used the E4D technology, F(2, 73)=2.68, p=0.075.

Table 3 shows the quantitative results of the first two surveys given to the students (post-training survey and post-competency exam survey). Based on a 1 to 5 scale, with 1=very easy and 5=very difficult, the students indicated the easiest aspect of utilizing this technology was the orientation of the models, while the most difficult was the overall navigation of the software. The results of the final survey indicated


Table 1. Comparison of practical/competency exam grades between students in Group A and Group B

Group A, N=38 Group B, N=38 Time Mean (SD) Mean (SD) p-value (t-test)

Practical 1 80.77 (5.89) 81.25 (6.71) 0.74Practical 2 82.83 (4.96) 80.88 (6.57) 0.15Practical 3 84.28 (3.97) 84.76 (4.21) 0.62

Table 2. Comparison of practical/competency exam grades between students who utilized E4D technology and those who did not

Signed into E4D Did Not Sign into E4D Time Mean (SD) Mean (SD) p-value (t-test)

Practical 1 83.30 (4.32) N=16 80.26 (6.52) N=60 0.04Practical 2 85.04 (3.04) N=7 81.53 (5.99) N=69 0.13 (power=73.5%)Practical 3 87.31 (2.86) N=4 84.36 (4.09) N=72 0.16 (power=49.8%)

Table 3. Students’ self-reported difficulty in four categories of utilizing E4D technology

Self-Reported Difficulty Mean (SD)

Post-Training Survey Post-Competency Exam Survey Aspect of Technology (n=13) (n=41)

Scanning 2.9 (1.1) 3.2 (0.8)Orientation 2.5 (0.8) 3.1 (0.9)Alignment 2.8 (0.7) 3.3 (0.7)Navigation 3.5 (0.8) 3.4 (0.9)

Note: Difficulty was rated on a scale from 1=very easy to 5=very difficult. The Navigation category was missing two responses.

Table 4. Students’ self-reported helpfulness of E4D technology in four aspects of crown preparation (n=13)

Aspect of Crown Preparation Self-Reported Helpfulness Mean (SD)

Occlusal 3.1 (1.0)Proximal 2.9 (1.0)Facial-lingual 3.4 (1.0)Margins 2.9 (1.2)

Note: Helpfulness was rated on a scale from 1=not helpful at all to 5=extremely helpful.

Table 5. Average minutes used per student and overall percent utilization of E4D for three practical/competency exams

Practical 1 Practical 2 Practical 3

Average use (min) per individual 94 80 95Percent utilization 44.7% 18.5% 5.3%


perform the necessary steps to make the technology truly beneficial. Such concerns may suggest why the utilization rate was so low. Alternatively, students’ survey responses may have been artificially inflated due to a social desirability bias.

Even so, when given the opportunity to utilize the technology in preparation for the competency exam, surprisingly few students participated. The actual utilization rates (Table 5) were much less than one might anticipate and much lower than the percentage of students indicating interest on the sur-veys. We should emphasize that participation in this study was voluntary as was the amount of time each student spent using the technology when it was made available. Anecdotally, faculty members working in the simulation lab noted that students were more apt to request feedback from the specific professor who would be grading the competency exam than to visualize the difference themselves utilizing the E4D technology. This may suggest students were more concerned about the grade they would receive on the competency exam than actually improving their ability to prepare the tooth more accurately. These findings were consistent with what others have reported when assessing students and may provide insights into ways in which to increase student mo-tivation to use CAD/CAM in vivo.9,16-18

A recent study reported significant improve-ment in students’ performance due to the utilization of CAD/CAM technology in the grading of crown preparations.9 Since the technology was apparently used as part of a grading rubric, student participation in that study was mandated. Other studies reported less dramatic results using other computer-assisted teaching modalities, most of which did not mandate participation, though each commented positively on the potential of those technologies.9,10,12,19,20

One notable benefit of utilizing the E4D technology, which most of the other technology be-ing used for this purpose does not offer, is that the skills gained from this experience will translate di-rectly into clinical productivity. As an added benefit, Browning et al. reported a significant reduction in laboratory costs through the use of CAD/CAM sys-tems in the student clinics.21 The development of E4D Compare was a direct extension of the E4D Design Center as a technology created almost exclusively for clinical application. As the technology continues to advance, accurate assessment of clinical performance will most certainly be the next area of investigation.

Limitations to our study include the small sample size. In retrospect, additional training would

DiscussionCAD/CAM technology makes it possible for

students to see the difference between their tooth preparation and an ideal tooth preparation, magni-fied and in 3D. One cannot help but think this abil-ity would be beneficial to a certain percentage of dental students if not all. In their study, Cho et al. demonstrated that better performing students tended to underevaluate their own performance, whereas underachievers were more likely to overestimate the quality of their work.12 The ability to see the relative differences between the student’s preparation and the ideal has the potential to bring both the overestimator and underevaluator more close to reality.

This pilot study does not support the effective-ness of CAD/CAM technology as implemented as a self-study aid in our curriculum. Even though the sample size for the effectiveness analysis was small (thirty-eight in each group), the mean competency exam scores were very similar between the interven-tion groups, with the mean being slightly higher in the non-CAD/CAM group at both Time 1 and Time 2. Therefore, even if we had a larger sample size, it would be unlikely that the study would demonstrate the effectiveness of CAD/CAM as implemented. Our further analyses suggest that the main reason for the lack of effect of CAD/CAM was the extremely low utilization rate of the technology for self-study. Al-though it was available, the students for the most part did not use it. The survey responses from the students, however, clearly suggested its potential and indicated an interest by the students in utilizing this technol-ogy as an aid to learning and improving their tooth preparation skills. Student comments on the surveys indicated both positive and negative impressions of using the technology. A consistent concern expressed was difficulty in obtaining a digital impression. The results of the efficacy analysis suggested the potential of CAD/CAM since students who reported using the technology had higher mean competency exam scores than those who did not, although the difference did not reach statistical significance overall. The dif-ference was significant at Time 1, but post hoc power analyses (Table 2) indicated our sample size was not sufficient to show differences at Times 2 or 3.

Considering that this study was the students’ first exposure to this technology and that only a lim-ited time for instruction was given, it is not surprising that the majority of their concerns revolved around technical difficulties and the time it took them to


and capacity restraints. At: www.aacn.nche.edu/news/articles/2009/09enrolldata. Accessed: February 8, 2014.

5. Gottlieb R, Lanning SK, Gunsolley JC, Buchanan JA. Faculty impressions of dental students’ performance with and without virtual reality simulation. J Dent Educ 2011;75(11):1443-51.

6. Buchanan JA. Experience with virtual reality-based tech-nology in teaching restorative dental procedures. J Dent Educ 2004;68(12):1258-65.

7. Ben Gal G, Weiss EI, Gafni N, Ziv A. Preliminary assess-ment of faculty and student perception of a haptic virtual reality simulator for training dental manual dexterity. J Dent Educ 2011;75(4):496-504.

8. Callan RS, Blalock JS, Cooper JR, Coleman JF. Reliability of CAD/CAM technology in assessing crown preparations in a preclinical dental school environment. J Dent Educ 2013;78(1):40-50.

9. Renne WG, McGill ST, Mennito AS, et al. E4D compare software: an alternative to faculty grading in dental educa-tion. J Dent Educ 2013;77(2):168-75.

10. Gadbury-Amyot CC, Purk JH, Williams BJ, Van Ness CJ. Using tablet technology and instructional videos to enhance preclinical dental laboratory learning. J Dent Educ 2014;78(2):250-8.

11. Murphy RJ, Gray SA, Straja SR, Bogert MC. Student learning preferences and teaching implications. J Dent Educ 2004;68(8):859-66.

12. Cho GC, Chee WWL, Tan DT. Dental students’ ability to evaluate themselves in fixed prosthodontics. J Dent Educ 2010;74(11):1237-42.

13. Hubbard IJ, Neilson C, Carey LM. Task-specific training: evidence for and translation to clinical practice. Occup Ther Int 2009;16(3-4):175-89.

14. Wierinck E, Puttemans V, Swinnen S, van Steenberghe D. Effect of augmented visual feedback from a virtual reality simulation system on manual dexterity training. Eur J Dent Educ 2005;9:10-6.

15. Weeks DL, Anderson LP. The interaction of observational learning with overt practice: effects on motor skill learn-ing. Acta Psychol 2000;104(2):259-71.

16. Gunderman RB. Ingredients of successful medical school student teaching. Am J Roentgenol 2004;182:1115-7.

17. Gunderman RB, Kanter SL. Perspective: how to fix the premedical curriculum revisited. Acad Med 2008;83: 1158-61.

18. Voyer S, Pratt D. Feedback: much more than a tool. Med Educ 2011;45:862-5.

19. Jasinevicius TR, Landers M, Nelson S, Urbankova A. An evaluation of two dental simulation systems: virtual reality versus contemporary non-computer-assisted. J Dent Educ 2004;68(11):1151-62.

20. Maggio MP, Buchanan JA, Berthold P, Gottlieb R. Vir-tual-reality-based technology (VRBT) training positively enhances performance on practical examinations. J Dent Educ 2005;69(1):161(Abstract).

21. Browning WD, Reifeis P, Willis L. Kirkup ML. Includ-ing CAD/CAM dentistry in a dental school curriculum. J Indiana Dent Assoc 2013;92(4):40-5.

probably have increased the number of students who utilized the technology when it was made available to them. Additional instruction in image acquisition is highly recommended as well as a tutorial on im-age interpretation of the overlays. Another limitation is that there is always some skepticism concerning the accuracy of utilization time when recorded by students for themselves. It is difficult to mandate participation when the technology has yet to be validated for a particular purpose.

ConclusionThis study highlights important considerations

for in vivo use of CAD/CAM technology in a dental curriculum. To be able to fully determine the effec-tiveness of CAD/CAM as an aid to teaching tooth preparation skills, we plan to repeat this study with a larger sample size and to attempt to increase student motivation to use the technology. To increase utiliza-tion, it is imperative that students be introduced to this technology as early in the curriculum as possible. The incremental introduction to the various aspects of the process, juxtaposed to its usefulness within various courses, will help familiarize students with the technology and make them more comfortable with its application. We also recommend that the results derived from the comparison of the students’ attempt of an ideal tooth preparation to the actual preparation be somehow quantified and utilized as part of the grading rubric. Additional study into the accuracy, repeatability, and validity of this aspect of this technology is paramount to its application for grading purposes.

DisclosureThe authors did not receive any benefits from

the sales of any equipment, technology, or products mentioned in this study, nor did they receive any free products in order to conduct it.

REFERENCES 1. Pyle M, Andrieu SC, Chadwick DG, et al. The case for

change in dental education. J Dent Educ 2006;70(9):921-4.2. Bailit HL, Beazoglou TJ, Formicola AJ, et al. U.S.

state-supported dental schools: financial projections and implications. J Dent Educ 2006;70(3):246-57.

3. Thibault GE, Neill JM, Lowenstein DH. The acad-emy at Harvard Medical School: nurturing teaching and stimulating innovation. Acad Med 2003;78:673-81.

4. Student enrollment expands at U.S. nursing colleges and universities for the 9th year despite financial challenges

Documents

Effectiveness and Feasibility of Utilizing E4D Technology ... · Abstract: Recent efforts have been directed towards utilizing CAD/CAM technology in the education of future dentists