Upload
frank-quinn
View
214
Download
0
Embed Size (px)
Citation preview
A pilot study comparing the effectiveness of conventional
training and virtual reality simulation in the skills
acquisition of junior dental students
Frank Quinn1, Paul Keogh1, Ailbhe McDonald2 and David Hussey3
1Dublin Dental School and Hospital, Lincoln Place, Dublin 2, Republic of Ireland; 2Eastman Dental Institute for Oral Healthcare Sciences, 256 Grays InnRoad, London WC1X 8LD, UK; 3Royal Victoria Dental School and Hospital, Royal Victoria Hospital Group Trust, Grosvenor Road, Belfast BT12 6BP, UK
The use of virtual reality (VR) in the training of operative dentistry isa recent innovation and little research has been published on itsefficacy compared to conventional training methods. To evaluatepossible benefits, junior undergraduate dental students wererandomly assigned to one of three groups: group 1 as taughtby conventional means only; group 2 as trained by conventionalmeans combined with VR repetition and reinforcement (withaccess to a human instructor for operative advice); and group3 as trained by conventional means combined with VR repetitionand reinforcement, but without instructor evaluation/advice,which was only supplied via the VR-associated software. Atthe end of the research period, all groups executed two class1 preparations that were evaluated blindly by ‘expert’ trainers,under traditional criteria (outline, retention, smoothness, depth,wall angulation and cavity margin index). Analyses of resultingscores indicated a lack of significant differences between thethree groups except for scores for the category of ‘outlineform’, for group 2, which produced significantly lower (i.e. better)scores than the conventionally trained group. A statisticalcomparison between scores from two ‘expert’ examiners
indicated lack of agreement, despite identical written and visualcriteria being used for evaluation by both. Both examiners,however, generally showed similar trends in evaluation. An anon-ymous questionnaire suggested that students recognized thebenefits of VR training (e.g. ready access to assessment, erroridentification and how they can be corrected), but the majorityfelt that it would not replace conventional training methods(95%), although participants recognized the potential for devel-opment of VR systems in dentistry. The most common reasonscited for the preference of conventional training were excessivecritical feedback (55%), lack of personal contact (50%) andtechnical hardware difficulties (20%) associated with VR-basedtraining.
Key words: virtual reality; skill acquisition; self-directed and self-paced learning; real-time feedback; objective evaluation.
�Blackwell Munksgaard, 2003Accepted for publication 6 March 2002
FOR many decades, mechanical simulation devices
have been used in training exercises, such as in
aerial and marine aviation. They have also been
employed for the rehearsal of reaction protocols in
hazardous or potentially hazardous situations, thereby
eliminating the risk of injury, loss of life or damage to
expensive equipment. The earliest widespread use of
this technology was in competitive or ‘hazardous’
arcade games, e.g. aerial combat and car racing.
Associated with the rapid increase in computing
speeds and miniaturization of components (1), the
application of virtual reality (VR) has increased dra-
matically, particularly due to the ubiquitous presence of
the home computer. In the healthcare area, VR has had
limited integration into the acquisition of surgical skills
(e.g. suturing and ‘keyhole’ surgery) and planning of
individual surgical procedures, particularly implant,
craniofacial and neurosurgical procedures (2, 3). With
respect to skills acquisition, VR-based training has been
utilized for the repeated use of a standardized simu-
lated patient (4).
Dental students are expected to be competent at a
large array of procedures on certification; indeed, in
many countries, no further training is compulsorily
required. This is very different from medical graduates
who are required to undergo extensive postgraduation
vocational training. The large number of undergradu-
ate dental procedures, which must be mastered, have
traditionally been tested by practical examinations. The
subjective nature of these tests, and their associated
stress, has led to many dental faculties replacing them
with ‘competence tests’ (5, 6), which are organized,
within certain limits, at the student’s convenience
and assessed under objective or semiobjective criteria
Eur J Dent Educ 2003; 7: 13–19Printed in Denmark. All rights reserved
13
(7, 8). The use of standardized VR scenarios may be
beneficial in the preparation and reinforcement of
required procedures (9). There may be additional appli-
cations for postgraduate students, to revise operative
and fixed prosthodontic procedures, and continuing
dental education. Once assessment criteria have been
universally accepted within the faculty, VR prepara-
tions and evaluations may be used to standardize
teachers and improve conventional teaching.
Computing speeds and costs have remained a con-
straining factor for the widespread use of specialized
systems to relatively small consumer groups, such as
dental undergraduates. In the last 5 years, however, a
dental ‘patient’ simulator has been developed and
marketed for the training of undergraduate dental
students. This simulator consists of a conventional
torso and movable head, with integral maxillary and
mandibular jaws, which carry conventional plastic
teeth. There is an operatory-type light, suction and a
bracket table, carrying an ultra-high speed handpiece
and air/water syringe. With these components, train-
ing in operative dentistry can be undertaken in a
manner identical to conventional methods.
The novel components are light emitting diodes on
both the head and the handpiece and a tracking camera
that monitors head and handpiece movements (Fig. 1).
There are two personal computers: one to collate track-
ing information and the second to integrate all data and
execute the associated VR software. Finally, there is a
video screen which presents real-time virtual images to
the operator (Fig. 2).
At the time of this project, the system’s software
allowed several operative and fixed prosthodontic pro-
cedures. It is postulated that the use of these options
would be superior to conventional training, with plastic
teeth, as the individual layers of the natural tooth are
represented in the virtual tooth, including dentinal
carious lesions. Preparations may be viewed from
many angles and at varying magnification; these fea-
tures may improve student understanding and self-
assessment (10, 11). The image of the virtual mouth and
tooth provide real-time information feedback of the
actual preparation cut on the tooth analogue. The
operator-controlled magnification may improve visua-
lization of cavity detail (12).
The software has an integrated database of theoretical
information, ranging from conventional principles of
cavity preparation to the pathology of the carious
lesion. This database is continuously accessible during
training sessions. In addition, there is a glossary of
dental terms, with associated multimedia explanations.
A final, and perhaps most important, benefit of this
simulation unit is that the software will analyze the
preparation, on request, and provide detailed written
and two-dimensional graphic evaluation of the pre-
paration (Fig. 3). Alterations may then be made and
the preparation re-evaluated. This encourages self-
directed and ‘deep’ learning and self-paced skills
Fig. 1. Operator, torso with handpiece and display of VR mouth.
Fig. 2. The real-time display with magnified virtual tooth pre-paration.
Fig. 3. VR-based analysis of cavity preparation, compared to preset‘ideal parameters’.
Quinn et al.
14
acquisition (11, 13, 14). It is postulated that these
benefits will aid the different styles and rates of learning
(15). This self-direction and self-pacing has been advo-
cated by many authorities due to the overloaded nature
of modern dental curricula and the ‘bulimic’ nature of
conventional teaching methods (16). This term refers to
the cramming of large amounts of theoretical informa-
tion, in excessive detail and regurgitating it for an all
or nothing assessment. All too often, the information
is then discarded and never reviewed again. This
approach favours ‘shallow’ or poorly retained learning.
The system administrator/course coordinator can
control access to specific, appropriate ‘lessons’. This
is managed via a programmable user database and
unique passwords. Operator access to ‘lessons’ may
be limited to specific groups or during a particular time
frame. The duration allowed per individual procedure
may also be limited, if so desired. The procedures, once
selected, present clinical details and radiograph(s) for
perusal by the student; they may then proceed to the
virtual mouth and operative procedures completed.
This integration of clinical information and procedures
is believed to lead to improved knowledge acquisition
and retention.
Individual sessions for students may be stored for
later review, either by the student or by an instructor.
The review is in the form of a real-time video playback,
with fast forward and rewind functions. The recorded
sessions may act as a positive feedback to the student,
illustrating improved technique with individual prac-
tice. If desired, recorded sessions may be used as
evidence of ‘competence’ in clinical practice.
The Dublin Dental School received four Dental
Simulation Units and, finding only anecdotal evidence
of the benefits of VR in operative training, a pilot
research project was undertaken to quantify the bene-
fits, if any.
Methods and materials
The second year dental undergraduate class, consisting
of 32 students, was randomly assigned (by lottery) to
one of three groups. All students were given the same
introductory lecture and demonstration on the design
and instrumentation of conventional class 1 cavity
preparation. All students received ‘conventional’ opera-
tive practice in the undergraduate laboratory. All were
supplied with a millimetre graduated periodontal
probe, a mouth mirror and a sharp probe. All prepara-
tions were completed with the same ISO standard pear-
shaped tungsten carbide bur (245), used at ultra-high
speeds and with continual water spray.
The group 1 students worked exclusively with con-
ventional phantom heads and teeth. These students had
continual access to an instructor, to provide feedback
and evaluation. Students performed repeated class 1
preparations for approximately 21 h.
Group 2 students also worked on conventional phan-
tom heads but, in addition, received 1 h of instruction
on the use of the dental simulation unit, and 4 h of
preparing multiple class 1 preparations on the lower
left first molar on the VR units. The students had real-
time feedback from the VR unit (Fig. 2) and had con-
tinual access to the same instructor (PK), but only for
technical instruction, real-time feedback and evalua-
tion. The students did not have access to preparation
evaluation options in the software. In addition to the
VR simulators’ exposure, the students had approxi-
mately 16 h of conventional phantom head teaching.
Group 3 students had identical conventional training
time as group 2, identical instruction in the use of the
dental simulation units and 4 h for preparation of multi-
ple class 1 preparations of the lower left first molar.
These students had access to the same instructor (PK),
but only for technical questions: feedback and prepara-
tion evaluation were provided by the units’ software.
The students also had access to the information data-
base and glossary. In addition to exposure to the VR
simulators, the students had approximately 16 h of
conventional phantom head teaching.
When all students had completed the allocated
operative time, all groups executed two class 1 cavities.
The individual teeth were coded anonymously and
submitted to two independent scorers. These two
experienced restorative academic trainers were not
informed of the nature of the project, but were supplied
with the ‘ideal’ cavity dimensions and illustrations of
the desired cavity shape and dimensions on the lower
left first molar (17, 18).The scorers were requested to provide an ordinal
score (0–3 or 0–4) for the following aspects of the cavity
design: outline form, retention form, depth, smooth-
ness, cavosurface angulation (17) and ‘cavity margin
index’ (19). The scores were qualitative, with 0 indicat-
ing an ideal preparation for that parameter, and 4 (or 3)
representing the aberrant performance.
The criteria for cavity evaluations were based on ‘best
practice’ information (17, 18). These criteria, although
subjective, were intended to be as clear-cut as possible.
The criteria and scores are appended below:
Statistical analysesOn return of the scores, the code was broken and non-
parametric analyses (Kruskal–Wallis/Wilcoxon Rank
Sum) of the data were undertaken with respect to the
15
Virtual reality in the training of operative dentistry
three groups. In addition, the agreement between the
two scorers was examined utilizing Kappa analyses.
Jump-in Software was employed for statistical analyses
(JMP IN, SAS Institute Inc).
A structured anonymous questionnaire was given to
groups 2 and 3, after they had completed all conventional
and VR exercises. This was designed to be non-directive
and to elucidate the subjective comparative opinions of
students who have been exposed to both conventional
operative training and the VR training units.
Results
The results data were not normally distributed. Non-
parametric analyses failed to show statistically signi-
ficant differences between the three groups (Tables 1
and 2), except for the criterion ‘outline form’ (p¼ 0.037).
The difference appears to be between groups 1 and 2
(Wilcoxon Rank Sum test).
Comparison between two ‘expert’ examiners indicated
poor agreement. The best agreement was for ‘cavity
smoothness’, which gave a Kappa statistic of 0.156 and
the poorest was for ‘cavity margin index’, with the
Kappa statistic of 0.012. Nonetheless, both scorers failed
to show statistically significant difference for the eva-
luation criteria, except for ‘outline form’. One examiner
reported a relatively greater difference between the
three groups for this criterion, while the other exam-
iner’s scores fell just short of statistical significance at
the p¼ 0.05 level. This disparity, combined with expo-
sure to conventional training by all groups, may have
masked differences between the three groups.
The anonymous questionnaire indicated that stu-
dents identified a relatively limited access to instruc-
tors, and an inconsistency in instructor evaluation, as
being unfavourable aspects of conventional training
(Table 3). Ninety-five percent of the individuals in
experimental groups 2 and 3 felt, however, that VR-
based training will not replace conventional training in
operative dentistry. The respondents selected technical
hardware difficulties, excessively critical feedback and
lack of personal interaction as the limitations of this VR-
based system (Tables 4 and 5). When asked to select
TABLE 1. Wilcoxon/Kruskal–Wallis test for differences betweencontrol (1) and experimental groups (2 and 3), for the factor outlineform
Group Count Scoresum
Scoremean
(Mean�Mean0)/SD0
1 24 942 39.2500 2.4642 20 532.5 26.6250 �1.8643 20 605.5 30.2750 �0.701
w2¼ 6.5718; d.f.¼ 2; p¼ 0.0374.
TABLE 2. Kruskal–Wallis test for differences between control (1)and experimental groups (2 and 3), for the factors retention,smoothness, cavity depth, cavosurface angulation and cavitymargin index scores
w2 d.f. Probability
Retention� 4.1832 2 0.1235Smoothness� 3.2622 2 0.1957Cavity depth� 4.1692 2 0.7909Cavosurface angulation� 5.1259 2 0.0771Cavity margin index score� 1.5397 2 0.4631
�One-way test, w2 approximation.
TABLE 3. Percentage frequencies of the two most commonquestionnaire responses to: ‘list everything you dislike by theconventional training’
Supervisors are too busy Supervisor inconsistency
85% 40%
TABLE 4. Percentage frequencies of three most common ques-tionnaire responses to: ‘list everything you dislike by the VR-basedtraining’
Too critical Too easy to blockLED sensors
Inconsistent easeof use
55% 20% 20%
TABLE 5. Percentage frequencies of three most common ques-tionnaire responses to: ‘list everything you like about conventionaltraining’
Personal toucheasier to learnabout errors
Interaction/discussion
Demonstration ofhow to preparecavity correctly
45% 50% 55%
TABLE 6. Percentage frequency of selected descriptors to thequestion: ‘select which words best describe VR-based trainingand conventional training?’
Descriptor VR-based training Conventional training
Easy 20 45Stimulating 40 60Uninvolving 35 15Satisfying 25 55Frustrating 85 25Helpful 50 95
TABLE 7. Percentage frequencies of the three most commonquestionnaire responses to: ‘list everything you like about theVR-based training
VR presentationis interesting
Feedbackaids erroridentification
Feedback shows whereerrors are, and how tocorrect them
60% 30% 30%
16
Quinn et al.
words to describe the participants experience with VR-
based and conventional training, the most common
descriptors were ‘frustrating’ for VR-based and ‘help-
ful’ for conventional training (Table 6). They identified,
however, interesting presentation, improved access to
feedback and consistency in evaluation as benefits of
VR-based training (Tables 7 and 8). When requested to
make direct comparisons of VR-based and conventional
training, 40% of respondents selected VR-based as
being ‘much better’ for ‘providing feedback’, whereas
70% selected for ‘increasing confidence in cavity pre-
paration’ as being ‘much better’ for conventional train-
ing (Table 9).
Discussion
The effective use of students’ time has become an
increasing necessity with the current rapid expansion
of dental curricular contents (16). The use of VR in
undergraduate training in operative dentistry, particu-
larly where the unit also provides objective formative
evaluation on demand, may prove advantageous either
in a more rapid rate of skills acquisition or in the
development of superior quality of preparation. These
suggested benefits would have attractions for both
dental educators and students.
Within the limitations of this protocol, however, the
study failed to demonstrate any consistent differences
between the three groups, in quality of class 1 cavity
preparation, when conventional operative assessment
criteria were used, except for the category ‘outline
form’. For this category, the group who had real-time
visual feedback during VR procedures and conven-
tional instructor advice performed significantly better
than the conventionally trained, but not compared to
those who had real-time VR feedback and software
‘objective’ evaluation. Separate analyses of the two
‘expert’ assessors indicated a strong statistical signifi-
cance for one, with the other’s scores falling just short of
significance at the p¼ 0.05 level.
The lack of other significant advantages may have
arisen due to the relatively small sample sizes, the rela-
tively large proportion of conventional training versus
VR time in groups 2 and 3, the relative simplicity of the
class 1 cavity preparation or the imprecision of the
implementation of the evaluation criteria. Alternatively,
the results may reflect an actual lack of systematic differ-
ence in the methods of skills acquisition. Although
larger sample sizes might have exhibited larger inter-
group differences, it is likely that such differences
would have been of limited clinical importance.
Participants in the two VR groups (2 and 3), had 4 h of
practice on the simulation units and 16 h devoted to
conventional training. The control group (1) had 21 h of
conventional training only. This occurred for logistical
and standardization reasons only. This relative dispro-
portion in time spent on conventional versus VR train-
ing may have resulted in a dilution of any difference in
the rate of operative skills acquisition but is unlikely to
have reduced any qualitative benefits gained from real-
time feedback and on-demand objective evaluation.
This problem is being addressed in a current study
where students will have access to conventional train-
ing only or to VR training only, both for identical time
duration. These students have no professional experi-
ence of operative dentistry. The study will be reported
in 2002.
It was interesting that the two VR groups showed a
tendency towards more accurate control of cavity wall
angulation but this difference failed to show statistical
significance (p¼ 0.0771). This may reflect improved
TABLE 8. Percentage frequencies of the descriptor ‘very good’ for the above headings in relation to conventional operative training and VR-based training
Immediate feedback Self-paced learning Providing more thoroughassessment
Increasing confidencein cavity preparation
Conventional 15% 40% 30% 65%VR 60% 25% 50% 20%
TABLE 9. Percentage frequency of selected descriptors on relative superiority of VR-based training or conventional training
VR-based traininga Conventional traininga
Much better? A bit better? Much better? A bit better?
Providing feedback 40 35 15 5Allows self-paced learning 5 35 20 40More thorough understanding 35 40 20 5Allowing independent work 10 55 10 20Increasing confidence in cavity preparation 5 5 70 15
aWhich was better? How much?
17
Virtual reality in the training of operative dentistry
critical skills associated with the use of real-time virtual
display during preparations and in-depth evaluation of
cavity design (12). This will be investigated further in a
future study.
Although there was poor agreement between the two
‘experts’ for individual scores, in general, the trends
were similar and failed to indicate significant differ-
ences between the three groups, except for outline form.
This lack of reproducibility in detailed critical clinical
appraisal has been reported by other workers (20, 21)
and has been cited as a frustrating aspect of training for
dental students, particularly inexperienced students.
Alternatively, the VR software presents extensive
detailed and reproducible analyses of cavity ‘errors’.
To the inexperienced student, without the benefit of
human guidance, this proved to be relatively dis-
couraging and apparently insurmountable. Thus, a
combination of approaches may prove to be more
advantageous.
The student questionnaire confirmed that the
majority of the VR students recognized the benefits
of this technology (Tables 3 and 7). One such notable
benefit, in the virtual preparation, is that of three-
dimensional visualization of the anatomical layers of
teeth, including the pathological distribution of the
virtual carious lesion. Machine calibration, excessively
critical assessment and technical problems, however,
were cited as dislikes for the VR groups (Table 4).Personal interaction and demonstration of how to
prepare the cavity correctly were cited as favourable
aspects of conventional training (Table 5).
As might be expected, the results of the questionnaire
indicated that the VR units were superior to conven-
tional training in providing ‘immediate feedback’ (60%
vs. 15% for ‘very good’ and in ‘providing more thor-
ough assessment of . . . cavity preparation’ (50% vs. 30%
for ‘very good’) (Table 8). Unexpected findings were,
however, that conventional training was ‘much better’
for ‘increasing confidence in cavity preparation’ in 70%
of the responses (Table 9) and that when asked to select
words to describe VR experience and conventional
training, ‘helpful’ was the selected descriptor for 95%
of the conventional and only 50% of the VR respon-
dents. Similarly 85% of respondents selected ‘frustrat-
ing’ for the VR experience whereas this was selected by
only 25% for the conventional training (Table 6). On
questioning, the participants felt the two aspects of
frustration were restricted operator position (so as
not to block the tracking camera’s light emitting diodes)
and that the VR screen did not appear to display
accurately what they perceived as the cavity outline.
This latter factor may be due to the limited resolution of
the real-time display.
A daily ‘fine tuning’ was necessary for each of the
simulation units. The duration of this preparatory pro-
cedure ranged between 5 min to over 1 h. Upgrades of
hardware and software during the project led to a
marked reduction in the time occupied in this tedious
exercise. The developers have now introduced an auto-
mated checking procedure, which has been actively
encouraged by all current system users. In addition,
access to information support (IS) services was needed
on a regular basis, particularly in the early stages of
integration. Open access to the VR units proved to be
impossible due to the necessity for technical and
procedural advice requested by the participants.
Nonetheless, the majority of students found software
and graphic interfaces relatively accessible and navig-
able.
Unfortunately, at present, modification of the clinical
‘scenarios’ is not possible locally, by the system
administrator/faculty, but must be customized by
the developers: this involves considerable time and
cost. This may limit the number and variations of
individual lessons available, e.g. shallow versus deep
cavities, and minimal versus conventional cavity
designs.
Summary
1. Within the constraints of the study, there was no
statistically significant difference between conven-
tional operative training alone or in combination
with VR simulations, in terms of cavity quality as
assessed by conventional criteria.
2. Students favour a combination of conventional ‘sub-
jective’ instructor, and VR simulation with objective
feedback.
3. Consistency in VR system calibration was a signifi-
cant problem early in the study: this deficiency needs
to be thoroughly addressed if students’ confidence is
to be acquired and maintained by this new and
promising technology.
4. A major benefit in the system would be the ability to
modify the clinical scenarios and requirements
locally.
5. Independent assessors showed similar scoring trends
but exhibited poor individual score agreement.
6. Simulation training via VR units shows great pro-
mise in undergraduate and postgraduate dental
training, but it is unlikely that it will replace human,
subjective interaction.
7. Further study is needed to evaluate the benefits of
VR training when used alone, particularly in relation
to speed of skills acquisition.
18
Quinn et al.
Acknowledgements
The authors would like to thank Ms. Sarah Keogh and
Alan Kelly (Senior Biostatistician).
References
1. Munnelly B, Holden P. A complete coursebook of theEuropean computer driver’s licence. London: PearsonEducation Books, 2000: 2–5.
2. Seipel S, Wagner IV, Koch S, Schneider W. Oral implanttreatment planning in a virtual reality environment.Comput Meth Programs Biomed 1998: 57: 95–103.
3. Wagner A, Rasse M, Millessi W, Ewers R. Virtual realityfor orthognathic surgery: the augmented reality environ-ment concept. J Oral Maxillofac Surg 1997: 55: 456–462.
4. Joly B, Grant J. Patient-based assessment-simulatedpatients. In: The good assessment guide. London: TheJoint Centre for Education in Medicine, 1997: 96–97.
5. Mossey P. Competency based teaching and assessmentin dentistry: the context. In: Mossey P, Stirrups D, eds.Clinical competencies in dentistry: exploring the issues.London: Medical and Dental Education Network, 1997: 8.
6. Dunkley P. Assessment of psychomotor skills. In:Mossey P, Stirrups D, eds. Clinical competencies indentistry: exploring the issues. London: Medical andDental Education Network, 1997: 19–22.
7. Yip HK, Smales RJ, Newsome PRH, Chu FCS, Chow TW.Competency-based education in a clinical course inconservative dentistry. Br Dent J 2001: 191: 517–522.
8. Joly B, Grant J. Reliability. In: The good assessmentguide. London: The Joint Centre for Education inMedicine, 1997: 34.
9. Joly B, Grant J. Constructive feedback. In: The goodassessment guide. London: The Joint Centre for Educa-tion in Medicine, 1997: 32–33.
10. Stirrups D. Assessment as a tool for supportive learning.In: Mossey P, Newton JP, Mason A, Stirrups A, eds.Clinical competencies in dentistry conference: assessingin competence. London: Medical and Dental EducationNetwork, 1998: 46–48.
11. Joly B, Grant J. Records of practice-portfolios. In: Thegood assessment guide. London: The Joint Centre forEducation in Medicine, 1997: 77–78.
12. Robinson PB, Lee JW. The use of real time videomagnification for the pre-clinical teaching of crownpreparations. Br Dent J 2001: 190: 506–510.
13. Norman GR, Schmidt HG. The psychological basis ofproblem-based learning: a review of the evidence. AcadMed 1992: 67: 557–565.
14. Ferrier BM. Problem-based learning does it make adifference? J Dent Educ 1990: 54: 550–551.
15. Honey P, Mumford A. Manual of learning styles.Berkshire, UK: Honey P, 1982: 59–73.
16. Preston JD. Editorial: bulimic education. Int J Prostho-dont 1995: 8: 301.
17. Summitt JB, Robbins JW, Schwartz RS. Fundamentals ofoperative dentistry. Chicago: Quintessence PublishingCo. Inc, 2001: 306–313.
18. Kidd EAM, Smith BGN, Pickard HW. Pickard’s manualof operative dentistry. Oxford: Oxford University Press,1996: 112–113.
19. Tronstad L, Leidel TI. Scanning electron microscopy ofcavity margins finished with chisels or rotating instru-ments at low speed. J Dent Res 1973: 53: 1167–1174.
20. Smales RJ, Creaven PJ. Evaluation of clinical methods forassessing the surface roughness of restorations. J ProsthetDent 1979: 42: 45–52.
21. Gray C. How to evaluate assessment methods. In:Mossey P, Newton JP, Mason A, Stirrups D, eds. Clinicalcompetencies in dentistry conference: assessing incompetence. London: Medical and Dental EducationNetwork, 1998: 22–23.
Address:
Frank QuinnDepartment of Restorative Dentistry and PeriodontologyDublin Dental SchoolLincoln PlaceDublin 2Republic of Ireland
Tel: þ353 1 6127312
Fax: þ353 1 6127297
e-mail: [email protected]
19
Virtual reality in the training of operative dentistry