7
A pilot study comparing the effectiveness of conventional training and virtual reality simulation in the skills acquisition of junior dental students Frank Quinn 1 , Paul Keogh 1 , Ailbhe McDonald 2 and David Hussey 3 1 Dublin Dental School and Hospital, Lincoln Place, Dublin 2, Republic of Ireland; 2 Eastman Dental Institute for Oral Healthcare Sciences, 256 Grays Inn Road, London WC1X 8LD, UK; 3 Royal Victoria Dental School and Hospital, Royal Victoria Hospital Group Trust, Grosvenor Road, Belfast BT12 6BP, UK The use of virtual reality (VR) in the training of operative dentistry is a recent innovation and little research has been published on its efficacy compared to conventional training methods. To evaluate possible benefits, junior undergraduate dental students were randomly assigned to one of three groups: group 1 as taught by conventional means only; group 2 as trained by conventional means combined with VR repetition and reinforcement (with access to a human instructor for operative advice); and group 3 as trained by conventional means combined with VR repetition and reinforcement, but without instructor evaluation/advice, which was only supplied via the VR-associated software. At the end of the research period, all groups executed two class 1 preparations that were evaluated blindly by ‘expert’ trainers, under traditional criteria (outline, retention, smoothness, depth, wall angulation and cavity margin index). Analyses of resulting scores indicated a lack of significant differences between the three groups except for scores for the category of ‘outline form’, for group 2, which produced significantly lower (i.e. better) scores than the conventionally trained group. A statistical comparison between scores from two ‘expert’ examiners indicated lack of agreement, despite identical written and visual criteria being used for evaluation by both. Both examiners, however, generally showed similar trends in evaluation. An anon- ymous questionnaire suggested that students recognized the benefits of VR training (e.g. ready access to assessment, error identification and how they can be corrected), but the majority felt that it would not replace conventional training methods (95%), although participants recognized the potential for devel- opment of VR systems in dentistry. The most common reasons cited for the preference of conventional training were excessive critical feedback (55%), lack of personal contact (50%) and technical hardware difficulties (20%) associated with VR-based training. Key words: virtual reality; skill acquisition; self-directed and self- paced learning; real-time feedback; objective evaluation. ß Blackwell Munksgaard, 2003 Accepted for publication 6 March 2002 F OR many decades, mechanical simulation devices have been used in training exercises, such as in aerial and marine aviation. They have also been employed for the rehearsal of reaction protocols in hazardous or potentially hazardous situations, thereby eliminating the risk of injury, loss of life or damage to expensive equipment. The earliest widespread use of this technology was in competitive or ‘hazardous’ arcade games, e.g. aerial combat and car racing. Associated with the rapid increase in computing speeds and miniaturization of components (1), the application of virtual reality (VR) has increased dra- matically, particularly due to the ubiquitous presence of the home computer. In the healthcare area, VR has had limited integration into the acquisition of surgical skills (e.g. suturing and ‘keyhole’ surgery) and planning of individual surgical procedures, particularly implant, craniofacial and neurosurgical procedures (2, 3). With respect to skills acquisition, VR-based training has been utilized for the repeated use of a standardized simu- lated patient (4). Dental students are expected to be competent at a large array of procedures on certification; indeed, in many countries, no further training is compulsorily required. This is very different from medical graduates who are required to undergo extensive postgraduation vocational training. The large number of undergradu- ate dental procedures, which must be mastered, have traditionally been tested by practical examinations. The subjective nature of these tests, and their associated stress, has led to many dental faculties replacing them with ‘competence tests’ (5, 6), which are organized, within certain limits, at the student’s convenience and assessed under objective or semiobjective criteria Eur J Dent Educ 2003; 7: 13–19 Printed in Denmark. All rights reserved 13

A pilot study comparing the effectiveness of conventional training and virtual reality simulation in the skills acquisition of junior dental students

Embed Size (px)

Citation preview

Page 1: A pilot study comparing the effectiveness of conventional training and virtual reality simulation in the skills acquisition of junior dental students

A pilot study comparing the effectiveness of conventional

training and virtual reality simulation in the skills

acquisition of junior dental students

Frank Quinn1, Paul Keogh1, Ailbhe McDonald2 and David Hussey3

1Dublin Dental School and Hospital, Lincoln Place, Dublin 2, Republic of Ireland; 2Eastman Dental Institute for Oral Healthcare Sciences, 256 Grays InnRoad, London WC1X 8LD, UK; 3Royal Victoria Dental School and Hospital, Royal Victoria Hospital Group Trust, Grosvenor Road, Belfast BT12 6BP, UK

The use of virtual reality (VR) in the training of operative dentistry isa recent innovation and little research has been published on itsefficacy compared to conventional training methods. To evaluatepossible benefits, junior undergraduate dental students wererandomly assigned to one of three groups: group 1 as taughtby conventional means only; group 2 as trained by conventionalmeans combined with VR repetition and reinforcement (withaccess to a human instructor for operative advice); and group3 as trained by conventional means combined with VR repetitionand reinforcement, but without instructor evaluation/advice,which was only supplied via the VR-associated software. Atthe end of the research period, all groups executed two class1 preparations that were evaluated blindly by ‘expert’ trainers,under traditional criteria (outline, retention, smoothness, depth,wall angulation and cavity margin index). Analyses of resultingscores indicated a lack of significant differences between thethree groups except for scores for the category of ‘outlineform’, for group 2, which produced significantly lower (i.e. better)scores than the conventionally trained group. A statisticalcomparison between scores from two ‘expert’ examiners

indicated lack of agreement, despite identical written and visualcriteria being used for evaluation by both. Both examiners,however, generally showed similar trends in evaluation. An anon-ymous questionnaire suggested that students recognized thebenefits of VR training (e.g. ready access to assessment, erroridentification and how they can be corrected), but the majorityfelt that it would not replace conventional training methods(95%), although participants recognized the potential for devel-opment of VR systems in dentistry. The most common reasonscited for the preference of conventional training were excessivecritical feedback (55%), lack of personal contact (50%) andtechnical hardware difficulties (20%) associated with VR-basedtraining.

Key words: virtual reality; skill acquisition; self-directed and self-paced learning; real-time feedback; objective evaluation.

�Blackwell Munksgaard, 2003Accepted for publication 6 March 2002

FOR many decades, mechanical simulation devices

have been used in training exercises, such as in

aerial and marine aviation. They have also been

employed for the rehearsal of reaction protocols in

hazardous or potentially hazardous situations, thereby

eliminating the risk of injury, loss of life or damage to

expensive equipment. The earliest widespread use of

this technology was in competitive or ‘hazardous’

arcade games, e.g. aerial combat and car racing.

Associated with the rapid increase in computing

speeds and miniaturization of components (1), the

application of virtual reality (VR) has increased dra-

matically, particularly due to the ubiquitous presence of

the home computer. In the healthcare area, VR has had

limited integration into the acquisition of surgical skills

(e.g. suturing and ‘keyhole’ surgery) and planning of

individual surgical procedures, particularly implant,

craniofacial and neurosurgical procedures (2, 3). With

respect to skills acquisition, VR-based training has been

utilized for the repeated use of a standardized simu-

lated patient (4).

Dental students are expected to be competent at a

large array of procedures on certification; indeed, in

many countries, no further training is compulsorily

required. This is very different from medical graduates

who are required to undergo extensive postgraduation

vocational training. The large number of undergradu-

ate dental procedures, which must be mastered, have

traditionally been tested by practical examinations. The

subjective nature of these tests, and their associated

stress, has led to many dental faculties replacing them

with ‘competence tests’ (5, 6), which are organized,

within certain limits, at the student’s convenience

and assessed under objective or semiobjective criteria

Eur J Dent Educ 2003; 7: 13–19Printed in Denmark. All rights reserved

13

Page 2: A pilot study comparing the effectiveness of conventional training and virtual reality simulation in the skills acquisition of junior dental students

(7, 8). The use of standardized VR scenarios may be

beneficial in the preparation and reinforcement of

required procedures (9). There may be additional appli-

cations for postgraduate students, to revise operative

and fixed prosthodontic procedures, and continuing

dental education. Once assessment criteria have been

universally accepted within the faculty, VR prepara-

tions and evaluations may be used to standardize

teachers and improve conventional teaching.

Computing speeds and costs have remained a con-

straining factor for the widespread use of specialized

systems to relatively small consumer groups, such as

dental undergraduates. In the last 5 years, however, a

dental ‘patient’ simulator has been developed and

marketed for the training of undergraduate dental

students. This simulator consists of a conventional

torso and movable head, with integral maxillary and

mandibular jaws, which carry conventional plastic

teeth. There is an operatory-type light, suction and a

bracket table, carrying an ultra-high speed handpiece

and air/water syringe. With these components, train-

ing in operative dentistry can be undertaken in a

manner identical to conventional methods.

The novel components are light emitting diodes on

both the head and the handpiece and a tracking camera

that monitors head and handpiece movements (Fig. 1).

There are two personal computers: one to collate track-

ing information and the second to integrate all data and

execute the associated VR software. Finally, there is a

video screen which presents real-time virtual images to

the operator (Fig. 2).

At the time of this project, the system’s software

allowed several operative and fixed prosthodontic pro-

cedures. It is postulated that the use of these options

would be superior to conventional training, with plastic

teeth, as the individual layers of the natural tooth are

represented in the virtual tooth, including dentinal

carious lesions. Preparations may be viewed from

many angles and at varying magnification; these fea-

tures may improve student understanding and self-

assessment (10, 11). The image of the virtual mouth and

tooth provide real-time information feedback of the

actual preparation cut on the tooth analogue. The

operator-controlled magnification may improve visua-

lization of cavity detail (12).

The software has an integrated database of theoretical

information, ranging from conventional principles of

cavity preparation to the pathology of the carious

lesion. This database is continuously accessible during

training sessions. In addition, there is a glossary of

dental terms, with associated multimedia explanations.

A final, and perhaps most important, benefit of this

simulation unit is that the software will analyze the

preparation, on request, and provide detailed written

and two-dimensional graphic evaluation of the pre-

paration (Fig. 3). Alterations may then be made and

the preparation re-evaluated. This encourages self-

directed and ‘deep’ learning and self-paced skills

Fig. 1. Operator, torso with handpiece and display of VR mouth.

Fig. 2. The real-time display with magnified virtual tooth pre-paration.

Fig. 3. VR-based analysis of cavity preparation, compared to preset‘ideal parameters’.

Quinn et al.

14

Page 3: A pilot study comparing the effectiveness of conventional training and virtual reality simulation in the skills acquisition of junior dental students

acquisition (11, 13, 14). It is postulated that these

benefits will aid the different styles and rates of learning

(15). This self-direction and self-pacing has been advo-

cated by many authorities due to the overloaded nature

of modern dental curricula and the ‘bulimic’ nature of

conventional teaching methods (16). This term refers to

the cramming of large amounts of theoretical informa-

tion, in excessive detail and regurgitating it for an all

or nothing assessment. All too often, the information

is then discarded and never reviewed again. This

approach favours ‘shallow’ or poorly retained learning.

The system administrator/course coordinator can

control access to specific, appropriate ‘lessons’. This

is managed via a programmable user database and

unique passwords. Operator access to ‘lessons’ may

be limited to specific groups or during a particular time

frame. The duration allowed per individual procedure

may also be limited, if so desired. The procedures, once

selected, present clinical details and radiograph(s) for

perusal by the student; they may then proceed to the

virtual mouth and operative procedures completed.

This integration of clinical information and procedures

is believed to lead to improved knowledge acquisition

and retention.

Individual sessions for students may be stored for

later review, either by the student or by an instructor.

The review is in the form of a real-time video playback,

with fast forward and rewind functions. The recorded

sessions may act as a positive feedback to the student,

illustrating improved technique with individual prac-

tice. If desired, recorded sessions may be used as

evidence of ‘competence’ in clinical practice.

The Dublin Dental School received four Dental

Simulation Units and, finding only anecdotal evidence

of the benefits of VR in operative training, a pilot

research project was undertaken to quantify the bene-

fits, if any.

Methods and materials

The second year dental undergraduate class, consisting

of 32 students, was randomly assigned (by lottery) to

one of three groups. All students were given the same

introductory lecture and demonstration on the design

and instrumentation of conventional class 1 cavity

preparation. All students received ‘conventional’ opera-

tive practice in the undergraduate laboratory. All were

supplied with a millimetre graduated periodontal

probe, a mouth mirror and a sharp probe. All prepara-

tions were completed with the same ISO standard pear-

shaped tungsten carbide bur (245), used at ultra-high

speeds and with continual water spray.

The group 1 students worked exclusively with con-

ventional phantom heads and teeth. These students had

continual access to an instructor, to provide feedback

and evaluation. Students performed repeated class 1

preparations for approximately 21 h.

Group 2 students also worked on conventional phan-

tom heads but, in addition, received 1 h of instruction

on the use of the dental simulation unit, and 4 h of

preparing multiple class 1 preparations on the lower

left first molar on the VR units. The students had real-

time feedback from the VR unit (Fig. 2) and had con-

tinual access to the same instructor (PK), but only for

technical instruction, real-time feedback and evalua-

tion. The students did not have access to preparation

evaluation options in the software. In addition to the

VR simulators’ exposure, the students had approxi-

mately 16 h of conventional phantom head teaching.

Group 3 students had identical conventional training

time as group 2, identical instruction in the use of the

dental simulation units and 4 h for preparation of multi-

ple class 1 preparations of the lower left first molar.

These students had access to the same instructor (PK),

but only for technical questions: feedback and prepara-

tion evaluation were provided by the units’ software.

The students also had access to the information data-

base and glossary. In addition to exposure to the VR

simulators, the students had approximately 16 h of

conventional phantom head teaching.

When all students had completed the allocated

operative time, all groups executed two class 1 cavities.

The individual teeth were coded anonymously and

submitted to two independent scorers. These two

experienced restorative academic trainers were not

informed of the nature of the project, but were supplied

with the ‘ideal’ cavity dimensions and illustrations of

the desired cavity shape and dimensions on the lower

left first molar (17, 18).The scorers were requested to provide an ordinal

score (0–3 or 0–4) for the following aspects of the cavity

design: outline form, retention form, depth, smooth-

ness, cavosurface angulation (17) and ‘cavity margin

index’ (19). The scores were qualitative, with 0 indicat-

ing an ideal preparation for that parameter, and 4 (or 3)

representing the aberrant performance.

The criteria for cavity evaluations were based on ‘best

practice’ information (17, 18). These criteria, although

subjective, were intended to be as clear-cut as possible.

The criteria and scores are appended below:

Statistical analysesOn return of the scores, the code was broken and non-

parametric analyses (Kruskal–Wallis/Wilcoxon Rank

Sum) of the data were undertaken with respect to the

15

Virtual reality in the training of operative dentistry

Page 4: A pilot study comparing the effectiveness of conventional training and virtual reality simulation in the skills acquisition of junior dental students

three groups. In addition, the agreement between the

two scorers was examined utilizing Kappa analyses.

Jump-in Software was employed for statistical analyses

(JMP IN, SAS Institute Inc).

A structured anonymous questionnaire was given to

groups 2 and 3, after they had completed all conventional

and VR exercises. This was designed to be non-directive

and to elucidate the subjective comparative opinions of

students who have been exposed to both conventional

operative training and the VR training units.

Results

The results data were not normally distributed. Non-

parametric analyses failed to show statistically signi-

ficant differences between the three groups (Tables 1

and 2), except for the criterion ‘outline form’ (p¼ 0.037).

The difference appears to be between groups 1 and 2

(Wilcoxon Rank Sum test).

Comparison between two ‘expert’ examiners indicated

poor agreement. The best agreement was for ‘cavity

smoothness’, which gave a Kappa statistic of 0.156 and

the poorest was for ‘cavity margin index’, with the

Kappa statistic of 0.012. Nonetheless, both scorers failed

to show statistically significant difference for the eva-

luation criteria, except for ‘outline form’. One examiner

reported a relatively greater difference between the

three groups for this criterion, while the other exam-

iner’s scores fell just short of statistical significance at

the p¼ 0.05 level. This disparity, combined with expo-

sure to conventional training by all groups, may have

masked differences between the three groups.

The anonymous questionnaire indicated that stu-

dents identified a relatively limited access to instruc-

tors, and an inconsistency in instructor evaluation, as

being unfavourable aspects of conventional training

(Table 3). Ninety-five percent of the individuals in

experimental groups 2 and 3 felt, however, that VR-

based training will not replace conventional training in

operative dentistry. The respondents selected technical

hardware difficulties, excessively critical feedback and

lack of personal interaction as the limitations of this VR-

based system (Tables 4 and 5). When asked to select

TABLE 1. Wilcoxon/Kruskal–Wallis test for differences betweencontrol (1) and experimental groups (2 and 3), for the factor outlineform

Group Count Scoresum

Scoremean

(Mean�Mean0)/SD0

1 24 942 39.2500 2.4642 20 532.5 26.6250 �1.8643 20 605.5 30.2750 �0.701

w2¼ 6.5718; d.f.¼ 2; p¼ 0.0374.

TABLE 2. Kruskal–Wallis test for differences between control (1)and experimental groups (2 and 3), for the factors retention,smoothness, cavity depth, cavosurface angulation and cavitymargin index scores

w2 d.f. Probability

Retention� 4.1832 2 0.1235Smoothness� 3.2622 2 0.1957Cavity depth� 4.1692 2 0.7909Cavosurface angulation� 5.1259 2 0.0771Cavity margin index score� 1.5397 2 0.4631

�One-way test, w2 approximation.

TABLE 3. Percentage frequencies of the two most commonquestionnaire responses to: ‘list everything you dislike by theconventional training’

Supervisors are too busy Supervisor inconsistency

85% 40%

TABLE 4. Percentage frequencies of three most common ques-tionnaire responses to: ‘list everything you dislike by the VR-basedtraining’

Too critical Too easy to blockLED sensors

Inconsistent easeof use

55% 20% 20%

TABLE 5. Percentage frequencies of three most common ques-tionnaire responses to: ‘list everything you like about conventionaltraining’

Personal toucheasier to learnabout errors

Interaction/discussion

Demonstration ofhow to preparecavity correctly

45% 50% 55%

TABLE 6. Percentage frequency of selected descriptors to thequestion: ‘select which words best describe VR-based trainingand conventional training?’

Descriptor VR-based training Conventional training

Easy 20 45Stimulating 40 60Uninvolving 35 15Satisfying 25 55Frustrating 85 25Helpful 50 95

TABLE 7. Percentage frequencies of the three most commonquestionnaire responses to: ‘list everything you like about theVR-based training

VR presentationis interesting

Feedbackaids erroridentification

Feedback shows whereerrors are, and how tocorrect them

60% 30% 30%

16

Quinn et al.

Page 5: A pilot study comparing the effectiveness of conventional training and virtual reality simulation in the skills acquisition of junior dental students

words to describe the participants experience with VR-

based and conventional training, the most common

descriptors were ‘frustrating’ for VR-based and ‘help-

ful’ for conventional training (Table 6). They identified,

however, interesting presentation, improved access to

feedback and consistency in evaluation as benefits of

VR-based training (Tables 7 and 8). When requested to

make direct comparisons of VR-based and conventional

training, 40% of respondents selected VR-based as

being ‘much better’ for ‘providing feedback’, whereas

70% selected for ‘increasing confidence in cavity pre-

paration’ as being ‘much better’ for conventional train-

ing (Table 9).

Discussion

The effective use of students’ time has become an

increasing necessity with the current rapid expansion

of dental curricular contents (16). The use of VR in

undergraduate training in operative dentistry, particu-

larly where the unit also provides objective formative

evaluation on demand, may prove advantageous either

in a more rapid rate of skills acquisition or in the

development of superior quality of preparation. These

suggested benefits would have attractions for both

dental educators and students.

Within the limitations of this protocol, however, the

study failed to demonstrate any consistent differences

between the three groups, in quality of class 1 cavity

preparation, when conventional operative assessment

criteria were used, except for the category ‘outline

form’. For this category, the group who had real-time

visual feedback during VR procedures and conven-

tional instructor advice performed significantly better

than the conventionally trained, but not compared to

those who had real-time VR feedback and software

‘objective’ evaluation. Separate analyses of the two

‘expert’ assessors indicated a strong statistical signifi-

cance for one, with the other’s scores falling just short of

significance at the p¼ 0.05 level.

The lack of other significant advantages may have

arisen due to the relatively small sample sizes, the rela-

tively large proportion of conventional training versus

VR time in groups 2 and 3, the relative simplicity of the

class 1 cavity preparation or the imprecision of the

implementation of the evaluation criteria. Alternatively,

the results may reflect an actual lack of systematic differ-

ence in the methods of skills acquisition. Although

larger sample sizes might have exhibited larger inter-

group differences, it is likely that such differences

would have been of limited clinical importance.

Participants in the two VR groups (2 and 3), had 4 h of

practice on the simulation units and 16 h devoted to

conventional training. The control group (1) had 21 h of

conventional training only. This occurred for logistical

and standardization reasons only. This relative dispro-

portion in time spent on conventional versus VR train-

ing may have resulted in a dilution of any difference in

the rate of operative skills acquisition but is unlikely to

have reduced any qualitative benefits gained from real-

time feedback and on-demand objective evaluation.

This problem is being addressed in a current study

where students will have access to conventional train-

ing only or to VR training only, both for identical time

duration. These students have no professional experi-

ence of operative dentistry. The study will be reported

in 2002.

It was interesting that the two VR groups showed a

tendency towards more accurate control of cavity wall

angulation but this difference failed to show statistical

significance (p¼ 0.0771). This may reflect improved

TABLE 8. Percentage frequencies of the descriptor ‘very good’ for the above headings in relation to conventional operative training and VR-based training

Immediate feedback Self-paced learning Providing more thoroughassessment

Increasing confidencein cavity preparation

Conventional 15% 40% 30% 65%VR 60% 25% 50% 20%

TABLE 9. Percentage frequency of selected descriptors on relative superiority of VR-based training or conventional training

VR-based traininga Conventional traininga

Much better? A bit better? Much better? A bit better?

Providing feedback 40 35 15 5Allows self-paced learning 5 35 20 40More thorough understanding 35 40 20 5Allowing independent work 10 55 10 20Increasing confidence in cavity preparation 5 5 70 15

aWhich was better? How much?

17

Virtual reality in the training of operative dentistry

Page 6: A pilot study comparing the effectiveness of conventional training and virtual reality simulation in the skills acquisition of junior dental students

critical skills associated with the use of real-time virtual

display during preparations and in-depth evaluation of

cavity design (12). This will be investigated further in a

future study.

Although there was poor agreement between the two

‘experts’ for individual scores, in general, the trends

were similar and failed to indicate significant differ-

ences between the three groups, except for outline form.

This lack of reproducibility in detailed critical clinical

appraisal has been reported by other workers (20, 21)

and has been cited as a frustrating aspect of training for

dental students, particularly inexperienced students.

Alternatively, the VR software presents extensive

detailed and reproducible analyses of cavity ‘errors’.

To the inexperienced student, without the benefit of

human guidance, this proved to be relatively dis-

couraging and apparently insurmountable. Thus, a

combination of approaches may prove to be more

advantageous.

The student questionnaire confirmed that the

majority of the VR students recognized the benefits

of this technology (Tables 3 and 7). One such notable

benefit, in the virtual preparation, is that of three-

dimensional visualization of the anatomical layers of

teeth, including the pathological distribution of the

virtual carious lesion. Machine calibration, excessively

critical assessment and technical problems, however,

were cited as dislikes for the VR groups (Table 4).Personal interaction and demonstration of how to

prepare the cavity correctly were cited as favourable

aspects of conventional training (Table 5).

As might be expected, the results of the questionnaire

indicated that the VR units were superior to conven-

tional training in providing ‘immediate feedback’ (60%

vs. 15% for ‘very good’ and in ‘providing more thor-

ough assessment of . . . cavity preparation’ (50% vs. 30%

for ‘very good’) (Table 8). Unexpected findings were,

however, that conventional training was ‘much better’

for ‘increasing confidence in cavity preparation’ in 70%

of the responses (Table 9) and that when asked to select

words to describe VR experience and conventional

training, ‘helpful’ was the selected descriptor for 95%

of the conventional and only 50% of the VR respon-

dents. Similarly 85% of respondents selected ‘frustrat-

ing’ for the VR experience whereas this was selected by

only 25% for the conventional training (Table 6). On

questioning, the participants felt the two aspects of

frustration were restricted operator position (so as

not to block the tracking camera’s light emitting diodes)

and that the VR screen did not appear to display

accurately what they perceived as the cavity outline.

This latter factor may be due to the limited resolution of

the real-time display.

A daily ‘fine tuning’ was necessary for each of the

simulation units. The duration of this preparatory pro-

cedure ranged between 5 min to over 1 h. Upgrades of

hardware and software during the project led to a

marked reduction in the time occupied in this tedious

exercise. The developers have now introduced an auto-

mated checking procedure, which has been actively

encouraged by all current system users. In addition,

access to information support (IS) services was needed

on a regular basis, particularly in the early stages of

integration. Open access to the VR units proved to be

impossible due to the necessity for technical and

procedural advice requested by the participants.

Nonetheless, the majority of students found software

and graphic interfaces relatively accessible and navig-

able.

Unfortunately, at present, modification of the clinical

‘scenarios’ is not possible locally, by the system

administrator/faculty, but must be customized by

the developers: this involves considerable time and

cost. This may limit the number and variations of

individual lessons available, e.g. shallow versus deep

cavities, and minimal versus conventional cavity

designs.

Summary

1. Within the constraints of the study, there was no

statistically significant difference between conven-

tional operative training alone or in combination

with VR simulations, in terms of cavity quality as

assessed by conventional criteria.

2. Students favour a combination of conventional ‘sub-

jective’ instructor, and VR simulation with objective

feedback.

3. Consistency in VR system calibration was a signifi-

cant problem early in the study: this deficiency needs

to be thoroughly addressed if students’ confidence is

to be acquired and maintained by this new and

promising technology.

4. A major benefit in the system would be the ability to

modify the clinical scenarios and requirements

locally.

5. Independent assessors showed similar scoring trends

but exhibited poor individual score agreement.

6. Simulation training via VR units shows great pro-

mise in undergraduate and postgraduate dental

training, but it is unlikely that it will replace human,

subjective interaction.

7. Further study is needed to evaluate the benefits of

VR training when used alone, particularly in relation

to speed of skills acquisition.

18

Quinn et al.

Page 7: A pilot study comparing the effectiveness of conventional training and virtual reality simulation in the skills acquisition of junior dental students

Acknowledgements

The authors would like to thank Ms. Sarah Keogh and

Alan Kelly (Senior Biostatistician).

References

1. Munnelly B, Holden P. A complete coursebook of theEuropean computer driver’s licence. London: PearsonEducation Books, 2000: 2–5.

2. Seipel S, Wagner IV, Koch S, Schneider W. Oral implanttreatment planning in a virtual reality environment.Comput Meth Programs Biomed 1998: 57: 95–103.

3. Wagner A, Rasse M, Millessi W, Ewers R. Virtual realityfor orthognathic surgery: the augmented reality environ-ment concept. J Oral Maxillofac Surg 1997: 55: 456–462.

4. Joly B, Grant J. Patient-based assessment-simulatedpatients. In: The good assessment guide. London: TheJoint Centre for Education in Medicine, 1997: 96–97.

5. Mossey P. Competency based teaching and assessmentin dentistry: the context. In: Mossey P, Stirrups D, eds.Clinical competencies in dentistry: exploring the issues.London: Medical and Dental Education Network, 1997: 8.

6. Dunkley P. Assessment of psychomotor skills. In:Mossey P, Stirrups D, eds. Clinical competencies indentistry: exploring the issues. London: Medical andDental Education Network, 1997: 19–22.

7. Yip HK, Smales RJ, Newsome PRH, Chu FCS, Chow TW.Competency-based education in a clinical course inconservative dentistry. Br Dent J 2001: 191: 517–522.

8. Joly B, Grant J. Reliability. In: The good assessmentguide. London: The Joint Centre for Education inMedicine, 1997: 34.

9. Joly B, Grant J. Constructive feedback. In: The goodassessment guide. London: The Joint Centre for Educa-tion in Medicine, 1997: 32–33.

10. Stirrups D. Assessment as a tool for supportive learning.In: Mossey P, Newton JP, Mason A, Stirrups A, eds.Clinical competencies in dentistry conference: assessingin competence. London: Medical and Dental EducationNetwork, 1998: 46–48.

11. Joly B, Grant J. Records of practice-portfolios. In: Thegood assessment guide. London: The Joint Centre forEducation in Medicine, 1997: 77–78.

12. Robinson PB, Lee JW. The use of real time videomagnification for the pre-clinical teaching of crownpreparations. Br Dent J 2001: 190: 506–510.

13. Norman GR, Schmidt HG. The psychological basis ofproblem-based learning: a review of the evidence. AcadMed 1992: 67: 557–565.

14. Ferrier BM. Problem-based learning does it make adifference? J Dent Educ 1990: 54: 550–551.

15. Honey P, Mumford A. Manual of learning styles.Berkshire, UK: Honey P, 1982: 59–73.

16. Preston JD. Editorial: bulimic education. Int J Prostho-dont 1995: 8: 301.

17. Summitt JB, Robbins JW, Schwartz RS. Fundamentals ofoperative dentistry. Chicago: Quintessence PublishingCo. Inc, 2001: 306–313.

18. Kidd EAM, Smith BGN, Pickard HW. Pickard’s manualof operative dentistry. Oxford: Oxford University Press,1996: 112–113.

19. Tronstad L, Leidel TI. Scanning electron microscopy ofcavity margins finished with chisels or rotating instru-ments at low speed. J Dent Res 1973: 53: 1167–1174.

20. Smales RJ, Creaven PJ. Evaluation of clinical methods forassessing the surface roughness of restorations. J ProsthetDent 1979: 42: 45–52.

21. Gray C. How to evaluate assessment methods. In:Mossey P, Newton JP, Mason A, Stirrups D, eds. Clinicalcompetencies in dentistry conference: assessing incompetence. London: Medical and Dental EducationNetwork, 1998: 22–23.

Address:

Frank QuinnDepartment of Restorative Dentistry and PeriodontologyDublin Dental SchoolLincoln PlaceDublin 2Republic of Ireland

Tel: þ353 1 6127312

Fax: þ353 1 6127297

e-mail: [email protected]

19

Virtual reality in the training of operative dentistry