Seediscussions,stats,andauthorprofilesforthispublicationat:https://www.researchgate.net/publication/308750082

Areviewofcomputer-aidedoralandmaxillofacialsurgery:planning,simulationandnavigation

ArticleinExpertReviewofMedicalDevices·September2016

DOI:10.1080/17434440.2016.1243054

CITATIONS

0

READS

30

4authors,including:

Someoftheauthorsofthispublicationarealsoworkingontheserelatedprojects:

RarecaseofacerebralabscesscausedbyPorphyromonasGingivaliswithunderlyingparodontitis

andperiodontitis.Viewproject

DrugprotocolinPostsurgicalneuropathicpainandtheroleofQSTinpatientmanagement.View

project

YiSun

UniversitairZiekenhuisLeuven

52PUBLICATIONS266CITATIONS

SEEPROFILE

ConstantinusPolitis

UniversitairZiekenhuisLeuven

185PUBLICATIONS909CITATIONS

SEEPROFILE

AllcontentfollowingthispagewasuploadedbyConstantinusPolitison30September2016.

Theuserhasrequestedenhancementofthedownloadedfile.

Full Terms & Conditions of access and use can be found athttp://www.tandfonline.com/action/journalInformation?journalCode=ierd20

Download by: [Cornell University Library] Date: 30 September 2016, At: 05:29

Expert Review of Medical Devices

ISSN: 1743-4440 (Print) 1745-2422 (Online) Journal homepage: http://www.tandfonline.com/loi/ierd20

A review of computer-aided oral and maxillofacialsurgery: planning, simulation and navigation

Xiaojun Chen, Lu Xu, Yi Sun & Constantinus Politis

To cite this article: Xiaojun Chen, Lu Xu, Yi Sun & Constantinus Politis (2016): A review ofcomputer-aided oral and maxillofacial surgery: planning, simulation and navigation, ExpertReview of Medical Devices, DOI: 10.1080/17434440.2016.1243054

To link to this article: http://dx.doi.org/10.1080/17434440.2016.1243054

Accepted author version posted online: 29Sep 2016.

Submit your article to this journal

View related articles

View Crossmark data

Publisher: Taylor & Francis

Journal: Expert Review of Medical Devices

DOI: 10.1080/17434440.2016.1243054

A review of computer-aided oral and maxillofacial surgery: planning, simulation and navigation

Xiaojun Chen1,*, Lu Xu1, Yi Sun2, Constantinus Politis2

1 Institute of Biomedical Manufacturing and Life Quality Engineering, State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China 2 Faculty of Medicine, KU Leuven, Leuven, Belgium Address correspondence to: Xiaojun Chen, PhD Room 805, School of Mechanical Engineering, Shanghai Jiao Tong University, Dongchuan Road 800, Minhang District, Shanghai, China Post Code: 200240 E-mail: xiaojunchen@163.com

Tel: (+86) 21-34204851 Fax: (+86) 21-34206847

Abstract

Introduction: Currently, oral and maxillofacial surgery (OMFS) still poses a significant challenge for

surgeons due to the anatomic complexity and limited field of view of the oral cavity. With the great

development of computer technologies, he computer-aided surgery has been widely used for

minimizing the risks and improving the precision of surgery.

Areas covered: The major goal of this paper is to provide a comprehensive reference source of current

and future development of computer-aided OMFS including surgical planning, simulation and

navigation for relevant researchers.

Expert commentary: Compared with the traditional OMFS, computer-aided OMFS overcomes the

disadvantage that the treatment on the region of anatomically complex maxillofacial depends almost

exclusively on the experience of the surgeon.

Keywords oral and maxillofacial surgery; preoperative planning; surgical simulation; surgical

navigation; computer-assisted surgery

1. Introduction

Nowadays, the oral and maxillofacial surgery (OMFS) has been widely applied for the treatment of

the diseases and disorders of the face and jaws including trauma, congenital and acquired defects, facial

infections, cancers in the head and neck region [1]. However, due to the anatomic intricacies in the

maxillofacial region, the OMFS is still poses a significant challenge for the surgeons [2].

Over the past decades, with the great development of computer technologies, the computer-aided

surgery have been widely used for minimizing the risks and improving the precision of the surgery.

Compared with the traditional OMFS, the computer-aided OMFS overcomes the disadvantage that the

treatment on the anatomically complex maxillofacial region depend almost exclusively on the

experience of the surgeon [3]. For example, the surgeon can perform a surgical simulation based on a

patient model that has previously been 3D-reconstructured with the actual patient 2D dicom data [4].

Furthermore, the computer-aided surgical navigation system enables the real-time motion tracking of

surgical instruments. On the basis of the established correspondence between features in the

preoperative images and the surgical scene, a three-dimensional map can be provided for the surgeon to

conduct a precision surgery [5].

In this paper, the key technologies of computer-aided OMF surgery are discussed including the

surgical planning, simulation and navigation for relevant researchers.

2. Vision-based surgical planning

The vision-based surgical planning has become more and more important for OMFS since it enables

surgeons to design and optimize the surgical scheme intuitively. It encompasses the procedures of

image processing (such as segmentation and registration), three-dimensional visualization and

preoperative planning. Currently, these procedures are usually completed in the commercially available

software tools including Mimics®, SurgiCase® (Materialise, Leuven, 3001, Belgium), SimPlant®

(Dentsply, York, 17401, USA), Nobel GuideTM (Nobel Biocare, Zürich-Flughafen, CH-8058,

Switzerland), iPlan (BrainLab AG, Feldkirchen, 85622, Germany), VoXim® (IVS Technology GmbH,

Chemnitz, 09125, Germany), Analyze (AnalyzeDirect, Inc., Overland Park, 66085, USA), Amira

(FEITM, Hillsboro, 97124, USA) and 3dMD (3dMD LLC, Atlanta, 30339, USA) [6, 7].

Table 1 shows some clinical applications of using commercially available softwares for preoperative

planning in OMFS. For example, Zheng et al. (2016) performed the preoperative planning for

maxillary reconstruction through Mimics 10.01 software (Materialise) [8]. In his study, the skull and

the fibula were firstly 3D-reconstructed, and the critical anatomical structures such as tumor and blood

vessels were manually segmented and visualized. According to these data, the simulations of maxillary

resection, fibula cutting, and repositioning were accomplished. In the field of oral implantology, Shen

et al. (2015) presented the usage of Simplant Pro 11.04 (Dentsply) to conduct the preoperative planning

including the evaluation of bone mass, the selection of suitable implant sizes, and the determination of

the insertion depth and orientation [9]. Komiyama et al. (2011) carried out surgical planning via

Procera® software (Nobel Biocare) for 30 patients with an edentulous maxilla, mandible or both

between September 2003 and May 2007, and the planning data were then used for the subsequent

template design and manufacturing [10]. Sadiq et al. (2012) presented two cases of “waferless”

stereotactic maxillary positioning [11]. Before the surgery, the orthognathic planning was performed

through Voxim® software (IVS Technology GmbH), and then the operation was accomplished with the

aid of the Voxim® navigation system. Voss et al. (2016) developed a computerized technique based on

the software of Amira Version 5.4.3 (FEITM), which allowed three-dimensional modeling, distinction of

the fracture fragments, and virtual fracture reduction [12]. Ullah et al. (2015) predicted the

postoperative facial appearance after orthognathic surgery through 3dMD Vultus (3dMD LLC) [13].

According to the accuracies of 13 clinical cases, it demonstrated that 3dMD Vultus produced clinically

satisfactory of three-dimensional facial soft tissue predictions after Le Fort I advancement osteotomy.

However, these commercially 3-dimenasional planning software tools were expensive and lack of

software expandability. Therefore, today, some open-source pre-surgical planning software package

have been used for clinical applications and presented in the literature. In general, these software tools

are programmed on the basis of some widely used open-source toolkits such as VTK (Visualization

Toolkit), ITK (Insight Toolkit), CTK (Common Toolkit) and QT. For example, 3D Slicer (Brigham

Women’s Hospital, Boston, 02115, USA), Gimias (Centre for Computational Imaging and Simulation

Technologies, University of Sheffield, Sheffield, S10 2TN, United Kingdom) and CamiTK

(TIMC-IMAG, Grenoble, 38706, France) are some well-known, powerful software packages for

3-dimensional visualization, medical image computing and planning interventions. Zhan et al. (2016)

proposed an interactive method for model clipping and developed a loadable module named

SmartModelClip based on 3D Slicer platform [14]. Figure 1 shows planning of three types of Le Fort

fractures in orthognathic surgery through the user-friendly module with high reliability and efficiency.

In addition, some homemade planning softwares have also been developed by research groups. For

example, Chen et al. (2010) presented clinical case study for oral implantology planning using the

self-developed software named “CAPPOIS” (Computer Assisted Preoperative Planning for Oral

Implant Surgery, Shanghai Jiao Tong University, Shanghai, 200240, China) [15]. Olsson et al. (2013)

presented cranio-maxillofacial surgery planning system that combines stereoscopic 3D visualization

with haptic feedback [16]. The system allows the surgeons to plan the restoration of skeletal anatomy

in facial trauma patients, and conduct the preliminary testing of alternative solutions for restoring bone

fragments to their proper positions.

In addition, although the three-dimensional cephalometry is not always presented in 3D planning

software, it is sometimes indispensable as an important feature for the 3D diagnosis, treatment planning,

and evaluation of post treatment results. For example, Gordon et al. (2014) developed a

computer-assisted planning and execution (CAPE) workstation for facial transplantation [17]. With the

support of this platform, the static cephalometric analysis was performed so that the surgeon can

evaluated different placements for the donor’s face-jaw-teeth alloflap on the recipient’s face in relation

to orbital volumes, airway patency, facial projection, and dental alignment. On the basis of

three-dimensional analysis in Vectra XT Imaging System (Canfield Scientific, Inc., Fairfield, 07054,

USA), Davidson and Kumar (2015) evaluates changes in nasal aesthetics after Le Fort I advancement

in patients with cleft lip and palate [18].

3. Surgical simulation and training

On the basis of the virtual planning result, the surgeons can be provided an optimal scheme. However,

due to the intricate anatomical structures, it requires high operative skills before carrying out the

surgery on the actual patient for the surgeons. Therefore, it is one of the most important tasks facing

medicine today that how to transfer the surgical planning to the real patient. The surgeons (especially

the next generation of surgeons) should be trained to reduce the human error on the operating table [19].

The traditional surgical practicing method is the usage of the rapid prototyping models, animals and

cadavers, which is a complex, expensive, and time-consuming process. Furthermore, the anatomy of

animals differs from that of humans, and the cadaveric practice is under ever-increasing scrutiny by the

media, government and public.

Over the past decades, the surgical simulation system based on the virtual reality (VR) and haptic

feedback techniques has been developed rapidly. Compared with the method of training with plastic

models, VR-based simulation system can provide a virtual environment which contains different

anatomic structures, and the surgeon can simulate different procedures on these virtual models through

haptic feedback devices such as Omega series (Force Dimension, Nyon, CH-1260, Switzerland) and

PHANTOM® Desktop (Sensable, Wilmington, 01887, USA) [20]. Furthermore, this training system

can be reused many times which offers a cost-effective and efficient alternative to traditional training

methods [21].

Table 2 shows some applications of surgical simulation and training systems in OMFS. For example,

Wu et al. (2014) described a virtual training system for maxillofacial surgery utilizing a 3D immersive

workbench (Display 300, SenseGraphics, Kista, 16455, Sweden) and a six degrees-of-freedom (DOF),

high –fidelity force feedback haptic device of Omega.6 (shown in Figure 2) [22]. With the use of this

training system, the operations of maxilla and mandible (such as cutting, drilling and milling) were

simulated via a self-developed algorithm, and the vivid 3D stereo effect was realized. Wang et al. (2012)

presented a haptic-based dental simulator (iDental) with both force and torque feedback [23]. During

the evaluation experiments, the system allowed the surgeons to simulate the periodontics operations

such as the pocket probing examination, calculus detection and calculus removal. However, the

simulations of prosthodontics and endodontics operations were not included. Konukseven et al. (2010)

developed a Vision-Haptic Device Integrated Dental Training Simulation System for simulating basic

dental operations such as the diagnosis of caries and drilling [24]. Pohlenz et al. (2010) described a new

educational tool called Voxel-Man simulator that was originally designed for dental school [25]. The

feedback from students involved in the pilot-test indicated that the force feedback, spatial 3D

perception, and image resolution of the simulator were sufficient for virtual training of dental surgical

procedures. Tse et al. (2010) designed a virtual dental training system (hapTEL) based on haptic

technology [26]. With the use of this system, dental students were able to learn and practice procedures

such as dental drilling, caries removal and cavity preparation for tooth restoration. In addition, an

open-source framework named SOFA (https://www.sofa-framework.org/) was developed by four teams

at INRIA (Institute for Research in Computer Science and Automation, Rocquencourt, 78150, France),

focusing on real-time simulation, with an emphasis on medical simulation.

Currently, although the simulations of drilling and cutting procedures in OMF surgery have high

fidelity, the rigorous modelling of deformable body (such as soft tissues and blood vessels) is one

major challenge for VR-based simulation system [27]. The deformable simulation is also called

“collision detection” in the literature as the progressive deformation is caused by the forces resulting

from the surgical instruments [28]. Since the constitutive behavior of soft tissue under surgical tools’

interaction is nonlinear and time-dependent, the linear small strain kinematic formulations are not

strictly valid [29]. Over the past decade, various kinds of feasible modelling approaches have been

developed for deformation simulation, and they can be classified into three categories, namely,

mass-spring model (MSM), finite-element model (FEM) and mass-tensor model (MTM) [30]. For

example, Duan et al. (2016) proposed a volume preserved mass-spring model with novel constraints for

soft tissue deformation [31]. With the use of this MSM model, the large deformations can be handled

with nonlinear elasticity for a multi-object system. Xu et al. (2011) presented a nonlinear viscoelastic

mass-tensor visual model for more realistic surgery simulation [32]. Han et al. (2012) developed a

patient-specific biomechanical modelling framework based on a nonlinear finite element (FE) solver

for predicting large breast deformation [33]. In addition, Cheng et al. (2015) introduced another novel

method of using the properties of BP neural network to predict facial soft tissue changes in patients

after complete denture prosthesis [30].

Compared with the simple and fast MSM, the FEM has high precision but is complicated requiring

long calculation time. Therefore, it is meaningful to use graphics processing unit (GPU) techniques in

more complex simulation systems for increasing the computing power [31]. For instance, on the basis

of GPU-accelerated NVIDIA FleX position-based dynamics framework, Camara et al. (2016) presented

a real-time simulation platform that allows for a plausible and realistic deformation of soft tissue [34].

Johnsen et al. (2015) developed a GPU-based nonlinear finite element toolkit called NiftySim to

provide high-performance soft tissue simulation capabilities for medical image computing and surgical

simulation applications [35].

4. Surgical navigation

Although the surgeons’ knowledge on anatomy and operating skills can be improved though the

surgical training system, the surgical procedure in oral and maxillofacial region may be difficult in

complex operation sites. In order to transfer the preoperative planning to the actual surgical site

precisely, the computer-aided surgical navigation system has been employed for OMFS since its

introduction in 1986 [36]. With the support of modern image-guided intervention techniques, the

surgical instruments (such as drill and saw) can be tracked during the operation, and their movements

relative to the patient anatomy are rendered on the computer screen in real time [37].

Figure 3 shows a clinical case application for the treatment of temporomandibular joint ankylosis

using BrainLab surgical navigation system. The patient was a five-year-old girl with the mouth opening

of 8mm. Firstly, on the basis of the panoramic digital X-Ray image and three-dimensional volume

rendering data, the preoperative planning was completed and then evaluated on a 3D printed model.

For example, the optimal location of the osteotomy line was designed to avoid damaging the

surrounding anatomic structures. Then, the reference frame was fixed on the patient and surface based

registration was performed. During the operation, the direction and the position of osteotomy line was

checked through the location pointer, and the ankylotic bone was resected precisely according to the

preoperative trajectory under the interactive guidance of 2D and 3D image rendering environment.

After the image-guided surgery, the patient’s mouth opening width reached to 30mm and no

complications occurred during the follow-up investigation.

Currently, the major surgical navigation systems used in OMFS can be divided into two groups: 1.

The utilization of commercial softwares; 2. The self-developed navigation systems by the research

groups. Table 3 shows some applications of surgical navigation systems in OMFS. Gui et al. (2013)

conducted five clinical cases of using STN navigation system (Stryker®, Kalamazoo, 49002, USA) for

removal of foreign bodies in the complex, deep maxillofacial region, and the accuracy was less than

0.8mm [38]. Furthermore, compared with the conventional non-navigated technique, the surgical time

was reduced approximately 40% of using image-guided system. Li et al. (2016) presented nine

applications of intraoperative navigation for the reconstruction of mandibular defects with

microvascular fibular flap [39]. With the support of BrainLab navigation system, the osteotomy of the

fibular flap was easily performed and the maximal mean linear error was 3.4±1.3mm. Sun et al. (2014)

evaluated the accuracies of BrainLab image-guided system for maxillary positioning in bimaxillary

surgery, which were 0.44±0.35mm, 0.50±0.35mm and 0.56±0.36mm respectively in sagittal, vertical,

and mediolateral direction [40]. Stein (2015) described a case of minimally invasive retrieval of a

broken dental needle using a Medtronic StealthStation® S7 (Medtronic, Minneapolis, 55432-5604,

USA) surgical navigation system [41]. Under the intraoperative 3-dimensional guidance, the accuracy

of the localizing and removing procedures was improved. Essig et al. (2013) demonstrated the

applications of navigation-assisted orbital wall reconstruction using VoXim® [42]. Compared with the

virtual planning, the maximal deviations of the reconstructed medial orbital wall, orbital floor and

lateral orbital wall were, respectively, -0.05±0.70mm, 0.27±0.70mm and -0.23±0.75mm. Chen et al.

(2009) developed an image guided oral implantology system (IGOIS, Shanghai Jiao Tong University)

[2]. The results of its application in the placement of zygoma implants showed that the average distance

deviations at entry and exit point were, respectively, 1.36±0.24mm and 1.57±0.24mm, while the

average angle deviation between the axis of the planned and the actual placed implant is 4.1°±0.90°. In

orthognathic surgery, the 2-splint technique is typically used for the positioning of the maxilla and the

mandible, which may be inaccurate. Therefore, Li et al. (2014) performed five cases of image-guided

free-hand repositioning of the maxillomandibular complex, and the mean absolute errors of the

maxillary position were clinically acceptable (<1.0mm) [43]. In addition, Strong et al. (2008) evaluated

the accuracy of three commercially available optical navigation systems (StealthSatation (Medtronic),

VectorVision (BrainLab AG), and VoXim® (IVS Technology GmbH)) for maxillofacial reconstruction

[44]. The target registration errors of the navigation systems were all less than 1.5mm, and the

differences are small.

5. Expert commentary

Vision-based surgical diagnosis and planning has been considered a routine procedure in the field of

OMFS since it can provide detailed information regarding to the surgical overview, especially the

critical anatomic structures to the surgeons such as the maxillary sinus and the inferior alveolar nerve.

With the support of two- and three-dimensional data sets, preoperative planning can be performed,

including 2D/3D geometrical measurements, bone density analysis, cutting path determination, drilling

trajectory design, osteotomy simulation and repositioning of bony segment [45]. Nowadays, some

open-source planning systems can reduce the cost for the users, especially for small hospitals or clinics.

However, they may lead to errors and unnoticed problems due to the lack of a quality control

mechanism. In contrast, the commercially available systems usually verify the data, and validate their

accuracy and planning aspects in terms of applicability. This is not the case in open systems, and some

unexperienced clinicians may face problems during the surgery because they may not notice a problem

in the “virtual planning”.

The virtual-reality-based surgical simulation system using haptic feedback device has proven useful

for surgical education, planning, and rehearsal. Compared with the conventional 3d printed model or

cadaver approach, this interactive platforms can be efficient and reduce the costs to the trainee, faculty

and society significantly. Currently, one technical challenge with the development of VR-based training

simulators is the prediction of soft tissue nonlinear deformations owing to respiratory movements or

surgical manipulations [46]. Thus, some research groups have proposed various kinds of algorithms for

the deformation modelling, and the computational time can be reduced many times when using GPU

capabilities [47]. In addition, owing to many different bacteria species harboring on teeth, tongue,

gingiva, and the surrounding soft tissues, the oral environment is inhabited [48]. Over the past years,

numerous studies have shown that, compared with the traditional dental surgery, the computer-assisted

flapless surgery can decrease the incidence of surgery-related bacteremia, complications and

discomfort in the immediate postoperative period significantly [49, 50]. For the purpose of improving

the surgical skills, the training system may be beneficial for the surgeons to perform the flapless

surgery.

In order to transfer the preoperative planning to the actual surgical site accurately, the surgical

navigation system has been used in OMFS since 1990s. Under the interactive guidance of 2D and 3D

image rendering, the surgery can be performed without damaging the critical structures. According to

the characteristics of the localization sensors, the tracking systems can be classified into three

categories: 1. Optical active and passive infrared. 2. Electromagnetism. 3. Ultrasound. Among them,

the infrared tracking device is the most used in navigation system due to its high technical precision, in

the range 0.1-0.4mm [51]. The registration accuracy is of vital importance to the navigation system,

and the artificial point-based registration has a higher accuracy than the anatomic landmark-based

registration. In addition, surface-based registration is other alternative with markless technique, but has

low reliability [52]. Although the “rigid body” registration simplifies the registration process, it has

quite limited applicability since the deformation or distortion of anatomical and pathological structures

of interest are not considered between image acquisitions [53]. Currently, some non-rigid registration

algorithms have been proposed aiming at compensating for tissue deformation, or aligning images from

different subjects. For example, Marami et al. (2014) presented a non-rigid image registration

algorithm employing a dynamic FE-based linear elastic model of tissue deformation, which was

applicable to 3D-3D and 3D-2D, single and multi-modality image registration problems [54]. However,

the further development requires a higher accuracy for the actual clinical applications.

Even under the surgical navigation system, the possible human errors cannot be avoided during the

surgery. In recent years, surgical robot has been an ongoing active area of research for assisting the

surgeons to perform complex procedures, but most applications are now in the field of laparoscopic or

endoscopic surgery [55-57]. For example, with the support of several interactive robotic arms (Da

Vinci Xi®, Intuitive Surgical, Inc., Sunnyvale, 94086-5304, USA), the blood loss and complications can

be significantly reduced in spleen-preserving laparoscopic distal pancreatectomy [58]. Therefore, it has

a potential in terms of robot-assisted OMFS in the future.

6. Five-year view

In the past, owing to the anatomic complexity in the maxillofacial region, there was a big challenge

for the surgeons. Today, the computer-aided OMF surgery including surgical planning, simulation and

navigation has become increasingly important to improve the surgical precision, safety and reliability.

A variety of commercial and self-developed preoperative planning software tools have been applied

for the OMFS. However, since various imaging techniques, e.g. CT, MRI (Magnetic Resonance

Imaging) and PET (Positron Emission Tomography), can provide different information for the

treatment plan, it becomes increasingly important to combine these sources to a fused image data set in

one planning platform. In terms of the haptics-assisted surgical planning system, it enables analysis,

planning, and preoperative testing of alternative solutions for the surgery, especially when minimally

invasive techniques are used. Nevertheless, the simulation algorithms of deformable body with high

fidelity and efficiency is still under development. In addition, although the recent advanced haptic

feedback can provide six input DOFs and three output DOFs, the torque feedback is still not available,

which is a critical component in the surgical simulation systems.

During the image-guided surgery, one of the limitations is that the surgeons have to switch between

the actual operation site and computer screen which is inconvenient and impact the continuity of

surgery [59]. Nowadays, with the use of wearable augmented reality (AR) devices, the real objects and

virtual (computer-generated) objects are mixed in a real environment. Therefore, numerous researchers

are focusing on the development of AR-based surgical navigation system, and some pilot studies in the

field of OMFS have been reported. For example, Badiali et al. (2014) presented a head-mounted

system offering augmented reality information to the surgeons [60]. During the augmented-reality

assisted LeFort1 maxillary repositioning, the virtual planning was overlaid on a real patient which was

a significantly useful strategy for the operators. On the basis of phantom experiments, Vigh et al. (2014)

demonstrated that the accuracies of a navigation system for oral implantology do not differ

significantly using either an AR-head-mounted display or a monitor as a device for visualization [61].

However, as for the real clinical applications, it still requires the improvements not only in terms of the

accuracy and reliability, but also the comfort of wearable hardware device. In addition, it is significant

to simplify the workflow of surgical navigation system with novel minimal invasive approaches.

Key issues

The vision-based surgical planning system has proven as an efficient planning tool for oral and

maxillofacial surgery, but more specialized and user-friendly software platform is required.

Non-rigid registration method is an active area of research for 3D-3D and 3D-2D, single and

multi-modality image fusion in the surgical planning, simulation and navigation systems.

Although the existing virtual-reality-based surgical training system offers powerful simulation in

cutting and drilling procedures, it highlights the need to develop simulation algorithms of

deformable body with high fidelity and efficiency.

Clinical data demonstrates that the commercial and self-developed surgical navigation systems can

significantly minimize the risks and improve the precision of OMFS, but future research work is

ongoing to simplify the workflow of the surgical navigation system.

The augmented reality (AR) is a promising technology for the next generation of surgical

navigation system, but more intuitive and accurate result is required for real clinical applications.

Funding

This study was supported by the Natural Science Foundation of China 181171429, 81511300891,

Foundation of Science and Technology Commission of Shanghai Municipality 114441901002,

15510722200, 164419084001 and SJTU-KU Leuven Fund.

Declaration of Interest

The authors have no relevant affiliations or financial involvement with any organization or entity with

a financial interest in or financial conflict with the subject matter or materials discussed in the

manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert

testimony, grants or patents received or pending, or royalties.

Reference

Papers of special note have been highlighted as:

* of interest

** of considerable interest

1. Andersson L, Kahnberg KE, Pogrel MA. Oral and maxillofacial surgery.

WILEY-BLACKWELL, Oxford, United Kingdom (2010).

2. Chen X, Ye M, Lin Y, Wu Y, Wang C. Image guided oral implantology and its application in the

placement of zygoma implants. Computer Methods and Programs in Biomedicine 93, 162-173

(2009).

3. Hassfeld S, Zoller J, Albert FK, Wirtz CR, Knauth M, et al. Preoperative planning and

intraoperative navigation in skull base surgery. Journal of Cranio-Maxillofacial Surgery 26,

220-225 (1998).

* Early study demonstrates that 3D-visualizaiton, simulation of surgical procedures and

intraoperative navigation can effectively support the surgeon in diagnosis of skull base surgery.

4. Hassfel S and Muhling J. Computer assisted oral and maxillofacial surgery-a review and an

assessment of technology. Int. J. Oral Maxillofac. Surg. 30, 2-13 (2001).

* One of the first papers on the review of computer-assisted oral and maxillofacial surgery.

5. Nijmeh AD, Goodger NM, Hawkes D, Edwards PJ, McGurk M. Image-guided navigation in

oral and maxillofacial surgery. British Journal of Oral and Maxillofacial Surgery 43, 294-302

(2005).

6. Bell RB. Computer planning and intraoperative navigation in orthognathic surgery. J. Oral

Maxillofac. Surg. 69, 592-605 (2011).

7. Neugebauer J, Stachulla G, Ritter L, Dreiseidler T, Mischkowski RA, et al. Computer-aided

manufacturing technologies for guided implant placement. Expert Rev. Med. Devices 7,

113-129 (2010).

8. Zheng G, Wang L, Su Y, Liao G, Zhang S, et al. Maxillary reconstruction assisted by

preoperative planning and accurate surgical templates. Oral Surg. Oral Med. Oral Pathol. Oral

Radiol. 121, 233-238 (2016).

9. Shen P, Zhao J, Fan L, Qiu H, Xu W, et al. Accuracy evaluation of computer-designed surgical

guide template in oral implantology. Journal of Cranio-Maxillo-Facial Surgery 43, 2189-2194

(2015).

10. Komiyama A, Pettersson A, Hultin M, Nasstrom K, Klinge B. Virtually planned and

template-guided implant surgery: an experimental model matching approach. Clin. Oral Impl.

Res. 22, 308-313 (2011).

11. Sadiq Z, Collyer J, Sneddon K, Walsh S. Orthognathic treatment of asymmetry: two cases of

“waferless” stereotactic maxillary positioning. British Journal of Oral and Maxillofacial

Surgery 50, e27-e29 (2012).

12. Voss JO, Varjas V, Raguse JD, Thieme N, Richards RG, et al. Computed tomography-based

virtual fracture reduction techniques in bimandibular fractures. Journal of

Cranio-Maxillo-Facial Surgery 44, 177-185 (2016).

13. Ullah R, Turner PJ, Khambay BS. Accuracy of three-dimensional soft tissue predictions in

orthognathic surgery after Le Fort I advancement osteotomies. British Journal of Oral and

Maxillofacial Surgery 53, 153-157 (2015).

14. Zhan Q, Chen X. Boolean combinations of implicit functions for model clipping in

computer-assisted surgical planning. PLOS ONE 11, e0145987 (2016).

15. Chen X, Yuan J, Wang C, Huang Y, Kang L. Modular preoperative planning software for

computer-aided oral implantology and the application of a novel stereolithographic template: a

pilot study. Clinical Implant Dentistry and Related Research 12, 181-193 (2010).

16. Olsson P, Nysjo F, Hirsch JM, Carlbom IB. A haptics-assisted cranio-maxillofacial surgery

planning system for restoring skeletal anatomy in complex trauma cases. Int. J. CARS 8,

887-894 (2013).

* Interesting study of the use of haptic feedback in cranio-maxillofacial surgery planning system.

17. Gordon CR, Murphy RJ, Coon D, Basafa E, Otake Y, et al. Preliminary development of a

workstation for craniomaxillofacial surgical procedures: introducing a computer-assisted

planning and execution system. J. Craniofac. Surg. 25, 273-283 (2014).

18. Davidson E, Kumar AR. A preliminary three-dimensional analysis of nasal aesthetics following

Le Fort I advancement in patients with cleft lip and palate. The Journal of Craniofacial Surgery

26, e629-633 (2015).

19. Schaverien MV. Selection for surgical training: An evidence-based review. Journal of Surgical

Education 73, 721-9 (2016).

20. Escobar-Castillejos D, Noguez J, Neri L, Magana A, Benes B. A review of simulators with

haptic devices for medical training. J. Med. Syst. 40, 104-126 (2016).

** Critical review of recent medical training systems using haptic devices.

21. Heng PA, Cheng CY, Wong TT, Xu Y, Chui YP, et al. A virtual-reality training system for knee

arthroscopic surgery. IEEE Transactions on Information Technology in Biomedicine 8, 217-226

(2004).

22. Wu F, Chen X, Lin Y, Wang C, Wang X, et al. A virtual training system for maxillofacial surgery

using advanced haptic feedback and immersive workbench. Int. J. Med. Robotics Comput.

Assist Surg. 10, 78-87 (2014).

* Detailed description of a virtual surgical training system for maxillofacial surgery.

23. Wang D, Zhang Y, Hou J, Wang Y, Lv P, et al. iDental: A haptic-based dental simulator and its

preliminary user evaluation. IEEE Transactions on Haptics 5, 332-343 (2012).

24. Konukseven EI, Önder ME, Mumcuoglu E, Kisnisci RS. Development of a visio-haptic

integrated dental training simulation system. Journal of Dental Education 74, 880-891 (2010).

25. Pohlenz P, Grobe A, Petersik A, Von Sternberg N, Pflesser B, et al. Virtual dental surgery as a

new educational tool in dental school. Journal of Cranio-Maxillo-Facial Surgery 38, 560-564

(2010).

26. Tse B, Harwin W, Barrow A, Quinn B, Diego JS, et al. Design and development of a haptic

dental training system – hapTEL. EuroHaptics 2010, Part II, LNCS 6192, 101-108 (2010).

27. Wang S, Yang J. Simulating cranio-maxillofacial surgery based on mixed-element

biomechanical modelling. Computer Methods in Biomechanics and Biomedical Engineering 13,

419-429 (2010).

28. Mattei TA. Nonhomeomorphic topological transformations and the challenge of collision

detection in virtual reality simulation in neurosurgery. World Neurosurgery 81, 206-214 (2014).

29. Taylor ZA, Cheng M, Ourselin S. High-speed nonlinear finite element analysis for surgical

simulation using graphics processing units. IEEE Transactions on Medical Imaging 27,

650-663 (2008).

30. Cheng C, Cheng X, Dai N, Jiang X, Sun Y, et al. Prediction of facial deformation after complete

denture prosthesis using BP neural network. Computers in Biology and Medicine 66, 103-112

(2015).

31. Duan Y, Huang W, Chang H, Chen W, Zhou J, et al. Volume preserved mass-spring model with

novel constraints for soft tissue deformation. IEEE Journal of Biomedical and Health

Informatics 20, 268-280 (2016).

32. Xu S, Liu XP, Zhang H, Hu L. A nonlinear viscoelastic tensor-mass visual model for surgery

simulation. IEEE Transactions on Instrumentation and Measurement 60, 14-20 (2011).

33. Han L, Hipwell JH, Tanner C, Taylor Z, Mertzanidou T, et al. Development of patient-specific

biomechanical models for predicting large breast deformation. Phys. Med. Biol. 57, 455-472

(2012).

34. Camara M, Mayer E, Darzi A, Pratt P. Soft tissue deformation for surgical simulation: a

position-based dynamics approach. Int. J. CARS 11, 919-928 (2016).

35. Johnsen SF, Taylor ZA, Clarkson MJ, Hipwell J, Modat M, et al. NiftySim: A GPU-based

nonlinear finite element package for simulation of soft tissue biomechanics. Int. J. CARS 10,

1077-1095 (2015).

* Presentation of the GPU-based nonlinear finite element surgical simulation system.

36. Roberts DW, Strohbehn JW, Hatch JF, Murray W, Kettenberger H. A frameless stereotaxic

integration of computerized tomographic imaging and the operating microscope. J. Neurosurg.

65, 545-549 (1986).

37. Chen X, Lin Y, Wu Y, Wang C. Real-time motion tracking in image-guided oral implantology.

Int. J. Med. Robotics Comput. Assist Surg. 4, 339-347 (2008).

38. Gui H, Yang H, Shen SGF, Xu B, Zhang S, et al. Image-guided surgical navigation for removal

of foreign bodies in the deep maxillofacial region. J. Oral Maxillofac. Surg. 71, 1563-1571

(2013).

39. Li P, Xuan M, Liao C, Tang W, Wang X, et al. Application of intraoperative navigation for the

reconstruction of mandibular defects with microvascular fibular flaps-preliminary clinical

experiences. J. Craniofac. Surg. 27, 751-755 (2016).

40. Sun Y, Luebbers HT, Agbaje JO, Lambrichts I, Politis C. The accuracy of image-guided

navigation for maxillary positioning in bimaxillary surgery. The Journal of Craniofacial

Surgery 25, 1095-1099 (2014).

41. Stein KM. Use of intraoperative navigation for minimally invasive retrieval of a broken dental

needle. J. Oral Maxillofac. Surg. 73, 1911-1916 (2015).

42. Essig H, Dressel L, Rana M, Rana M, Kokemueller H, et al. Precision of posttraumatic primary

orbital reconstruction using individually bent titanium mesh with and without navigation: a

retrospective study. Head & Face Medicine 9, 18-25 (2013).

43. Li B, Zhang L, Sun H, Shen SGF, Wang X. A new method of surgical navigation for

orthognathic surgery: optical tracking guided free-hand repositioning of the maxillomandibular

complex. The Journal of Craniofacial Surgery 25, 406-411 (2014).

44. Strong EB, Rafii A, Holhweg-Majert B, Fuller SC, Metzger MC. Comparison of 3 optical

navigation systems for computer-aided maxillofacial surgery. Arch Otolaryngol Head Neck

Surg. 134, 1080-1084 (2008).

* Interesting study of the comparison of three commercially available surgical navigation systems

for maxillofacial reconstruction.

45. Chen X, Lin Y, Wang C, Shen G, Zhang S, et al. A surgical navigation system for oral and

maxillofacial surgery and its application in the treatment of old zygomatic fractures. Int. J. Med.

Robotics Comput. Assist Surg. 7, 42-50 (2011).

46. Marescaux J, Diana M. Next step in minimally invasive surgery: hybrid image-guided surgery.

Journal of Pediatric Surgery 50, 30-36 (2015).

** Critical review of the key technologies including virtual reality, augmented reality and surgical

robotics in minimally invasive surgery.

47. Idkaidek A, Jasiuk I. Toward high-speed 3D nonlinear soft tissue deformation simulations using

Abaqus software. J. Robotic Surg. 9, 299-310 (2015).

48. Arisan V, Bölükbaşi N, Öksüz L. Computer-assisted flapless implant placement reduces the

incidence of surgery-related bacteremia. Clin. Oral Invest. 17, 1985-1993 (2013).

49. Rousseau P. Flapless and traditional dental implant surgery: an open, retrospective comparative

study. J. Oral Maxillofac. Surg. 68, 2299-2306 (2010).

50. Hultin M, Svensson KG, Trulsson M. Clinical advantages of computer-guided implant

placement: a systematic review. Clin. Oral Implants Res. 23, 124-135 (2012).

51. Khadem R, Yeh CC, Sadeghi-Tehrani M, Bax MR, Johnson JA, et al. Comparative tracking

error analysis of five different optical tracking systems. Computer Aided Surgery 5, 98-107

(2000).

52. Sun Y, Luebbers HT, Agbaje JO, Schepers S, Vrielinck L, et al. Evaluation of 3 different

registration techniques in image-guided bimaxillary surgery. J. Craniofac. Surg. 24, 1095-1099

(2013).

53. Hill DLG, Batchelor PG, Holden M, Hawkes DJ. Medical image registration. Phys. Med. Biol.

46, R1-R45 (2001).

54. Marami B, Sirouspour S, Capson DW. Non-rigid registration of medical images based on

estimation of deformation states. Phys. Med. Biol. 59, 6891-6921 (2014).

55. Thaveau F, Nicolini P, Lucereau B, Georg Y, Lejay A, et al. Associated Da Vinci and Magellan

robotic systems for successful treatment of nutcracker syndrome. Journal of Laparoendoscopic

& Advanced Surgical Techniques 25, 60-63 (2015).

56. Morelli L, Guadagni S, Franco GD, Palmeri M, Caprili G, et al. Use of the new Da Vinci Xi®

during robotic rectal resection for cancer: technical considerations and early experience. Int. J.

Colorectal Dis. 30, 1281-1283 (2015).

57. Roviello F, Piagnerelli R, Ferrara F, Scheiterle M, Franco LD, et al. Robotic single docking total

colectomy for ulcerative colitis: first experience with a novel technique. International Journal

of Surgery 21, 63-67 (2015).

58. Liu Y, Ji WB, Wang HG, Luo Y, Wang XQ, et al. Robotic spleen-preserving laparoscopic distal

pancreatectomy: a single-centered Chinese experience. World Journal of Surgical Oncology 13,

275-283 (2015).

59. Chen X, Xu L, Wang Y, Wang H, Wang F, et al. Development of a surgical navigation system

based on augmented reality using an optical see-through head-mounted display. Journal of

Biomedical Informatics 55, 124-131 (2015).

60. Badiali G, Ferrari V, Cutolo F, Freschi C, Caramella D, et al. Augmented reality as an aid in

maxillofacial surgery: validation of a wearable system allowing maxillary repositioning. J.

Craniomaxillofac. Surg. 42, 1970-6 (2014).

* Presentation of the augmented-reality assisted maxillofacial bone surgery.

61. Vigh B, Muller S, Ristow O, Deppe H, Holdstock S, et al. The use of a head-mounted display in

oral implantology: a feasibility study. Int. J. CARS 9, 71-78 (2014).

Figure 1. Planning of three types of Le Fort fractures in orthognathic surgery using SmartModelClip

module based on 3D Slicer platform. (A) The main window of the graphic user interface of the module.

(B) Clipping of Le Fort I fractures. (C) Clipping of Le Fort II fractures. (D) Clipping of Le Fort III

fractures.

Figure 2. (A) A virtual training system for maxillofacial surgery using advanced haptic feedback of

Omega.6 and immersive workbench of Display 300. (B) Simulation for maxilla cutting. (C) Simulation

for fixation of titanium plate in the maxilla.

Figure 3. A clinical case application for the treatment of temporomandibular joint ankylosis using

BrainLab surgical navigation system. (A) Preoperative picture of a five-year-old girl with the mouth

opening of 8mm. (B) Acquisition of preoperative panoramic digital X-Ray image. (C) Volume

rendering of the CT scanning data. (D) Evaluation on the 3D printed model. (E) Fixing the reference

frames on the patient for the intra-operative navigation procedure. (F) Intra-operative picture of the

surgery. (G) Real-time navigation. (H) The screenshot of the navigation software. (I) The postoperative

picture of the patient with the mouth opening of 30mm.

Table 1. Some clinical applications of using commercially available softwares for preoperative planning

in OMFS.

Authors Type of surgery Planning system Distributor

Zheng et al. [8] Maxillary reconstruction Mimics 10.01 Materialise, Leuven,

3001, Belgium

Shen et al. [9] Oral implantology Simplant Pro

11.04

Dentsply, York, 17401,

USA

Komiyama et

al. [10]

Oral implantology Procera® Nobel Biocare,

Zürich-Flughafen,

CH-8058, Switzerland

Sadiq et al. [11] Orthognathic surgery VoXim® IVS Technology GmbH,

Chemnitz, 09125,

Germany

Voss et al. [12] Bimandibular fractures Amira FEITM, Hillsboro, 97124,

USA

Ullah et al. [13] Orthognathic surgery 3dMD Vultus 3dMD LLC, Atlanta,

30339, USA

Table 2. Some applications of surgical simulation and training systems in OMFS.

Authors Haptic device Simulator Distributor

Wu et al. [22] Omega.6, Force Dimension, Nyon,

CH-1260, Switzerland

VR-MFS Shanghai Jiao Tong

University, Shanghai, China

Wang et al. [23] Phantom Omni, Sensable,

Wilmington, 01887, USA

iDental Beihang University, Beijing,

China

Konukseven et al.

[24]

Sensable Phantom Premium 1.5,

Sensable, Wilmington, 01887, USA

Vision-Haptic Device

Integrated Dental Training

Simulation System

Middle East Technical

University, Ankara, Turkey

Pohlenz et al. [25] Phantom Omni, Sensable,

Wilmington, 01887, USA

Voxel-Man University Medical Center

Hamburg-Eppendorf, Hamburg , Germany

Tse et al. [26] Phantom Omni, Sensable,

Wilmington, 01887, USA

hapTEL University of Reading,

Berkshire, UK

Table 3. Some applications of surgical navigation systems used in OMFS.

Authors Type of surgery Navigation

system

Distributor Accuracy

Gui et al. [38] Removal of foreign bodies in

the deep maxillofacial region

STN navigation

system

Stryker®, Kalamazoo,

49002, USA

Less than 0.8mm

Li et al. [39] Reconstruction of

mandibular defects

BrainLab

navigation system

BrainLab AG,

Feldkirchen,

85622,Germany

Mean linear error of

3.4±1.3mm

Sun et al. [40] Maxillary positioning Kolibri navigation

system

BrainLab AG,

Feldkirchen, 85622,

Germany

0.44±0.35mm

(Sagittal)

0.50±0.35mm

(Vertical)

0.56±0.36mm

(Mediolateral

direction)

Stein [41] Retrieval of broken dental

needles

Medtronic

StealthSatation®

S7

Medtronic, Minneapolis, 55432-5604, USA

Improved

Essig et al. [42] Orbital wall reconstruction VoXim® IVS Technology GmbH,

Chemnitz, 09125,

Germany

-0.05±0.70mm

(Medial orbital wall)

0.27±0.70mm

(Orbital floor)

-0.23±0.75mm

(Lateral orbital wall)

Chen et al. [2] Oral implantology Image guided oral

implantology

system (IGOIS)

Shanghai Jiao Tong

University, Shanghai,

200240, China

1.36±0.24mm (Entry

point)

1.57±0.24mm (Exit

point)

4.1°±0.90° (Angle)

Li et al. [43] Repositioning of the

maxillomandibular complex

AccuNavi 2.0 Shanghai UEG Medical

Devices Co., Ltd,

Shanghai, 200135, China

0.76mm-1.12mm

(Vertical)

0.56mm-0.94mm

(Axial)

0.39mm-0.58mm

(Horizontal)