Virtual reality simulation of surgery with haptic feedback based on the boundary element method

www.elsevier.com/locate/compstruc

Computers and Structures 85 (2007) 331–339

Virtual reality simulation of surgery with haptic feedback basedon the boundary element method

P. Wang a, A.A. Becker a,*, I.A. Jones a, A.T. Glover b, S.D. Benford b,C.M. Greenhalgh b, M. Vloeberghs c

a School of Mechanical, Materials, Manufacturing Engineering and Management, University of Nottingham, Nottingham NG7 2RD, UKb School of Computer Science and Information Technology, University of Nottingham, Nottingham NG7 2RD, UK

c Academic Division of Child Health, School of Human Development, University of Nottingham, Nottingham NG7 2RD, UK

Received 9 November 2006; accepted 23 November 2006Available online 19 January 2007

Abstract

A virtual reality real-time simulation of surgical operations has been developed using the boundary element (BE) method. Othernumerical techniques and related approaches to real-time modelling of deformable objects are briefly reviewed. The challenges raisedby a key application – the simulation of neurosurgery – are identified. A brief review of the BE method is presented followed by a descrip-tion of its implementation for three basic actions of prodding, pinching and cutting deformable objects. An initial implementation ofthese techniques is described in which haptic and visual feedbacks are generated when these operations are carried out on simple deform-able objects.� 2006 Elsevier Ltd. All rights reserved.

Keywords: Boundary element method; Virtual reality; Surgery simulation; Haptic feedback; Cutting; Real-time

1. Introduction

Advanced haptic interfaces require techniques for simu-lating the deformation of complex objects as a result ofusers’ interactions through haptic devices. A variety ofsuch techniques have been developed, most notably theuse of the finite element (FE) method to model the struc-ture of a deformable object. However, the boundary ele-ment (BE) method is now well established as an accuratecomputational mechanics technique with its unique featureof surface-only modelling for elastic problems. The BEmethod has many attractive features that are suitable forimplementation in virtual reality simulations which requirea real-time response to the forces applied to a deformableobject.

This paper explores the potential of the BE method tosupport haptic interfaces and real-time simulation of defor-

0045-7949/$ - see front matter � 2006 Elsevier Ltd. All rights reserved.

doi:10.1016/j.compstruc.2006.11.021

* Corresponding author. Tel.: +44 115 951 3791; fax: +44 115 951 3800.E-mail address: [email protected] (A.A. Becker).

mations. The simulation of neurosurgery is chosen as achallenging driving application for this work as this requirestrainees to carry out a variety of actions – prodding, pinch-ing and cutting – on models of the highly deformable softtissues of the brain. It is also potentially a very benefi-cial application of haptic technologies. In this paper, theBE method is briefly reviewed in relation to previousapproaches, the requirements of neurosurgical simulationsare discussed and the use of the BE formulation to providehaptic feedback is presented. The paper is an extended ver-sion of the authors’ conference paper presented at the sev-enth international conference on computational structurestechnology, Lisbon, 7–9 September 2004 [1].

The paper is organised as follows. A brief review of thetechniques used for simulating deformable objects is pre-sented in Section 2, followed by a discussion in Section 3of the main challenges and difficulties encountered in sim-ulating surgical procedures in real-time. Section 4 presentsan overview of the BE formulation for displacement andtraction integral equations, while Section 5 summarises

mailto:[email protected]

332 P. Wang et al. / Computers and Structures 85 (2007) 331–339

the pre-solution BE technique used to simulate proddingand cutting. An overview of the implementation of the hap-tic simulation system is given in Section 6, followed by ageneral discussion and conclusions.

2. Techniques for simulating deformable objects

Various approaches have been proposed for modellingdeformable objects, many of them driven by the desire todevelop surgical simulators. James and Pai [2] and Gibsonand Murtich [3] review much of the published work,including several papers by Terzopoulos [4] and his co-workers who presented important early work on deform-able models. Techniques for modelling deformable objectsin computer simulations have included simple non-physicalmodels using splines and patches, and physical modelsusing mass-spring techniques.

Of greater interest here are the methods based uponcomputational continuum mechanics techniques, notablythe FE method [5]. There is an established literature onapplying the FE method to surgical simulation. Bro-Niel-sen [6] describes a number of approaches to the speeding-up of the FE process for use within surgical simulations.These include the condensation of a solid FE model to con-sider only the surface degrees of freedom and the use of theinverse of the stiffness matrix for repeated multiplicationswith the force vector. However, the cutting simulationwas not considered in the work because the condensationand inverse stiffness matrix would become invalidatedimmediately after cutting. Zhuang and Canny [7] discusstheir simulation system based upon a geometrically non-linear FE model to allow a human operator to performreal-time interaction with soft 3D objects that undergolarge global deformations, achieving the high update fre-quency (over 1000 Hz) required for force feedback simula-tion by interpolating between simulated states. Woodcocket al. [8] describe the use of adaptive mesh refinement/coarsening to achieve interactive rates with complexdeformable and fracturable FE models in a virtual surgicalenvironment. Song and Reddy [9] study the feasibility ofsimulating cutting in a virtual surgical environment. Ingeneral, the FE models have been thought to be too slowto achieve visual and haptic feedback in real-time.

The focus of this paper is on an alternative computa-tional mechanics technique, the BE method (see e.g., [10–12]). The BE method differs significantly from the FEmethod in that only the surface is discretised in order toapply boundary conditions to the continuum equationsgoverning the behaviour of the body bounded by the discre-tised surface. In elastic problems, the interior of the body isnot discretised, thus reducing the size of the system equa-tions to be solved, and also avoiding the need to generateinterior node and element data which do not contribute tosatisfying the requirements of a VR model. These factorsraise the intriguing possibility that the BE method mightoffer a more computationally scalable approach to model-ling real-time deformable objects than the FE method.

Very little has been published on the use of BE analysisfor real-time simulation applications. Examples of suchwork are James and Pai [2], Monserrat et al. [13] whichboth identify surgical simulation as a primary or possibleapplication. They also describe BE simulations of irregularobjects subjected to arbitrary ‘‘poking’’ loads. However,none of this published work has as yet considered the issuesof the cutting of a solid, large deformations, contactbetween flexible bodies and the enclosure of the object ina tough membrane, all of which are important consider-ations in surgical simulations. These issues would thereforeappear to be the critical ones that need to be addressedbefore BE-based surgical simulation can become a reality.Some aspects of the present BE surgery simulator havebeen published earlier [14,15].

3. The challenges of neurosurgical simulation

The long-term aim for this work is to develop a simula-tion system for the neurosurgery process, incorporating adeformable model of the tissue being operated upon,together with visual and haptic feedback for maximumrealism. This is intended both to aid the training of sur-geons and the planning of operations. These skills are cur-rently developed over years of surgery training andpractice. The training process involves gathering theoreti-cal knowledge, assisting an experienced surgeon, beingassisted by an experienced surgeon and ultimately comingto the ‘‘solo’’ experience. National standards (e.g., thoseof the UK’s Royal College of Surgeons) require trainee sur-geons to be exposed to and to perform an appropriatenumber and variety of operations before being ‘‘signedoff’’ as competent. In certain specialties, however, the num-ber of cases is relatively small and there may be limitedopportunities to achieve this. Surgical simulation is there-fore an attractive alternative and can take several differentforms:

(i) Operation planning tools such as [16], where there isno attempt to simulate the physical movements andforces occurring during the operation.

(ii) Simulation of operations on rigid structures suchas bones, prostheses and bone grafts such as [17]which simulates a wide range of procedures on thesestructures.

(iii) Simulation of surgery on deformable objects repre-senting human organs and soft tissues, either forthe simulation of the surgical process itself [18–20]or to understand the effects of surgery upon deform-able tissues [21]. Again, a wide range of surgical pro-cedures need to be simulated, including prodding,displacing, lifting and cutting.

Viewed from the modeller’s (rather than the surgeon’s)perspective, a simulation of the surgery process on anorgan, such as the brain, based upon the BE method must

P. Wang et al. / Computers and Structures 85 (2007) 331–339 333

be capable of representing the following features, listedapproximately in order of difficulty:

(i) Indentation of the tissue with a sharp object such as ascalpel or forceps prior to cutting, with visual feed-back at a suitable refresh rate (say 25–50 Hz) forthe whole body and haptic feedback at (say)1000 Hz for the points where interaction takes place.

(ii) Self-weight of the tissue – i.e., so that it naturallydeforms under its own weight.

(iii) Retraction of the cut tissue using a shaped andapproximately rigid implement having a finite surface.

(iv) Cutting of the tissue, either using a scalpel or (in thecase of neurosurgery) using a forceps-like device, thuscreating new surfaces resulting in additional deforma-tions due to the change in structural behaviour of thetissue under existing loads.

(v) The contact of the tissue with itself and with itssurroundings.

The approach adopted in this project is to tackle theserequirements approximately in the order given above, ini-tially making use of simplified test cases to try out ideasalong the way.

4. BE analysis of deformable objects

The deformable object is assumed to be a linear elasticbody and deformations are assumed to be small comparedto the object size, i.e. the superposition principle can beapplied.

BE analysis relies upon the existence of surface integralequations to model the continuum behaviour. A displace-ment integral equation (BIE) describes the displacementat a source point p (also called ‘‘load’’ point) within thedomain of the problem due to tractions t and displace-ments u on a surface point Q (also called a ‘‘field’’ point)on the surface S of a volume V as well as the contributionof body forces f as follows:

uiðpÞ þZ

ST ijðp;QÞujðQÞdSðQÞ

¼Z

SUijðp;QÞtjðQÞdSðQÞ þ

ZV

Uijðp; qÞfjðqÞdV ðqÞ ð1Þ

where Uij and Tij are called the displacement and tractionkernels, respectively, and are functions of the relative posi-tions of the points p and Q and the material properties butare independent of the boundary conditions. In threedimensional problems, they are defined as follows (see,e.g., Brebbia et al. [10], Becker [11] and Banerjee [12]):

U ijðp;QÞ ¼1

16pð1� mÞl1

r

� �ð3� 4mÞdij þ

oroxi

oroxj

� �T ijðp;QÞ ¼

�1

8pð1� mÞr2

oron

� �ð1� 2mÞdij þ 3

oroxi

oroxj

� �þ 1� 2m

8pð1� mÞr2

oroxj

ni �oroxi

nj

� � ð2Þ

where r is the distance between points p and Q, n is theoutward normal at the field point Q, m is the Poisson’sratio, and l is the shear modulus. All integrals are surfaceintegrals, except for the last integral in Eq. (1) which is avolume integral containing body forces fj(q).

In the case of an organ, its initial load is the gravita-tional body force and cannot be ignored. The gravitationalfield is represented by q, g, which is a constant body forceper unit volume, in which q is a mass density and g is thegravitational acceleration. Using Green’s functions, thevolume integral in Eq. (1) can be transformed into a sur-face integral, thus preserving the surface-only modellingcapability of the BE method, as follows:Z

VU ijðp; qÞfjðqÞdV ðqÞ ¼

ZS

P iðp;QÞdSðQÞ ð3Þ

in which,

P iðp;QÞ ¼1

8plfi

oroxm

nm �ni

2ð1� mÞ fmoroxm

� �ð4Þ

Therefore, for an elastic problem all integrals are evaluatedon the surface, and the interior of the domain does nothave to be discretised. In BE analysis, the surface is discret-ized into elements, and the displacements and tractionsover each element are described by an interpolation shapefunction. For simplicity, constant triangular surface ele-ments are used in this work where the displacement is as-sumed to be linear over each element and the elementscan be represented by a node placed at the centroid ofthe triangle. Other more sophisticated surface elementscan also be used such as quadratic eight-noded quadrilat-eral surface elements. The simplicity of the triangular sur-face elements means that fine meshes are needed tomodel awkward geometries.

By taking each node on the boundary in turn as the loadpoint p and performing the numerical integrations over allsurface elements in Eq. (1), a set of linear algebraic equa-tions is obtained as follows:

½A�fug ¼ ½B�ftg þ fRGg ð5Þwhere [A] and [B] are matrices that contain the numericalintegrations involving Tij and Uij, respectively, and {RG}contains the gravitational load obtained from the numeri-cal integrations in Eqs. (3) and (4).

After applying the constraint and load conditions, therelevant elements of the matrices [A] and [B] are swappedsuch that all the unknowns end up on the left hand sideof the equation and the known variables are on the righthand side, and Eq. (5) becomes

½A��fxg ¼ fRbcg þ fRGg ð6Þ

where [A*] is the solution matrix, {x} contains all unknownvariables (either displacements or tractions), and {Rbc}contains all prescribed values of displacements and trac-tions. After solving the algebraic equations, all tractionsand displacements on the boundary are known.


The stresses at any point p can be obtained by differen-tiating Eq. (1) for the displacements at point p and usingHooke’s law as follows:

rijðpÞ þZ

SSkijðp;QÞukðQÞdSðQÞ

¼Z

SDkijðp;QÞtkðQÞdSðQÞ þ

ZV

Dkijðp; qÞfkðqÞdV ðqÞ ð7Þ

Again, the volume integral involving gravitational bodyforces in Eq. (7) can be transformed into a surface integral,and written as follows:

rijðpÞ þZ

SSkijðp;QÞukðQÞdSðQÞ

¼Z

SDkijðp;QÞtkðQÞdSðQÞ þ

ZS

W ijðp;QÞdSðQÞ ð8Þ

where Skij and Dkij are stress kernels, see, e.g., Becker [11],and the gravitational term is expressed as follows:

W ijðp;QÞ ¼1

8proroxm

nm fioroxjþ fj

oroxi

� ��þ 1

1� mmdij

oroxm

nmfsoroxs� fmnm

� �� 1

16pð1� mÞr fmoroxm

nioroxjþ nj

oroxi

� ��þð1� 2mÞðfinj þ fjniÞ

�ð9Þ

5. Algorithms for prodding and cutting

This section presents a description of how the BEmethod can be used to model the actions of prodding(indenting), pinching and cutting. Unlike FE analysis, BEanalysis does not discretise the volume of the continuumbut instead relies upon the existence of surface integralequations to model the continuum behaviour. This is a dis-tinct advantage of the BE method when cutting progresses,since the propagation of a crack or a cut only locally affectsthe elements at the crack tip, whereas in FE analysis thiswould involve a substantial re-meshing of the inter-con-nected domain mesh around the crack tip. In a BE formu-lation, as cutting progresses, the system matrix is graduallyextended by the addition of a few more rows and columns.

5.1. Pre-solutions and superposition

When a surgery tool contacts an organ, it can be repre-sented as a non-zero displacement being applied on thedeformable model. Prior to the real-time simulation, alook-up table or a set of ‘‘pre-solutions’’ can be createdby applying, respectively, in x, y and z directions, a hypo-thetical unit displacement on all surface nodes in turn andsolving the system of equations without gravitationalloads. This results in the creation of a matrix (say [Up]),or a look-up table, for storing the relevant deformations,of the same size as the solution matrix [A*] and a matrix

(say [Tp]) for storing tractions. For a linear system, thesuperposition principle, together with the look-up table,can be used to arrive at the deformations resulting fromany set of applied displacements.

The initial surface of the object is usually finely meshedwith a large number of polygons (triangles), so that prod-ding or pinching can be applied at any point on the surfaceof the object. Fortunately, in practice the surgical opera-tion is usually focussed on only a small part of the organand the location of this small part can be identified by sometype of MRI scan before the real operation starts. In orderto reduce the total number of degrees of freedom, the sur-face of the object can be divided into several sub-regions,with only one sub-region (the part being operated on) beingfinely meshed while the other sub-regions can remain rela-tively coarse-meshed. Uneven meshing is a common mod-elling strategy in BE or FE analysis which requires thatadjacent element sizes gradually, not abruptly, change.

When the haptic device indents the object, the model isinstantly deformed while adding any existing displacementsdue to the gravitational load. The pre-solution methodassociates the least computational cost with rendering thedeformations in real-time, as the method involves retriev-ing numbers from a look-up table with a relatively smallnumber of mathematical operations.

5.2. Prodding algorithm

During the virtual (real-time) simulation, the node orelement being touched is first identified using conventionalray-intersection techniques as implemented in the Open-SceneGraph graphics library [22]. Assuming that the hapticdevice is moved to become in contact with element node j

and continues its motion to penetrate the undeformed sur-face by a movement vector d, the relevant pre-solutions arelocated in the columns (3j � 2), (3j � 1) and (3j) in the pre-solution matrix [Up]. The displacements of the object due tothe prodding can then be calculated as follows:

dispðiÞ ¼ dxU pði; 3j� 2Þ þ dyUpði; 3j� 1Þ þ dzU pði; 3jÞ;i ¼ 1; n ð10Þ

where n is the total number of degrees of freedom.These displacements are added to the displacements

from the self-weight. Similarly, the tractions at the touchedpoint can be calculated via the relevant stored tractionmatrix [Tp] and scaled by the above prodding components.These tractions are then converted into feedback forces bymultiplying them by the area of the touched element andreversing their sign. Pinching is the same as proddingexcept that the signs of the displacements and tractionsare reversed.

Alternatively, a surgical tool may be defined as anapproximately rigid implement having a finite surface andmathematically described as a plane polygon. This can beused, for example, to represent the retractors used to sepa-rate the cut tissue. As points inside the retractor polygon


touch any surface elements of the object, the surface ele-ments will experience the same movement as the rigidretractor. Identifying these elements requires calculationof intersections between the retractor polygon and the ele-ment triangles. Once the contacted elements are identified,the pre-solution look-up table can be used to obtain thedeformed model.

As an example, for a problem with 1600 degree of free-dom, the CPU time for obtaining the model deformationsand feedback forces using a look-up table is less than 10�7 susing double precision (real*8) parameters.

5.3. Cutting algorithm

5.3.1. Creation of a cut

Practical surgery involves the creation of a cut, similarto a crack in fracture mechanics, followed by the creationof new surfaces as cutting continues. To model an idealisedcrack, i.e., with an infinite stress at the crack tip, the ‘‘dual’’BE implementation, see e.g., [23,24], can be employed tomodel the crack propagation. However, this would requirea substantial amount of computation time, not currentlyfeasible in real-time.

In brain surgery, the cutting process is performed usinga special pair of forceps operating on the principle of dia-thermy (high-frequency electric current passing betweenthe tips). The aim is to pinch the brain tissue to be removedbut not to close the forceps completely. The material,which has been softened by the forceps, is sucked awaywith a suction tube. This suggests that it is reasonable totreat a crack as a small gap rather than a perfect crack withinfinite stress at the crack tip. When cutting starts, a gapcan be introduced where, in addition to new surface ele-ments created in the new cut surfaces, the geometries of afew existing surface elements around the cut must be alsomodified, see Fig. 1.

The cut simulation is initially carried out in an approx-imately straight line direction to a small depth perpendicu-lar to the surface, i.e., the cut is performed in anincremental piecewise linear direction determined by thedirection of the cutting tool contacting the model.

5.3.2. Extending the cut

If the surface surrounding the cut is finely meshed withsmall triangular surface elements, the gap is assumed to

Gap created to represent a cut

Fig. 1. A schematic representation of a cutting operation.

occur at the interface of the elements, without the needto sub-divide the surface elements. Along the cut depthdirection, two new surfaces are created and divided into tri-angular elements sharing some edges and nodes with thesurface elements. A new surface is also created at the bot-tom of the cut parallel to the original surface.

The starting point of the cut is first identified as thenearest node to the cutting tool. The travelling directionof the cutting tool is determined by checking the dot prod-ucts of the cutting tool direction and the edges of all theelements connected to the start node. The edge with thelargest dot product with the cutting tool direction is identi-fied as the cutting edge.

A small gap is created between the identified edges. Thecut is assumed to propagate in a perpendicular direction tothe surface. The cut proceeds in increments equal to theaverage size of the surface edges. A new mesh topology isused to create new elements on both sides of the cuttingsurface and on the bottom surface of the cut. Fig. 2 showsexamples of initiating and extending a cut.

The extension of the cut to a new depth is simulated bycreating more elements and nodes along the cut surfacesand adjusting the topology of the relevant elements.

5.3.3. Extending the system of equations

In the first stage of cutting, the two new cut surfaces arestill in contact with each other. New elements are createdonly in these new surfaces and the solution matrix isexpanded by adding extra rows and columns related tothe newly created elements as shown in Fig. 3.

An efficient algorithm based on the inverse of the initialsystem matrix can be developed by partitioning (see, e.g.,[25]). The new solution matrix [A*] is composed of the ini-tial solution matrix ½A�0� and the expanded partitions P, Qand S, as follows:

½A�� ¼A�0 Q

P S

� �ð11Þ

The initial inverse matrix ½A�0��1 is computed before simula-

tion starts and stored. To find the new inverse matrix ½A��1

for the new system matrix equation, matrix [A*] can be par-titioned in the same manner:

½A��1 ¼eA eQeP eS

" #ð12Þ

½A��1 can then be found using inversion by partitioning[25], as follows:eA ¼ ½A�0��1 þ ð½A�0�

�1QÞ � ðS � P ½A�0��1QÞ�1 � ðP ½A�0�

�1ÞeQ ¼ �ð½A�0��1QÞ � ðS � P ½A�0��1QÞ�1

eP ¼ �ðS � P ½A�0��1QÞ�1 � ðP ½A�0�

�1ÞeS ¼ ðS � P ½A�0��1QÞ�1

ð13Þ

Each step of cutting adds only a few elements, so that sub-matrices P, Q and S are relatively small compared to [A*].The parentheses in Eq. (13) highlight repeated factors and

Old matrix

New rows

New columns

Fig. 3. A schematic representation of the expansion of the BE systemmatrix following a cut.

Fig. 2. Examples of cutting simulations: (a) initiation of a cut and (b) extension of a cut.


are computed only once. The floating-point operations inthis case are limited to a few matrix multiplications. Defin-ing one float operation (flop) as consisting of one multipli-cation or division followed by one addition, the abovealgorithm involves (2n2m) flops, where n is the size of theold matrix and m is the number of new added rows or col-umns. If the new added size is (1/200) of the old size, thisalgorithm involves (1/100)n3 flops compared to n3 for usingthe LU decomposition method. Therefore, for small sizeproblems (500–800 degrees of freedom), it becomes possi-ble to complete the above algorithms in real-time.

However, if cutting extends beyond an initial narrow‘‘crack’’, the cutting action involves the modification of asmall number of elements as well as the creation of new ele-ments on the cut surfaces. Therefore, some rows and col-umns in the old portion of the matrix need to bemodified by re-integration on the relevant modified ele-ments, and the inverse of the initial system matrix, ½A�0�

�1,

Table 1Comparison of CPU time in seconds and iterations

Number of equations, n Simulation GMRES CPU (iteration

1080 New cut 0.06250 (3)Extended cut 0.14062 (9)



becomes invalidated. In this case, iterative methods areemployed to solve the equations as a new problem. Regard-ing the choice of iterative method, numerical experimentson the most popular iterative methods for non-symmetricsystems, e.g., GMRES [26], BiCGStab and BiCG [27], sug-gest that GMRES without restart is the most effectiveapproach as it required fewer iterations and less CPU timethan the others. This is consistent with the conclusion byJames and Pai [2] who performed numerical experimentswith MALAB. Iterations can be further reduced if the rel-evant pre-solutions are used as the initial guess for the iter-ation. Table 1 lists the CPU time and iterations evaluatedby different methods for several models experiencing cut-ting. It shows that the CPU time associated with up ton = 1500 is likely to be under 0.25 s using a high specifica-tion PC (3.06 GHz single processor, 1 Gb RAM).

6. Implementation of a haptic simulation incorporating

BE analysis

The above BE formulation has been implemented in asuite of computer programs resulting in a first basic simu-lation of deformable objects that supports the core actionsof prodding, pinching and cutting. The programming lan-guages used are Fortran (for the BE computations) andC++ (for the virtual reality simulation). The simulationprovides both real-time visual and haptic feedback. Fig. 4summarises this implementation.

The modelled object is represented in the system as a tri-angular surface mesh, which is free to deform under theinfluence of an appropriate tool, which represents the dif-

s) BiCGStab CPU (iterations) BiCG CPU (iterations)

0.234375 (8) 0.234375 (12)0.25000 (9) 0.296875 (14)

0.453125 (11) 0.578125 (20)0.468750 (11) 0.609375 (21)

0.890625 (13) 0.93750 (20)1.140625 (18) 0.93750 (21)

Model DataGraphical Display

BEM AlgorithmsHaptic Process

Phantom(x2)

Indenter

Force


BE AlgorithmsHaptic Device

Phantom (x2)

Indenter

Force

Geometry and material Properties


BEM AlgorithmsHaptic Process

Phantom(x2)

Indenter

Force


BE AlgorithmsHaptic Device

Phantom (x2)

Indenter

Force

Geometry and material Properties

Fig. 4. Schematic representation of the surgical simulation system.


ferent surgical operations within the simulation. In the cur-rent implementation, this tool may be used to perform thefollowing three actions.

(i) Prodding – The model deformations occur when theindenter has been deemed to have contacted theobject surface. The amount the applied indentationof the haptic device penetrates below the surfacedetermines the extent to which the surrounding poly-gons of the model are displaced. When the indenter isoutside the original confines of the model, contact isdeemed to be no longer taking place and the objectreverts to its original shape.

Fig. 5. Examples of prodding, pinching and cutting sim

(ii) Pinching – After surface contact has been deter-mined, deformations occur with reference to this ini-tial point of contact. This is akin to ‘‘pinching’’ anobject and manipulating or contorting it; it uses thesame BE technique as for prodding, but introducestensile rather than merely compressive forces.

(iii) Cutting – The tool is used to propagate a ‘‘cut’’within the model. The direction of the cutting toolprior to contact is used to determine the directionof the cut within the model. The cut travels throughthe model for a predetermined distance relating tothe surrounding mesh size and new facets are gener-ated and rendered according to the cutting algorithmdescribed above. Cutting is progressed incrementallyto either extend the length of the cut or to make itdeeper.

Fig. 5 shows examples of prodding, pinching and cuttingvarious simple objects. The current implementation man-ages real-time two-dimensional and three-dimensionaldeformation and cutting of objects. In Fig. 5, the imageson the left show simple meshes representing a cuboid, whilethe images on the right show more complex refined meshesof a 3D brain geometry.

The BE simulation system comprises a dual-processor1.8 GHz Xeon PC with a nVidia Quadro4 700XGL graph-ics card, although the application is capable of running onmachines of a much lower specification. The graphical sub-system has been implemented using the OpenSceneGraphgraphics library [22]. Haptic feedback is provided by aPhantom haptic device. The haptic process runs at approx-imately 1000 Hz and is responsible for updating thetool position and button state as well as generating the

ulations: (a) prodding, (b) pinching, and (c) cutting.


appropriate force feedback. The button on the Phantom sty-lus is used to either initiate the pinching action or performthe cutting operation. The haptic software used is a subsetof the GHOST (General Haptic Open Software Toolkit)API and provides the force feedback generation and updat-ing of the indenter position. The haptic and rendering pro-cesses are synchronised such that the model geometry isrendered before the next indenter position update occursand possibly causes further vertex displacements.

The current development test suite wraps the simulationin an interface that supports the configuration of variousparameters and display of the relevant forces generated.Key parameters include the rigidity of the model (i.e., itsmaterial properties), the ability to turn the haptic forceson/off, choosing the action performed by the surgical tool,and controlling the rendering of the model (solid, wire-frame, points). This interface is being utilised during thedevelopment stage of the project, where there is a need toalter the general configuration of the model’s behaviour.

7. Discussion and conclusions

Driven by the ultimate aim of creating an effective neu-rosurgical simulation, this work explores whether the BEmethod offers a potentially more scalable alternative tothe FE method for generating haptic and visual feedbackwhen different actions are applied to deformable objects.A review of the BE formulation is presented together withsome initial solutions to modelling the actions of prodding,pinching and cutting. An implementation of these solutionshas also been described. It is shown that the BE technique,with its unique surface-only modelling, has considerablepotential for virtual reality surgery simulation that requirereal-time deformations and surface re-meshing duringcutting.

For prodding and pinching simulation by using pre-cal-culated solutions, the CPU time is less than 10�7 s indicat-ing that much larger models can be used in this kind ofsimulation. However, associated with cutting, the numberof surface elements is limited in order to meet the require-ment of real-time (in practice, a neurosurgeon would takeabout 0.25 s to perform each incremental stage of the sur-gical cutting). Fortunately, in practice, the surgical opera-tion is usually focussed on only a small part of the organand the location of this small part can be identified by sometype of MRI scan before the real operation starts. In orderto reduce the total number of BE surface elements, the sur-face of the object can be divided into several sub-regions,with only one sub-region (the part being cut) being finelymeshed while the other sub-region can remain relativelycoarse meshed.

When cutting occurs, the surface geometry changes andnew surfaces are created. The current stored pre-solutionswould need to be modified. In principle, this requires thecomputation of a new set of pre-solutions based on thenew ‘cut’ geometry, a process which cannot at present becarried out in real-time, as it would require numerous

equation solving operations which would take minutes,even hours, rather than seconds. To maintain continuityof the haptic feedback at the high (around 1000 Hz) refreshrate required, currently the existing pre-solutions are stillemployed and modified using an interpolation techniqueto produce the new part of these pre-computed solutionsassociated with new elements and nodes. However, a moreaccurate solution can be obtained by using an iterativeequation solving method that uses the previous solutionas the initial guess. Such an approach would cost moreCPU time, of the order of 0.25 s in the current trial tests.

The current surgical simulator simplifies the analysis byassuming a linear elastic behaviour and utilising a smalldeformation theory, thus allowing the use of the superposi-tion principle which incorporates the pre-solution approach.However, to provide a more realistic haptic feel and moreaccurate deformations of human organs, more complexnon-linear formulations that also incorporate large defor-mations and pre-stresses have to be considered. To date, ithas not been possible to incorporate such non-linear BE for-mulations in real-time applications.

A possible solution to speeding up the updated pre-solu-tions for cut surfaces would be to implement parallel com-puting as a background task. Work is currently ongoing toimplement self-contact and large deformation in the BE dri-ven haptic simulation. Future work also includes extendingthe current implementation to improve the graphical userinterface and simulate two-handed surgery through theuse of two simultaneous PHANToM haptic devices.

Acknowledgements

This research is supported by the UK Engineering andPhysical Sciences Research Council (EPSRC) via GrantGR/R84030/01.

References

[1] Wang P, Becker AA, Glover AT, Benford SD, Greenhalgh CM,Vloeberghs M, et al. Application of the boundary element method tothe simulation of surgery including haptic feedback. In: ToppingBHV, Mota Soares CA, editors. Proceedings of the seventh interna-tional conference on computational structures technology. Lisbon:Civil-Comp Press; 7–9 September 2004. Stirling, paper 100, 2004.

[2] James DL, Pai DK. ArtDefo: Accurate real time deformable objects.In: Proceedings SIGGRAPH99; 1999. p. 65–72.

[3] Gibson SF, Murtich. A survey of deformable models in computergraphics, Technical report TR-97-19, Mitsubishi Electric ResearchLaboratories, Cambridge, MA, November 1997.

[4] Terzopoulos D, Platt J, Barr A, Fleischer K. Elastically deformablemodels. Comput Graph 1987;21:205–14.

[5] Zienkiewicz OC, Taylor RL. The finite element method. Oxford:Butterworth-Heinemann; 2000.

[6] Bro-Nielsen M. Finite element modelling in surgery simulation. ProcIEEE 1998;86(1):490–503.

[7] Zhuang Y, Canny J. Haptic interaction with global deformations.Proc IEEE Int Conf on Robot Auto 2000;3:2428–33.

[8] Woodcock R, Phillips C, Attikiouzel Y. Adaptive refinement andcoarsening for virtual surgical environments. Proc Aust NZ ConfIntel Inform Syst 1994:447–51.


[9] Song GJ, Reddy NP. Towards surgical simulation of cutting in theVR environment. ASME Bioeng BED 1994;28:149–50.

[10] Brebbia CA, Telles JCF, Wrobel LC. Boundary element tech-niques. Berlin: Springer Verlag; 1983.

[11] Becker AA. The boundary element method in engineering. Lon-don: McGraw-Hill; 1992.

[12] Banerjee PK. The boundary element methods in engineering. NewYork: McGraw-Hill; 1994.

[13] Monserrat C, Meier U, Alcaniz M, Chinesta F, Juan MC. A newapproach for the real-time simulation of tissue deformationsin surgery simulation. Comput Meth Prog Biomed 2001;64:77–85.

[14] Wang P, Becker AA, Glover AT, Jones IA, Benford SD, VloeberghsM, et al. Designing a virtual reality simulator for neurosurgery. In:Richir S, Taravel B, editors. Proceedings of the seventh interna-tional conference on virtual reality. Laval, France; 19–22 April 2005.p. 35–9.

[15] Wang P, Glover T, Becker A, Jones A, Benford S, Greenhalgh C, et al.Virtual reality surgery simulation using haptics and boundaryelement techniques. In: Proceedings of the world haptics conference.Pisa, Italy; 18–20 March 2005. [Poster reproduced in proceedings,2005].

[16] Meyer U, Stamm T, Meier N, Joos U. First experience with a publicdomain computer-aided surgical system. Brit J Oral Max Surg2001;40:96–104.

[17] Tsai M, Hsieh M, Jou S. Virtual reality orthopedic surgery simulator.Comp Biol Med 2001;31:333–51.

[18] Delingette H, Cotin S, Ayache N. Efficient linear elastic models ofsoft tissues for real-time surgery simulation. Stud Health TechnolInformat 1999;62:100–1.

[19] Moutsopoulos K, Gillies D. Deformable models for laparoscopicsurgery simulation. Comput Network ISDN Syst 1997;29:1675–83.

[20] Hansen KV, Brix L, Pedersen CF, Haase JP, Larsen OV. Modellingof interaction between a spatula and a human brain. Med Image Anal2004;8:23–33.

[21] Keeve E, Girod S, Kikinis R, Girod B. Deformable modelling offacial tissue for craniofacial surgery simulation. Comput AidedSurgery J Int Soc Comput Aided Surgery 1998;3:228–38.

[22] http://www.openscenegraph.org, [Last accessed 15 September 2006].[23] Portela A. Dual boundary element analysis of crack growth. In:

Brebbia CA, Connor JJ, editors. Topics in engineering, vol.14. Southampton: Computational Mechanics Publications; 1993.

[24] Portela A, Aliabadi MH. The Dual boundary element method:effective implementation for crack problems. Int J Numer Meth Eng1992;33:1269–87.

[25] Press WH, Teukolsky SA, Vetterling WT, Flannery BP. Numericalrecipes in FORTRAN: the art of scientific computing. 2nd ed. Cam-bridge University Press; 1992.

[26] Saad Y, Schultz MH. GMRES: a generalized minimum residualalgorithm for solving non-symmetric linear systems. SIAM, J Sci StatComput 1986;7:859–69.

[27] Barret R, Berry M, Chan TF, Demmel J, Donato JM, Eijkhout V,et al. Templates for the solution of linear systems: building blocks foriterative methods. Philadelphia PA: SIAM; 1994.

http://www.openscenegraph.org

Documents

Virtual reality simulation of surgery with haptic feedback based on the boundary element method