Tablet Driven Paradigm for Hybrid Reality Surgery Interaction

A Tablet- Driven Paradigm For Hybrid Reality Surgery Interaction

A Tablet- Driven Paradigm For Hybrid Reality Surgery Interaction

Presenters: Andrei Marroquin, Hamzah Shannag, Huyen Le, Matthew DunningAdvisor: Dr. Mahesh ShenaiCo- advisor: Dr. Juan Cebral

Extra slide to guide us for the presentation

Focus on the clinical needs:sophisticated visualization methods via hybrid/augmented reality some operations are done blindly (no guidance).In minimally invasive neurosurgeries, surgeons are blind to the interior of the head when introducing instruments (like catheters)The idea: superimpose images on a the physical head during surgery to see the relative placement of the instruments with respect to anatomical structures.To: allow surgeons to be more precise and have more information to guide the procedureGive a few surgical procedures that would benefit from this system. Hydrocephalus is one of themspend the last 50% of your presentation discussing your solution at a level of detail that is easy for a general audience to understand, but with more than general statements.

1

What is Hybrid Reality?The technology of the virtual reality world of computers.

Entering simulated worlds without leaving the physical environment.

IntroductionRelevanceProblem StatementDesignProcessEngineering ChallengesFuture DirectionCopyright businessinsider.comCopyright playgoogle.com

The Clinical NeedVisualization methods via hybrid/augmented reality in the OR.

To address various surgical methods in:Neurosurgery

Impact: Increasing surgical precision and visualization during minimally invasive procedures.

IntroductionRelevanceProblem StatementDesignProcessEngineering ChallengesFuture DirectionCopyright from Dr. Shenai

OR stands for Operating RoomTo address various surgical methods in:Neurosurgery:Stereotactic placement of medical instrumentsLocalization of tumors and other intracranial lesions

3

Possible Applications for Hybrid Reality in NeurosurgeryPlacement of drainage (burr) holes for hematomas.

Localization of tumors and other intracranial lesions.

Stereotactic placement of catheters/electrodes, biopsy needles

IntroductionRelevanceProblem StatementDesignProcessEngineering ChallengesFuture DirectionCopyright medicalimages.comCopyright youtube.com

so the possible applications where hybrid reality could be applied are:

Stereotactic placement of catheters/electrodes, biopsy needles.Localization of tumors and other intracranial lesions.Placement of drainage (burr) holes for hematomas.

Hematoma is swelling of clotted blood within the tissue. The drainage is done where the swelling occurs here point at the left side of the brain picture.4

Stereotactic Placement When Addressing Hydrocephalus.Hydrocephalus:Excessive accumulation of fluid in the brainCauses: Brain damage, loss in mental and physical abilities, and even death

Surgery: Ventriculostomy

Copy right at aboukidhealth.ca

IntroductionRelevanceProblem StatementDesignProcessEngineering ChallengesFuture DirectionCopyright ipcphysicaltherapy.com

Hydrocephalus is a condition where the normal flow of Cerebrospinal Fluid (CSF), produced in the ventricles is interrupted (either by obstruction or a failure of the body to re-absorbed the CSF) causing intracranial pressure to increase and the ventricle to enlarge.

Ventriculostomy surgeryVentriculostomy is the process of draining the excess CSF from the brain to release increased pressure. Incision of the catheter is done blindly, using landmarks to estimate the area for where incision should be done. 5

Ventriculostomy

Done Blindly

Landmarks

IntroductionRelevanceProblem StatementDesignProcessEngineering ChallengesFuture DirectionCopyright orion-group.net

Theres no better way than placing landmarks on the head and assuming proper placement for incision. Wouldnt it be beneficial to superimpose the image on the head and to properly register it for guidance of this surgery? To know exactly where incisions should be made.6

Stealth Navigation

IntroductionRelevanceProblem StatementDesignProcessEngineering ChallengesFuture DirectionExisting MethodologiesCopyright medtronic.com

Copyright brainlab.comBrainLAB Navigation

(Mention that they do not superimpose)

Interfaces with multiple imaging systems (iCT, iMRI)Navigational Camera or Electromagnetic SystemHigh ResolutionEasy to access pre-op exams

7

Available DevicesHead mounted displays (HMD)

Semitransparent mirror reflection

IntroductionRelevanceProblem StatementDesignProcessEngineering ChallengesFuture DirectionCopyright helio-TEKD.comCopyright at Researchgate.net

Head Mounted Displays are one of the most effective optical devices that have been used in various forms to assist surgeons and other medical personnel to support and improve visualization of the work site

Semitransparent mirror reflection: Placed above the patient. The mirror reflects the virtual scene while the surgeon looks through the glass to see the patient. Control software refreshes the virtual images so that the projected scene matches the action in the OR by tracking everything that moves during a procedure.- Should mention that there are no marketed versions of these used today in medicine. And that Researchers have attempted to address these needs.

8

Problems with Existing MethodsSuperimpositionPrice:Stealth Navigation Stealthstation costs 1 million dollars.BrainLAB Navigation Vector Vision-BrainLAB costs $225,000.

Proposed solution:Augmented reality concept (superimposition)Cheaper

IntroductionRelevanceProblem StatementDesignProcessEngineering ChallengesFuture Direction

Copyright trailx.comCopyright audaxceph.com

Superimposition: both the methodologies do not superimpose the images onto the head during surgery, they are only used to display where the surgeon desires to look on one of the methodologys screen next to the operating bed.

Price:Stealth Navigation The Stealthstation roughly costs 1 million dollars.BrainLAB Navigation Vector Vision-BrainLAB costs $225,000.

Proposed solution:Augmented reality concept (superimposition)The device built along with the software is significantly cheaper for hospitals to purchase.

Engineering Problem: Aligning/registering medical images with physical objects simultaneously.9

IntroductionRelevanceProblem StatementDesignProcessEngineering ChallengesFuture DirectionCopyright orion-group.netCopyright synapse.korwamed.org

Theres no better way than placing landmarks on the head and assuming proper placement for incision. Wouldnt it be beneficial to superimpose the image on the head and to properly register it for guidance of this surgery? To know exactly where incisions should be made.10

45o

h13D-printed clampTripodOperating TableSemi- reflective screenTableth2Foam headSurgeon


So what will this project bring?Develop a digital platform, which displays pre-operative images onto a surgical field in an anatomically correct method.Provide digital landmarks for real-time surgical procedures.Observe internal anatomy without incisions nor surgical exposure.Reduce patient risk associated with unguided procedures.

Adjustable angle depends on:Height of the physician h1 which ranges 47- 74Adjustable height of the tripod holding the tablet h2. ranges 16- 72Height of the operating bed h3Angle is approximated by the equation:Measuring the angle from the face of the tablet to the horizontal operating bed and subtracting 45 from it45 is the angle between the semi-reflective glass and the tablet.So is most often approximately 45 degreesThe CT scan is resliced at angle to view the desired angle of the rendered volume.

Tablet size at least 10 inches for the size of the head or foam head

11

Pictorial Demonstration


This is our setting that we have set up in dr cebrals lab. We shot it from different angles to show how its put together and the different parts.12

How are we conducting the project ?


CT-scan ImagesSoftwareOptical DeviceImaging Re-slicing

RegisteringSuperimposingRendering Projecting


Rendering the 3D images obtained from a CT scan of the foam head. (MATLAB).Utilizing an optical device to superimpose the images onto the foam head. (Tripods, reflective plexigalss, tablet, 3D-printed clamp, foam head).Optical device screen must be perpendicular to the surgeons eye-sight.Tablet is 45* degrees from the optical device screen.Brain images displayed on the tablet and are reflected onto the perpendicular screen for real time guidance of ventriculostomy.SooDisplaying images obtained from a CT scan of the foam headBuilding an optical device to support the superimposition processUsing the fiducials on the foam head to align and register the images when superimposed onto the foam head

13

CT scan of the Foam Head




CT-scan ImagesSoftwareOptical DeviceImaging

Re-slicing




15

Methods - Programming


16



CT-scan ImagesSoftwareOptical DeviceImaging

Re-slicing




17

Methods Optical DeviceTripods

Adjustable Reflective screen & tablet

Foam head - fiducials

Machine Shop3D-printed table clampTools: drilling, cutting, etc.


Were mounting the tablet and semi-reflective screen to the tripods for three dimensional orientation, since surgeons can be different heights18

Foam Head Fiducials



This shows the image being displayed on the tablet that is superimposed onto the foam head.20

Physical Prototype - Video

Future Plans


Improve Code:Develop GUI- Graphical User InterfaceIncrease speed

Improve Optical device:Improve stabilityReduce area taken by the device

Validation- Testing:CT-scan cadaver head with fiducialsRun code & superimposeAttempt ventriculostomyCT-scan again with catheter insertedMeasure distance between the ROI and the placement of the catheterDetermine the percent error of incision

Time Spent & BudgetTime: Each member spent approximately 8 hours a week.Budget: $200 Windows tablet$100 optical device$10 foam head

Dr. Cebral Guidance for the physical design, methodology of the code, and how to prepare a professional PowerPoint.Dr. Shenai Aim and purpose of the project, CT scan of the foam head.Dr. Bray Guidance to where we stand and ideas regarding the project.Dr. Cressman Using the equipment at the machine shop and guidance to designing the required table clamp.Dr. Chitnis Thoughts regarding the code.George Mason University Providing the machine shop.

ReferencesFacts about hydrocephalus, National Hydrocephalus Foundation, http://nhfonline.org/facts-about-hydrocephalus.htm(n.d.). Retrieved November 30, 2015, from http://www.nhs.uk/conditions/chiari-malformation/documents/etv.pdfBrain Health Community. (2012, August 21). Retrieved November 30, 2015, from http://www.wellsphere.com/brain-health-article/hydrocephalus-and-treatment-shunts-and-endoscopic-third-ventriculostomy/1727082Jones RFC, Kwok BCT, Stening WA, & Vonau M: The current status of endoscopic third ventriculostomy in the management of non-communicating hydrocephalus. Minim. Invas. Neurosurg, v. 37, pp. 28-36, 1994.Perneczky A, Tschabitscher M, & Resch KDM: Endoscopic Anatomy for Neurosurgery. New York: Thieme Medical Publishers, Inc., 1993. Assessment of ETV patency with the 3D-SPACE technique. (2015). Daily Bibliographic and Video Review of the Neurosurgery. doi:J Neurosurg 122:13471355, 2015A Novel Protocol of Continuous Navigation Guidance for Endoscopic Third Ventriculostomy. (2014). Daily Bibliographic and Video Review of the Neurosurgery. doi:10:514524, 2014

26

Extra - 3D-Printed Clamp

Extra - Reflective-Mounted Screen

Extra - Tablet-HolderDouble-sided sticky pads

Extra - Tripods

CT-scan Re-slices (proof that our software works)

90 degrees45 degrees