Enabling Peripheral Vision in Laparoscopic surgery using a Robotic arm integrated with Camera

Abstract

Our design idea presents a novel proposition for enabling peripheral vision in laparoscopic surgery. It

focuses on a new system of camera assembly employing fish-eye lens with reduced spherical aberration,

camera positioning using a robotic arm and integration of the robotic arm with the existing trocar. An

electro-mechanically controlled robotic arm enabling efficient camera positioning, camera locking and

emergency camera removal is designed to fit and lock into a trocar groove without hindering surgical

instrument handling and control. Thus, the design model eliminates the present need of having a separate

trocar dedicated for endoscopy. As our camera is placed in an optimum position providing a peripheral

vision, the continuous need of adjusting the camera’s position during the surgery is absolutely eliminated.

( Fig.1 - An overall diagram explaining our design model )

* All drawings in this abstract have been sketched by hand and scanned.

Background of the invention

The existing endoscope system in laparoscopic surgery is typically used to provide tubular vision of the

internal operating organs. This system limits the field of vision of the surgeon and also constrains the

position of camera placement limiting the surgeon’s efficiency.

In order to overcome these constraints, a possible solution of assembling a combination of lenses that

would replicate the cornea of the human eye was analyzed. The problem of varying refractive index and a

high spherical aberration in the cornea made us negate this option. In the present day scenario, fish eye

lens assembly is considered to be the best option for providing peripheral vision with reduced spherical

aberration. After analyzing various fish eye lens combinations, the best possible combination was chosen

to serve our miniature camera requirements.

Even though this combination of lens assembly increased the field of vision, the camera assembly and

camera position was limited by the angle of insertion of the trocar and the trocar placement. To overcome

this problem, an electro-mechanically controlled arm is designed that can position the fish eye lens camera

assembly optimally to provide peripheral vision in the region of surgeon’s interest.

In the existing system, a separate trocar is dedicated to the use of an endoscope. As it increases the pain

and the healing time for the patient, we extended our design that eliminates the requirement of a

separate trocar for endoscope by integrating the robotic arm carrying the camera assembly into one of the

trocars used for surgical instruments.

Horizontal Extension Arm

Console

Image Processing Power Unit

Cropping Filters

Surgical Instruments Peripheral Projector

Analog TV Output

Digital Output

( Fig.2 – Components involved in the design )

Control Handle Modified Trocar

Individual components explained

The camera assembly

Lens: An 1800 fisheye lens assembly with the aperture ratio F/4 has been selected for use in our camera assembly. The fish eye lens system includes: a pair of negative meniscus lenses (L1 & L2), a negative cemented doublet composite lens (L3) with negative refractive power, and a positive fourth lens (L4) having a convex surface on the object side. A filter (L5) is positioned before the diaphragm. A second lens group is positioned on the image side of the diaphragm and consists of three lenses (L6, L7 & L8) of which at least two are cemented doublet lenses. Replacing the conventional lens with a fish eye lens enables us to converge light at the periphery subtended at an angle of 180o. The lens assembly reduces spherical aberration to a maximum and is convenient for miniaturization.

Lighting: As shown in fig.4, the light coming from an LED is directed onto a collimator, which transforms the light source into a parallel beam. This parallel beam is incident on a partially reflecting mirror and reflected through 900 which are fed into the L8 lens of the fish eye lens assembly. This system of using a fish eye lens for in-vivo illumination provides an even distribution of light around the periphery.

Imaging device comprises of a miniature CCD (Charge Coupled Device) which captures light at the focus of the fish eye lens assembly and is sent to the image processing setup (console) through the cabling. A conventional autofocus mechanism can be used to position the CCD for optimal video sharpness.

Dual axis control: In spite of the peripheral vision provided by the camera assembly, it is necessary that the camera is initially rotated to view the region of surgeon’s interest and the position is locked. A miniature spherical pointing motor attached to the camera assembly orients it in the required direction.

The assembly: The lens, illuminating source (LED) and the CCD are assembled as shown in fig.4. The partially reflecting mirror permits the use of the same fish eye lens for both illumination and video capture. The power input is provided to a common circuit board, which distributes the power to the LED and the imaging apparatus. A wiper is incorporated at the outer periphery of the assembly to prevent fogging and is controlled at the control handle.

LED Illumination Source

Fish-eye lens Assembly

Collimator CCD Circuit Board

Partially Refracting Mirror

Wiper Control Mechanism

( Fig.3 – Fish-eye lens system)

( Fig.4 – The Camera Assembly)

In-vivo surveillance arm

The following presents a step-wise diagram indicating the positioning of the surveillance arm inside the body:

Step 1: Insertion of vertical arm Step 4: Horizontal arm linkage Locking Step 2: Vertical arm-trocar groove locking Step 5: Horizontal arm extension Step 3: Lifting of horizontal arm Step 6: Surgical instrument insertion

Linear insertion arm (Vertical arm): The vertical arm is

2mm in diameter and is provided with a slider

mechanism at the periphery to control the upward and

downward movement of the horizontal arm. The

vertical arm has two magnets at the periphery, which

get attached to an electro magnetic plate provided

within the groove of the trocar.

( Fig.5 – Step-wise diagram indicating the positioning of the surveillance arm )

( Fig.6 – Modified Trocar groove for vertical arm insertion)

Motor assembly for arm extension/retraction

Telescopic Horizontal Arm Camera

Vertical Arm (2 mm dia)

Horizontal arm (7 mm dia)

Trocar bore

Electro-magnetic lock plate

Trocar groove

Step 1 Step 2 Step 3

Step 4 Step 5 Step 6

Skin

Trocar groove

Trocar

Trocar Lock

Surgical Instrument

Linkage

Horizontal-Vertical Linkage: A metallic string moves the slider along a thin slot that controls the upward and downward motion of the horizontal arm. The joint is covered with a protective elastic membrane to prevent accumulation of impurities within the system.

Extendable arm (Horizontal arm):

The fig.8 (a) indicates the position of the horizontal arm when it is fully retracted within the system. By controlling a motor placed as shown, the rotation of the motor causes each extendable telescopic rod to move out thereby moving the camera to an optimal position for best video capture. Each telescopic extendable piece is provided with an end-hooking facility (typically used in a radio aerial antenna) which makes the next outer telescopic sleeve to move along with the extending arm. The gear wheel connected with the motor is spring loaded from the top to continually ensure a downward force to keep it in contact with the telescopic outer periphery teeth.

The fig.8 (b) indicates the position of the horizontal arm when it is fully extended. After the horizontal arm is fully extended to the required position, the entire system is locked by means of a universal lock system present in the control handle (discussed below).

( Fig.7 – Horizontal arm lifting )

Motor

Down-Force Spring

Camera

Retracted Arm Extended Arm

( Top View )

Teeth for Linear motion

( Front View )

( Side view )

( Fig.8 – Squiggle motor controlled horizontal arm extension )

Fig.8 (a)

Fig.8 (a)

Fig.8 (b)

Fig.8 (b)

Metal String

Slot

Metallic linkage

Slider

Elastic membrane

Teeth for Linear motion

Gear wheel

Transmission belt

Trocar Modification:

Existing Design Functionality: The surgeon places the trocar by skin incision on the body cavity and the appropriate

surgical instrument is then inserted into the trocar. Presently, several trocars are inserted into the patient’s body

making the area look more like a pin cushion stand. A minimum of three trocars is essential presently - one

separately for insertion of the tubular camera for in-vivo viewing and atleast a couple more for surgical instrument

insertion.

Design Modifications for the trocar: The modified trocar design assembly will be provided with a 3mm diameter

groove at one side of the outer periphery of the existing bore having a locking mechanism for fastening the in-vivo

surveillance vertical arm (2 mm diameter) to the trocar.

A control handle is integrated at the head of the trocar having several functions for controlling the movement of the

surveillance arm.

Modified Trocar

Horizontal Extension Arm

Emergency Button

Wiper Activation

Linkage Control Extension Arm Control

Surgical Instrument

Universal Lock

Camera Rotation Control

Groove Lock LED

Trocar Lock

Surgical Instrument

Electrical output to console

( Fig.9 – Modified Trocar )

Trocar Groove with Vertical Arm

Vertical Arm

The Control Handle

The control handle is optimally designed to control the functions of positioning and placing the camera for best viewing, activating a wiper system for sweeping the outer spherical dome, providing an electro-magnetic supply for fixing the vertical insertion arm to the trocar groove, a universal lock for holding the assembly in a fixed position and an emergency retrieval system for retracting the entire system assembly.

Wiring and metallic strings: Fig.11 indicates the wiring scheme used in the surveillance arm. A system of pulleys at

the joint of the horizontal and vertical arm ensures smooth movement of the metallic strings when it is bent from

the vertical to the horizontal arm. The red colored lines represent the electrical wiring and the blue lines represent

the mechanically controlled strings. All the electric wires are bundled together through a single cable (red colored

and enlarged in the figure).

Horizontal Extension Arm

Emergency Button

Wiper Activation

Linkage Control Extension Arm Control

Surgical Instrument

Universal Lock

Camera Rotation Control

Groove Lock LED

Modified Trocar

Trocar Lock

Surgical Instrument Electrical output to console

( Fig.10 – Trocar with control handle)

( Fig.11 – Wiring and metallic strings )

Electrical wire bundle Magnified

Power (CCD & LED)

CCD Output

Power (rotation-tilt)

Pulley aiding string movement

Control handle

Vertical arm

Mechanically controlled Metallic string

Groove-vertical arm locking: The inner periphery of the groove is fitted with an electro-magnetic plate. Two

magnets placed at a strategic position on the outer vertical arm get attracted and fix onto the electro-magnetic plate

when a minimum charge is applied to the electro magnetic plate by a push button provided on the handle.

Positioning and Placing of Camera: A rotary cylindrical dial is calibrated with a mechanical stop. Rotation of a dial at

the handle winds and unwinds the metallic string around the cylinder to move a slider along a thin slot that controls

the upward and downward motion of the horizontal arm through a linkage as shown in the fig.7.

The extension and retraction of the horizontal arm is controlled by another calibrated push button which when

activated controls the rotation of a motor placed on top of the camera that extends each of the telescopic arm as it

rolls out in one direction. The rotation of the motor in the opposite direction retracts the telescopic arm.

Vertical Arm

Motor assembly for arm extension/retraction

Telescopic Horizontal Arm

Modified Trocar Bore

Trocar groove with electro-magnetic plate

( Fig.12 – Groove-vertical arm locking )

( Fig.13 – Positioning and Placing of Camera )

Camera

Wiper Activation: Due to the problems of fogging and condensation on the outer surface of the camera dome, a

wiper system along the outer contour is provided with a foam material for the wiping action. The wiper is

mechanically actuated in one direction using metallic string from the handle. After sweeping 1800 around the outer

contour, a spring-loaded mechanism pushes the wiper back to its original position.

Universal Lock: A spring-loaded clamp is used to hold the metallic strings that hold the horizontal arm linkage. This

clamp ensures that the whole horizontal assembly is held at a fixed position without proving a hindrance to the

surgical instrument movement.

Emergency Button: An emergency button is provided at the bottom of the control handle for quick retrieval of the

entire assembly out of the trocar. When the emergency button is pressed, a programmed microchip is integrated

into the control handle to perform the following sequence of operations:

1) The motor controlling the horizontal arm is rotated in the opposite direction to quickly retract the arm.

2) The universal clamp releases the wires holding the linkage to the horizontal arm causing the slider to make the

horizontal arm fall back to its original vertical position.

3) The electric supply to the electro magnetic plate at the inner periphery of the groove is switched off to unlock

the vertical arm from the trocar groove.

4) A LED glows to signal that the sequence of operations is complete and the arm maybe pulled out of the trocar.

Spring-loaded Return Mechanism

Mechanically controlled wiper string

Wiper Arm

( Lateral View ) ( Front View )

( Fig.14 – Wiper Mechanism)

Peripheral video projection

After a thorough study of the existing peripheral

projection methods, the following two projection

principles have been chosen for optimal peripheral

view.

In the first method, the image from the CCD device is

directly fed onto a fish-eye projection lens without

applying any filters in the image processing stage and

the peripheral image is retained. The fish-eye lens

projects the image onto a hemi-spherical dome screen,

which is located at a position which is best suitable for

viewing by the surgeon.

If a compromise can be made on the

peripheral dome imaging option and the

surgeons feel that the same image

projected on a curved screen would prove

sufficient, a simpler and an economically

more viable solution maybe opted. In this

method, the video from the CCD device is

processed with anti-spherical filters in the

console and is projected onto a curved

screen using a conventional LCD projector.

Additional Analog Output:

Processed video output from the console is made available simultaneously both on an analog TV device and a

computer for recording, freezing frames during surgery and detailed examination.

( Fig.15 – Hemi-spherical dome video projection )

( Fig.16 – Alternative flat screen video projection )

11 | P a g e

Economic feasibility

The final design model proposed above has been arrived at after careful synthesis and analysis at every

stage for its economic feasibility and ease of material availability.

In the first stage, usage of various lenses for obtaining peripheral vision was studied. Though liquid lenses

can be effectively customized to replicate the human cornea, this option was negated due to the cost

factor involved in fabrication and assembly setup. The above constraints also prohibited us from using

dimpled lenses which is again a promising option for enabling peripheral vision. The extensive application

of the fish-eye lens assembly in everyday usage like: door pinhole lens, Spy CCD cameras, etc. makes it an

easily available and accessible component. Moderate market pricing and extent of miniaturization already

achieved made us choose this option.

In the second stage, when various operational modes of the robotic arm like pneumatic, hydraulics,

electronics and mechanics were taken into consideration, mechanically controlled system was chosen due

to its reliability, cost effectiveness, higher precision and ease of operation.

In the last stage of video projection, considering the economic demand and space constraints for a

peripheral dome setup, the output video is presented on a curved screen using a conventional LCD

projector after applying various image processing filters.

12 | P a g e

Patents Referred

United States Patent #4070098 – Fisheye projection lens system for 35mm motion pictures

United States Patent #4966454 – 3-D Motion Picture Projector

United States Patent #4685450 – Endoscope

United States Patent #4403605 – Endoscope Photographing System

United States Patent #64716371- Image orientation for endoscopic video displays

United States Patent #5949430 – Peripheral lenses for simulating peripheral vision on a display device

United States Patent #5475420 – Video Imaging System with Image Processing optimized for small diameter endoscopes

United States Patent #6734893 – Endoscopy Illumination system for Stroboscopy

United States Patent #4616226 – Peripheral Vision Artificial Horizon Device

United States Patent #4855838 – Remotely Controlled Pan and Tilt Television Camera

United States Patent #4678289 – Apparatus for Deflection of a Light Beam

United States Patent #4639772 – Focusable Video Camera for Use with endoscopes

United States Patent #5147316 – Laparoscopic Trocar with Self-locking port sleeve

United States Patent #7256834 – Digital Camera with Panning/Tilting functionality

United States Patent #6844991 – Fish-Eye Lens

United States Patent #4647161 – Fish-Eye Lens system

United States Patent #6301058 – Wide Angle Lens

United States Patent #5122122 – Locking Trocar Sleeve

United States Patent #4890713 – Pan and Tilt Motor for Surveillance Camera

United States Patent #4656506 – Spherical Projection System

United States Patent #4654030 – Trocar

United States Patent #4601710 – Trocar Assembly

United States Patent #5693967 – Charged Coupled device with micro-lens